I’m at the PAPIs (Predictive Applications and APIS) [twitter: papistotio] conference at the NERD Center in Cambridge.

The first speaker is Cynthia Rudin, Director of the Prediction Analysis Lab at MIT. Her topic is “Regulating Greed over Time: An Important Lesson for Practical Recommender Systems.” It’s about her Lab’s entry in a data mining competition. (The entry did not win.) The competition was to design a better algorithm for Yahoo’s recommendation of articles. To create an unbiased data set they showed people random articles for two weeks. Your algorithm had to choose to show one of the pool of articles to a user. To evaluate a recommender system, they’d check if your algorithm recommended the same thing that was shown to the user. If the user clicked on it, you could get an evaluation. [I don’t think I got this right.] If so, you sent your algorithm to Yahoo, and they evaluated its clickthrough rate; you never got access to Yahoo’s data.

This is, she says, a form of the multi-arm bandit problem: one arm is better (more likely to lead to a pay out) but you don’t know which one. So you spend your time figuring out which arm is the best, and then you only pull that one. Yahoo and Microsoft are among the companies using multi-arm bandit systems for recommendation systems. “They’re a great alternative to massive A-B testing

• ” [Alternative view

] [No, I don’t understand this. Not Cynthia’s fault!.].

Because the team didn’t have access to Yahoo’s data, they couldn’t tune their algorithms to it. Nevertheless, they achieved a 9% clickthrough rate … and still lost (albeit by a tiny margin). Cynthia explains how they increased the efficiency of their algorithms, but it’s math so I can only here play the sound of a muted trumpet. But it involves “decay exploration on the old articles,” and a “peak grabber”: If any articles gets more than 9 clicks out of the last 100 times they show the article, and they keep displaying it: if you have a good article, grab it. The dynamic version of a Peak Grabber had them continuing to showing a peak article if it had a clickthrough rate 14% above the global clickthrough rate.

“We were adjusting the exploration-exploitation tradeoff based on trends.” Is this a phenomenon worth exploring?The phenomenon: you shouldn’t always explore. There are times when you should just stop and exploit the flowers.

Some data supports this. E.g., in England, on Boxing Day you should be done exploring and just put your best prices on things — not too high, not too low. When the clicks on your site are low, you should be exploring. When high, maybe not. “Life has patterns.” The Multiarm Bandit techniques don’t know about these patterns.

Her group came up with a formal way of putting this. At each time there is a known reward multiplier: G(t). G is like the number of people in the store. When G is high, you want to exploit, not explore. In the lower zones you want to balance exploration and exploitation.

So they created two theorems, each leading to an algorithm. [She shows the algorithm. I can’t type in math notation that fast..]

Be the first to comment »

At the IAB conference, Frances Donegan-Ryan from Bing begins by reminding us of the history of online search.

We all leave digital footprints, she says. Every time we search, data is recorded. The sequence of our searches gives especially useful information to help the engine figure out what you’re trying to find out. Now the engines can refer to social graphs.

“But what do we do with data?”

Bing Predicts
looks at all the data it can in order to make predictions. It began by predicting the winners and losers in American Idol, and got it 100% right. For this election year, it tried to predict who would win each state primary or caucus in the US. Then it took in sentiment data to figure out which issues matter in each state, broken down by demographic groups.

Now, for example, it can track a new diabetes drug through the places people visit when logged into their browser. This might show that there are problems with the drug; consider for example people searching for unexpected side effects of it. Bing shares the result of this analysis with the CDC. [The acoustics where I was sitting was poor. I’m not sure I got this right.]

They’re doing the same for retail products, and are able to tell which will be the big sellers.

Frances talks about Cortana, “the only digital system that works across all platforms.” Microsoft is working on many more digital assistants — Bots
— that live within other services. She shows a temporary tattoo
made from gold leaf that you can use as a track pad, and other ways; this came out of MIT.

She says that the Microsoft version of a Fitbit can tell if you’re dehydrated or tired, and then can point you to the nearest place with water and a place to sit. Those shops could send you a coupon.

She goes quickly over the Hololens since Robert Scoble covered it so well this morning.

She closes with a story about using sensor data to know when a cow is in heat, which, it turns out, correlates with them walking faster. Then the data showed at what point in the period of fertility a male or female cow is likely to be conceived. Then they started using genetic data to predict genetically disabled calves.

It takes enormous computing power to do this sort of data analysis.

Be the first to comment »

I’m at the IAB conference in Toronto. Canada has a privacy law, PIPEDA law (The Personal Information Protection and Electronic Documents Act) passed in 2001, based on OECD principles.

Barbara Bucknell
, the director of policy and research at Office of the Privacy Commissioner where she worries about how to protect privacy while being able to take advantage of all the good stuff data can do.

A recent large survey found that more than half of Canadians are more concerned about privacy than they were last year. Only 34% think the govt is doing enough to keep their privacy safe. Globally, 8 out of 10 are worried about their info being bought, sold, or monitored. “Control is the key concern here.” “They’re worried about surprises: ‘Oh, I didn’t know you were using my information that way!'”

Adam Kardash [this link
?] says that all the traditional approaches to privacy have be carefully reconsidered. E.g., data minimization says you only collect what you need. “It’s a basic principle that’s been around forever.” But data scientists, when asked how much data they need for innovation, will say “We need it all.” Also, it’s incredibly difficult to explain how your data is going to be used, especially at the grade 6-7 literacy rate that is required. And for data retention, we should keep medical info forever. Marketers will tell you the same thing so they can give you information about you what you really need.

Adam raises the difficulties with getting consent, which the OPC opened a discussion about. Often asking for consent is a negligible part of the privacy process. “The notion of consent is having an increasingly smaller role” while the question of control is growing.

He asks Barbara “How does PEPIDA facility trust?”

Barbara: It puts guardrails into the process. They may be hard implement but they’re there for a reason. The original guidelines from the OECD were prescient. “It’s good to remember there were reasons these guardrails were put in place.”

Consent remains important, she says, but there are also other components, including accountability. The organization has to protect data and be accountable for how it’s used. Privacy needs to be built into services and into how your company is organized. Are the people creating the cool tech talking to the privacy folks and to the legal folks? “Is this conversation happening at the front end?” You’d be surprised how many organizations don’t have those kind of programs in place.

Barbara: Can you talk to the ethical side of this?

Adam: Companies want to know how to be respectful as part of their trust framework, not just meeting the letter of the law. “We believe that the vast majority of Big Data processing can be done within the legal framework. And then we’re creating a set of questions” in order for organisations to feel comfortable that what they’re doing is ethical. This is very practical, because it forestalls law suits. PEPIDA says that organizations can only process data for purposes a reasonable person would consider appropriate. We think that includes the ethical concerns.

Adam: How can companies facilitate trust?

Barbara: It’s vital to get these privacy management programs into place that will help facilitate discussions of what’s not just legal but respectful. And companies have to do a better job of explaining to individuals how they’re using their data.

Be the first to comment »

I’m at a IAB conference in Toronto. The first speaker is Robert Scoble, who I haven’t seen since the early 2000s. He’s working at Upload VR that gives him “a front row seat on what’s coming.”

WARNING: Live blogging. Not spellpchecking before posting. Not even re-reading it. Getting things wrong, including emphasis.

The title of his talk is “The Fourth Transformation: How AR and AI change everything.”

First: The PC.

Second: Mac and GUI. Important companies in the first went away.

Third: Mobile and touch. Companies from the second went away.

We’re now getting a taste of the fourth: Virtual Reality and Augmented Reality. Kids take to VR naturally and with enthusiasm, he notes.

“Most people in the world are going to experience with VR with a mobile phone because the cost advantages of doing that are immense.” This Christmas Google will launch its Tango sensors that map the world in 3D. Early games for the Tango phone will give a taste of AR: mapping the physical space and put virtual things into it. Robert shows what’s possible with the Tango phone. Retail 411 is working on bringing you straight to the product you want in a physical store. This tech will let us build new games, but also, for example, put a virtual blue line on a floor to show you where your meeting is. Or, in a furniture store it can show you the items in a vision of your home.

Robert calls AR “Mixed Reality” because he thinks AR refers to the prior generation.

Vuforia was designed for mobile phones, placing virtual objets in real space. But soon we’ll be doing this with glasses, Robert says. Genesis [?] puts a virtual window on your wall. Click on it, and zombies crawl through it and come toward you.

Magic Leap got huge investments because the optics of the glasses they;re building are so good. He points out that the system knows to occlude images by interfering real world objects, e.g., the couch between you and the zombie.

He shows a Hololens app preview. Dokodemo Teleportation Door, made in Unity. You place a door on the ground. Open it. There’s a polygonal world inside it. Walk through the door and you’re in it.

Robert says Apple ditched the headphone jack in order to put advanced audio computing in your head, replacing ambient sound with processed sound that may include virtual audio.

Eyefluence builds sensors for eyes. Robert shows video of someone navigating complex screens of icons solely with his eyes. “Advertisers will be able to build a new kind of billboard in the street and know who looked at it.” [Oh great.]

ActionGram puts holograms into VR. [If you need a tiny George Takei in your living room — and who doesn’t? — this is for you.]

SnapChat bought a company that puts a camera in glasses. SnapChat is going to bring out a connected camera. It could be the size of a sugar cube.

Sephora has an app that shows you how their makeup looks like on your face, color matched.

Robert talks about the effect on sports. E.g, Nascar has 100+ sensors in cars already Researchers are putting sensors in NFL players’ tags for “next gen stats.”

“We’re in the Apple II stage” of this. It wasn’t great but kicked off a trillion dollar industries. Robert’s been told that we’re two years away, but says maybe it’s four years. “The new Ford cards are all built in virtual reality…If you don’t have a team thinking about working in this new world, you’ll be at a disadvantage soon.”

“This is the best educational technology humans have ever invented.”

This is intensely social tech, he says. You can play basketball or ski jumping with your friends over the Internet. He shows a Facebook demo. You can share things with others, things with media inside of them. E.g., go to a physical space and see it together. [Very cool demo. I think this is it:]

Be the first to comment »

Top 10 new names for Ben & Jerry’s coffee ice cream to convince them to bring it back. #BringBackCoffee @benandjerryspdx

10. Coffee Hold the Gimmicks

9 . Coffee with OMG SO MUCH Cream and Sugar. Also, It’s Frozen.

8. Coffee Uncrunchy

7. St. Agnes‘ Coffee Purity

6. Coffee Coffee Reanimation

5. Larry David’s I Said I Don’t Want Anything In My Cone Except Coffee

4. Coffee Shutup

3. Jack Nicholson’s Coffee and Chicken Salad Sandwich on Wheat Toast

2. What Part of Coffee Do You Not Understand?

1. Just Fucking Coffee

Be the first to comment »

We made the odd choice of replacing our miracle composite kitchen floor — zero upkeep — with a knotty pine floor. Pine is beautiful, and we think it helps make our kitchen look more inviting, but it’s very soft wood.

So we’re conditioning it with tung oil. Tung oil penetrates the wood and polymerizes, hardening it while enriching the color and giving it a satiny sheen. The floor will still be softer than our miracle composite, but you have to resign yourself to thinking of the dents and scratches as signs of its being lived in. Or on. Also, because tung oil penetrates the wood and isn’t a layer on top, you can sand out some dings, and you can always wipe on a little more oil. We may live to regret it, but we like it so far.

Unfortunately, tung oil is a pain in the tuchus when compared to, say, polyurethane. You brush poly on, you let it dry, you lightly scratch it up with brillo or sandpaper, and you do it again. Boom done. Tung oil takes several coats, it smells, it takes longer to dry before the next coat, and it takes much longer to fully polymerize.

After a lot of research — Thank you, Internet — we decided on going with Real Milk Paint tung products. The rational reasons are:

1. They seem to have high quality products. Since there are various types of tung oil pretenders on the market, that counts.

2. For the initial applications, the Internet recommends cutting the tung oil with a solvent. Real Milk has a pre-mixed prep called Half and Half that cuts the tung with citrus oil. True to the claims, we found that it dries quickly and doesn’t smell bad — sort of citrusy, unsurprisingly. (Nevertheless, we trained a fan over the floor while it dried to blow the odor away from us humans.)

But the real reason we went with Real Milk is that they seem like Real People who know their tung oil. I came to this conclusion by reading their discussion boards and watching their videos. They seem to be craftspeople who love finishes that bring out the beauty of the wood they have just worked. They are straightforward and non-defensive. They are on the side of their customers.

I confirmed this minutes ago by calling customer support with a question and talking with a couple of folks there. Our third coat wasn’t drying. They told me what to do about it (dry it) and reassured me that this is in fact a sign that the wood has been saturated. Now we just have to walk carefully on it for a month until it’s fully set.

Could I be wrong about the people and the company? Absolutely. I’m wrong about most things. Maybe they’ll turn out to be the robotic face of a Big Tung, a mega-corporation peddling relabeled motor oil drained from Chernobyl. But I will have at least been fooled for the right reasons.

And that counts for a lot.

3 Comments »

I heard that Microsoft has some excellent $0.10 deals for Windows 10 owners like me. So I checked Bing:

The top hit (an ad by Microsoft) takes you to a page for corporate sales of Windows phones.

The second hit (an ad by Microsoft) takes you to the generic Microsoft Store front page from which it is virtually impossible to find the $0.10 sales.

None of the rest of the results on the first page of the Bing search gets you anywhere close.

Same search at Google:

The top hit (a Microsoft ad) takes you to the same generic front page of the Microsoft Store as the second hit on Bing, which makes no mention of the $0.10 sales.

The following Google results take you to pages about the $0.10 sales from which you can actually get to the goddamn sale.

Yes, these sales are real. For example, this is from the Microsoft.com site this afternoon:

I got there by going to the WindowsCentral.com post listed in the Google results….although right now the Windows site is telling me that something is wrong and I should come back later.

PS: To get to the Hitman Go sale, my best advice is to go to the Windows Store on your Windows 10 machine. The $0.10 sales are featured there. Or search there for Hitman Go.

1 Comment »

Adblocking is, as Doc Searls claims, “the biggest boycott in human history.” Since August 12, Doc’s been posting what I can only call an in-depth, analytical, evidence-based rant. It is not to be missed.

Be the first to comment »

Back in the early 1980s—yes, children, it’s time for an anecdote from the Dark Ages—WordPerfect was my writing tool. I was a power user and was quite attached to it. But there were some things I thought they could do better. So, I wrote a four page letter that was (as I recall) very appreciative of the program overall — not a set of gripes, but a fan’s notes. I sent it to the WordPerfect corporation.

I never heard anything back. Not even the form letter I expected.

That was back then.

On my Mac I frequently use Sync2Folders “its techie rawness is one of the reasons I like it”to, well, sync two folders. It does exactly what I want, and it’s free, although donations are suggested. (I’ve donated the suggested €6 more than once.)

In terms of the look and feel, Sync2Folders isn’t slick, and in its functionality it tends towards the techie. But it’s simple enough that I can do the basic things that I want to do. In fact, its techie rawness is one of the reasons I like it: It does a job that’s not trendy, and it does it without gussying itself up.

Also, and perhaps more important, it looks like something that a developer created and put out in the world for free. Which is exactly what it is.

A couple of days ago I got an automated email from the developer, Thomas Robisson when I donated for the third time. I’d like to pretend that I’m just that generous, but the truth is that I’m just that forgetful. So, I appreciated that the developer noted the duplication, told me how to avoid the app’s request for fiscal aid, and reminded me that a single license can be used on multiple computers.

I responded by email to thank Thomas, and also to point out a feature that I’d like and that I’d thought was in an earlier version. I was confident that this was going to turn out to be a DUM— a dumb user mistake — and at least I was right about that.“ The Net occasions the generosity of people like Thomas” Over the course of a couple of emails in which Thomas asked for some basic debugging info, it turned out that, yes, I had simply missed the button that did what I was asking for. D’oh.

I know that the Internet is the defiler of youth and the death of civilization. But it also occasions the generosity and creativity of people like Thomas.

Further, before the Net, there was only the slightest chance that a user and a product creator could engage. And if they did it was likely to be in the stilted, inhuman voice of the Marketing department.

So, thank you, Thomas. And thank you, Internet.

2 Comments »

We should be grateful that Facebook has renamed its Internet access service from Internet.org to Free Basics by Facebook. The idea is that if you’re in the developing world, you’ll get access to the “Internet” which is really access to Facebook and all that it permits.

Calling that arrangement “Internet.org” was as Orwellian as marketing gets, like advertising Snickers as a “lunch bar.” No no no. A Snickers bar may be delicious, and may even give you enough of a burst of energy that for the final fifteen seconds of your Powerpoint presentation at the weekly status meeting you have an overbearing confidence that alienates your boss’s boss who happens to have dropped by, dooming your long-term prospects at that company, but it is not lunch. It lacks all the essential properties of lunch, even if you may at some point eat one because you forgot your lunch and your wallet and have no friends who will share with you.

The Facebook service is to the Internet as Snickers is to lunch: a poor replacement that lacks all of the essential elements that make a lunch a lunch and the Internet the Internet.

The new name has the advantage of sounding like an hypoallergenic mascara that’s hired Christie Brinkley as its spokesmodel.

1 Comment »