March 2010 Archives

One of the latest ideas to hit the buzz circuit is the concept of "digital intuition" - introduced by my6sense, a company which has developed a tool that serves up the most relevant information for us. They've developed a recommendation engine which TechCrunch says "separates the signal from the noise and helps users shift their attention to the content they care about most."

The application learns what you like, then finds more.

The Independent describes it as follows:

It compiles RSS feeds, tweets, and online blog articles - intuitively ranking them according to your browsing habits. Within a few days (and without any explicit intervention on your behalf) the application optimizes the content to fit your specific tastes; the most relevant and interesting information is automatically displayed at the top.

In my view, m6sense is misleading when calling this type of assistance "Digital Intuition."  It's a recommendation engine, plain and simple. And it may be a very good one at that.  But let's not call this intuition.

Let's look at the difference more closely. 

A recommendation engine basically tells you what you want to know based on your past behavior.  So it learns by watching you, your clickstream, and creates inferences based on your habits. It learns your habitual preferences by predicting the "questions" you are likely to ask, and tracking a history of "answers" you deem sufficient.

Intuition is exactly the opposite.  We rely on intuition to make decisions in new, unfamiliar situations. No recommendation engine can do this for us, period. How can it?  In the case of intuition, we may not even know the right "questions" to ask, let alone the "answers."

Intuitive Intelligence, as we explain earlier: lies beyond the boundaries of science and analytics. It bridges the realms of reality and imagination, reason and instinct, material and spiritual dimensions of human existence. Intuitive Intelligence is non-linear, a key skill for success in the new economy, an economy driven by constant disruption and chaos.

Thus the phrase "digital intuition" is a misnomer.  A good marketing ploy, perhaps, but not intuition.

This leads me to a second point: Trust.  In this case, let's look at "digital trust."

Can you trust a recommendation engine maintained by a third-party (in this case in Israel) to track your online behavior, study your patterns, and feed you information it "selects" for you?  How will you know these recommendations are unbiased? How will you know that they're not marketing feeds from marketers anxious to understand your buying behavior?  How can you be assured of privacy?  Where will your behavioral data be stored? Who can access it?  Will it be sold?

These are the questions that need to be answered before a service like my6sense goes mainstream.  Incidentlly, these are the same questions all cloud based services are going to have to answer - from Google, to Microsoft, to Facebook, to my6sense.

And you will have to use some of that intuitive intelligence to answer the question: "can I trust them?"  

How does an analytic company like Google make its most important decisions?

 If we are to believe the Google myth, we learn, first and foremost, that they test everything:

We test everything at Google. While any company would prefer real-life data to hunches and guesses, Google is more focused than most (or any) on getting conclusive proof that a new feature or function improves the user experience. We release many of our products in beta on Google Labs to get this kind of feedback early in the process so that we can influence the design and iterate quickly.

The ability to test lots of products and features on hundreds of millions of users is enormously valuable. This test-bed of users (otherwise known as google.com) provides Google with an incredible advantage over enterprise-only search vendors. Bad ideas can be discarded quickly and great ideas can be implemented rapidly, because we have confidence and data to show that they'll improve the user experience.

Of course, when all decision-making is data-driven, it can lead to "madness."

 Here's how Douglas Bowman explains why he quit Google:

When a company is filled with engineers, it turns to engineering to solve problems. Reduce each decision to a simple logic problem. Remove all subjectivity and just look at the data. Data in your favor? Ok, launch it. Data shows negative effects? Back to the drawing board. And that data eventually becomes a crutch for every decision, paralyzing the company and preventing it from making any daring design decisions.

In the end, said Bowman, he "won't miss a design philosophy that lives or dies strictly by the sword of data."

The testing culture doesn't end there.  On the Google Testing blog, James Whittaker describes the testing frameworks he's observed among the job applicants he's looking to hire:

• Input Domain Framework
• Divide and Conquer Framework
• Fishbowl Framework
• Storybook Framework
• Pessimists Framework

Which one of these frameworks will be best for Google, asks Whittaker.

 Which leads us to the topic of this blog post: Just how do the executives at Google make decisions?

Do they base their decisions on the data?  Let's look at one well publicized executive decision and the executive decision-maker: Eric Schmidt and his decision to buy YouTube.

On October 9, 2006, in a deal valued at $1.65 billion, Google outbid a number of other competitors to snag YouTube, the online video site which was growing at a rate far outpacing Google's own Video site.

The official Google line was as follows:

The YouTube team has built an exciting and powerful media platform that complements Google's mission to organize the world's information and make it universally accessible and useful," said Eric Schmidt, Chief Executive Officer of Google.  "Our companies share similar values; we both always put our users first and are committed to innovating to improve their experience. Together, we are natural partners to offer a compelling media entertainment service to users, content owners and advertisers."

 So how did Eric Schmidt value Google?  Was he analytical, precise, objective?

By his own admission, Schmidt says, in a deposition by lawyers in the Viacom copyright lawsuit, that there was very little revenue coming into YouTube to justify the price his company paid.

Schmidt says that he told his company's board of that YouTube was worth $600 million to $700 million.

Via CNET, we get Schmidt's own words:

Viacom attorney Stuart Jay Baskin: And what was management's valuation?

Eric Schmidt: Much lower than we paid for it.

Baskin: And how was that communicated to the board?

Schmidt: I told them.

Baskin: So why don't you tell us what you remember telling the board in connection with the valuation?

Schmidt: I believe YouTube was worth somewhere around $600 million to $700 million.

...
Baskin: What methodology did you use to come up with that number?

John P. Mancini, an attorney working for Google, objects.

Schmidt: My judgment.

Baskin: Was it based on cash flow analysis? Comparable companies? What were you using as the basis for your judgment?

Mancini objects.

Schmidt: It's just my judgment. I've been doing this a long time.

...
Baskin: I'm not very good at math, but I think that would be $1 billion or so more than you thought the company was, in fact, worth.

Mancini objects.

Schmidt: That is correct.

Later...

Baskin: Can you tell us what reasoning you explained?

Schmidt: Sure, this is a company with very little revenue, growing quickly with user adoption, growing much faster than Google Video, which was the product that Google had. And they had indicated to us that they would be sold, and we believed that there would be a competing offer--because of who Google was--paying much more than they were worth. In the deal dynamics, the price, remember, is not set by my judgment or by financial model or discounted cash flow. It's set by what people are willing to pay. And we ultimately concluded that $1.65 billion included a premium for moving quickly and making sure that we could participate in the user success in YouTube.

What this tells us is that even in the most analytic company in the world, the big decisions are still made on intuition.

Here's why. Analytics can only tell us about the past. We have no data on the future.  So Eric Schmidt was making an intuitive decision about Google's own future through examining the intangibles.

For example:

1. YouTube's popularity was sky-rocketing, making it the runaway market leader among video-sharing sites.
2. It was crushing his company's own site, Google Video.
3. YouTube was up for auction and would be sold to a competitor unless Google jumped first.
4. Google overbid to ensure YouTube didn't fall into rival hands.

And that doesn't take into account two other points which made the deal a winner.  From the very beginning, the Google philosophy has been - get attention first, then monetize it.  And that is what this bet was all about.

Schmidt saw the attention trajectory in YouTube's growth, and he knew that if anyone could monetize that attention it would be Google.  To leave YouTube for Murdoch, Microsoft, or Yahoo was not an option.

In hindsight, it may have been a brilliant move.  Although the monetization has proved difficult, Google is breaking even today, which is far better than what has happened at MySpace, for example.

Hats off to Grist's David Roberts for putting together a thought-provoking line of thinking in Why Bill Gates is wrong. And no, he's not talking about Bing.

At the core, Roberts challenges the hubris of viewing all society's problems through the lens of innovation.  That is, he says that Gates is fundamentally off mark when he sees the solution to our environmental crisis as primarily technical.  Innovate our way out of this mess, says Gates.  Not so fast, says Roberts.

Innovation ≠ technology, is what Roberts states, and he's right. Here's how he describes the sustainable city of the future:

Everything is linked up in a smart, integrated communications, power, and transportation network. The city "knows" which roads are congested and which parking spots are free. It can communicate to individuals what combination of walking, transit, and individual vehicles will get them where they're going fastest. Vehicles are small, electric, modular, and--via sensors, GPS, and broadband wireless--intelligent, so they can pilot and park themselves. They can be charged by parking-integrated stations or even electromagnetic coils embedded in curbs, and since they're interchangeable and easily customizable, they can be public goods (like today's car-sharing services), easily swapped out and thus continuously in use. The city uses the vehicles' batteries as distributed energy storage, along with other storage options including pumped hydro integrated into the sewer system. Rooftops, parking lots, and other marginal lands are covered with solar panels; small-scale wind turbines are perched on bridges and towers; cogeneration systems are attached to every industrial facility. Through smart design and sensing, every building and neighborhood maximizes efficiency. The city senses power demand, knows where power is being produced and stored, and continuously balances supply and demand.

So what's holding this vision up? By a wide margin, says Roberts, the biggest barriers to creating such bright green cities are social.

I agree. It's not technology - all of the technology needed already exists - but political will that is holding us back; remember Copenhagen?

Roberts describes the social innovations that need to take place:

Building a city that behaves like an integrated organism means developing a holistic, long-term plan that will coordinate multiple agencies and levels of government. Big, long-term thinking is not exactly an American strong suit these days. Also--and this is a underappreciated problem--cities are cripplingly dependent on the financial largesse of state and federal authorities. They have very little autonomy to borrow money and invest in their own futures.

There are all kinds of collective action and first-mover problems: Who puts the charging stations in if there aren't electric cars on the road yet, and vice versa? Who pays for a smart grid before distributed generation is in place, and vice versa? How can public infrastructure and private market development be coordinated?

Many of the investments involve high upfront costs that are paid back slowly over time. New financing models will be needed both for private individuals and companies and for cities themselves.

Changes in the way individuals live, work, communicate, and travel must be introduced in a way that maintains social cohesion and political support for further changes. That requires research in social psychology and other behavioral disciplines (sorely lacking in much policymaking). How these things are introduced matters just as much as what they are.

OK. So, let's ask again, what's the holdup? Why can't we address the challenges - both socio-political and technical?  The answer: the structure of our democracy is damaged.  When lobbyists have taken over every aspect of the debate, and the Supreme Court seconds this type of behavior, it doesn't take a genius to see that "the future" doesn't stand a chance.

The "inconvenient truth" has once again been buried.  And to say that technology and innovation will take care of it is a huge mistake, one that might even cost us a lot in our near future.

As much as I respect Bill Gates's exceptional achievements there is always something fundamental missing for me to feel fully engaged when i hear his highly analytical presentations at TED. Always very seductive intellectually, extremely well analyzed and thought out, very well documented, supported by a wealth of data and facts, with solutions to complex issues made simple to understand ... but has human complex evolution ever been sorted out by analysis alone?