Can you really measure innovation with a Top Ten list?
The end of another year is nigh, meaning, of course, an avalanche of “Top Ten” and “Best-Of” lists. We’ll soon find out the sporting moment of the year, learn which movie topped the box office, discover which country mismanaged its economy the most, and revisit whatever moment most embarrassed an actor, celebrity or politician. In 30 days, we’ll know where Harry Potter and the Deathly Hallows Part 2, Greece, Lindsay Lohan and Silvio Berlusconi rank.
The Top Ten World Welcomes Innovation
Forbes.com and Thomson Reuters recently jumped into the “Top Ten” waters by releasing their analyses of the world of inventions, innovation and patents. Forbes.com published The World’s Most Innovative Companies list in October, while Thomson published 2011 Top 100 Global Innovators last week (even applying for a service mark to protect the project title from innovation-data-bandwagon jumpers).
One Topic. Two Approaches.
What’s most interesting about the two lists is how dissimilar they are. Which should strike you as pretty strange given that they, at least from their titles, should be measuring the same thing. Their respective results, though, are wildly different (see below).
First, let’s take a look at their respective methodologies.
Companies making Thomson’s 2011 Top 100 Global Innovators list were selected based on the following criteria, which all relate exclusively to patenting:
1. Volume: Includes all organizations with at least 100 “innovative” patents secured during most recent three-year period.
2. Success: Ratio of published applications to granted patents over most recent three-year period.
3. Global: Companies that sought patent protection from The Chinese Patent Office, The European Patent Office, The Japanese Patent Office and The United States Patent and Trademark Office.
4. Influence: Number of citations of an organization’s patents over the most recent ten-year period, excluding self-citations.
To summarize, companies that made Thomson’s list were those that successfully pursued patent protection in China, Japan, the EU and the United States for a large portfolio of patents that were frequently cited by others.
Forbes’ selection criteria for its World’s Most Innovative Companies list were very different – based almost entirely on quantification and stock market popularity. Here is how Forbes editors explained it:
“Our method relies on investors, who vote with their wallets, to identify the companies they expect to be innovative today and in the future. [...]It is calculated first by projecting a company’s income (cash flows, in this case) from existing businesses, plus anticipated growth from those businesses, and look at the net present value (NPV) of those cash flows. We compare the NPV of cash flows from existing businesses with a current market capitalization: Companies with a current market cap above the NPV of cash flows have an innovation premium built into their stock.”
To summarize, companies that made Forbes’ list were companies that have been until now successfully managed, and remain popular with investors.
The Results: Measuring Innovation
Each list comprises 100 companies. Combine the results and you’re looking at 187 different companies. Only 13 companies appear on both lists. Here are they are, along with a random sampling of 17 other companies that appear only once, on either the Forbes or Thomson lists.
|Daikin Industries, Ltd.||Yes|
|Rockwell Automation, Inc.||Yes|
|Sandvik Intellectual Property AB||Yes|
|Analog Devices, Inc.||Yes|
|Applied Materials, Inc.||Yes|
|Exxon Mobil Corporation||Yes|
|Toyota Motor Corporation||Yes|
And the Winner Is...
Forbes’ choice for the most innovative of The World’s Most Innovative Companies is:
- a San Francisco software company founded in March 1999,
- a company listed on the New York Stock Exchange with annual revenues in its latest fiscal year of $ 1.6 billion
- a fast-growing company that has secured a grand total of 30 patents from the United States Patent And Trademark Office.
Give it up for Salesforce.com.
Yet, Salesforce.com does not appear anywhere on Thomson’s list; with such a tiny patent portfolio, how could it? In addition, the Thomson list does not include a single BRIC company. This means no Brazilian, Russian, Chinese, or Indian companies. Instead, its list is heavily weighted to the Old World as you can see below in the country representation:
Surely, there is at least one company from China, India, Taiwan, the UK (notwithstanding Unilever, which is co-located in The Netherlands and the UK), Canada, Australia, Finland, Denmark or Norway that should crack a Top 100 list of the world’s most innovative companies. Perhaps their patent budgets are lower?
Invention and Innovation Are Not Synonymous
Alfred Armand Montapert wisely wrote, “Do not confuse motion and progress” in his Supreme Philosophy of Man: The Laws of Life (1970). Similarly, don’t confuse patents and innovation and stock market performance; they’re all different.
Think of an invention as an idea or concept that you can protect via a patent. In contrast, innovation is the successful commercialization of a concept in the form of a product or service. Because its rankings are entirely based on patenting activity, one could therefore argue that Thomson’s Top 100 Global Innovators methodology is flawed.
The Organization for Economic Co-operation and Development (OECD) concurs (http://oecdwash.org/innovationmapper; Patents & Innovation)
“Monitoring and analyzing patenting activities is one of the most common ways to trace innovation. However, patents are not a perfect indicator of capturing innovation potential. A patent is not a guarantee for a successful product innovation. In fact, very often inventions turn out to have no or very limited market value.”
In addition, patent activity varies widely among industries; certain sectors like the pharmaceutical industry invest heavily in pursuing legal protection for their inventions. In contrast, the IT industry often evolves faster than the legal protection that patent filing can provide. In other industries, companies choose to keep their discoveries secret instead of seeking legal protection.
And patent quality vs. quantity? There’s a good reason why someone long ago coined the term “junk patents.” Just because you work for a frequent patent filer doesn’t mean you by default work for an innovative company.
So what about Forbes? Its World’s Most Innovative Companies methodology certainly isn’t without reproach, either. The authors acknowledge that “our method relies on investors, who vote with their wallets, to identify the companies they expect to be innovative today and in the future.” Sounds a lot like a popularity contest based on stock price. Enron’s energy trading activities were at one time considered very innovative, its stock was popular, and the price duly reflected the admiration. Today? Not so much.
There just isn’t a linear relationship between inventions and patents and innovation and stock market performance that is truly objective. Unfortunately, they don’t fall nicely into line to be counted or neatly summarized in a Top Ten or Top 100 report. Innovation can’t be quantified as neatly as the number of Lindsay Lohan’s arrests between 2009-2011, or downloads of a Lady Gaga song on iTunes.