We’re continuing our patent quality discussion by tackling the issue of measurement. How to measure patent quality is an appealing topic for everyone who especially subscribes to a “you can’t manage what you can’t measure” philosophy. Yet, measuring patent quality is fraught with subjectivity as our guest explains.
An IPfolio customer, he’s VP of Intellectual Property for a U.S. software company and agreed to speak with us under a veil of anonymity. If you missed the first half of our discussion with the pseudonymous Paul, we suggest reading What is Patent Quality? before continuing below.
IPfolio: Paul, we started discussing patent quality by asking the obvious “what is it?” question. Let’s do the same with measurement. How can you measure patent quality?
Paul: Firstly, there is no real patent quality metric. Nothing has been widely and universally accepted yet. Yes, it is qualitative and subjective, but there is a way to see glimpses of quantifiable metrics that may be representative of patent quality. Then you could weight accordingly to what you want.
Are you talking about value in the context of quality?
Well, they’re two ways I like to think about breaking down metrics. The first metric type is the “open loop” metric. This looks at metrics of the patent application. Examples are; number of words in the specification, number of words in the independent claims, ratio of independent claims to dependent claims, number of figures, number of alternative embodiments, numbers of hours spent by engineer, number of hours spent by attorney on the draft, number of annotations per figure, original word count that is not duplicated in any other patent application, and length of invention disclosure. There are others, of course, but you can see the variety.
Now, independently, none of them represent patent quality. However, in a version of “the sum is greater than the whole of the parts,” by combining several then appropriately weighting each, they can give you an indication.
The subjectivity isn’t in the answers because all of them are numbers. The subjectivity obviously lies with the weighting.
No question. If you can arrive at a somewhat accurate weighting methodology, open loop metrics will help produce a value determination for you reasonably quickly. A much longer alternative is “‘closed loop” metrics, which are based on the issued patent. They’re much more valuable but also require way more time. You’ll have a very slow feedback loop to make changes. Examples of open loop metrics are number of citations, allowance rate, number of office actions to allowance, and number of words per issued claim.
In the first part of our discussion, you mentioned the value of “adding more words” if necessary, is this what you were referring to?
Before going further, it’s important to understand that patent quality is easy if you have a ten-year feedback loop. If you don’t have a decade, it’s harder to justify your budget. Some people like to count words. A higher word count could mean - not necessarily but potentially - that you have a lot more that you wrote about. You spent a lot more time writing about it and supplied important details. This is one facet of patent quality.
Is it convincing as far as lobbying your boss to justify a budget increase?
As I said before, patent quality is very hard to assess. If you go to your boss and say, “I need to improve the quality of these patents, which will raise the cost,” you might get justification push back in return. Something to the effect of wanting to see some reportable metrics that align with evidence of value.
Leveraging Peer Feedback
What’s a good answer for the boss?
Something I’ve refined over time is an alternative quality valuation process based on figure counts. I create side-by-side comparisons of our portfolio vs. competitors, test them on employees, and gauge their feedback.
I randomly sample 10 of our patent applications and 10 from a competitor in the same space. Then I group our figures onto a PowerPoint slide against figures pulled from our competitor’s patents. I show this presentation to engineers and others in the company without any identifying information or hints. It’s very much a “first impression” kind of exercise. I ask them which ones look more detailed. By tracking how often they choose ours, I can get a sense of the level of detail we’ve included in our draft.
It you haven’t had any response from the patent office, I’ve found this to be an excellent way to determine patent quality in advance. I know it takes some time, but you’ll get some good feedback.
What is it called?
I call it a feedforward loop. It’s useful if your patent hasn’t matured and you lack citations. If you have citations, a feedback loop will be familiar to your readers. You look at citations such as people who have cited you, as well as your coverage and the value in your coverage.
In terms of patent quality, a feedback loop is a lot more objective than subjective; you don’t need the metric of having a lot of technical definition. You need more metrics of how many people are citing you, indicating they have encountered the same problem. This indicates how many people have run into the citation, or how many people are trying to solve the inventive step the same way as you.
How common do you think feedforward is among your peers?
Not very. People are definitely missing out. I’ve used the process or methodology at several companies, and can point to it as a clear source of patent quality. I just think people try to focus on patent quality without any explicitly good way of showcasing it. When you have a portfolio in its infancy, most accept there is no real measurement. Even though that is the case in general, there can be minimal efforts taken to showcase the effort you are putting into your patents.
Begin With the Scope In Mind
Are there any other formal exercises you do to quantify quality?
Yes. I also like to use something called scope intention. It’s a somewhat labor intensive process, but well worth doing. You begin with clarity about the scope you want to get:
“This is the scope I am claiming. This is an A because this is what I want to get, or this is more of a B because this is what I want to get and I want to be practical. I am going to claim this coverage. If I get it, I’ll be happy and all will be good.”
It’s a narrowing process; if I get this coverage and this coverage and this coverage, I get narrower and narrower and narrower. As I’m doing this, I can rate the coverage I want ahead of time. As we are prosecuting the applications, and responding to office actions, I can compare the coverage received against our big picture.
By mapping everything, you can say “well, I am getting alignment of everything I want and the coverage I want, and I am getting higher allowance rate on patents in general.” All of these are very good quality indicators.
How do you integrate these exercises into your IP program? Are they part of your regular activity and something you do with both internal staff and outside contractors?
Yes. If you’re a company with a young and growing portfolio, I think you have to define quality as something organic to be improved over time. In the beginning, you have to make short-term decisions. Think of this as your initial patent quality phase.
How are we showing that we are improving patent quality? Look at the amount of attorney time on it. Look at the number of words, look at the number of figures. While these steps collectively add value, I understand that sceptics will always push back, such as:
“The patent attorney is spending more time on it.”
“The patent attorney is just putting extra figures that are not necessary.”
“The patent attorney is just putting more words that are not necessary.”
Without investigation, none of these are valid. Every in-house counsel should be checking them anyway as part of an internal quality review process. If you don’t address patent quality, you open yourself up to abuse. If you have good people reading their patents and understanding the business strategy, it’s very easy to confirm whether there is abuse.
My overall advice is to get really serious about patent quality during patent drafting. You’ll get more of what you’re paying for and your portfolio quality will increase. By following some of my suggestions, you’ll be able to measure the improvements.
We wanted to send our thank you to our mystery guest, thanking him for his time.
If you have a specific topic you’d like to read about in the IPfolio blog, let us know. We have incredibly interesting customers all over the world who are running fascinating R&D programs that result in the products and services that we use daily. We love introducing you to them. Send your suggestions to [email protected]