Patent Quality Markers From IPRs, District Courts, and Examination at the Patent Office

There are about 600,000 utility patent applications filed in the U.S. each year, feeding into over 8,000 patent examiners at the U.S. Patent and Trademark Office. Each of these patent applications reaches an end-of-life, by being granted, by being abandoned, or by having the patent term expire before grant, which is equivalent to abandonment but much more expensive. Typically, these final dispositions are reached by the patent owners and patent examiners without the involvement of third parties. With all of this data, there are a lot of significant lessons that can be learned by looking at statistics from patent examination. In fact, as indicated in my prior posts and presentations, I think these lessons learned can help us decide where to focus our claims, what to add to our specifications, and even in what spaces we should be inventing more often.

In contrast, there are about 4,000 patent cases filed in district courts each year, with about 10-14% reaching a decision on the merits (see also Patent Litigation Year in Review 2017, available from Lex Machina). In other words, data sets derived from district court activity have less than about a thousand patents per year across all technology sectors, even after discounting all of the bias and non-patent-quality-factors that come into play before a case is litigated then allowed by the parties to proceed to a final decision. In order to inform efficient patent drafting practices, district court data is often too light to support strong conclusions.

There are also about 1,500 Inter Partes Review (IPR) petitions with the Patent Trial and Appeal Board (PTAB) filed each year, with about 61% ending from a PTAB decision and 25% from a final decision. Similarly to the district court sample, data sets derived from IPR activity have less than a thousand patents per year across all technology sectors, even after discounting all of the bias and non-patent-quality-factors that come into play before an IPR petition is received then allowed by the parties to proceed to a final decision. For these reasons, it is more challenging to glean patent drafting wisdom from IPR and District Court data sets than it is from patent application data sets.

That said, in an article entitled "Determinants of Patent Quality: Evidence from Inter Partes Review Proceedings," by Brian J. Love, Shawn P. Miller, Shawn Ambwani, the authors have been able to draw some high-level conclusions about patent drafting from an IPR data set. It was also helpful to see the IPR-based conclusions lining up with prior patent office studies. After excluding the conclusions that relate to invention quality (forward and backward citations) and to the financial backing and commitment of the patent owner (NPE versus non-NPE, small entity, large firm, etc.), there are two conclusions that meaningfully touch on patent drafting practices, each discussed below.

A. Longer Claims Are Generally Slightly Stronger: Patents with first claims that are ten words longer are 1% less frequently instituted.
  • "Patents with more total words per claim and patents with more unique words in claim 1 are both less likely to be instituted, with an increase of 1,000 total words per claim or an increase of ten additional words in claim 1 each associated with a one percent decrease in the chance of institution;"
This conclusion makes a lot of sense, as longer claims tend to have more limitations. A greater number of combined limitations tend to be harder to find in the prior art. From my prior study on claim length (N ~ 100,000), it is also clear (at least in the software space) that the benefits of adding length begin to drop off after 800 characters (about 200 words), and there are diminishing returns to adding length beyond 1600 characters (about 400 words). It would be interesting to see whether these diminishing returns on longer claims are also present in the IPR-based dataset.

Also from the prior study, it is clear (at least in the software space) that claims having less than 300 characters (about 75 words) are in a different category of risk. If patents are ever granted on cases in or near this category, and if any of those patents are the subject of an IPR, it would be interesting to see if they have a different category of institution rates. Then again, given that the allowance rate is roughly 0% for this category, it would be difficult to learn much from the downstream IPR data.

B. Buyer Beware With High Allowance Examiners: Patents reviewed by examiners with 10% higher allowance rates are 2.5% more likely to be instituted.
  • "Patents reviewed by more experienced examiners, patents reviewed by examiners with higher allowance rates, and patents reviewed by examiners in art units with higher allowance rates are all more likely to be instituted, with a roughly 2.5 percent increase in the likelihood of institution associated with each additional 1,000 applications under an examiner’s belt, each 10 percent increase in an examiner’s allowance rate, and each 10 percent increase in an art unit’s allowance rate."
This also seems consistent with logic and the data derived from the patent office. In my last post, I explained how factors unrelated to prior art could potentially explain 99.9% of the examination results from Art Unit 2138. If prior art did not drive outcomes during prosecution at the patent office, it makes sense that it would be easier to later knock out these patents with prior art. I would expect this to be true for all art units where the outcomes of most abandoned cases can be explained with reference to factors unrelated to prior art.

Also, while the article mentions the "promotion effect," it would have been interesting to see if examiner experience was significantly related to IPR outcomes independently of examiner allowance rates (a question the article could have more directly answered). My guess is that the relationship between examiner experience and IPR outcomes was dependent on examiner allowance rates. In other words, I expect allowance rates, not examiner experience, to be the driving factor here.

I would love to hear your thoughts on the article and this post in the comments section!


  1. Hi Eric, all interesting studies. Thanks for linking them. Intrigued by your final paragraph, I did a quick graph of institutions vs examiner allowance rate:

  2. Thanks for sharing. Although I did not read through the Love et al. paper in depth, I can't help but think that predicting institution rates--and evaluating patent quality in general--is a more complicated problem than claim length or examiner allowance rates. The paper explains in Section V ("Multivariate Analysis") that a given variable's correlation with institution rate loses significance when the other variables considered are controlled in a multivariate regression. I think this suggests that each of the variables (claim length, art unit, number of prior art references cited, etc.) are interrelated in a way that is difficult to summarize with a few practice tips. On Page 68 of the paper, the authors state that one of the only surviving statistically significant results in the multivariate regression is that PAE patents are nearly 8 percent more likely to be instituted. That being said, the practice tips are helpful and a good starting point in gauging patent quality.

    This type of data (with over 20 variables considered) seems like an excellent candidate for deep learning. Intuitively, a deep neural network will reduce the 20+ dimensions into fewer and fewer dimensions, until reaching the output stage that would output a confidence value (e.g., % change of institution). It seems possible that, after training the network, the weights could be examined to understand which combinations of variables have a stronger influence on the output. Running test data sets through the trained network could then provide some validation for these conclusions. While I'm no data scientist, I think that linear regression is too limiting for such complex data.

    I agree with your point that allowance rates, rather than examiner experience, is the stronger indicator of IPR institution rates. One would expect that a higher allowance rate means that more patents with shorter claims (i.e., with no or few amendments during prosecution) reach allowance--which in turn would lead to a higher institution rate, as the shorter, broader claims are more likely to be invalid.

  3. such a nice post i really apprciate your writting skills and thanks for providing me all

    information regarding ardrntip.Patent drafting support

  4. such a nice post i really apprciate your writting skills and thanks for providing me all

    information regarding ardrntip.Patent drafting support

  5. very interesting to read this article.I would like to thank you for the efforts you had made for writing this awesome article. Looking for the best English speaking course in Riyadh? Enhance your language skills with Ziyyara’s tailored online English classes in Saudi Arabia.
    For more info visit Spoken English Classes in Saudi Arabia or Call +971505593798


Post a Comment