Prof. Robert P. Merges, University of California at Berkeley School of Law
October 27, 2014
The Software Freedom Law Center said that, in its opinion in CLS Bank v. Alice,1 the Supreme Court “took one more step towards the abolition of patents on software inventions. Upholding its previous positions, the Court held that abstract ideas and algorithms are unpatentable. It also emphasized that one cannot patent ‘an instruction to apply [an] abstract idea … using some un-specified, generic computer.’”2 Though very little in the Alice opinion is clear or free from doubt, this is one oft-heard interpretation of the case. Software patents are in the dock, awaiting sentencing. Many predict a swift execution.
Certainly the post-Alice cases point in that direction. No patent claiming software-related technology seems to have survived §101 since the Alice opinion was issued.3 Faced with the high-concept “test” or “approach” of Alice – which provides very little concrete guidance – district courts have so far done what appears to be the safe thing. They have taken refuge in the basic holding of the case. Whatever else the opinion says or fails to say, at the end of the day the patent was declared invalid. So it appears that, understandably, trial courts have taken this to be the true lesson of Alice: These sorts of patents are now invalid.
While these decisions make sense in the immediate aftermath of the Alice case, the Federal Circuit must make a course correction. The chief patent court must not allow Alice to stand for the simple proposition that software patents are invalid under §101. That is not what the case says. And it would not be good policy to apply Alice that way. There are patents in the software field that are not, under any reasonable meaning of the term, “abstract ideas.” The Federal Circuit needs to wait until one of these cases pops up on appeal. And then it needs to reverse the district court, which will surely have invalidated the claim or claims in question.
What Kind of Case Could Put Alice in Perspective?
The ideal vehicle for a course correction would have three features: (1) The claims would cover real technology; they would embrace a clever and novel invention in an area of interest to computer scientists and/or computer programmers (as opposed to marketing or other business people); (2) the invention would be recognized as significant or important in a technology field and/or business market; and (3) the patent would be owned by a company that actually makes and markets technology-based products, ideally a smaller company for which the presence of a patent can be shown to enhance its economic viability and its ability to compete with larger companies in its field.
Fortunately for those of us who follow this sort of thing, a recent case out of the Central District of California just might fit the bill. In McRO vs. Activision Publishing, Inc.,4 one of a series of cases involving the same patents asserted against numerous accused infringers, Judge George H. Wu of the Central District of California invalidated the patent in the case as an abstract idea under §101, and ruled from the bench that the trial was therefore over. But a close look at the technology involved, and the overall context of the case, shows an invention that only by a great stretch might be deemed “abstract.” The nature of the technology in McRO is so far removed from the banal business methods of Bilski and Alice, and forms part of such an innovative sector of the animation field, that the case is simply begging to be reversed by the Federal Circuit. McRO represents the best recent vehicle for the Federal Circuit to use to begin applying some counter-pressure to the trend that Alice set in motion.
Looking Into the Technology: Automating Facial Animation for the Entertainment Industry
The technology in the McRO case comes from the animation field. It translates spoken words into mouth movements in three-dimensional animated images. This technology helps to automate the process of adapting an animated image to mouth words without having to draw or manually program the animated movements of a character’s mouth. The software takes as its input recordings of spoken speech, and provides as output a sequence of animated images of a mouth speaking the words in the recorded input.
According to a story in the trade press, automated mouth and lip animation is an important aspect of the “holy grail” of pure, automated facial animation.5 This field presents formidable challenges:
With its complexity, nuance and sophistication, the human face – its expressions and speech – poses seemingly insurmountable challenges to replicate in CGI. Nonetheless, software developers and visual effects facilities have focused resources, research and development to conquer the difficulties of both generating believable facial expressions and lip synching – in a way that’s flexible, realistic and cost-effective.6
This same story quotes one experienced animator:
Three years ago, Maury Rosenfeld, president of Los Angeles visual effects facility Planet Blue created Tulip, a combination facial animation and speech modeling system. “At the core of our system is speech recognition,” said Rosenfeld. “We take an extraction of both the phonetic content of what people are saying, along with pitch and rhythm.” A facial expression model, which integrates with the speech recognition model, is guided by the animator who “directs” the mood, with the system automatically doing the rest.
“Our interface is an indefinite number of live tracks – you can layer the animation in real time,” Rosenfeld explained. “The animator may want to add detail to the left eyebrow for a moment and, on the next pass, he can make it twitch a little. Our system lets the animator get very detailed.” Projects include an hour and a half of speech for the James Bond game “The World Is Not Enough,” and on tap for the future, said Rosenfeld, is work on “interactive media with next-generation communication technologies.”
Planet Blue, the company cited by animator Rosenfeld, is the operating division of the plaintiff in the McRO case, McRO, Inc. And Tulip is the commercial embodiment of the invention claimed in the two McRO patents asserted in McRO v. Activision. To get a sense of the technology involved, consider one of the McRO patents, U.S. Patent 6,307,576, “Method for Automatically Animating Lip Synchronization and Facial Expression of Animated Characters, issued October 23, 2001, with a priority date of October 2, 1997.
In the language of this patent, a spatial model of an image of a mouth begins with an initial position, and undergoes a change in shape – a “morph” – during the animation sequence. The animator can input a value that affects the magnitude of the morph; this is called a morph “weight.” The end state of an animation sequence is called a “morph target” – it is the mouth shape at the end of a particular sound.
All sounds in spoken language are broken down into simple units called “phonemes,” and each phoneme is associated with a certain mouth shape. (Think of the round mouth of an “Ohhh” sound versus the pursed lips of a “Ppppp” sound.) A stream of recorded spoken words is broken down into phonemes, and these are associated with mouth shapes; the transitions from one mouth shape (phoneme) to the next are the morphs.
This rudimentary background should allow a basic understanding of Claim 1 of the ’576 patent, which reads as follows:
1. A method for automatically animating lip synchronization and facial expression of three-dimensional characters comprising:
obtaining a first set of rules that define output morph weight set stream as a function of phoneme sequence and time of said phoneme sequence;
An apparatus for automatically animating lip synchronization and facial expression of three-dimensional characters comprising:
obtaining a timed data file of phonemes having a plurality of sub-sequences; generating an intermediate stream of output morph weight sets and a plurality of transition parameters between two adjacent morph weight sets by evaluating said plurality of sub-sequences against said first set of rules;
generating a final stream of output morph weight sets at a desired frame rate from said intermediate stream of output morph weight sets and said plurality of transition parameters; and
applying said final stream of output morph weight sets to a sequence of animated characters to produce lip synchronization and facial expression control of said animated characters.7
With the technology and claim in mind, let us now consider the Alice case and how it applies to the §101 issue in McRO.
Alice, Put Simply
The Court in Alice invalidated the claims at issue because they were too abstract; they violated the “abstract idea” exclusion from patentable subject matter under §101. The Court explained:
On their face, the claims before us are drawn to the concept of intermediated settlement, i.e., the use of a third party to mitigate settlement risk. Like the risk hedging in Bilski, the concept of intermediated settlement is “ ‘a fundamental economic practice long prevalent in our system of commerce.’ ”8
This holding, the Court said, was consonant with “the pre-emption concern that undergirds our §101 jurisprudence.”9 This is the by-now familiar worry that overly broad claims, drawn at too high a level of abstraction, will preempt or cut off specific, concrete patentable invention in the future. This is expressed, alternatively, as a desire on the part of the court not to tie up “building blocks of human ingenuity,” that is, basic or fundamental contributions that can be deployed in a wide array of settings to form components of specific, concrete inventions.
These considerations coalesce, in the Alice opinion, into a two-step test. The Court wrote:
In Mayo Collaborative Services v. Prometheus Laboratories, Inc., … 132 S. Ct. 1289 (2012), we set forth a framework for distinguishing patents that claim laws of nature, natural phenomena, and abstract ideas from those that claim patent-eligible applications of those concepts. First, we determine whether the claims at issue are directed to one of those patent-ineligible concepts…. If so, we then ask, “[w]hat else is there in the claims before us?” Id., at 132 S. Ct., at 1297. To answer that question, we consider the elements of each claim both individually and “as an ordered combination” to determine whether the additional elements “transform the nature of the claim” into a patent-eligible application. Id., at 132 S. Ct., at 1298, 1297. We have described step two of this analysis as a search for an “ ‘inventive concept’ ” – i.e., an element or combination of elements that is “sufficient to ensure that the patent in practice amounts to significantly more than a patent upon the [ineligible concept] itself.”
To summarize, then, in the context of a case such as McRO, we look to see whether the claim covers an abstract idea. If so, we ask whether the “what else” in the claim suggests an “inventive concept,” something that makes the claimed invention amount to “significantly more” than just a patent on the abstract idea.
Applying Alice to the Claimed Invention in McRO
I do not think Claim 1 of the ’576 patent covers an abstract idea. I say this for two reasons. First, the claim represents a concrete, real-world invention. It is not a broad, category-encompassing concept. It covers a particular or concrete species of a broader identifiable genus. Second, and as a consequence, the claim does not preempt an entire field. It does not cover a basic building block but a single brick in the edifice of a larger technological construct or “building.”
Even if the claim were said to cover an abstract idea, however, I think it embodies an inventive concept that amounts to something that is “significantly more” than the abstract idea itself. In the fine points of the claim, it adds detail, structure, and particularity to the general conceptual category it is part of.
So either way, as a concrete invention or as an abstract idea-plus, Claim 1 should pass through the filter of §101. It goes without saying that the claim may be obvious, may have problems with novelty, may encounter problems under enablement or written description. Passing through §101 is but one of the many filters in the patent system.10 But there is no doubt in my mind that under Alice it is patentable subject matter.
Analyzing Claims: Levels of Abstraction
Probably the most difficult aspect of applying Alice is the issue of abstraction. Because abstractness is a conceptual quality, any description of it has the unfortunate feature of being difficult to grasp, ethereal, transcendental. The Oxford English Dictionary includes this definition of abstract: “Withdrawn or separated from matter, from material embodiment, from practice, or from particular examples. Opposed to concrete.” So how do we know when a patent claim has this quality of abstractness?
One answer might be that, because all of software has the quality of being “separated from matter,” and certainly can be represented in a way that is “separated from … material embodiment”, the Supreme Court in Alice must have meant to say that all software is “abstract,” and therefore excluded from §101 eligibility. Certainly that is the spirit of the sources cited at the beginning of this article. On the other hand, the Court says that the simple concept of mitigating settlement risk covered by the claims in Alice fail §101, at least with respect to the “system” claims, because they “merely require generic computer implementation.”11 And while the claims in Gottschalk v. Benson,12 invalidated by the Court in its 1972 opinion, were analyzed in Alice, those claims were described as “an algorithm” or “mathematical principle.”13
The simple fact is that the Alice opinion never mentions the word “software”. The Court was eminently capable of simply sweeping away software patents by equating “abstract” with “software.” But it did not. I conclude that it is not software that is prohibited under §101, but “abstract ideas” (however claimed or embodied), including of course “mathematical principles.”
So we know that the claim in McRO is not invalid simply because it is software. But how do we know that the claim is not abstract? The answer I think lies in constructing a series of hypothetical claims in the general technological space that includes the claimed invention. Then comparing these claims, first to the claims struck down in Bilski and Alice, then to the actual claim in the patent. We ask: On the “hierarchy of abstractness” that stretches from extremely abstract to highly concrete, where does the actual claim fall? If it is closer, conceptually, to the claims in Bilski and Alice, then out it goes. But if it is much closer to the middle or bottom of the hierarchy, it survives. That means it is more concrete. It is not abstract according to the OED definition above, because it is not “withdrawn or separated from practice … or particular examples.” It is, in fact, a practicing embodiment or application of the abstract idea, a particular example rather than an abstract category.
The general approach I am advocating here has much in common with the “levels of abstraction” test, whose origin lies in an influential opinion by Judge Learned Hand in the case of Nichols v. Universal Pictures.14 Though Hand was dealing there with the “idea-expression” dichotomy in copyright law, that issue has much in common with the “abstract idea” issue in patent law. Idea-expression and abstract ideas both guard against the risk that IP rights will over-protect. And they both work by drawing a line between concrete specifics and over-broad generalities.
Constructing Hypothetical Abstract Claims
The place to start is with the field of endeavor of the claimed invention. I would say it is automated animation. This is the field with which the §101 test of Alice would be concerned. The Alice analysis would ask: Is this field preempted by the claim at issue? To the extent this field is a “basic building block” of modern animation, does the claim in effect cover that building block?
Here are some claims that, in my opinion, fail the test of Alice:
1. A method (or system) for automatically mapping dynamic inputs onto a changing sequence of animated images.
2. A method (or system) for automating animation comprising the steps of: (a) taking as a first input a series of still images; (b) taking as a second input parameters relating to general features of desired transitions between said still images; and (c) rendering as output a resulting series of animated images presenting viewers with a continuous sequence of animated movement between said still image inputs.
3. A method (or system) for rendering computer generated imagery comprising the steps of: (a) constructing a database of rest state images from preexisting imagery sources; (b) receiving instructions regarding the desired sequence of rest state images drawn from said database; (c) receiving instructions regarding speed and style parameters desired for automatic intervening images generated according to this method; (d) following said parameters in rendering automatic intervening images; and (e) outputting a combination of rest state images and automated intervening images.
Referring now back to Claim 1 of the ’576 patent, notice the following detailed elements:
- Just in the preamble, the claim is limited to “automatically animating lip synchronization and facial expression of three-dimensional characters”; it does not cover all animation, only deals with lip synchronization, and only applies to three-dimensional (CGI) animation, not two-dimensional traditional cartoon animation.
- It employs a very specific approach to the automation of CGI lip-synching: the use of user-inputted rules for “output set morph weights” (sets of user-assigned coefficients to specify characteristics of the end-state of a movement between one mouth shape and another).
- It breaks down the initial sequence into a series of subsequences and generates an intermediate sequence of images, based on the timed sequence of phoneme inputs, plus “transition parameters” for producing the final output stream from the intermediate sequence.
My simple conclusion from this is that Claim 1 of the ’576 patent is very different, and much more specific, than the three hypothetical claims listed above. In my view these three claims are roughly similar to the over-broad claims that were invalidated in Bilski and Alice. But they are pitched at a much higher level of abstraction than the actual Claim 1. These hypothetical claims would have almost completely preempted the field of automated animation. But Claim 1 does not; it covers a specific application in one sector of the overall field. It is a brick – maybe a good one, maybe not, but a brick – and not a basic building block.
The ’576 patent in the McRO case provides an excellent vehicle for clarifying the law of §101 patentability post-Alice. It is a specific and detailed invention occupying one corner of the field of automated three-dimensional facial animation for lip-synching. It is a very far cry from the highly abstract claims of Bilski and Alice, which in effect claimed the high-level concepts of risk hedging and third-party transactional guarantors. The ’576 patent is a real invention in a real industry that does not preempt or occupy any large field of endeavor. It may be anticipated, obvious, or not enabled; but it is not an abstract idea.