I write this post on the three-year anniversary (Cheers!) of Judge Andrew Peck’s Da Silva Moore v. Publicis Groupe et al, S.D. New York, 11-1279, 2-24-2012 opinion, widely cited as the first case ruling to endorse the use of predictive coding or “technology-assisted review” (TAR) as a discovery tool.
TAR is the process of training a computer system to make decisions about the responsiveness of a document that would otherwise be reviewed and coded by a manual reviewer. With TAR, human effort is not eliminated, but rather used throughout the review process to train the system on what is responsive and what is not. The documents used to train the system are called the “training set” or “seed set.” Once the system is trained, the computer reviews and codes the documents.
Since Da Silva Moore, the use of TAR in cases has gained some traction with litigants and courts. Commentary on the cost-savings and increased accuracy of TAR versus human review is relatively old news, and it seems well-established in case law that, as a general matter, TAR is an appropriate method for reviewing electronic data. But the defensibility of the particular TAR process used in a specific case is not yet predictable (pun intended). For example:
- Do I need the opposition’s consent or the Court’s approval to use TAR?
- Can I switch to using TAR in the middle of my case?
- How much do I have meet and confer with the other side about my TAR methodologies?
- Do I have to disclose my training set to the other side?
Here’s what we know as of now:
Consent: In his most recent TAR case, Rio Tinto PLC v. Vale, S.A. et al, S.D. New York, 14-3042, 3-2-2015, Judge Peck cautioned against holding TAR to a higher standard than manual review or keyword search, neither of which require consent or approval. It now seems unlikely that an opinion requiring approval as a prerequisite to TAR is in the immediate future, at least not from our TAR pioneer. That said, Judge Peck stated in Da Silva Moore that the party’s transparency in its proposed protocol helped ensure the Court’s approval.
Buyer’s Remorse: If you want to deviate from an ESI protocol or discovery order by switching from manual review to predictive coding, check out Progressive Casualty Insurance Company v. Delaney et al, D. Nev., 11-678, 7-18-2014 and Bridgestone – Bridgestone Americas, Inc. v. Intl’ Bus. Machines Corp., M.D. Tenn. , 13-1196, 7-22-2014. In Progressive, the Court said no. In Bridgestone, the Court allowed Plaintiff to “switch horses midstream.” The difference? You guessed it — openness and transparency. Had Progressive been cooperative and transparent, the Court would “not hesitate” to approve a protocol for the use of predictive coding, but Progressive was unwilling. In Bridgestone, however, plaintiff notified defendant of the change and promised to be open and transparent in what it was doing.
Transparency and Disclosure of Training Sets: Must you disclose the training set used to train your system? Wouldn’t that be discovery of irrelevant documents used to tell the system what NOT to find? The Court in Biomet M2A Magnum Hip Implant Products Liability Litigation, N.D. Ind., 12-md-2391, 8-21-2013 thought so. The Biomet Court did not believe it had the authority to compel information that was not discoverable under the Federal Rules of Civil Procedure, even though it was displeased with Biomet’s overall lack of transparency and cooperation. The Eastern District of Pennsylvania in In re Drywall Antitrust – In re Drywall Antitrust Litig., E.D. Pa., 13-md-2437, 11-26-2013 similarly declined to require the producing party to turn over its training set, but did require that it be preserved. What about work product—doesn’t your training set embody protected mental impressions? This issue has not yet been squarely addressed by the courts. Though we may not yet have a Court that will require you to turn over seed sets, judges will strongly encourage discussion and cooperation. Failure to act accordingly could impact the Court’s discretion in other ways.
Takeaways:
- TAR is here to stay. According to Judge Peck, TAR case law is sufficiently developed now that when the parties want to use TAR in a case, courts will generally permit it. Embrace TAR on large volume document reviews.
- Openness and transparency will simplify the process for your opponent and for the Court—which means it will simplify the process for you. In these relatively early days of TAR, “transparency” may equate to agreement of the parties. But if the opposition won’t agree, don’t be reluctant to seek Court approval of your TAR protocol, and be prepared to tell the Court that you offered full disclosure of your process.
- Watch for future rulings that may clarify how the attorney work product doctrine can be reconciled with discovery of training sets, and how the precise training set methodology you use in your case will affect how transparent—and when—you’ll need to be about your TAR process.