In the MDL In Re: Lithium Ion Batteries Antitrust Litigation, No. 13-MD-02420 YGR (DMR) (N.D. Cal., Feb. 24, 2015), the parties agreed to a general search term protocol but disagreed about what steps the parties needed to take if they could not resolve a disagreement over a particular term. Specifically, Plaintiff wanted Defendant to conduct a randomized qualitative sampling of documents retrieved by searching for any disputed terms, and to then allow Plaintiff to review the resulting documents following a privilege review. Defendant refused and the parties ultimately brought the dispute to the court.
Plaintiffs contended that a qualitative sampling would offer insight into why a seemingly relevant search term had returned a disproportionate number of irrelevant documents. Plaintiffs argued that the parties could then use this data to refine the searches and improve precision in identifying relevant documents, a process which Plaintiffs argued would benefit both parties by allowing them to focus their searches on relevant, discoverable material.
Defendants objected to Plaintiffs’ proposal, arguing that it would force Defendants to produce to Plaintiffs a number of irrelevant and non-responsive documents, thus imposing an impermissible discovery obligation on Defendants. Defendants also argued that Plaintiffs already had insight into Defendants’ production process via Defendants’ cooperation with Plaintiffs in developing and applying the search terms. Defendants further argued that Plaintiffs had failed to make a showing that Defendants’ production was incomplete.
The Northern District of California rejected Defendants’ arguments, ordering that Defendants must comply with Plaintiffs’ proposed qualitative sampling process, and agreeing with Plaintiffs that such a process “incorporates best ESI practices.” Citing Judge Peck’s seminal predictive coding decision Da Silva Moore v. Publicis Group, 287 F.R.D. 182, 191 (S.D.N.Y. 2012), the court concurred with Judge Peck that keyword searches are often “over-inclusive, that is, they find responsive documents but also large numbers of irrelevant documents.” In Moore, the court ordered the defendants to produce a random sample of irrelevant documents “to allow calculation of the approximate degree of recall and precision of the search and review process used.” Id. at 202. The Northern District of California concluded that the principles expressed in Moore regarding the importance of quality control testing to ensure accuracy and reduce the number of false positives should apply equally to the search term dispute at issue.
In ordering the qualitative sampling, the court concluded that it could mitigate Defendants’ fears that Plaintiffs would have access to irrelevant documents as follows:
- Plaintiffs agreed that Defendants could review the random qualitative sample first, and remove any irrelevant document for any reason provided that they replace the document(s) with an equal number of randomly generated document(s);
- Defendants would conduct the qualitative sampling only after they had exhausted an agreed-upon quantitative evaluation process;
- The parties would only use the randomly generated irrelevant documents to resolve search term disputes and the documents could not be used for any other purpose in the litigation;
- Any irrelevant documents would be destroyed within 14 days of dispute resolution;
- Only a certain number of attorneys would be allowed to review the random sample.
ILS – Plaintiff Electronic Discovery Experts