Point of View: Managing eDiscovery Reviews with Predictive Coding - Promises & Challenges

New technologies often offer lawyers a competitive advantage and provide legal managers promises of improved operational effectiveness. But at the same time, they also can introduce new challenges and potential problems. Such is the case with eDiscovery and predictive coding – a powerful tool applied to a sizable data set to identify similar or related electronically stored information (ESI) or documents deemed "hot" within the larger population.

Predictive coding is being employed with increasing regularity by corporations, their outside counsel and government agencies to facilitate the identification of potentially responsive data, reduce the review of non-responsive documents and, where appropriate, conduct quality control. However, organizations are encountering certain hot spots and blind spots connected with predictive coding that are cause for concern.

Potential Pitfalls

Consider the following potential challenges and consequences:

  • Judicial and opposing counsel pressure to disclose training criteria and documents used to perform active machine learning without revealing their case strategies or work product

  • Implications of key term searching and predictive coding sequencing

  • Incorporation of new data after initial seeding

  • Reliance on Optical Character Recognition (OCR) and/or alternative data extraction to create text searchable documents; and the impact of inadequate or improper processing prior to machine learning

  • Quality control – how to address bias or simply judgment calls between reviewers and the subject matter expert (SME)

  • The pros and cons of using a single subject matter expert or “crowd sourcing” the decision-making process

As the growth of big data continues unabated, organizations and key stakeholders still struggle to rein in eDiscovery and legal compliance costs. While the possibilities and promises of predictive coding are becoming more compelling and appealing, the challenges and considerations are becoming more concerning.

Our Point of View

Implementing a sound predictive coding approach may not be easy, but it doesn’t need to be overly difficult. It requires asking the right questions within and between the legal and IT stakeholders, such as:

  • Under what circumstances is predictive coding the appropriate solution in a matter?

  • What statistical sampling techniques will be employed? What are the specific confidence levels and intervals to be employed? What levels of precision and recall determine success? Are determinations defensible?

  • What is the profile of the trainer of the machine learning system? What are the capabilities that qualify him or her to do so? How will confirmation bias be eliminated?

  • How will privileged documents be screened and accounted for, knowing this must be a rigorous process and take into account existing case law, party agreements and uncertainty regarding what constitutes “reasonable steps”? In the U.S., will it demonstrate compliance with Federal Rule of Evidence 502 if privileged material is inadvertently produced? What about cross-border issues?

  • What is the position of the firm/corporation on integrating keyword culling and predictive coding? There are technological and logical considerations that need to be substantiated.

Document Review -- An Ingrained Process

Having the right mix of expertise and resources available is essential in effectively managing eDiscovery matters.

If you decide you need to augment in-house knowledge with experienced third-party eDiscovery specialists, make sure your outside provider can deliver effective solutions that offer simplicity, cost-effectiveness, compliance with both internal and external requirements, defensibility, predictability, and transparency; and in particular, enable document reviews to become an ingrained and sustainable process within your organization.

Have you used predictive coding successfully in your firm's eDiscovery engagements? Please share your thoughts and best practices in the comment box below.

Contact Us

Whether legal organizations are responding to eDiscovery requests or proactively developing and implementing strategies for data retention and management, Robert Half Legal eDiscovery Services offers the expertise and resources to help navigate the challenges and complexities of eDiscovery matters. We help organizations evaluate and implement effective predictive solutions while mitigating the technical risks and delivering repeatable and sustainable results.

We recognize the need to drive consistency and defensibility across cases which often have unique predictive coding requirements. We deliver the necessary eDiscovery project and process expertise to make your investment in predictive coding work. Our focus on process integration and risk management provides immediate and direct benefits and delivers real results and value.

Contact us directly or visit our website to learn more.

Frank Wu, Managing Director, Robert Half Legal eDiscovery Services

Office: +1.714.322.6677; Email: [email protected]

Joel Wuesthoff, Director, Robert Half Legal eDiscovery Services

Office: +1.718.663.9852; Email: [email protected]

Tags: eDiscovery