Informed Buyers of E-Discovery: Why General Counsel Must Become Tech Savvy

Introduction

Litigants are struggling more than ever to complete civil discovery in an efficient manner.  The volume of electronically stored information has swelled to the point that it cannot be managed without the assistance of computing.  This is not unique to litigation: heads of marketing find themselves inundated with millions of data points on customer behavior and buying patterns, petroleum engineers face gargantuan databases of geographical data, and retailers are asked to keep track of a maze of merchandising statistics.  However, while legal departments slowly transport this heavy load of information as if by pack mule, business units wield this information like a rapier.  Some firms not only avoid getting bogged down by “big data,” they use it as an income-generating asset.  This paper suggests that changes in-house will allow legal departments to do the same with their electronically stored information.

In Part I, I discuss the fundamental problem: civil discovery has gone far afield of its original goals, and the administrative problem of preserving a client’s electronically stored information, reviewing it for privilege and relevance, and producing it to opposing counsel, is getting too large.  This difficulty is not for a lack of technological solutions, but because existing technological solutions are not being effectively adopted.  The reasons for this slow adoption are many, including institutional sluggishness, misperceptions of the technology’s weaknesses, and disincentives to change.  In Part II, I present a solution to these problems, focusing primarily on the role of general counsel in setting the pace for outside counsel, courts, and rule makers. I propose that general counsel’s incorporation of one or two low-level tech-people, combined with an increased openness to and insistence on the use of cost-saving technology, will solve the problems presented in Part I.

I.         The Problem of Electronic Discovery

A. Background: How Discovery Went Wrong

The enactment of the Federal Rules of Civil Procedure transformed American pretrial litigation from a system of severe limitation, to one of potentially tremendous breadth.[1]  The origins of civil discovery were noble: shared knowledge of all relevant facts was hoped to allow claimants to fairly build their case, and to allow defendants to find the basis for their opponent’s claims.[2]  Civil litigation is not limited to the facts alleged in the pleadings; rather, it is supplemented by discovery intended to define and clarify the issues in the case.[3]  Broad civil discovery has been controversial since its inception—and it has remained so throughout its entire existence.[4]  Some scholars have argued for reforms, while other scholars have argued that the system is fine the way it is.[5]

Whatever the origins of civil discovery, it is now uncontroversial that electronic discovery poses new and difficult challenges to the pre-trial exchange of information.[6]  The volume of documents to be exchanged has grown exponentially.[7]  As of 2002, greater than 90% of corporate information was stored electronically, and emails generated 400,000 terabytes of information per year.[8]  In a 2008 antitrust litigation, both plaintiffs and defendants agreed that the defendant’s production alone could be “somewhere in the neighborhood of a pile 137 miles high.”[9]  The cost of such a production would be staggering: the RAND Corporation recently estimated the cost just for review at $18,000 per gigabyte.[10]  According to another research group, the amount spent on e-discovery can be expected to continue to rise.[11]  In particular, the costs of human review of discovery documents have become gargantuan: “human review of documents as part of responding to discovery requests consumes about 73 cents of every dollar spent on the production of [electronically stored information (ESI)].”[12]

Electronic discovery is expanding beyond simply high costs during one stage of litigation and now threatens the substantive resolution of lawsuits in general. As litigation costs rise, they play an increasingly important role in the decisions to file and/or settle a lawsuit.[13]  Indeed, even before the electronic revolution, it was observed that increasing incentives to avoid trial could completely separate the resolution of a case from its strength on the merits.[14]  In short, the costs of the once noble pretrial exchange of information is poised to consume the entire litigation process from end to end. Discovery is no longer the gateway to litigation; discovery is litigation.[15]

B. The Barrier to Cost-Effective Discovery: Slow Adoption of Technology

The legal system has adopted advanced technology for the use in discovery at a snail’s pace.  From watching the way litigants and courts treat metadata and predictive coding, reviewing the development of Federal Rule of Evidence 502(d), and listening to industry professionals, it is clear there is a large roadblock between the status quo and a technically savvy legal world.

1. Evidence of Slow Adoption: The Metadata Saga

Metadata is now presumptively discoverable.[16]  This statement, however, was not so easy to make a few years ago. Rule 34(a) of the Federal Rules of Civil Procedure states that any electronically stored information “stored in any medium from which information can be obtained either directly, or if necessary after translation . . . into a reasonable usable form is discoverable.”[17]  Although this rule, looking back, seems to be an obvious reference to native format documents,[18] courts and parties took quite a while to catch on.  It was not enough that Rule 34(b)(2) specifically states “[a] party must produce documents as they are kept in the usual course of business.” [19]

Metadata is data about data. Properly attached to an electronic file, metadata can tell you when the file was created, who created it, who edited it, and what type of file it is.  For instance, in Microsoft Windows, right-clicking and viewing the “properties” of a file will tell you some of that file’s metadata.  Similarly, the header to an email is metadata.  Metadata is library information that helps in tracking and sorting electronic information.  Documents in native format (i.e., with their metadata) stand in contrast to, for instance, TIFF images of documents that are more difficult to search and categorize.[20]

In Wyeth v. Impax Laboratories, Inc., the Middle District of Florida denied a motion to compel production of documents in their native format.[21] Defendants had produced TIFF files, but following a “general presumption against the production of metadata,” the court declined to force the production of the native files.[22]  The standard announced would deprive litigants of a useful tool in the categorization and sorting of documents: “[I]f the requesting party can demonstrate a particularized need for the native format of [ESI], a court may order it produced.”[23]  The Sedona Principle at the time falsely stated that “in most cases metadata will have [little or] no evidentiary value.”[24]

Other courts were quicker to come around to the idea that metadata should be presumptively discoverable,[25] and the District of Kansas in 2005 set the standard that prevails today: “[W]hen a party is ordered to produce electronic documents as they are maintained in the ordinary course of business, the producing party should produce the electronic documents with their metadata intact.”[26]  In light of this new understanding, Sedona Principle 12 was revised to state that productions should be made “taking into account the need to produce reasonably accessible metadata.”[27]  Thus, the legal community slowly recognized that metadata is a necessary part of litigation.

2. Evidence of Slow Adoption: The Predictive Coding Saga

Computers provide vast resources for the discovery of  evidence—resources that have been largely untapped by litigators.[28]

For at least thirteen years, scholars have been asking whether computers can aid in the discovery process.[29]  Predictive coding, a method of clustering documents by a decades-old tool called latent semantic indexing, is proposed as the next generation of search technology for lawyers.[30]  So even though predictive coding is the “new kid on the block,” leveraging computers to reduce the costs of discovery is an old concept.[31]  Despite this, computer-assisted review has failed to take hold in the profession.

2012 was a big year for predictive coding: the method was approved in a case in the Southern District of New York, a Delaware Vice Chancellor ordered parties to use the method, and the method’s potential was heavily featured at law and technology conferences.[32]  In 2011, an important article showed that predictive coding can mean not only cost savings, but also superior results to manual review by human reviewers.[33]  And in 2012, AbovetheLaw named “predictive coding” the buzzword of the year.[34] But guess what: the buzzword of the year in 2006 was “concept searching”-essentially a synonym for “predictive coding.”

Despite years of attention, recent focus, and evidence of its strengths, predictive coding has not been widely adopted: cases mentioning it are the exception, and it has largely been relegated to academic journals and conferences.[35]  RAND’s cost of review report includes a section arguing that very few litigants have adopted computer-assisted review.  None of the companies RAND polled used computer-assisted review, and its researchers were able to find no references to predictive coding outside of vendor advertisements and just two cases in which predictive coding had been used to perform a  live document review.[36]

Much like it was with metadata, the legal system has been very slow to adopt predictive coding.

3. Evidence of Slow Adoption: Federal Rule of Evidence 502(d)

Federal Rule of Evidence 502(d) now allows parties to seek an order from the court to prevent waiver of attorney-client privilege and the work product doctrine despite that party’s disclosure of such information.[37]  Arriving at this “claw-back” rule was quite a struggle.[38]  From 2006 to 2008, the proposed changes were debated, sent to public hearing, commented on, amended, and reviewed.[39]  Despite complaints and federalism concerns, proposed rule 502 was eventually transmitted to Congress for approval.[40]  Finally, in 2008, President Bush signed the bill into law.[41]  Although the process of Congressional adoption was uncharacteristically speedy, the slow movement by the Judicial Conference shows that even small changes to the Federal Rules are restricted by a tremendous amount of inertia.

4. Evidence of Slow Adoption: Professional Opinions

Having read, seen, and spoken with academics, judges, practitioners, consultants, legal technologists, independent consortiums, and discovery vendors, it is apparent to me that blockages stand between today’s world and a world in which the legal profession embraces technology.  At Georgetown, I took a class titled Electronic Discovery Seminar,[42] during which I sat through lectures by a number of practitioners in the e-discovery field.[43]  During that semester, I also attended an online presentation on Big Data, Big Analytics—Attacking ESI Volume with More than Just TAR.[44]  Finally, I attended the Advanced eDiscovery Seminar in McLean, Virginia, one of the nation’s largest electronic discovery conferences.[45]  Despite the relative popularity of that event, though, I was frequently told that judges and lawyers interested in e-discovery are in the extreme minority. I was told that most practitioners and judges are largely ignorant to the e-discovery problem and its solutions.  From these conversations, my general impression was that lawyers feel the legal field is slow to adopt new technology.

My impression that the legal field is slow to adopt technology is not only based on personal experience and anecdotal evidence.  The metadata, predictive coding, and Federal Rules of Evidence examples I have given show this on their own.  In Section C, I consider the causes of this phenomenon.

C. Reasons For the Slow Adoption of Discovery Technology

A perfect system of electronic discovery would see a seamless interaction between client and outside counsel and between litigants and the court. Parties would cooperate to create value and make the process efficient.[46]  The court would apply on-point case law and statutes, and the law would reflect the common goal of efficiency sought by both parties.  Unfortunately, the reality is that none of these entities are performing efficiently: (1) courts and (2) the Judicial Conference are not cutting edge institutions; (3) prevailing negative views of e-discovery technology restrict its popularity; and most importantly, (4) the bar does not seem interested in aggressively seeking new technology.

1. Institutional Underpinnings: The Judiciary

At present, in very many cases, if we want to know why a rule of law has taken its particular shape, and more or less if we want to know why it exists at all, we go to tradition.[47]

The study of law, as Justice Holmes’ famous essays suggests, is the study of history: court-made law is a series of intransigent doctrines evolving in slow-moving bodies.  In contrast, electronic discovery is an interdisciplinary and cutting edge practice. Done right, electronic discovery bands together data-mining scientists, computer scientists, hardware forensics engineers, and lawyers.[48]  While the court system is properly structured to gauge the credibility of litigants and, at its best, to push parties toward speedy resolutions of their disputes, litigants do not takes their computers to the courthouse for technology support.  The judiciary has not proven itself an efficient adopter of technology or expertise.

The traditional solution to this concern is the use of non-Article III judges, such as Article I magistrates or special masters,[49]  or their quick-moving and adaptable nature.[50]  Indeed, some magistrate judges have become impressively well-versed in the supervision of high technology disputes.  Judge Grimm’s discussion in Victor Stanley provides a good example: “[T]he net effect of accessing the Registry Editor and running the Disk Defragmenter program after deleting files and running the Disk Cleanup program was to ensure that deleted files could not be recovered.”[51]

Despite these enjoyable examples of judicial expertise, non-Article III judges will not be able to solve this problem on their own for at least four reasons.  First, e-discovery expert judges are rare, and frustration with discovery decision-making dwarfs the impressive work done by the three or four vanguard judges.  Second, the job of the judiciary has never been to babysit parties: The United States relies on an adversary system, and the more judges get involved, the less the system works.[52]  Third, magistrate judges can never hope to achieve the highest levels of technological know-how.  Innovation is, at its base, an exercise in testing the unknown and answering difficult questions: Nobel prizes are not awarded to scientists doing last year’s experiment; they are reserved for truly new ideas.  A judge with an organic chemistry PhD[53] is excellent, but still probably unable to truly master a physics patent.  Finally, Article I judges have largely failed to solve electronic discovery problems: E-discovery continues to loom large, despite the honorable work done by technologically-inclined judges.

2. Institutional Underpinnings: The Judicial Conference

The Federal Rules of Civil Procedure were amended in 2006 to reflect dramatic changes in the way discovery was conducted.  Before 2006, the words “electronically stored information” were nowhere to be found in the Federal Rules. Although it was a great feat to pass the 2006 Amendments, one cannot expect the Federal Rules to track the cutting edge of technology.  However, the body tasked with altering the Federal Rules, the Judicial Conference,[54] is by no means a quick-moving body.  The conference is comprised of judges from the appellate courts, district courts, and the Supreme Court.[55]  The Rules, therefore, will always be slightly out-of-date, and will not force parties to comply with the latest techniques.  A prime example is the current state of Federal Rule of Civil Procedure 37(e).

Rule 37(e) now provides that “[a]bsent exceptional circumstances, a court may not impose sanctions under these rules on a party for failing to provide electronically stored information lost as a result of the routine, good-faith operation of an electronic information system.”[56]  In other words, courts can impose little or no incentive on litigants to maintain a diligent records-keeping system.  Even though technology has made such policies easier—digital storage costs continue to drop—courts have no way to punish litigants for failing to keep up with the times.

Perhaps if Rule 37(e) were updated to allow for further preservation sanctions, courts would be less restricted in dealing with, for example, evidence spoliation cases.[57]  However, one cannot expect government regulation—particularly, the Federal Rules, a very protracted form of government regulation—to be the sole reason for corporate change.  The Judicial Conference is far too cumbersome to be leading the way with new technology.

3. Prevailing Negative Views of New E-Discovery Technology

Technology is often looked at hypercritically by practicing lawyers.  A recent case is instructive.  In late February 2012, United States Magistrate Judge Andrew Peck issued an order in the Da Silva Moore case approving the use of and setting limits on the scope of discovery by predictive coding.[58]  Judge Peck recognized the technique’s obvious advantages:

Unlike manual review, where the review is done by the most junior staff, computer-assisted coding involves a senior partner (or [small] team) who review and code a “seed set” of documents. The computer identifies properties of those documents that it uses to code other documents. As the senior reviewer continues to code more sample documents, the computer predicts the reviewer’s coding. (Or, the computer codes some documents and asks the senior reviewer for feedback.)[59]

Unfortunately, what started as an elegant and scientific solution became a media circus.  Plaintiffs, who eventually moved to recuse Judge Peck from the case in light of his speaking engagements on the CLE circuit, criticized Judge Peck’s order as biased.[60]

Whatever the merits of that motion, Da Silva Moore has unfortunately taken what might be a showcase for the latest in e-discovery technology, and shown the world that pain is what you get for trying to push the envelope: The parties had their claims delayed, and the court came under fire.[61]

The drama surrounding Da Silva Moore is exemplary of the bar’s negative view of predictive coding, which can be summed up as follows: it is difficult, it doesn’t make any sense, it is expensive, and it’s risky.  E-discovery software has been pitched as a black-box technology that can compute and process electronic discovery with little human help.[62]  This is exactly the opposite of what a good lawyer wants to hear: a lawyer’s training is precisely to meddle in a human and subjective way; the law is an exercise in applying human reactions to objective facts and data.[63]

The perception remains that, by analogy, a man can break through a mountain with a sledgehammer just as well as can a machine.[64]  This is no longer the case,[65] but it seems to remain true in many lawyers’ minds.

4. Resistance to Change at the Bar 

Men at some time are masters of their fates:
The fault, dear Brutus, is not in our stars,
But in ourselves.[66]

Maybe Shakespeare is right. While there are barriers to the efficient use of technology both in the judiciary and in the existing rules, it may be that the source of the problem starts earlier—with practicing lawyers themselves.  The problems listed above—problems with judges, rules, and prevailing views on the technology—are not the main barrier to the effective adoption of electronic discovery tools.  Rather, I propose that the greatest hurdle begins with general counsel and their outside attorneys.

Part of the problem is that lawyers may be ignorant of technology.  However, a more troubling problem is that lawyers may actually be incentivized to retain their obsolete e-discovery models for as long as possible.

i. Problems at the Bar: Ignorance of Technology

First, the law firm structure is one of hierarchy,[67] typically dominated by partners many years senior of their associates.  This makes sense from an experiential standpoint, but is not ideal for the adoption of technology: one’s seniors “tend to be less proficient in [the] use of technology.”[68]

Secondly, lawyers went to Georgetown, Yale, and Harvard, not MIT.  The study of law traces its lofty origins not to the scientific laboratory, but to ancient principles of philosophy and governance.[69]  It is axiomatic that lawyers are not technologists.  Indeed, it has been shown that lawyers need not establish themselves in the laboratory to reach the heights of their profession, even in technologically complex industries and cases.[70]  Thus, it may be difficult for lawyers to learn new technology.  But it may also be that lawyers actively resist learning new technology.

ii. Problems at the Bar: Disincentive to Change

Despite high costs, at least three factors make lawyers disinterested in updating their e-discovery technology: (a) they believe their current system is defensible; (b) they believe the current system works, and (c) the current system makes them tremendously wealthy.

(a) The Current System Is Defensible

That current e-discovery norms are defensible is beginning to change.  In In re Fannie Mae Securities Litigation, the Office of Federal Housing Enterprise Oversight (OFHEO) was subpoenaed as a third party to produce documents.[71]  OFHEO spent $6 million hiring fifty contract attorneys to review their massive production, but nevertheless failed: They were forced by the court to submit to sanctions.[72]  Judges are getting irritated with ineffective discovery: in the Gross case, Judge Peck saw “the latest example of lawyers designing keyword searches in the dark, by the seat of the pants.” [73]  The Gross case concerned discovery of an extensive database of emails.[74]  Because the parties had argued over the emails without conferring or adequately understanding the database, Judge Peck saw his order as “a wake-up call to the Bar in [the Southern District of New York]” to change its discovery practices.[75]

(b) The Current System Works

Does it? It has now been shown that “managed review,” or the human, eyes-only review of privileged documents, is both less effective and more costly than computer-assisted review.[76]  In 2006 the National Institute of Standards and Technology, a federal agency created to measure American technology,[77] began a competition to assess various methods of retrieving electronic business records for use in civil litigation.[78]  The competition—named the Legal Track at the Text Retrieval Conference (“TREC”)—has taken place every year since 2006, inviting competitors to test their review schemes against other automated, or human-based, review teams.[79]  The TREC results have shown, contrary to some skeptics’ expectations, that automated review is both cheaper and better than exhaustive manual review.[80]

(c) The Current System Pays Lawyers

This is the most negative reason to believe lawyers have been slow to adopt new technology.  It is not a popular opinion,[81] but that does not mean that it is not a real motivator.[82]  Simply put, staffing a document review team is a tremendous source of wealth for whoever is given the task, be it a law firm, a vendor, or an electronic discovery consultant.  Any business venture given such a revenue stream is unlikely to make efforts to stop the stream from flowing.

There are many reasons lawyers have been slow to adopt new discovery technology.  But we cannot expect these problems to resolve themselves: men are not angels,[83] and lawyers are probably worse.  It is inconsistent with our system of commerce and government to rely on cooperation between parties, especially between adversaries.[84]  Free market principles present the ideal solution to this problem.  In Part II, I propose that in-house counsel can drive the industry to technological fluency by beginning with its own staff.

II. The Solution to Electronic Discovery

Managing enormous stores of information has become a primary concern for the world’s top businesses.[85] This field is called “big data,” the study of data sets so large that they cannot be searched or used by traditional means.[86]  To business groups, big data is not a distant, hypothetical solution, but a concrete space where talented technologists can add value.[87]  Companies who have mastered the use of large quantities of data tell impressive stories.  For instance, Amazon has cut the pain out of customer service calls,[88]  Sears tailors coupons to specific customers,[89] and Google allows even small businesses to assess the relevance of advertisements to end-users.[90]

Not only has technology inundated the corporate world, but the information age has transformed all of our lives individually.  I grew up using Windows 3.1[91] and now flirt with which Android operating system I want on my cellular phone.  What got me from point A to point B?  Rapid advances in computer science and technology certainly played a vital role.  But more importantly, I was actually taught each new technology along the way.  Either a person—for me, usually my father—or an intuitive user-experience—think AOL, Google, or the iPhone—has gently ushered each of us into the world of new technologies.  These companies were not randomly inspired to facilitate an easy way to get on the Internet, a simple way to use advanced search, or a slick way to “feel” the device: They were incentivized by the existence, or potential for, a market.  Google stole users from Altavista and Yahoo by the thousands in what seemed like the span of a week.[92]  Apple released its original iPhone to lines around the block.[93]  Informed buyers—like my father for AOL, or modern tech geeks for the iPhone—played a key role in getting the most out of new technology.  It is these buyers that brought the technology revolution—smart buyers who taught their coworkers and friends about the new tools available to them.  To bring technology to the discovery space, a similar process will need to occur.

In the following section I propose that the substantive solution to the discovery problem is smooth incorporation of technology into electronic discovery, and argue that the framework to achieve the solution starts by bringing tech people in house.

A. The Substantive Solution: Step-by-Step Adoption of Technology

The long-term solution to the electronic discovery problem is, of course, more technology. Only the adoption of helpful technology in each step of e-discovery can lessen the burden of handling large document sets.  The Electronic Discovery Reference Model (“EDRM”) is widely accepted as a basic framework for how discovery proceeds in most cases.[94]  As seen in Figure 1, the model tracks the evolution of a discovery task from a high-volume, low-relevance set of documents, to a low-volume, high-relevance set of documents.  Throughout this process, lawyers and their staff “identify” the proper sources of information, “preserve” that information, assess (“processing, review, analysis”) which documents are relevant and which are privileged, and transmit (“produce”) only relevant, nonprivileged documents to opposing counsel.

Didday Figure 1

Figure 1(Used with permission.)

As the legal industry has evolved, more and more of the EDRM became automated.  For instance, collection is no longer done by loading filing cabinets onto trucks, but by forensically imaging hard drives and backup tapes.  Similarly, information management is no longer tasked to a single librarian, but is handled by a library-like team of technologists and programs that sort a company’s information, like Microsoft Outlook and iManage.  But to further reduce costs, technology needs to play a role in more parts of the EDRM, and play a larger role in the parts it already occupies. In other words, to save money, computers need to do a better job of taking discovery work away from lawyers.

The first, second, third, fourth and fifth steps of the EDRM are already quite automated.  For instance, “collection” is largely a forensics exercise now,[95] and preservation is the human-sized problem of issuing a litigation hold and maintaining hard drives and backup tapes.[96]  Technological advances in these areas will save money. For example, tighter information management that avoids the creation of duplicate documents would shrink the overall document pool and lessen discovery obligations. But the biggest cost-saving opportunity lies in step 3 of the EDRM.

The lack of processing, review, and analysis technology stands out.  These three are at bottom one task: defining documents as relevant and/or privileged.  To this day, much of this task is still done by large teams of junior or contract attorneys tasked with running their eyes over as many documents as they can in a day, for many days.  While this is a tremendous source of wealth for staffing agencies, this form of review is very costly to clients. Review and processing, by far, are the largest expenses in an electronic discovery project.[97]  Cost-saving technology in this area would mean significant reductions in litigation expenses.

Didday Figure 2

Figure 2[98]

The entire EDRM needs to be “iPhoned.”[99] We need applications helping us at every stage. If contract attorneys are the best solution, those attorneys need tools to make their reviews quicker and more efficient.  Even when creating seed sets for a predictive coding model, we need algorithms to help make selecting documents quick and easy. These, and other ideas, will be generated by software people. However, the more important and more difficult question is how we can get lawyers to buy into adopting this technology.

B. The Procedural Solution: In-House Counsel As Agents of Change

The future of legal services lies in [the hands of General counsel]. Most law firms are unlikely to change their ways without steady pressure from in-house lawyers. More than this, General Counsel must also be prepared to bring root-and-branch change to the workings of their own departments.[100]

I have outlined problems with the legal system, the legal profession, and pervading views on technology that combine to slow the adoption of new legal technology.  Judges will not do it,[101] the Judicial Conference will not do it,[102] and outside counsel will not do it.[103]  In-house counsel must be the agent of change in legal technology because no one else will.  Pace and Zakaras’ “Where the Money Goes” 2012 article gave the profession an empirical shot in the arm.[104] Hard data supported what was suspected anecdotally: Human review is very expensive, but no better than computer-assisted review.[105]  Importantly, that article also made a recommendation to solve this problem:

We believe that the most effective solution would be for forward-thinking, sophisticated organizational litigants to take a leap of faith and decide at the start of selected cases that their review obligations will be discharged using predictive-coding technology and to do so in a most public and transparent manner.[106]

We should listen to this recommendation.  I suggest a two-pronged approach to implement it: (1) hire tech-savvy employees in-house and (2) demand the use of effective cost-saving technology by outside counsel.

1. Hire Tech People In-House

As established in Part I, lawyers are not technologists.[107]  Bringing technology to a group that does not understand it requires help.[108]  Like the child configuring his parents’ VCR, or Apple bringing mobile computing to the masses, there must be a conduit between the problem (massive discovery obligations) and the solution (effective technology).[109]  The role of the in-house technologist would be to make the legal department an informed buyer of e-discovery services.

If technology is to be added into the EDRM, legal departments will need a staff comfortable with learning about the various tools they will be purchasing and implementing.  Electronic discovery conference materials, like modern magazines, are jammed almost cover-to-cover with advertising materials for vendors.  Without in-house technical prowess, a corporation can be tricked into buying what it does not need.  Those advertising materials can become tools of deception.

Fixing this problem does not mean turning the entire legal department into a tech group.  Indeed, everyone today has a smart phone, but only a small minority of buyers wait in line for a new iPhone on the day of its release.  In-house technologists would be akin to early adopters of technology—those waiting in line for the iPhone.  They would introduce tech knowledge to their supervisors and coworkers.  Ideally, the role would not have to be a full-time position, but would be a specialization for those legal employees who are particularly tech oriented.  Send a twenty-something to the e-discovery conferences, and watch what happens.

Finally, obtaining in-house tech personnel means clients will be able to assess the viability of technological legal solutions without having to rely entirely on the words of salespeople.  This paper has purposefully excluded a recommendation of any particular vendor or software option.  It has been my experience that vendors, with something to sell, have the same incentive that law firms have to maintain managed review: automated review makes them money.  If all of a client’s information on discovery solution comes from outside vendors, the client will be pulled in many directions at once.  In-house counsel must strive to understand what it needs on its own terms.

2. Demand the Use of Cost-Saving Technology

In-house counsel is the master of the legal services market. If in-house buyers buy only from outside counsel who are competent and aggressive with e-discovery technology, other outside counsel will adapt to that demand and devote more resources and energy to refining e-discovery techniques.[110]  Rather than hope for an excellent Article I judge or rely on the sluggish Judicial Conference to solve a client’s problems, outside counsel would be incentivized to fix them from the start.  In-house counsel must make it their job to provide the corporation with constantly improving, cutting edge litigation technology, and to demand the same from their contractors.  This means getting creative about where advances can be made and shortcuts created.  If the need for effective discovery comes from the purchasers of discovery services themselves—corporate clients—the sellers of discovery services will be forced to provide it.  From an informed perspective, in-house will be able to demand exactly the technological services they need.

C. Counterarguments

I have encountered two primary counterarguments against my recommendations.  First, in-house counsel’s primary interest in hiring an outside lawyer is often not discovery expertise, but “finding the best antitrust lawyer in town.”  Second, technology is not the messiah I have described.

1. Counterargument: Clients Look Only For the Best Lawyer

According to the first argument, since clients seek good lawyers, not tech people, outside counsel’s hiring pitch will always focus on legal talent and trial/settlement results, leaving the client to simply hope their lawyers can get through discovery without too much cost.  I believe this misses the point.  If litigation today depends so much on discovery—indeed, if, as I posited earlier, discovery is litigation[111]—a client looking for a good lawyer is looking for a good technology team, or at least a lawyer who knows a good vendor.  Furthermore, I am not convinced in-house counsel is so naïve or cost insensitive that they do not review many aspects of the decision to hire outside counsel for a new litigation.  Many in-house lawyers were previously firm lawyers and know how big of a role discovery plays in modern litigation.  Some senior in-house lawyers may have transitioned to corporate practice before discovery grew to its current size, but this is only another reason that in-house legal departments should hire young, tech-savvy lawyers.

2. Counterargument: Technology Is Not the Ultimate Solution

This argument comes in two forms. First, technology is not a complete savior, because human beings will always be a necessary part of lawyering.  Second, slow adoption of technology is not the real problem with discovery; rather, the problem is faulty rules or intransigent parties.

As to the suggestion that technology is not a complete savior, I agree.  Discovery will always require the eyes of lawyers.  Litigation is an inherently subjective, human-based endeavor.  The best way to conquer an electronic discovery project is with a team of superior technologists and excellent lawyers.[112]  Indeed, my solution is human itself: in-house counsel does not need a new software platform—it needs a person who understands new software platforms.  Finally, technology may not be a perfect answer, but is it the only answer currently available.

As to the argument that the lack of technology is not the true problem, I also agree in part.  This paper, however, is concerned with answering a narrow question in the machine context: how can we best manipulate and exploit a colossal set of electronic documents?  The answer to that question must involve machines.  This paper is not concerned with the broader, human questions, such as whether the Federal Rules of Civil Procedure should prescribe smaller discovery, or whether our legal system should move away from its adversarial nature.

D. Conclusion and Recommendation For First Steps

The modernization of the EDRM I recommend above[113] will happen slowly over time.  For now, though, legal departments interested in lowering costs should focus on the primary cost-creators in e-discovery: processing, review and analysis.[114]  These are the elephants in the room.  The reason the explosion of document volume is such a problem is that review costs have kept steady while storage and preservation costs have sunk just as quickly as document volumes have increased.  This is not sustainable, and the current bodies relied on to fix the problem—courts, rules makers, and outside counsel—have not effectively responded.

The answer needs to begin with corporate counsel.  Corporate counsel should favor tech-savvy candidates in hiring with particular attention to big data experience, and counsel should have these people in the room when evaluating law firms, vendors, and discovery consultants.  They should leverage this knowledge to buy the products that are right for them. Despite the obstacles in the way of the efficient adoption of discovery technology in the legal arena, a solution exists.  The legal market is, at base, a market, and will evolve just like any market once it is forced to evolve.  That evolution will not start in courtrooms, in legislatures, or even in law firms.  It has to start in-house.

 

 

 



         [1].    Stephen N. Subrin, Fishing Expeditions Allowed: The Historical Background of the 1938 Federal Discovery Rules, 39 B.C. L. Rev. 691, 694–95, 719–29 (1998);  see also The Rules Enabling Act of 1934, ch. 1, 48 Stat. 1064 (1934). (Current version at 28 U.S.C. 2072.)

         [2].    See, e.g., Hickman v. Taylor, 329 U.S. 495, 507 (1947) (stating “[n]o longer can the time-honored cry of ‘fishing expedition’ serve to preclude a party from inquiring into the facts underlying his opponent’s case”).

         [3].    See Oppenheimer Fund, Inc. v. Sanders, 437 U.S. 340, 351 (1978) (“Consistently with the notice-pleading system established by the Rules, discovery is not limited to issues raised by the pleadings, for discovery itself is designed to help define and clarify the issues.”).

         [4].    See Subrin, supra note 1, at 730 (reciting misgivings contemporaneous with the expansion of discovery contemplated by the Rules Enabling Act); John H. Beisner, Discovering A Better Way: The Need for Effective Civil Litigation Reform, 60 Duke L.J. 547, 554–63 (2010) (tracing reform efforts against broad civil discovery from the adoption of the Federal Rules, through their application by courts, and following their amendment in 1970).

         [5].    Beisner, supra note 4, at 584; see also Linda S. Mullenix, Discovery in Disarray: The Pervasive Myth of Pervasive Discovery Abuse and the Consequences for Unfounded Rulemaking, 46 Stan. L. Rev. 1393, 1397 (1994) (arguing “the pervasive myth of discovery abuse . . . is but one aspect of a larger myth of American litigiousness, itself a pervasive belief that has seized the public consciousness in spite of the existence of contrary evidence”).

         [6].    See, e.g., David K. Isom, Electronic Discovery Primer for Judges, 2005 Fed. Cts. L. Rev. 1 (2005) (“It is a fact of modern life that an enormous volume of information is created, exchanged, and stored electronically.”); Hon. Lee H. Rosenthal & James C. Francis IV, Managing Electronic Discovery: Views from the Judges, 76 Fordham L. Rev. 1, 2 (2007) (empaneling judges to discuss the challenges of e-discovery).

         [7].    Mohammad Iqbal, The New Paradigms of E-Discovery and Cost-Shifting Determining Who Pays for Electronic Discovery, 72 Def. Couns. J. 283 (2005).

         [8].    One terabyte is 1,000 gigabytes. Today, a 3 terabyte harddrive costs about $150. See Seagate Barracuda 7200.14, NewEgg.com, http://www.newegg.com/Product/Product. aspx?Item=N82E16822148844 (last visited Nov. 7, 2012). To store all of the emails from 2002 on Apple’s latest iPod Nano, you would need 25 million Nanos—about half of all Nanos sold last year. See iPod Sales Chart, Wikipedia, http://en.wikipedia.org/wiki/File:Ipod_ sales_per_quarter.svg (last visited Dec. 19, 2012); see also How Much Information? 2003, University of California, Berkeley, School of Information Management and Systems, http://www2.sims.berkeley.edu/research/projects/how-much-info-2003/execsum. htm (last visited Nov. 7, 2012).

         [9].    In re Intel Corp. Microprocessor Antitrust Litig., 258 F.R.D. 280, 283 (D. Del. 2008).

       [10].    Nicholas M. Pace & Laura Zakaras, RAND Corp., Where the Money Goes: Understanding Litigant Expenditures for Producing Electronic Discovery (2012), available at http://www.rand.org/content/dam/rand/pubs/monographs/2012/RAND_MG1208.pdf.

       [11].    Gartner Newsroom, Gartner Says E-Discovery Software Marketplace is Set to Continue High-Growth Pace, Gartner, http://www.gartner.com/it/page.jsp?id=1257113 (last visited Dec. 19, 2012).

       [12].    Judge Herbert B. Dixon, Jr., Automating the Search and Review of ESI, 51 No. 3 Judges’ J. 36, 36 (2012).

       [13].    See Jonathan T. Molot, How U.S. Procedure Skews Tort Law Incentives, 73 Ind. L.J. 59, 116 (1997) (concluding that “litigation costs play such a significant role that they may well overshadow the merits when plaintiffs decide to file, and parties decide to settle, lawsuits”).

       [14].    Janet Cooper Alexander, Do the Merits Matter? A Study of Settlements in Securities Class Actions, 43 Stan. L. Rev. 497, 528 (1991).

       [15].    Ralph C. Losey, Introduction to e-Discovery: New Cases, Ideas, and Techniques 36 (2008) (opining that, given that 98% of cases settle and 98% of all records today are ESI, “[l]itigators are, like it or not, not really trial lawyers at all: they are discovery lawyers, negotiators and mediators”).

       [16].    See, e.g., In re Verisign, Inc. Securities Litigation, No. C 02-02270 JW, 2004 WL 2445243, at *1 (N.D. Cal. Mar. 10, 2004) (quoting the magistrate judge’s order that ruled “production of TIFF version alone is not sufficient,” and “the electronic version must include metadata as well as be searchable”).

       [17].    Fed. R. Civ. P. 34(a) (2012).

       [18].    A document in its native format includes the document’s metadata.

       [19].    Fed. R. Civ. P. 34(b)(2)(i); see also Fed. R. Civ. P. 34(b)(2)(ii) (“If a request does not specify a form for producing electronically stored information, a party must produce it in a form or forms in which it is ordinarily maintained or in a reasonably usable form or forms.” (emphasis added)).

       [20].    See Conferences and Workshops, Dublin Core Metadata Initiative, http://dublincore.org/workshops/ (last visited Dec. 19, 2012) (Dublin Core is a librarian conference on the topic of metadata). TIFF stands for Tagged Image File Format, a file that is a high-quality image of a document. A TIFF is not the native document itself, but rather the equivalent of a hard-copy printed page.

       [21].    Wyeth v. Impax Labs, Inc., 248 F.R.D. 169, 171 (D. Del. 2006).

       [22].    Id.

       [23].    Id.

       [24].    The Sedona Conference, The (2004) Sedona Principles: Best Practices, Recommendations & Principles for Addressing Electronic Document Production: The Sedona Conference, Working Group on Electronic Document Retention and Production, AZ, 5 Sedona Conf. J. 151, 193 (2004) (a previous version of Sedona Principle 12).

       [25].    See, e.g., In re Verisign, Inc. Securities Litigation, No. C 02-02270 JW, 2004 WL 2445243, at *1 (N.D. Cal. Mar. 10, 2004).

       [26].    Williams v. Sprint/United Mgmt. Co., 230 F.R.D. 640, 652 (D. Kan. 2005).

       [27].    The Sedona Principles after the Federal Amendments, The Sedona Conf., https://thesedonaconference.org/publication/The%20Sedona%20Principles (last visited Dec. 20, 2012).

       [28].    Mark D. Robins, Computers and the Discovery of Evidence—A New Dimension to Civil Procedure, 17 J. Marshall J. Computer & Info. L. 411, 412 (1999).

       [29].    See id.

       [30].    Jason R. Baron, Law in the Age of Exabytes: Some Further Thoughts on ‘Information Inflation’ and Current Issues in E-Discovery Search, 17 Rich. J.L. & Tech., 1, 29–31 (2011).

       [31].    Dixon, supra note 12, at 36 (listing nicknames for computer-assisted review including, “technology-aided review (TAR), machine-aided review (MAR), and, finally, the new kid on the block, predictive coding”).  “The introduction of computers and software applications that allow for ‘on-line’ review replaced the flipping of pages with the somewhat more efficient clicking of a mouse.  More useful still were the term searches that quickly became possible.  Term searches could be used to help find relevant documents more quickly.” Id.

       [32].    Da Silva Moore v. Publicis Groupe & MSL Group, 287 F.R.D. 182 (S.D.N.Y. 2012); see also Order Approving the Use of Predictive Coding for Discovery, Global Aerospace, Inc. v. Landow Aviation, L.P., No. CL 61040, 2012 Va. Cir. LEXIS 50 (Va. Cir. Ct. Apr. 23, 2012);  Transcript of Motion for Partial Summary Judgment, EORHB, Inc. v. HOA Holdings, LLC (2012), available at http://pdfserver.amlaw.com/legaltechnology/ predictive_coding_order_delaware.pdf (quoting the Judge at oral arguments as saying, “I would like you all, if you do not want to use predictive coding, to show cause why this is not a case where predictive coding is the way to go.”); see, e.g., Symposium, Advanced E-Discovery Institute: Process, Paradigms, and Pragmatism, Georgetown L. Continuing Legal Education, available at http://www.law.georgetown.edu/cle/pdfs/278.pdf (devoting three separate panels to “Technology Assisted Review”).

       [33].    See generally Maura R. Grossman & Gordon V. Cormack, Technology-Assisted Review in E-Discovery Can Be More Effective and More Efficient Than Exhaustive Manual Review, 17 Rich. J.L. & Tech. 11 (2011).

       [34].    The Next Big ‘Buzz Words’ in Legal Technology, AbovetheLaw, http://above thelaw.com/2011/02/the-next-big-buzz-words-in-legal-technology/ (last visited Dec. 20, 2012).

       [35].    See also Pace & Zarakas, supra note 14, at 71. (stating “it is reasonable to ask why it is not being adopted by more litigants. None of the companies participating in our data collection, for example, employed a computer-categorized review strategy, despite having firsthand knowledge of how expensive review can be.”)

       [36].    Pace & Zakaras, supra note 10, at 68.

       [37].    See Fed. R. Evid. 502(d).

       [38].    See Henry S. Noyes, Federal Rule of Evidence 502: Stirring the State Law of Privilege and Professional Responsibility with A Federal Stick, 66 Wash. & Lee L. Rev. 673, 691 (2009).

       [39].    Id. at 693–94.

       [40].    Id. at 697–98.

       [41].    Id. at 700.

       [42].    Courses: Electronic Discovery Seminar, Georgetown Law, http://apps.law. georgetown.edu/curriculum/tab_courses.cfm?Status=Course&Detail=174

(last visited Dec. 19, 2012).

       [43].    Lecturers included Magistrate Judge John Facciola of the U.S. District Court for the District of Columbia, expert witness Dan Regard of iDiscovery Solutions, Chris Adams from Huron Consulting, and Jeff Kangas from H5. The Professors of the course, Mike Hitsky of Latham & Watkins and Peter Gronvall of Huron Consulting, were also especially helpful.

       [44].    Litigation & Support Careers, eDiscovery Journal Free Webinar: Big Data, Big Analytics: Attacking the ESI Volume Problem with More than Just TAR, LinkedIn, http://www.linkedin.com/groups/eDiscoveryJournal-Free-Webinar-Big-Data-1296357.S.18 8296814?qid=9bad4e03-bed3-4995-a075-8ef0dd34d08f&goback=.gmp_1296357 (last visited Dec. 21, 2012).

       [45].    Advanced eDiscovery Institute, Georgetown L. CLE, http://apps.law.george town.edu/continuing-legal-education/showEventDetail.cfm?ID=278 (last visited Dec. 21, 2012).

       [46].    Cooperation Proclamation, The Sedona Conference, https://thesedona conference.org/cooperation-proclamation (last visited Dec. 19, 2012); see also Hon. John M. Facciola & Jonathan M. Redgrave, Asserting and Challenging Privilege Claims in Modern Litigation: the Facciola-Redgrave Framework, 4 Fed. Cts. L. Rev 1, 44–45 (2009) (seeking corporation between parties and pushing for multiple meet-and-confers to hash out e-discovery issues).

       [47].    Oliver Wendell Holmes, The Path of the Law, 10 Harv. L. Rev. 457 (1897).

       [48].    See Ralph C. Losey, Overview of the Problems Posed by E-Discovery and the Team-Based Solution, in e-Discovery Current Trends and Case (2008).

       [49].    See, e.g., The Honorable Shira A. Scheindlin & Jonathan M. Redgrave, Special Masters and E-Discovery: The Intersection of Two Recent Revisions to the Federal Rules of Civil Procedure, 30 Cardozo L. Rev. 347 (2008).

       [50].    Morton Denlow, Should You Consent to the Magistrate Judge? Absolutely, and Here’s Why, 37 No. 2 Litig. 3, 5 (2011) (explaining that a magistrate judge may have more “expertise and experience in [a] particular type of case”).

       [51].    Victor Stanley, Inc. v. Creative Pipe, Inc., 269 F.R.D. 497, 511 (D. Md. 2010); see also D’Onofrio v. SFX Sports Group, Inc., 247 F.R.D. 43, 48 (D.D.C. 2008) (Magistrate Judge Facciola stating, “[n]evertheless, it is clear that the Instruction, if applicable to electronic files, permits production of the Business Plan in a non-native form without accompanying metadata”).

       [52].    See Wayne Brazil, The Adversary Character of Civil Discovery, 31 Vand. L. Rev. 1295 (1978) (“[T]he modern rules of discovery apparently failed to appreciate how tenaciously litigators would hold to their adversarial ways and the magnitude of the antagonism between the principal purpose of discovery . . . and the protective and competitive instincts that dominate adversary litigation.”).

       [53].    See, e.g., Alan D. Lourie, Circuit Judge, U.S. Ct. of Appeals for the Fed. Cir., http://www.cafc.uscourts.gov/judges/alan-d-lourie-circuit-judge.html (last visited Dec. 20, 2012).

       [54].    28 U.S.C. §§ 2072–2073.

       [55].    The Judicial Conference of the United States: Membership, U.S. Cts., http://www.uscourts.gov/FederalCourts/JudicialConference/Membership.aspx (last visited Dec. 20, 2012).

       [56].    Fed. R. Civ. P. 37(e).

       [57].    In D’Onofrio, Judge Facciola proposed using the inherent power of the trial court to issue such sanctions. D’Onofrio v. SFX Sports Group, Inc., No. 06-687, 2010 WL 3324964 (D.D.C. Aug. 24, 2010).

       [58].    Da Silva Moore v. Publicis Groupe & MSL Group, 287 F.R.D. 182 (S.D.N.Y. 2012).

       [59].    Id. at 184.

       [60].    See Memorandum of Law in Support of Plaintiff’s Motion for Recusal and Disqualification, Da Silva Moore v. Publicis Group, No. 11-CV-1279, 2012 WL 1446534, at *10 (S.D.N.Y. Apr. 13, 2012).

       [61].    See, e.g., Update: Judge Andrew Peck Refuses Recusal in ‘Da Silva Moore’ Order, Law.com, http://www.law.com/jsp/lawtechnologynews/PubArticleLTN.jsp?id=120255985 0 200&Update_Judge_Andrew_Peck_Refuses_Recusal_in_Da_Silva_Moore_Order_ (last visited Dec. 20, 2012); see also Plaintiffs Attack Judge Peck’s Da Silva Moore Predictive Coding Order Again, eDiscovery News, http://ediscoverynewssource.blogspot. com/2012/03/update-plaintiffs-attack-judge-pecks-da.html (last visited Dec. 20, 2012).

       [62].    See E-Discovery Steps Outside the Black Box, The Metropolitan Corp. Couns., http://www.metrocorpcounsel.com/articles/21330/e-discovery-steps-outside-black-box (last visited Dec. 21, 2012) (explaining “[s]ome providers claim their product is not a black box because you can see what comes out of it”).

       [63].    See William P. Barnette, Ghost in the Machine: Zubulake Revisited and Other Emerging E-Discovery Issues Under the Amended Federal Rules, 18 Rich. J.L. & Tech. 11 (2012) (beginning with the Churchill quote, “‘I am all for your using machines, but do not let them use you.’”).

       [64].    Cf. American Folklore, John Henry: The Steel Driving Man, http://american folklore.net/folklore/2010/07/john_henry.html (last visited Dec.20, 2012).

       [65].    See Grossman & Cormack, supra note 33; see infra text accompanying note 81.

       [66].    William Shakespeare, Julius Caesar act 1, sc. 2; see also Milberg LLP, Hausfeld LLP, E-Discovery Today: The Fault Lies Not in Our Rules . . ., 4 Fed. Cts. L. Rev. 131, 132 (2011).

       [67].    Duncan Kennedy, Legal Education and the Reproduction of Hierarchy: A Polemic Against the System (2007).

       [68].    Old People and Technology, Glench.com, http://glench.com/articles/old-people-and-technology.html (last visited Dec. 20, 2012).

       [69].    See, e.g., Max Radin, Maintenance by Champerty, 24 Cal. L. Rev. 48 (1936) (tracing the origins of a particular legal doctrine from ancient Greece, through Rome, to medieval England).

       [70].    For instance, between Morrison Foerster’s and Wilmer Hale’s respective star patent litigators, neither holds a scientific degree. People: Harold McElhinny, Morrison Foerster, http://www.mofo.com/harold-mcelhinny/ (last visited Dec. 20, 2012); People: William F. Lee, WilmerHale, http://www.wilmerhale.com/william_lee/ (last visited Dec. 20, 2012).

       [71].    In re Fannie Mae Sec. Litig., 552 F.3d 814, 816 (D.C. Cir. 2009).

       [72].    Id. at 823.

       [73].    William A. Gross Const. Assocs., Inc. v. Am. Mfrs. Mut. Ins. Co., 256 F.R.D. 134, 135 (S.D.N.Y. 2009)

       [74].    Id. at 135.

       [75].    See id. at 134.

       [76].    See Grossman & Cormack, supra note 33.

       [77].    See Grossman & Cormack, supra note 33; see also About, Nat’l Inst. of Standards & Tech., http://www.nist.gov/public_affairs/nandyou.cfm (last visited Dec. 20, 2012).

       [78].    See Grossman & Cormack, supra note 33; see also TREC Legal Track, About the Legal Track, http://trec-legal.umiacs.umd.edu/#about (last visited Dec. 20, 2012).

       [79].    About, Nat’l Inst. of Standards & Tech., http://www.nist.gov/public_ affairs/nandyou.cfm (last visited Dec. 20, 2012).

       [80].    See Grossman & Cormack, supra note 33, at 61; see also Jason R. Baron, The Trec Legal Track: Origins and Reflections on the First Year, 8 Sedona Conf. J. 251, 251 (2007) (“Though NARA resources were severely strained, the experience was highly instructive on several scores . . . I found that there was little in the way of present-day research showing what search and information retrieval methods were objectively better to use in a legal context.”).

       [81].    For example, in a CLE I recently attended, the participants were asked for reasons why adoption of technology is slow in the legal field, but none of the possible responses was based on dollar terms.

       [82].    William W. Belt et al., Technology-Assisted Document Review: Is It Defensible?, 18 Rich. J.L. & Tech. 10 (2012).

       [83].    The Federalist No. 51, (Alexander Hamilton) (“If men were angels, no government would be necessary. If angels were to govern men, neither external nor internal controls on government would be necessary.”).

       [84].    See Brazil, supra note 53.

       [85].    A recent survey of Fortune 1000 executives found that 85% of the organizations polled have big data initiatives planned or in progress.  Paul Barth & Randy Bean, Who’s Really Using Big Data?, Harvard Business Review: HBR Blog Network, http://blogs.hbr.org/cs/2012/09/whos_really_using_big_data.html (last visited Dec. 20, 2012); see also Thomas H. Davenport & D.J. Patil, Data Scientist: The Sexiest Job of the 21st Century, Harvard Business Review, http://hbr.org/2012/10/data-scientist-the-sexiest-job-of-the-21st-century/ar/1 (last visited Dec. 20, 2012).

       [86].    John Pavolotsky, Demistifying Big Data, Bus. L. Today (2012), available at http://apps.americanbar.org/buslaw/blt/content/2012/11/article-03-pavolotsky.pdf.

       [87].    Jeanne Harris, Data is Useless Without Skills to Analyze it, Harv. Bus. R.: HBR Blog Network (Sept. 13, 2012 9:00 AM), http://blogs.hbr.org/cs/2012/09/data_is_ useless_without_the_skills.html

       [88].    The following narrative is instructive:

Last month, I talked to Amazon customer service about my malfunctioning Kindle, and it was great.  Thirty seconds after putting in a service request on Amazon’s website, my phone rang, and the woman on the other end—let’s call her Barbara—greeted me by name and said, “I understand that you have a problem with your Kindle.”  We resolved my problem in under two minutes, we got to skip the part where I carefully spell out my last name and address, and she didn’t try to upsell me on anything.  After nearly a decade of ordering stuff from Amazon, I never loved the company as much as I did at that moment.

Sean Madden, How Companies Like Amazon Use Big Data To Make You Love Them, FastCo Design (last visited Dec. 20, 2012), http://www.fastcodesign.com/1669551/how-companies-like-amazon-use-big-data-to-make-you-love-them.

       [89].    How Sears Uses Big Data to Get a Handle on Pricing, CIO Journal, http://blogs.wsj.com/cio/2012/06/14/how-sears-uses-big-data-to-get-a-handle-on-pricing/ (last

visited Dec. 20, 2012).

       [90].    See generally Get More Value From Your Online Content, Google AdSense, http://www.google.com/adsense/start/ (last visited Dec. 20, 2012); Taming Big Data, Wikibon, http://wikibon.org/blog/wp-content/uploads/2012/05/big-data-infographic. html (last visited Dec. 20, 2010) (an infographic on various uses of big data in the corporate marketplace).

       [91].    kitkatbar3003, Windows 3.1 Startup, YouTube (Jul. 11, 2008), http://www. youtube.com/watch?v=hSJDIGiepgU.

       [92].    See Harry McCracken, How Long Did It Take For the World to Identify Google as an AltaVista Killer?, Technologizer, http://technologizer.com/2009/05/22/how-long-did-it-take-for-the-world-to-identify-google-as-an-altavista-killer/ (last visited Dec. 19, 2012).

       [93].    See Richard Menta, iPhone: Hundreds Come, Lines Orderly,  mp3newswire.net, http://www.mp3newswire.net/stories/7002/iPhone-line.html (last visited Dec. 19, 2012) (chronicling some of the first iPhone sales in 2007).

       [94].    See What is EDRM?, EDRM.net, http://edrm.net/files/EDRM_backgrounder.pdf  (last visited Dec. 20, 2012) (explaining that 125 e-discovery organizations collaborated on the framework).

       [95].    See, Collection Guide, EDRM.net, http://www.edrm.net/resources/guides/edrm-framework-guides/collection (last visited Dec. 20, 2012).

       [96].    See Preservation Guide, EDRM.net, http://www.edrm.net/resources/guides/edrm-framework-guides/preservation (last visited Dec. 20, 2012).

       [97].    See infra Figure 2.

       [98].    See Pace & Zakaras, supra note 10, at XV. Used with permission.

       [99].    See, e.g., Daegis Launches Mobile Apps for E-Discovery Review, Law.com, http://www.law.com/jsp/lawtechnologynews/PubArticleLTN.jsp?id=1202579502426&Daegis_Launches_Mobile_Apps_for_EDiscovery_Review=&et=editorial&bu=LTN&cn=20121128&src=EMC-Email&pt=Law%20Technology%20News&kw=Daegis%20Launches%20 Mobile%20Apps%20for%20E-Discovery%20Review&slreturn=20121028093645 (last visited Dec. 20, 2012); Xerox Litigation Services Supports Mobile Access to OmniX, Law.com, http://www.law.com/jsp/lawtechnologynews/PubArticleLTN.jsp?id=120257686 4283 (last visited Dec. 20, 2012); c.f. Pension Comm. of Univ. of Montreal Pension Plan v. Bank of Am. Sec., LLC, 685 F. Supp. 2d 456, 461 (S.D.N.Y. 2010) (Judge Scheindlin noting that “[t]he answer [to exponentially growing e-discovery] lies principally in culture change (i.e., fostering cooperation strategies), combined with savvier exploitation of a range of sophisticated software and analytical techniques”).

     [100].    Richard Susskind, The End of Lawyers? Rethinking the Nature of Legal Services l(ii) (2010).

     [101].    Supra Part I.C.1.

     [102].    Supra Part I.C.2.

     [103].    Supra Part I.C.4.ii.

     [104].    See Pace & Zakaras, supra note 10.

     [105].    Id. at 66.

     [106].    Id. at 81.

     [107].    Supra Part I.C.4.ii.

     [108].    See supra Part II.

     [109].    Indeed, I expect any tech-savvy person is used to this story. My brother and I were tasked from the beginning with setting up VCRs, fixing printers, and getting our mother’s email to “work.” As I grew older, this skill became valuable in the workplace: as an undergraduate researcher, I helped seasoned biological scientists operate modern microscopes and research databases; and as a paralegal, I helped class action attorneys operate mass-mailing software to convey notice to thousands of potential class members with one click of the mouse.

     [110].    Cf. Economics Basics: Supply and Demand, Investopedia, http://www.investo pedia.com/university/economics/economics3.asp#axzz2FXQkpWxB (last visited Dec. 19, 2012).

     [111].    See supra Part I.A.

     [112].    See Losey, supra note 48, at 3.

     [113].    See supra Part II.A.

     [114].    See supra Figure 2.