Most litigators, whether trial lawyers or appellate specialists, are regularly asked to estimate the chances of a particular case result. So you can imagine the reaction in the legal blog world when four professors — Jane Goodman-Delahunty, Maria Hartwig, Par Anders Granhag and Elizabeth F. Loftus — published a study concluding that many lawyers may not be very good at it. Titled Insightful or Wishful: Lawyers’ Ability to Predict Case Outcomes, [pdf] the article appeared in the American Psychological Association’s Psychology, Public Policy & Law.
Over the past few weeks, the study’s been everywhere: The Chronicle of Higher Education, Point of Law, Disputing, the Volokh Conspiracy, the ABA Journal and the Wall Street Journal Blog. [Thanks to the WSJ Blog for bringing the study to my attention.]
The professors surveyed nearly five hundred litigators from forty-four states across the country, each of whom had a case expected to go to trial during the study period. Each attorney was asked two questions: (1) what would be a win in terms of the attorney’s minimal goal for the case; and (2) what is the probability that you will achieve this outcome or something better? After the cases were settled or tried, the professors conducted follow-up interviews about the case outcome and the lawyers’ subjective perception of the outcome.
Overall, only one third of all participants reported a case result which matched their original goal. Forty-four percent of the litigators — including many who reported a high level of confidence in their expected result — did less well than they expected. Interestingly, the researchers did not find that more years of experience resulted in more accurate predictions. Civil attorneys performed no better than criminal attorneys, nor did predictions tend to improve as trial drew closer. The only positive correlation the professors found was gender: women lawyers tended to make more accurate predictions than men did.
The researchers suggested several possible reasons for the participants’ overconfidence: (1) litigators’ role requires them to project maximum confidence in their position; (2) lawyers’ commitment to, and emotional investment in, clients might lead to overconfidence; (3) wishful thinking; or (4) litigators’ perception that they have considerable control over events, whether circumstances suggest that’s true or not.
Although this isn’t the place to analyze the researchers’ methodology, their basic conclusion — that objective analysis of likely trial outcomes can be challenging – seems plausible, given the demands of trial work, with the years of preparation, discovery and motion practice and the emotional investment in the case and the client that comes with it.
So would the researchers have reached the same conclusion in a study limited to experienced appellate specialists?
I’m not aware of any objective, peer-reviewed research on that question, but my anecdotal impression, after sixteen years as an appellate specialist, is that the researchers would not have found the same overconfidence error in such a study. Join me below the jump for some reasons why.
Continue Reading