Most litigators, whether trial lawyers or appellate specialists, are regularly asked to estimate the chances of a particular case result. So you can imagine the reaction in the legal blog world when four professors — Jane Goodman-Delahunty, Maria Hartwig, Par Anders Granhag and Elizabeth F. Loftus — published a study concluding that many lawyers may not be very good at it. Titled Insightful or Wishful: Lawyers’ Ability to Predict Case Outcomes, [pdf] the article appeared in the American Psychological Association’s Psychology, Public Policy & Law.
Over the past few weeks, the study’s been everywhere: The Chronicle of Higher Education, Point of Law, Disputing, the Volokh Conspiracy, the ABA Journal and the Wall Street Journal Blog. [Thanks to the WSJ Blog for bringing the study to my attention.]
The professors surveyed nearly five hundred litigators from forty-four states across the country, each of whom had a case expected to go to trial during the study period. Each attorney was asked two questions: (1) what would be a win in terms of the attorney’s minimal goal for the case; and (2) what is the probability that you will achieve this outcome or something better? After the cases were settled or tried, the professors conducted follow-up interviews about the case outcome and the lawyers’ subjective perception of the outcome.
Overall, only one third of all participants reported a case result which matched their original goal. Forty-four percent of the litigators — including many who reported a high level of confidence in their expected result — did less well than they expected. Interestingly, the researchers did not find that more years of experience resulted in more accurate predictions. Civil attorneys performed no better than criminal attorneys, nor did predictions tend to improve as trial drew closer. The only positive correlation the professors found was gender: women lawyers tended to make more accurate predictions than men did.
The researchers suggested several possible reasons for the participants’ overconfidence: (1) litigators’ role requires them to project maximum confidence in their position; (2) lawyers’ commitment to, and emotional investment in, clients might lead to overconfidence; (3) wishful thinking; or (4) litigators’ perception that they have considerable control over events, whether circumstances suggest that’s true or not.
Although this isn’t the place to analyze the researchers’ methodology, their basic conclusion — that objective analysis of likely trial outcomes can be challenging – seems plausible, given the demands of trial work, with the years of preparation, discovery and motion practice and the emotional investment in the case and the client that comes with it.
So would the researchers have reached the same conclusion in a study limited to experienced appellate specialists?
I’m not aware of any objective, peer-reviewed research on that question, but my anecdotal impression, after sixteen years as an appellate specialist, is that the researchers would not have found the same overconfidence error in such a study. Join me below the jump for some reasons why.
Persuading an appellate panel involves a different skill set than persuading a jury of non-lawyers. An appellate specialist is focused on a different goal — not which party is "right," but the more objective question of whether a legal error did or did not occur. In addition, although appeals are certainly not resolved quickly in these days of crowded appellate dockets, they still are resolved faster than most trials are, measured from complaint to verdict.
The most important reason why appellate specialists offer an important insight, however, is the nature of the appellate lawyer’s task. As the professors pointed out, for at least thirty years psychological research has shown that participants’ social judgments are improved by requiring them to think through the opposing viewpoint.
That’s a large part of what appellate specialists do. If the appellate lawyer is representing the appellant, then she knows exactly what the strongest possible argument for the opposing side is: the trial court’s order or judgment. Even if the appellate lawyer’s client won before the trial court, by the time of the appeal, the record is full of the best possible arguments for the opponent — page after page of motion practice, not to mention the trial transcript. A large part of the process of preparing for appellate oral argument involves closely analyzing the record to determine the hardest questions an appellate panel could ask, and most appellate specialists believe that moot courts – mock arguments where colleagues pepper the lawyer with difficult questions – are an indispensable part of preparing for any important argument.
Insightful or Wishful offers empirical support for a cornerstone of the case for appellate specialists: by the time a case has gone all the way through trial, having a new attorney, experienced in the language and procedures of the appellate courts, offer a fresh analysis really does improve client outcomes.