Hindsight bias in medicolegal expert reports

Malpractice litigation costs are a significant problem in the provision of healthcare. In the year 1999–2000, medical malpractice claims against the United Kingdom government totalled £4 billion, almost 10% of the overall National Health Service budget2 and a threefold increase in one year. In Australia, rising indemnity costs are changing the face of medical practice, forcing government action towards tort law reform.

The cause of this steep escalation in claims is multifactorial. The view sometimes expressed that it is due to declining standards of medical care is countered by overwhelming evidence of improved outcomes for medical interventions. Medicine has become a victim of its own success, because improved outcomes have fuelled unrealistic community expectations, encouraged by the increased activity of personal injury lawyers.

Although recent Australian judgments, such as Rogers v Whitaker,3 might seem to have shifted the emphasis away from reliance on medical expert reports, these are still the cornerstones of most medical negligence cases. The High Court of Australia reinforced this view in Rogers v Whitaker by stating inter alia:

Courts are naturally sympathetic to injured patients and, in complex cases, just one expert report critical of a doctor's actions can provide sufficient basis for a judgment for the plaintiff. Many such decisions have left the medical profession feeling that doctors are often punished unjustly simply because there has been an adverse outcome.

There is evidence that the objectivity of expert witnesses is frequently tainted by hindsight bias, also known as outcome bias.4 Hindsight bias is not deliberate, but is induced by what one researcher describes as "creeping determinism", a process propelled by subconscious desires on the part of the expert to appear knowledgeable, intelligent and unambiguous.5

The fact that expert peer review of a case is necessarily retrospective is rarely given adequate recognition in the litigation process, even though judges occasionally refer to the aspect of hindsight. Doctors have long been aware of the pitfalls of the "retrospectoscope", but it is not generally known that considerable scientific evidence indicates that hindsight bias is inevitable when a reviewer is aware of an adverse outcome.4,6-9 Hindsight bias ensures that some reasonably acting defendants will be unfairly subjected to adverse liability judgments.10

Hindsight bias has been recognised by psychologists for decades, and has been studied more recently by investigators interested in quality assurance in healthcare. In one study,6 anaesthetist reviewers were provided with sets of cases with the same descriptive facts, but with outcomes randomly assigned to be either bad or neutral. The anaesthetists consistently rated the care in cases with bad outcomes as substandard, whereas they viewed identical care with neutral outcomes as being up to standard. The degree of bias is linked to the severity of the outcome — with severely injured patients, judgements by reviewers tend to be harsher.11 When reviewers know of an adverse outcome, they tend to trivialise the management dilemmas facing the doctor at the time, overlooking the uncertainties inherent in diagnosis and treatment.7 Expert opinions frequently include the phrase "it should have been obvious". It has been suggested that hindsight bias is almost always present when that expression is used.7

We know of no Australian medical malpractice case where scientific evidence of hindsight bias has been adduced, although it has been raised in the United States.5 There might be a need to increase awareness of this phenomenon among the legal profession. The New South Wales Court of Appeal has accepted the relevance of sources of confounding and bias in epidemiological evidence,12 and there seems no reason why evidence about hindsight bias in medical reports should not similarly be evaluated. In addition, a barrister who is aware of the phenomenon should be capable of exposing hindsight bias during cross-examination. Unfortunately, simply warning experts about hindsight bias is inadequate — the effect is present even when those making the evaluations have been warned and advised to guard against it.8

In other areas, the law recognises the need to withhold information that may bias judgement. For example, in criminal trials, information about the previous record of the defendant is withheld from the jury. Instead of the criminal law standard of "beyond reasonable doubt", decisions in malpractice litigation are made on the less stringent "balance of probability", a situation which calls for more rigorous exclusion of sources of bias, such as knowledge of outcome.

One way to improve the objectivity of experts is to work towards explicit opinions about appropriateness of care. Retrospective peer-review evaluations are usually implicit,9 depending on subjective evaluation by the reviewer. Because the reviewer is usually aware of the outcome, his or her view is based on assessment of both process (diagnosis and treatment) and outcome, whereas courts, in deciding whether a doctor was negligent, should seek an objective opinion on process alone. Explicit opinions are based on specific predetermined criteria, set by group agreement. Developing such criteria is expensive and time consuming, but they can yield valid and reliable measures of appropriate care for both medical and surgical services.13,14 However, the paucity of published clinical practice guidelines indicates the difficulty of developing such criteria. Furthermore, it is virtually impossible to provide a set of pathways for all clinical contingencies.

Another approach might be to use a "neutral" expert or panel to evaluate the appropriateness of decisions by a clinician in a given situation. Hindsight bias can be diminished by withholding information about the outcome of the case.15 Unfortunately, the very seeking of an expert opinion usually indicates that there has been an adverse outcome.

Completely neutral opinions are difficult to obtain even with de-identified items such as histological or cytological slides or x-ray films. The seeking of an expert opinion makes the reviewer more alert than if the slide or film were being reported routinely. As one radiological expert said, "Whenever an attorney sends me radiographs, the first and only question that comes to my mind is, what was missed on these films?".5 It is almost never possible for such items to be slipped into the pile of routine items for the reviewer, but the objectivity of an expert opinion on slides or x-rays is increased if information about the outcome of the case is withheld.

The question of hindsight bias should be included in the current debate about the role of expert witnesses. Expert conferencing, in which experts for both sides meet and prepare a report that specifies matters agreed and matters not agreed and the reasons for non-agreement, is described in the NSW Supreme Court Rules,16 and may be useful, especially if experts are required to apply the criteria of evidence-based medicine, stating the level of evidence for each assertion. However, the application of the NSW Supreme Court Rules for expert witnesses, gazetted in January 2000, must be regarded as a failure so far with regard to expert conferencing. Despite in-principle support for this procedure by most Australian judges,17 we know of no medical negligence case in which expert conferencing has been used. Australian judges have also expressed a strong desire for objective and reliable expert help;17 elimination of hindsight bias would help further this aim.

Hindsight bias is almost inevitable in retrospective peer-review reports. Bias is increased when reviewers know there has been an adverse outcome, and the degree of bias is proportional to the severity of the outcome. Bias may be reduced, but not eliminated, by educating reviewers about it and by withholding outcome information.

These facts should be taken into account in judicial evaluation of medical expert reports. It should be possible for scientific evidence about hindsight bias to be adduced, where appropriate, in medical negligence cases.