Methods for investigating decision making in health care network meetings

Health care networks can take many forms. Generally they are groups of health professionals from different services and perspectives who come together to improve health care. They are increasingly being used to bring about health service improvement. In Queensland, health care networks were formally established after the Queensland Health Systems Review in 2005,1 with the aim of engaging clinicians in health care decision making. Health care networks continue to be an important policy intervention in the 2010 National Health and Hospitals Network agreement.2 As improvements in the health care system depend on the success of network teams, health systems need to consider how network teams work to achieve their goals.

The way in which decision making is approached is recognised as an important function of leadership in health care.3 This increasing interest in the process of decision making as a function of leadership is evidence of a much larger shift in leadership thinking among the academic community.4-6 Although we traditionally associate leadership in health care with individuals, the complexity of change required in the health care system means a new paradigm of leadership is required.7 Relational approaches to leadership offer an alternative view; rather than being the function of an individual, leadership is considered to be an outcome of the processes that emerge when individuals interact.5 Relational processes are being recognised as just as influential in changing peoples’ attitudes, behaviours and actions as the direct influence of a formal leader.4

Within this relational leadership paradigm sits sense-making approaches to leadership;6-8 leaders are considered as sense makers who interact in ways that enable the nature of a problem to be understood by all those involved. The Cynefin (pronounced Ku-nev-in) framework (Box 1)9 is one example of a sense-making approach, and has been identified as useful for understanding decision making in the complex environment of health care delivery.

According to Fulop and Mark, approaches that rely on technical expertise and professional experience are favoured in health care, and health care leaders are therefore at risk of being trapped in the mindset of past successes.3 To avoid this, health care leaders should pause before deciding on a course of action, consider the decision-making context in which they are operating, and choose the most appropriate approach.3

In the light of this advice and with the aim of making a connection between the Cynefin framework and the day-to-day decision making of a network team, we have developed a process for real-time capture and analysis of a network team’s interactions. This can be used to create a “meeting interaction fingerprint” that describes the interaction patterns of a network team engaged in decision making. The fingerprint consists of metrics summarising different aspects of a network team’s interactions.

The metrics from different meeting fingerprints would enable the interaction patterns that emerge during a meeting to be compared. We would like the fingerprint to capture similarities and differences in interaction patterns between teams, or meetings or decisions, which emerge during decision making. In combination, all possible metrics could be likened to a DNA chain. Just as variation in a specific part of a DNA chain can result in different traits, variation in individual metrics could indicate different decision-making processes.

Here, we demonstrate the process of producing a meeting interaction fingerprint.

A meeting fingerprint with the metrics that we are interested in is shown in Box 2. The total number of utterances was 1031, which increased to 1274 utterances after subdividing into behaviours. The total duration of utterances in hours, minutes and seconds was 1:27:04, while the total duration of utterances subdivided into behaviours was 1:22:53 (a shorter duration because of removed silences). The total duration of the meeting was 1:20:25. Overlapped speech caused the total duration of the utterances to be greater than the length of the meeting.

The sections in Box 2 each represent a different metric. The metrics are grouped into aggregate statistics and sequence metrics. On the left of Box 2, sections i, iv, vii, viii, xi and xii describe speaker-related metrics. On the right of Box 2, sections iii, vi, ix, x, xiii and xiv describe behaviour-related metrics. Raw sequence metrics (sections vii, ix, xi and xiii) are to the left of the normalised sequence metrics (sections viii, x, xii, and xiv) that eliminate most effects except that of the sequence order. The upper aggregate statistics, sections i, ii and iii, are aggregations of utterances of all time lengths. The lower aggregate statistics, sections iv, v and vi, are aggregations of utterances with particular durations. A logarithmic time scale was chosen for the durations to give a histogram bin size proportional to the duration, and very short utterances were removed to produce a smooth curve.

We have demonstrated a process for generating a meeting interaction fingerprint and given some examples of metrics that could be included. The fingerprint presented here describes the interactions over a whole meeting, leading up to a critical decision point. However, as meetings often include multiple decision points, it is also likely that different parts of meetings may have different fingerprints. A video of the fingerprint could also be made, in which each frame shows the fingerprint calculated from data up to the time of that frame.

Aggregate statistic metrics (counts of various types of annotations) give an insight into the meeting interaction. For instance, there are several dominant speakers (Box 2, i) and the short utterances tend to be positive behaviours (Box 2, vi). The aggregate statistic metric plots could be subdivided further to show the contributions of the different stages of the meeting.

Sequence metrics (the order in which types of annotations occur) can be divided into the macroscopic and microscopic order of the annotations. The upper sequence metrics (Box 2, vii, viii, ix and x) describe the macroscopic order of the annotations. The time elapsing between speakers’ utterances or speakers’ behaviours that re-occur give an insight into whether these are evenly or unevenly distributed throughout the meeting. The lower sequence metrics (Box 2, xi, xii, xiii and xiv) describe the juxtaposition of annotations. These could be regarded as describing the microscopic order of the annotations.

By manually reviewing the automated annotations, we have found a high correspondence between manual and automated annotations. Review of the places where the automated and manual meeting annotations differ shows that, for the metrics reported here, the automated process is generally correct. This automated analysis process then allows for all meetings that have been recorded to be annotated. The fingerprints of each meeting can then be normalised and plots presenting per-speaker information could be sorted so that speakers making the largest contribution are placed first.

Once all meetings have been annotated, similarities and differences between interaction patterns associated with manually identified decision-making processes can be made. We expect that this will prove useful for describing the interaction patterns that emerge during decision making.

Decision making in the different decision-making domains in the Cynefin framework may manifest as different modes of interacting and networking.11 For example, issues with strong scientific evidence, such as the promotion of proper hand-washing techniques to prevent infection, would fall into the simple domain. In the simple domain, the solution is evident; therefore, networking is more formal and targets specific individuals or groups for the purpose of communication. In the complicated domain, there is the requirement for expert debate; for example, to determine the appropriate information to be included in a treatment plan intended for general practitioners. Consequently, the network of individuals involved, although again targeting those with technical expertise, is more widespread than in the simple domain. The complex domain requires “out of the box” thinking, which is achieved by bringing together individuals or groups, all experimenting with possible solutions, to share their discoveries. Issues such as how to respond to unexplained variation in patient outcomes would fall into this domain. In the complex domain, networking and high levels of interaction are encouraged. In the chaotic domain (eg, immediately following a serious adverse event), it is important that order is restored to the system as quickly as possible; therefore, networking is once again restricted to a few individuals who decide on an immediate course of action. The intention of a meeting interaction fingerprint is to describe the differences in the interaction patterns typical of the different approaches to decision making.

Our process for annotating meetings allows for the automatic generation of a partial fingerprint (ie, one that includes the metrics that do not rely on behaviour). To date, conducting a program of research of this type on a large scale has been constrained by the time taken to manually annotate meetings. The technology developed for this project by the Australian e-Health Research Centre has given the processing power to routinely “fingerprint” meetings.

We have demonstrated the concept of generating a meeting interaction fingerprint. Future publications will report on the reliability of the annotation process, both manual and automatic. Most importantly, we will report on the results of comparing the meeting interaction fingerprints of different network teams engaged in decision making. The fingerprint concept will be expanded to include qualitative (narrative) as well as quantitative analysis.

2 An interaction fingerprint of a meeting, with example metrics

Click for larger version of this graphic.