Attitudes of health professionals to using routinely collected clinical data for performance feedback and personalised professional development

The delivery of health care has been transformed by digital technologies,1 particularly by the introduction of electronic health records (eHRs) across all levels of care.2 This is reflected in Australia by the growing interest in digital hospitals; that is, hospitals that employ information management and communications technology to support clinical workflows and to improve safety and quality.3 At the national level, My Health Record will provide an online repository of summary health information for all participating Australians and treating clinicians.4

While the immediate quality and safety benefits for patient care associated with eHR data are increasingly well established,5 the introduction of eHRs has not universally been favourably received.6 Clinicians report that their workloads have increased rather than decreased7 and do not see the benefits outweighing the expense in time and effort involved in gathering electronic health data.8 Only limited clinical data are currently available to clinical teams for informing quality improvement or performance review.9

One area in which the secondary use of data may be of significant value is in reducing clinical variation and improving the alignment of practice with evidence. Many studies have found that a considerable proportion of care delivered in Australia is not evidence‐based and that there is unwarranted variation in care between jurisdictions and services.10

It has long been recognised that personal reflection on clinical practice is a key learning strategy for clinicians.11 This is consistent with evidence that audit and feedback influence behaviour and that educational interventions linked to practice are highly effective.12 However, existing audit and feedback programs are often time‐consuming and expensive, usually involving manual review of health records, which delays feedback.13 In addition, educational interventions are rarely tailored to the individual needs of a clinician or to their performance or scope of practice.14

The increasing volume and quality of data in eHRs may facilitate responsive and personalised educational programs that include audit and feedback components tailored to the individual's practice.15 But surprisingly little research in this area has been published, particularly given the current focus on developing a professional performance framework that more effectively links education with practice in Australia.16 Performance feedback to clinicians is largely restricted to dashboards presenting summary clinical data, with the aim of improving adherence to quality guidelines.17 Few, if any studies have supplied evidence that could guide how data are presented or linked to educational interventions and other activities.

Central to the successful development of systems that employ eHRs or other data sources for delivering targeted education about performance is engaging health professionals in their design.18,19 Accordingly, we explored the attitudes of health professionals, health informaticians, and information communication technology professionals to using eHR data for performance feedback and personalised professional development. The investigation is part of a broader study, Turning Point, that aims to design, implement, and evaluate programs for personalised professional development and performance feedback.

Methods

The study employed a co‐design framework,20 an approach that aims to actively engage end users throughout the project cycle to ensure that the final product meets their needs. Co‐design encourages ownership and support of the methodologies and the solutions developed, leading to increased uptake and sustained implementation of innovations.

We collected information about health professionals’ views on how eHR data should be used for performance feedback and personalised professional development. Co‐design workshops explored whether practitioners currently received data related to their practice; their views on how data could be best presented; what types of data are likely to have the most value; which systems could be developed for presenting data; how these systems could be linked to professional development and training; and barriers to and enablers of using clinical data for educational purposes.

Participants were selected by purposive sampling.21 A mixture of health professional groups were recruited for the workshops, with participants ranging from early career clinicians to experienced health professionals. Participants were assigned to workshops by professional group, and no participant was included in more than one workshop. A total of nine co‐design workshops were held in two major public hospitals in Sydney: three for nursing staff (ten participants in total), three workshops for doctors (15 participants), and one workshop each for information communication technology professionals (six participants), health informaticians (four participants), and allied health professionals (13 participants). A convenience sample was used for the workshops, but we restricted attendance to a maximum of 15 people. Workshops were conducted until data saturation was reached.

Each workshop was facilitated by an interviewer experienced in qualitative research (author TS) with a set of semi‐structured questions. The workshops were audio‐recorded, and one investigator (AJ) made detailed field notes. The interviewer and researcher individually reflected on the content of each focus group meeting, then compared and discussed their reflections, recording any differences in observations in the field notes. The audio recordings were transcribed, de‐identified, and subjected to thematic analysis.

Ethics approval

The study was approved by the Western Sydney Local Health District Human Research Ethics Committee (reference, HREC4953).

Results

Analysis of workshop transcripts identified six major themes: enthusiasm for feeding back clinical data; formative rather than punitive use; data use and access; capturing complex clinical narratives; peer comparison, benchmarking, and collaborative learning; and system design challenges (Box 1, Box 2).

Enthusiasm for feeding back clinical data

The participants in all workshops were interested in access to routinely collected clinical data for secondary purposes, particularly for performance review and personalised professional development. While several health professionals in the nursing workshop and the three workshops with doctors indicated they had some access to data about their practice and outcomes, most participants felt that it was difficult to access data in eHR systems for secondary purposes, particularly the five early career doctors, who were more frequently unsure about how to access clinical data. Some participants, especially early career doctors, felt that access to data for the purposes of reflection on practice was actively discouraged. A number of more experienced health professionals revealed that they often used performance feedback as the basis for teaching units, but rarely drew data directly from eHRs for this purpose.

Formative rather than punitive use

Participants from all health professions raised concerns about the punitive use of clinical data for identifying their errors. Workshop participants saw such use of performance data as a barrier to implementing any personalised professional development platform. In contrast, participants were excited about using data for instructive purposes, such as identifying the latest evidence or guideline changes, and about rapidly applying this information to clinical practice.

Data use and access

Participants had differing opinions about who should have access to performance data. Doctors in all workshops debated options for access to data by professional colleges, service organisations, government bodies, and clinicians, without reaching a consensus. Medical trainees were clearly worried about the access of professional colleges to information that could affect their career progression. In the workshop for health informaticians, participants suggested that data use was linked to the motivations for collecting and digitising information.

Capturing complex clinical narratives

Participants in all workshops questioned whether clinical data in eHRs captured the complex narratives of clinical practice. Many noted that patients were seen by a team of health care professionals, and that individual eHR data points may not adequately reflect how clinicians performed in certain situations. The difficulty of capturing the complex clinical narrative was considered a barrier to feeding back routinely collected clinical data, especially for professional development, particularly by nurses and allied health professionals.

Peer comparison, benchmarking, and collaborative learning

Participants were keen to use eHR data to support peer collaboration, comparison and benchmarking. All health professional groups saw the potential of eHR data for benchmarking and informing personalised professional development. However, the definition of “benchmarking” differed between disciplines, and the focus of early career doctors was often on individual benchmarking against peers in national and international organisations.

Participants in all workshops expressed interest in using routinely collected data for collaborative and team‐based learning, particularly nurses and allied health professionals, who were more focused on benchmarking of teams and wards than of individuals. They were also more likely to express concerns about making the process competitive, for fear it would lose value.

System design challenges

Participants in all workshops discussed the design of systems for feeding back routinely collected data, particularly information communication technology professionals and informaticians, who noted that health care organisations usually postponed discussing secondary applications, such as exploiting data for professional development, instead giving priority to solving the practical problems of eHR implementation. Participants also commented that it was difficult to exploit health data for secondary purposes without causing negative perceptions of information communication technology staff.

Participants in all workshops noted that systems were generally designed so that they could be used by staff without a high degree of information technology training, with the aim of ensuring that patient safety not be compromised. Designing systems that engage staff with greater technical skills but are still accessible to all was viewed as challenging. One participant suggested the solution was to design systems personalised to individual health professionals.

Discussion

Our findings indicate that health professionals support exploiting clinical data for performance feedback and personalised professional development, and that they believe there is a growing body of data in eHR systems that is underutilised in this respect. This is perhaps unsurprising, given that facilities in many countries, including Australia, are more concerned with implementing or optimising core eHR functions than with investing in secondary opportunities.

Extracting data and reporting performance problems is generally undertaken in the context of quality improvement, and is often related to high level organisational or government performance indicators.22 The data analysed in these programs is often administrative, and not derived from eHRs. Similarly, data analysed in local quality improvement initiatives is usually collected for a defined period of time and not used for professional development purposes.23

There are also significant challenges related to the range of data types collected during routine care delivery in terms of its potential exploitation for secondary purposes such as education. When eHR data are used for providing performance feedback to clinicians, they are usually presented on a static dashboard, rather than in an interactive manner that allows the data to be explored when and how it may be most beneficial for the individual.17 Accompanying the data with scaffolding that assists their interpretation has not been reported, nor has using them to drive programs, such as personalised education.

Our findings provide insights into the barriers to and facilitators of designing programs that support more sophisticated integration of data into interventions aimed at changing the behaviour of health professionals. Unsurprisingly, anxiety about who can access data related to clinical performance and the use of such data for evaluative purposes was a key reservation for participants in all workshops. This is a challenging problem, as managers will feel obliged to react to information about poor performance by clinicians in order to reduce the risk to patient care. It will be difficult for organisations to leave the resolution of identified performance deficits to professional development processes alone, given the effects the problems may have on patient experiences or outcomes of care. These barriers need to be overcome by applying user‐centred design principles and collaboration between clinical professionals and safety and quality regulators for co‐designing such programs. New procedures that enable the secondary use of eHR data should be implemented, including governance structures and tools for rapidly identifying data suitable for performance feedback. Any systems developed will naturally need to comply with local regulations and standards regarding privacy and consent. The types of protection related to disclosure that apply to mortality and morbidity meetings may perhaps be adapted for use in this context.24

The challenges of assessing clinical performance on the basis of eHR data are substantial. They include how to distinguish individual from team performance, and how to gather sufficient information to accurately benchmark performance. Possible approaches to overcoming these problems include initially focusing on team performance and starting with simpler data types, such as antibiotic orders.

Questions of privacy and consent related to the secondary use of data also need to be examined from the perspective of the patient. Legislation regarding the use of patient data for performance feedback and personalised professional development is yet to be developed and agreed upon.

Limitations

This study examined only the views of the health professionals who participated in the workshops. However, a number of workshops, including participants from a range of health disciplines, were conducted, and workshops were conducted until data saturation was reached. Occupation‐based samples were selected to explore topics in depth, and this may have reduced interdisciplinary exchanges of ideas. A further limitation is that the study was conducted at two metropolitan public hospitals; different results may have been reached at rural or private hospitals. Further investigations could be undertaken at a broader range of hospitals, and workshops that included patients and other non‐medical participants could capture their views on the use of their eHR data for the education and professional development of health professionals.

Conclusion

Further investigation and evaluation of the use of eHR data for developing programs that link performance data with behavioural interventions, such as personalised professional development programs, would be useful. This could significantly improve practice and quality care, and support the development of the professional performance framework for linking practice with education that is currently being discussed in Australia.16

Box 1 – Overview of the themes that emerged from workshops

---