Evaluation of the acceptability of standardised clinical handover tools at four Victorian health services

The Victorian Quality Council (VQC) is a ministerial advisory council that was established in 2001 to provide leadership in safety and quality for Victorian health services. The VQC’s second term commenced in 2005 and ended in 2008. One of the strategic goals of the second term was to “support the ongoing development of a workplace culture for all healthcare staff that values teamwork and individual competence in the delivery and continuous improvement of quality and safe patient care”.1 A component of this goal was to “enhance continuity of care through clinical staff handover”.1 Clinical handover (CH) refers to the transfer of professional responsibility and accountability, for some or all aspects of care for a patient or group of patients, to another person or professional group on a temporary or permanent basis.2

Ineffective CH is a recognised patient safety issue. Evidence of this can be found in sentinel event program annual reports,3 outcomes of coroners’ recommendations, and national4,5 and international literature.6,7 Although the importance of good CH has been recognised internationally, there is limited research to guide the development of best practice standards.

The lead-up to the CH pilot project described here involved gathering information on ways to develop a project that would have a meaningful impact on improving patient safety. Initial work on CH was undertaken in February and April 2006, with the development of a CH information sheet and the distribution of a survey to all Victorian public health services. The information sheet outlined generic concepts related to CH and contained information about dangers associated with ineffective CH, barriers to good CH, and ways to improve CH. The sheet was distributed to all health services, with an invitation to provide ideas back to the VQC about further possible work.8

The survey was also distributed to all health services, which were invited to provide information about individual service problems associated with CH. The results indicated that CHs from shift to shift and from acute care to community care were seen as the most problematic for health service staff, particularly for medical and nursing staff.9

The final initiative before the development of the project was a 1-day workshop held in November 2006. One component of the workshop was group work intended to review the content of a VQC-initiated standardised CH format and associated rules to be used as the basis for CH. The summary of the group work highlighted the importance of organisational support for CH. Workshop participants developed five tools: an organisational readiness assessment chart, a suggested organisational policy, a recom-mended organisational protocol, a CH template containing a minimum dataset to be collected, and a set of key performance indicators (KPIs).

The focus of the VQC’s CH project would be an evaluation of the acceptability and appropriateness of the five tools. The tools were designed to be adapted to the needs of individual health services to help provide safe and effective CH. The purpose of introducing the tools was to help identify organisational responsibilities for CH, to raise awareness of the required organisational commitment to CH, to increase knowledge of personal roles and responsibilities in relation to CH, to help identify organisational structures that would facilitate effective CH, and to develop indicators for measuring organisational performance with regard to CH. Thus, the aims of the project were to gain an understanding of the usefulness of the CH tools, to identify opportunities for improving the tools, to further develop the suggested KPIs via a consultation process, to identify enablers and barriers to implementing the CH tools, and to identify common issues that organisations would need to consider when implementing the CH tools.

Health services were invited to participate in the pilot project through an expression of interest process. Once selected, each site was provided with funding to appoint a project officer to manage the pilot project and to facilitate changes to CH practice during the trial. They were asked to involve senior leadership in the project and select two appropriate areas within the organisation to trial the tools at night medical handover.

The setting for the pilot project was CH to a covering night medical officer. These CHs can be particularly difficult because of limited formal structures in place and the number of different units that may be involved. The project specified inclusion of acute medical CH from day to night shift, CH from day staff to two separate night medical officer positions, and CH of sick patients and patients who were anticipated to require a review in the next shift. (VQC did not intend to alter the current practice of the trial areas in relation to the type of patients normally included in CH.) Staff were required to evaluate the tools in two areas of medical CH at each selected site. The areas selected to participate could be medical or surgical units or a combination of both. However, intensive care and high-dependency units were excluded, as were rural health services, because of the different nature of CH and different staffing requirements in these areas (highly specialised in the former, and potentially lacking night handover in the latter, where cover is often provided by the same doctor over a 24-hour period).

Project officers were asked to collect data throughout the duration of the project. They were also asked to ensure completion of an organisational readiness checklist, to introduce a CH policy, to suggest CH protocol/guidelines, to trial a CH template, to administer questionnaires to medical staff in the participating ward before and after implementation of the project, and to assess the usefulness of draft KPIs.10

The aims of the CH organisational readiness checklist were to raise awareness of the required organisational commitment to the CH process and to identify gaps in CH management at an organisation level. The checklist was used to evaluate the organisation’s CH practice from the perspectives of governance; leadership and culture; competence and education; and information management and reporting.

The project officers were also asked to complete three pre- and post-observation audits of day-to-night CH at different times during the project, and to ensure the completion of questionnaires on the usefulness of the tools by five senior personnel in the organisation.

Through the expression of interest process, three metropolitan and one regional Victorian health service were selected for the project. Each health service chose a medical and a surgical unit. In two services, this included a trauma and emergency unit. All organisational readiness checklists were completed, baseline and post-implementation data were collected, and medical interviews were conducted.

The questionnaires on the usefulness of the organisational checklist, CH policy and suggested KPIs were completed by senior personnel and quality managers not directly involved with the project. The majority found the CH organisational readiness checklist to be moderately to very useful, and 85% of respondents said they would recommend using it in their organisation. The respondents indicated that the checklist was a comprehensive guide for identifying governance requirements for executives and that it assisted them in identifying gaps in the CH process.

The suggested CH policy content outlined the commitment of the organisation to CH and the organisation’s expectations of its staff regarding CH. Eighty-nine per cent of respondents said they would recommend using the suggested policy content in their organisation. Two organisations chose to adapt the proposed policy content, and the other two to adapt the proposed guidelines or protocol/procedure content to guide CH practice in their organisation.

Overall, the participants considered that the CH template containing the minimum dataset included most aspects of CH and was a useful foundation for individual organisations. The template highlighted ongoing management issues for the oncoming medical staff. Staff suggestions were that further information be included in the template, such as the urgency of review and the time to next review.

Analysis of completed CH templates by the project officer at one health service demonstrated that the CH template was not being properly used by the participating medical staff. Just over 30% of the medical staff at that health service did not complete details relating to results pending and relevant examination findings, and 57% did not complete an ongoing management plan. In most cases the CH template sheets were only used for CH for one particular shift.

In relation to the suggested KPIs, the participating sites collected data on the number of medical emergency team (MET) calls during night shift in participating areas and the number of incident reports in participating areas during the pilot period. Sixty-three per cent of respondents found the draft KPIs to be not useful or only a little bit useful. Most sites suggested that the proposed KPIs were time-consuming, difficult to collect and difficult to interpret. However, there was some belief that the KPIs were fairly comprehensive and would give a good overview about whether the CH process was working over a longer period of time.

The number of medical staff who responded to the pre- and post-trial questionnaires varied between health services from four to 13 people. Although the intention was to interview the same staff before and after the trial, it was recognised that this may not be possible, due to medical staff rotation. Hence, staff interviewed before the trial were not always the same staff as those interviewed after the trial. Only three of the four health services completed both the pre- and post-trial questionnaires. One site was unable to complete the post-trial questionnaires due to the prolonged illness of the project officer.

Post-implementation questionnaires indicated a positive response with regard to the usefulness of the tools in two health services and a negative response in one health service (despite a positive response to one question). However, overall feedback from staff was positive, with most noting a significant improvement in CH, allowing them to use their time more efficiently.

Other questions asked of the medical staff related to education about the organisation’s policies and procedures. Before implementation of the project, 33%–50% of respondents felt they had knowledge of policies and procedures, compared with 60%–88% after implementation.

The project officers conducted three observational audits of day-to-night medical CH during the project. Results indicated that the most successful aspects of the CH process were that all required attendees were present at CH and that CH took the form of a face-to-face meeting. In general, CHs were free of interruption and night medical staff had enough opportunity to confirm their understanding of the CH for each patient.

The audits also revealed less successful aspects of the CH process. In none of the health services was the nurse unit manager one of the required attendees at CH. Few staff in each health service knew of an escalation policy for deteriorating patients, and in two health services, only 67% of participating staff felt that the level of urgency of patient review had been clearly communicated. There were varying (low) responses regarding the communication of resuscitation status and regarding the availability of pathology and radiology results. In one health service, it was perceived that a structured response to CH was not required and that only electronic solutions to CH would improve the process.

The health services found the organisational readiness checklist and the suggested content for policy and protocol/guidelines the most useful. The checklist was a good tool for identifying gaps and priority areas for improvement in CH practice. Health service personnel suggested that some additions may improve the efficacy of the checklist. These additions included more details on the educational requirements for CH, more direction on how to incorporate tools into existing systems within the organisation, and more information regarding evaluation methods.

The contents of the policy, protocol/guidelines and CH templates provided a baseline for organisations to adapt and develop the content further to suit their specific needs. Suggestions for refinements included adding a statement about the benefits of good CH, informing staff that CH is a core component of staff responsibility and that there are different levels of complexity at different medical levels.

The suggested KPIs were generally not considered useful because of the short timeframe of the project and the small sample size. MET calls and incident reporting were also seen as not useful in determining the effectiveness of CH. All participants commented that there were too few MET calls to determine any change. They also believed that the number of MET calls does not necessarily directly correlate with the effectiveness and the quality of CH. Furthermore, all sites indicated that staff rarely document or provide details about the quality of CH in the incident report. One health service suggested that the “number of times that a medical staff [member] is called to see a patient [when] they were not expecting the call” might be a more appropriate KPI for CH.

The project highlighted that organisational support and commitment and stakeholder engagement and involvement are essential for the implementation and sustainability of changes to the CH process. Executive commitment and support during the execution and planning phase of the project are required to promote a culture of accountability, change and leadership. Furthermore, the role of medical champions is a vital one for medical acceptance and for bringing about change in CH practice. It is important that the head of each unit monitor the changes and work with junior medical staff to make improvements. However, the project officers indicated that there was a need for organisation-wide support to ensure sustainability.

The project had a number of limitations, including a small sample size, variable interpretation of the questionnaires between sites, and an insufficient timeframe to assess long-term impacts. Only day-to-night CH between medical staff was chosen to pilot the tools. The impact on day medical, nursing or allied health CH was not measured. The medical roster also posed a challenge to the project. The constant rotations of medical staff meant that there was a risk that the CH template would be forwarded on to the next rotation without orientation to the process. Medical rotation also affected data collection, as it was difficult to interview the same medical staff before and after the trial.

Overall, the four participating health services found the VQC pilot project valuable. The CH organisational readiness checklist, as well as suggested content for policy, protocol/guidelines and CH templates, provided health services with a strong basis from which to adapt the tool and implement consistent organisation-wide CH practices. The results from the pilot project can form the basis of further refinements to the CH tools and help determine future steps in CH work, including the development of KPIs. Issues relating to broad communication and sustainability of efforts will need to be clearly articulated and addressed in future efforts around CH.