Peer review of cancer multidisciplinary teams: is it acceptable in Australia?

Multidisciplinary care is a key enabler of high-quality treatment,1 and is endorsed in Australia and overseas as best practice in cancer care.2-4 An essential component of multidisciplinary care is a multidisciplinary team (MDT) that holds regular meetings where doctors from relevant disciplines, nurses, and allied health professionals meet formally to present new and ongoing patient cases. At a multidisciplinary meeting (MDM), MDT members discuss results of investigations, establish diagnosis and staging, and decide on the most appropriate treatment plan for each patient.

Guidelines recommending structure and processes for both MDTs and MDMs have been adopted by Cancer Australia and most state health departments in Australia.5,6 While quality assurance processes have been recommended to optimise multidisciplinary care,5,7 no robust and ongoing mechanisms have been put in place for accreditation, monitoring of adherence to National Safety and Quality Health Service Standards, or quality improvement (QI).5,8,9

In Australia, little has been done to establish the effectiveness of individual MDTs or to assess whether they are functioning in accordance with best-practice guidelines. Previous research found that many institutions that provide cancer care did not have MDTs,10 and that numerous MDTs were functioning outside of best-practice guidelines.10,11 A recent study highlighted the variability in the operation, participation, quality, and support that exists between Australian MDMs.12 The authors suggested that variability could be reduced by improved MDM funding, the establishment of standard definitions and the implementation of a method to evaluate MDMs against established criteria.

Over the past 15 years, the United Kingdom National Health Service established a National Cancer Peer Review (NCPR) program for cancer services.13 The NCPR assesses compliance with a set of nationally agreed “quality measures” involving both self-assessment by cancer teams and external reviews conducted by professional peers. This has led to improvements in MDM structure and processes, improved outcomes for patients, and has influenced system-wide changes, such as restricting certain types of surgery to larger centres.14

We developed a peer-review framework based on the NCPR model (Appendix 1). The framework was pilot tested with three mature MDTs from different tumour streams in metropolitan Western Australia. The peer-review process and pilot study are described in Appendix 2. MDTs were peer reviewed using the framework and provided with a report outlining recommendations for QI; the report template is shown in Appendix 3. The three MDT reviews were accomplished in one 4-day period. We then conducted a qualitative study to explore health professionals' (peer reviewers and MDT members) perceptions of the feasibility and acceptability of the peer-review process, which we report in this article.

Methods

Semi-structured interviews were conducted between December 2012 and July 2013 with peer reviewers (reviewers) and consenting MDT members. Respondents' perceptions were sought regarding the appropriateness of the information requested from the MDT; expectations and experiences of the process; perceived impact of the peer-review process; views about future implementation of the process, and suggestions for future improvement. Interviews were conducted by telephone or face-to-face, and were audio-recorded and transcribed verbatim.

This study was approved by the ethics committees of the three hospitals involved in the project and by the University of Western Australia Human Research Ethics Committee.

Analysis

Analysis of the interviews was undertaken using the framework approach15 and thematic analysis.16 All data were processed against the framework using NVivo 10 (QSR International). A detailed description of the methods is provided in Appendix 2.

Results

Seventeen MDT members (10 doctors, five nurses and two allied health professionals) and all five reviewers were interviewed. Data were grouped into four a priori themes: the peer-review process; the report; reviewers' recommendations; and future directions for the peer-review process. Each theme explored reviewers' and MDT members' perceptions and responses to the different aspects of the peer-review process and ideas for any future implementation. Textual examples informing the four themes are provided in Appendix 4.

The peer-review process

Overall, the peer-review process was well accepted by reviewers and MDT members, who considered the process to be reasonable and constructive, encouraging MDTs to reflect on their current practices. Most MDT members commented that the information requested from the MDTs was appropriate and provided reviewers with a good understanding of how their service was performing. Several MDT members, however, reported that many processes occurring outside MDMs (communication with general practitioners; referrals to allied health professionals; informal communication between all involved in patient care) were not adequately appraised. Despite efforts to advise MDT members about the peer-review process, several felt uninformed. Reviewers felt well equipped to assess the MDTs and found the supporting documentation appropriate, but believed that they were not always provided with adequate information from the MDTs.

The reviewers and the MDTs regarded the observation of a MDM as an important part of the peer-review process. Some participants believed that the one MDM observed was not an adequate representation of their team work (Appendix 4, Quote 1) and one MDT requested review of an additional recording. Several participants suggested that the MDT questionnaire be completed by all members of the MDT (not only by the clinical lead), which would give a more accurate picture of the MDT.

Participation in an external review was seen by MDT members as motivation to reflect and reconsider every aspect of their MDT, in both formal and informal meetings. The peer-review process was viewed as an opportunity to improve services by implementing some of the suggested changes or through exploring alternative approaches (Appendix 4, Quote 2). A minority of participants considered the process to be time-consuming, merely a bureaucratic exercise, and did not foresee any changes resulting from the practice. Reviewers believed that the peer-review process would benefit the teams, providing a learning and professional development opportunity; however, reviewers expressed concern that critical feedback would not always be well received.

MDT members regarded the difference in care provision between the public and private sectors to be important, and several suggested that this should have been considered more when assessing the MDTs (Appendix 4, Quote 3).

The choice of reviewers was regarded as critical by the participants. Recruiting well recognised cancer specialists as reviewers was considered essential to the credibility of the peer-review process. Most participants reported that the tumour-specific expertise of the reviewers was not a concern, providing they had substantial knowledge and experience in cancer care (Appendix 4, Quotes 4 and 5).

The peer-review report

A draft report was released to the MDTs for comment before the final report to ensure that it represented an accurate evaluation of the service (Appendix 3). Several changes were made to the report's format in response to the feedback from the teams. For example, the ranking of the teams as “well developed”, “developing” or “underdeveloped” for each of the core domains was removed in the final report. As a subjective measure, MDT members and reviewers did not feel that it added value, and thought it detracted from positive engagement with MDTs in the peer-review process.

Most MDT members were accepting of the report. With few exceptions, interviewees found the report to be accurate and diplomatic. While the reviewers felt that they had a good understanding of the MDT and had made relevant recommendations, a minority of MDT members stated that the MDTs' maturity and the role of some members were underestimated (Appendix 4, Quote 6 and 7). Some interviewees reported that, at seven pages, the report was too long and favoured a shorter version with quantifiable outcomes.

Recommendations arising from the peer-review process

Responses to the recommendations in the report were mixed. Most participants thought that the recommendations were beneficial, adequate and appropriate. The majority considered some recommendations to be more helpful than others, but a minority viewed them to be of limited value. It was evident that several suggestions had already been considered by the teams. In a number of instances, several members of one MDT perceived a recommendation to be particularly relevant, while other members viewed it as impractical or of limited value. For example, participants had diverse opinions about recommendations to develop governance documents specifying the structure and function of the MDTs.

Some of the general recommendations made were considered impractical in the specific MDT setting, for example the option of claiming Medicare reimbursement in public hospitals (Appendix 4, Quote 8).

Many participants were unsure about the extent to which recommendations would be implemented. Issues such as lack of resources, lack of time, competing priorities within the MDT, and ongoing hospital infrastructure changes were cited as barriers to instigating change. Several reviewers and MDT members suggested that teams should be kept accountable for the implementation of the recommendations through a follow-up review (Appendix 4, Quote 9).

Future of the peer-review process

Opinions about the purpose and future of the peer-review process were diverse. Most interviewees perceived that peer review of MDTs could be implemented in their hospital, but felt it would be burdensome in the current busy health system. Hence, implementation would require additional resources and time.

Participants expressed divergent views about the format in which the process could be introduced. Many believed that the peer-review process had a role for QI and education or should be required as accreditation for cancer services (Appendix 4, Quotes 10 to 14). Participants recommended that any accreditation should have well developed standards against which services could be benchmarked, but that standards should be appropriate for different settings. For example, private and public sector MDMs function differently, and rural and metropolitan MDMs have access to different levels of specialisation. One participant argued that there is no clear evidence about what constitutes best practice for MDTs (Appendix 4, Quote 15).

Outcome measures

This study highlighted the challenges of introducing a procedure which aimed to objectively measure a practice that is a complex and subjective process (Appendix 4, Quote 16). Hence, we sought participants' opinions about appropriate measures to objectively evaluate MDT functioning (Box).

Discussion

This study is the first reported description and evaluation of peer review of cancer teams in Australia. We piloted the peer-review process with MDTs, which differed in terms of tumour stream, health sector, team structure, meeting organisation and dynamics. This allowed an evaluation of the peer-review process under variable conditions. We found that the peer-review process was possible to undertake and was acceptable to MDTs and reviewers, and that the process provided an appropriate assessment of the services.

The extent to which the participants valued the feedback and recommendations varied, and participants were unsure whether their MDT would adopt recommendations. Concerns expressed by participants highlight a need for systematic QI processes such as the peer-review process to be endorsed at a jurisdictional and executive level, and for there to be a commitment to provide teams with the appropriate resources and support necessary to conduct the reviews and implement relevant recommendations. If peer review is adopted, a process for monitoring the implementation of recommendations needs to be considered, otherwise uptake may be limited.

The potential benefits of the peer-review process as a QI activity were recognised in the study, but views about the use of the peer-review process for accreditation were mixed. While some considered the use of the peer-review process as accreditation overly bureaucratic, others were positive, citing the need to ensure that the quality of MDTs is consistently high. Participants identified the need for agreed standards in MDT performance. Although our framework was based on the standards recommended by the NCPR program and the National Breast Cancer Centre MDT guidelines, some of our criteria were considered idealistic or inappropriate, highlighting the need to develop guidelines and criteria specific to the setting and cancer stream. If a peer-review process is adopted for accreditation, the standard at which a service is considered safe (and therefore accreditable) needs to be identified and measured.

Our study raises the question of whether it is more effective to measure outcomes rather than processes, as were primarily assessed in our framework. Arguably, improved outcomes provide evidence that changes in processes have had the desired effect.17-19 Benchmarking using clinical data to assess and compare the providers' performance has been identified as the most effective method to drive improvements in quality and patient outcomes20 and may be a more acceptable and cost-effective way to promote QI in this setting. However, any such system should ideally adopt a combination of MDT-relevant process and outcome measures21 and incorporate the National Safety and Quality Health Service Standards.9

The peer-review process was logistically challenging. Both MDT members and reviewers found the process time-consuming. It was difficult to organise all components of the peer-review process around the clinical commitments of MDT members and reviewers. If adopted, the peer-review process would require substantial resources — to cover the clinical load of reviewers as well as appropriate travel and administrative support — in addition to the considerable cost of establishing and running MDMs.22 In view of these concerns, an alternative may include routine collection of minimal process and outcome measures and feedback to services. In this way, national standards could be identified and targets set for MDT improvement. The challenges, however, are to develop measures that are relevant to clinical practice and quality of life, which can be assessed in a standardised way, and to support the collection and synthesis of appropriate and credible data.

The current peer-review process was driven by the research team and relied on the goodwill of the participating teams. If routine peer reviews are implemented, MDTs would benefit from clearly defined training regarding the peer-review process, including what would be considered credible data for the peer-review process and the best way of presenting these data.

This pilot study is limited to qualitative data, and does not quantify our findings. Our sample size of 22 would be considered small for quantitative analysis, but not for exploratory, qualitative research. This is particularly so where in-depth information was sought from well informed people from different disciplines, providing diverse perspectives regarding a phenomenon about which little is known — as was the case here.23,24 Another potential source of bias was that the research team who developed the peer-review process also undertook the evaluation. However, reviewers with whom the research team interacted most closely were interviewed by an independent researcher. Data checks were also conducted with several MDT members and reviewers to verify our findings. The generalisability of the study may be limited due to the small number of MDTs that were reviewed in only one state and because the majority of MDT participants (10/17) were doctors. MDT meetings are, however, largely dominated by doctors and the response from other professions may reflect this.

This study offers a potential peer-review model, determines essential features required for the successful implementation of the process, and suggests it is feasible in Australia. The peer-review framework, however, requires extensive review and evaluation with MDTs from different states and tumour types.