Implementing a research governance framework for  clinical and public health research

Poustie, Stephanie J; Taylor, David McD; Forbes, Andrew B; Skiba, Marina A; Nelson, Mark R; McNeil, John J

doi:10.5694/j.1326-5377.2006.tb00732.x

ARTICLE
AUTHORS
REFERENCES

Topics

Health services administration

Abstract

Research conduct in Australia and worldwide is mostly unaudited.
The purpose of good research governance is to ensure integrity in research through accountability, transparency and responsibility.
Institutional responsibility for research governance has been adopted by Monash University’s Department of Epidemiology and Preventive Medicine, providing clear lines of accountability for researchers as well as support and guidance.
A research audit tool has been developed, identifying areas where practice could be improved especially among less experienced researchers; the most common adverse findings concerned research protocols and procedure manuals.
The need for participant confidentiality, privacy and data security was found to be understood, and adhered to widely by all researchers.
An evaluation of the effect of audit on researchers found that the process was well accepted.

The scientific community is dependent upon public goodwill and trust, and recent publications,1-3 including the disclosure of the scientific fraud of the South Korean stem cell researcher Hwang, can only erode the public’s confidence.2

While episodes of identifiable research misconduct or fraud are rare, breaches in research integrity are more common.1,4,5 These breaches can include non-adherence to an approved protocol, inappropriate authorship, failure to publish key findings, and changing, introducing or omitting a primary outcome.5,6 Research conduct remains largely unexamined and, although mechanisms for auditing exist in Australia,1,7-9 this rarely occurs.

Even if research has received ethical approval from a human research ethics committee (HREC), there is no guarantee that ethical research conduct naturally follows. HRECs are poorly equipped to exert governance beyond their primary role of the ethical review of research proposals. They may also lack the transparency required for good governance.10,11 It is our belief that it is the structures and processes used to govern the conduct of research that determine how effectively participants are protected. Better governance with improved audit may detect breaches and help prevent research misconduct.

The purpose of research governance is to ensure research integrity through accountability, transparency and responsibility.12,13 Good governance also seeks to ensure that research is carried out with the highest scientific and ethical standards, appropriate use of finances, and robust monitoring, review and evaluation processes.12

Research governance frameworks

Frameworks for research governance in the United Kingdom, the United States and Canada are well established.14-17 The UK Department of Health introduced the Research governance framework for health and social care in 2001, and revised it in 2005.16 The US Department of Health and Human Services has an Office for Human Research Protections (http://www.hhs.gov/ohrp/), which has a Code of Federal Regulations (Common Rule) that is legally binding.14 In Canada, research is guided by the Tri-Council policy statement Ethical conduct for research involving humans 1998 (with amendments in 2000, 2002 and 2005).17,18

In Australia, there has been a call for greater emphasis on an institutional model of research governance,10 in the absence of a national research governance framework. Much of the literature discusses the cornerstones of governance (transparency, accountability and responsibility) and their adoption into various governance frameworks.11-13 However, there is little in the literature describing the operational tools for these frameworks, nor do the frameworks include active audit of research conduct. Indeed, the US, followed by the UK, have increasingly adopted a more regulated legislative approach to research conduct.14,16

We describe here the framework for research governance of the Department of Epidemiology and Preventive Medicine (DEPM) at Monash University, its operational methods and tools, and an evaluation of those tools. The operational features of good governance have been described as including appropriate oversight, sound policies and guidelines, effective implementation of those policies and guidelines, and continuous evaluation and feedback.15 While these features are reflected in the DEPM’s research governance framework, we placed greater emphasis on audit of research conduct rather than the somewhat vague notion of “appropriate oversight”.

The DEPM research governance framework

The Head of Department at the DEPM used a risk management approach to ensuring research is carried out to the highest scientific and ethical standards. A research governance framework was established, based on seven guiding principles of good governance adapted from the OECD (Organisation for Economic Co-operation and Development) Principles of corporate governance 19 (Box 1).

We implemented a model of institutional responsibility, moving away from self-regulation and reliance on the HREC approval process to determine research conduct.1 We also aimed to create a culture of conscience and responsibility, measuring performance (what we actually do) rather than compliance (what we are supposed to do).1 Our organisational structure for research governance, reflecting clear lines of accountability, is given in Box 2.

Operational methods

Development of a research audit tool

Two operational methods have been employed. The first was the development of a research audit tool that can be applied, in part or full, to any research study or data registry. The tool assesses compliance with the standards required by the regulatory authorities,7-9 compliance with privacy legislation, and performance against a higher standard of conduct reflected in the DEPM’s A guide to good research practice.21 The tool relies heavily on the International Conference on Harmonisation/Guideline for Good Clinical Practice (ICH/GCP)8 that provides the only existing standard for quality improvement in research conduct.

The audit tool initially employed a tick-box mechanism. However, after auditing six projects, it was redesigned to give a more accurate delineation of findings, problems and required actions. It now allows for quicker audit, is easier and simpler to read and, importantly, enables the auditor to commend researchers for good research conduct, reinforcing the required standard. We also attempted to highlight any gaps or deficiencies in departmental support, particularly for students, and to assess researchers’ understanding and practice of data security and privacy. In these activities, we differed from the usual “monitoring” that occurs during a pharmaceutical industry-sponsored study.

While our audit tool was initially designed to evaluate specific components of research conduct, it will evolve to incorporate other components of good governance, such as financial responsibility, insurance, intellectual property protection, and complaints handling.

The areas audited were the study protocol, the participant information and consent form, the HREC approval, other relevant study documentation, data management and data confidentiality procedures, and overall study management. We promoted the actual audit process as educative for the research staff and as a “work in progress”. A study is not “signed off” until all actions required are completed, and staff are provided with whatever assistance or guidance they require.

Studies audited to date have included randomised controlled trials, observational studies, prospective epidemiological studies, registries, industry-sponsored trials and PhD projects.

A total of 18 studies were randomly selected for audit by the Committee during the period 2002–2005. Eighteen other studies were not audited for a variety of reasons. These included studies that were “low risk” and heavily governed by other authorities, or that it was too early in the life of the study for audit to occur and the project was re-visited later.

Effect of the audit process

The second operational method or tool involved evaluation of the effect of the audit process on researchers. The Good Clinical Research Practice Committee considered carefully the role of the Research Governance Coordinator as the auditor and potential conflicts associated with audit, and sought to assess the effect of the audit process on research staff.

During 2002–2004, 13 studies involving a total of 24 staff were audited.

Approval from Monash University’s Standing Committee on Ethics in Research Involving Humans was obtained to distribute an anonymous, voluntary questionnaire to researchers associated with the 13 studies. The questionnaire sought to determine:

how important or relevant audit was to researchers;
whether they learnt from the process;
whether they considered the audit to be thorough; and, for those staff who were not principal investigators,
whether they felt they had supervisor support during the audit process.

The self-administered questionnaire, which consisted of seven questions for all staff and two additional questions for supervised research staff, was returned by internal mail or by a self-addressed envelope. The Likert scale 23 was employed to measure responses to a range of statements. Questionnaires were distributed in three batches: November 2003, and January and March 2004.

Findings of research audits

The most common adverse findings of the audits concerned research protocols and procedure manuals. While the minimum standards according to ICH/GCP were generally met, most were missing one or more requirements; for example, dates signatures, version numbers, flow charts, timelines, and descriptions of staff roles. Protocols, protocol amendments and approvals were sometimes not located in hard copy in the study document file or with the correct (approved) version number and date. Where deficiencies were identified, the auditor worked with the research staff to meet the expected standards.

Differing interpretations of signing a consent form were also a common finding, with multiple version numbers and dates causing the most confusion. Just over half the audited studies met the DEPM standard. HREC approval was problematic if approval time was short (2 years) for projects funded for 3 or more years. Both the approving HREC and study personnel often missed the requirements for annual reports.

The need for research participant confidentiality, privacy and data security was well understood and adhered to widely by all researchers and research staff. However, the translation of this understanding into practice varied among different groups. The signing of confidentiality statements was monitored for compliance and, while compliance was high and staff understood the implications of the need for confidentiality in their work, many could not remember signing the document.

Evaluation of the effect of audit on research staff

Twenty-four staff were surveyed and 17 (63%) completed questionnaires were returned. Only five of 11 principal investigators surveyed responded. The poor response from principal investigators reflected that five of the 11 had left the DEPM and were unaware of the DEPM’s research governance activities, and poss-ibly considered their response irrelevant as they were not directly involved in the audit process. However, five of six principal investigators located within the DEPM did respond. Of the other research staff (postdoctoral staff, nurses, research assistants) 12 of 13 responded. Overall:

14/17 agreed or strongly agreed that audit is an important part of the research process;
13/17 agreed or strongly agreed that the process of audit was helpful to their understanding of good clinical research practice;
12/17 agreed or strongly agreed that audit was helpful to the general conduct of the study;
11/17 disagreed or strongly disagreed that they felt uncomfortable when audited;
15/17 agreed or strongly agreed that the audit was thorough; and
16/17 agreed or strongly agreed that the issues raised in the report of the audited research study were appropriate.
10/12 other research staff agreed or strongly agreed that they had the support of the principal investigators during the audit process.

Although there was a difference in response rates between principal investigators and other research staff, responses were similar across both groups.

Discussion

Although largely untested and based on consensus,24 the ICH/GCP is the most familiar standard for research practice worldwide. It was written for the pharmaceutical industry, but is referred to in the National Health and Medical Research Council’s National statement on ethical conduct in research involving humans,9 and the Therapeutic Goods Association bases its guidelines for therapeutic research on the ICH/GCP.7 Our literature search found only one example of the ICH/GCP being used systematically to audit research conduct.25 However, while it is viewed as the minimum standard, it remains the only guideline for auditing research conduct. Our audit found that the DEPM research staff met the ICH/GCP standard, with any deficiencies generally relating to the higher standard reflected in our guidelines.21 In addition to this, many of the components of ICH/GCP are not well suited to public health research and we modified our audit tool accordingly.

Our evaluation of the effect of audit on research staff indicated that the audit component of risk management planning was well accepted, non-threatening and that most staff were comfortable with the process. Further, we were satisfied that research staff felt that issues raised during audit were appropriate and that principal investigators strongly supported the audit process involving their staff.

Overall, we view audit not only as an essential tool of our research governance framework, but also as an important part of the research process itself. We found strong support for this, with audit assisting staff in the required conduct of their research.

The DEPM accepts the model of institutional responsibility for research governance in an environment in which there are calls for greater research governance, with some published frameworks available, but little, if any, measurement of implementation. Audit is a necessary adjunct to research activity and an important operational tool of the DEPM research governance framework. Audit aims to be educative and supportive, rather than policing or raising issues of conflict for researchers. It is well accepted by staff and is seen as a necessary part of the research process. The other operational tools of our framework, the DEPM’s A guide to good research practice and ongoing education in research conduct appear to be well accepted, while the final tool, risk analysis of research projects, is yet to be implemented. Our framework for good research governance, including its operational tools, is evolving.

1 Guiding principles and the research governance framework of the Department of Epidemiology and Preventive Medicine (DEPM)

1. Clearly defined accountability and responsibility

The DEPM (specifically the Head of Department) is ultimately accountable for the conduct of research in the Department
Management arrangements, including reporting lines, have been established to support the accountability of the Head of Department
The Monash University Standing Committee on Ethics in Research Involving Humans is part of these arrangements and responsible for ensuring research proposals are ethically sound
Research investigators are held accountable for their actions

2. Participation

A Good Clinical Research Practice Committee governs all research associated with the DEPM
The Committee comprises six staff members who may be experienced clinician researchers, research managers, PhD supervisors, or senior lecturers. Most have two or more of these roles in the Department
The Committee meets fortnightly and also reports to the Head of Department
The Committee appointed a Research Governance Coordinator in 2002 whose central role is research audit, review and evaluation. This was an external appointment on a part-time basis. She is an experienced researcher who serves on Monash University’s Standing Committee on Ethics in Research Involving Humans

3. Transparency

This is maintained through the DEPM’s reporting arrangements:

The DEPM’s Annual report21
The Good Clinical Research Practice Committee provides 3-monthly reports to the Head of Department, and 3-monthly newsletters to all DEPM staff on matters of research governance
Any conflict of interest of Committee members is declared

4. Technical and managerial competence in research conduct

Only suitably qualified staff are employed
Handbooks and training are provided for staff
All staff sign privacy statements ensuring the confidentiality of research participants and meet with the Research Governance Coordinator
Short courses in research process are provided for staff and the wider research community. Courses are compulsory for all PhD candidates

5. Organisational capacity

The Research Governance Coordinator’s role is to audit, report on and improve the standard of clinical and public health research undertaken by researchers within the DEPM
The Coordinator’s role provides a resource for the DEPM, informing research conduct and policy development
The Coordinator meets with every new staff member as part of the induction process and outlines her role

6. Compliance

Publication of A guide to good research practice22 establishes standard procedures for conducting DEPM research, and staff are expected to meet the standards it outlines
Guidelines are reviewed and revised constantly
Compliance with the AS/NZS Risk Management Standards23 is part of this process. An overall risk management strategy is included in the DEPM’s A guide to good research practice22
Risk analysis will be applied to research projects in 2006

7. Audit, review and evaluation

This is a core operational component of the DEPM framework and the central role of the Research Governance Coordinator
All researchers at DEPM are audited; this is well publicised among staff
An audit template has been developed
In 2005, PhD candidates were targeted, as individual scientists have been described as the most at risk in the literature2
There is continuous evaluation of the effect of the audit process on staff

2 Organisational structure for research governance at Monash University Department of Epidemiology and Preventive Medicine

Received 27 April 2006, accepted 20 August 2006

View this article on Wiley Online Library

Stephanie J Poustie¹
David McD Taylor²
Andrew B Forbes¹
Marina A Skiba¹
Mark R Nelson³
John J McNeil¹

1 Department of Epidemiology and Preventive Medicine, Monash University, Melbourne, VIC.
2 Austin Health, Melbourne, VIC.
3 Discipline of General Practice, School of Medicine, University of Tasmania, Hobart, TAS.

Correspondence: stephanie.poustie@bigpond.com

Competing interests:

John McNeil is currently Chair of the HREC at the Alfred Hospital, Melbourne.

1. Pickworth E. Should local research ethics committees monitor research they have approved? J Med Ethics 2000; 26: 330-333.
2. Van Der Weyden MB. Managing allegations of scientific misconduct and fraud: lessons from the “Hall affair” [editorial]. Med J Aust 2004; 180: 149-151. <MJA full text>
3. Gerber P. What can we learn from the Hwang and Sudbø affairs? [editorial] Med J Aust 2006: 184: 632-635. <MJA full text>
4. Caan W. Research bureaucracy in the United Kingdom. BMJ 2004; 329: 623-624.
5. Gøtzsche PC . Research integrity and pharmaceutical sponsorship [editorial]. Med J Aust 2005; 182: 549-550. <MJA full text>
6. Geggie D. A survey of newly appointed consultants’ attitudes towards research fraud. J Med Ethics 2001; 27: 344-346.
7. Henry D, Kerridge IH, Hill SR, et al. Medical specialists and pharmaceutical industry-sponsored research: a survey of the Australian experience. Med J Aust 2005; 182: 557-560. <MJA full text>
8. Therapeutics Goods Administration. Note for guidance on good clinical practice (CPMP/ICH/135/95). July 2000. http://www.tga.gov.au/docs/pdf/euguide/ich/ich13595.pdf (accessed Dec 2005).
9. International Conference on Harmonisation Steering Committee. ICH harmonised tripartite guideline. Guideline for good clinical practice. 1996. http://www.cosa.org.au/documents/ICH_GCP_%20guidelines.pdf (accessed Dec 2005).
10. National Health and Medical Research Council. National statement on ethical conduct in research involving humans. Canberra: NHMRC, 1999.
11. Walsh MK, McNeil JJ, Breen KJ. Improving the governance of health research. Med J Aust 2005; 182: 468-471. <MJA full text>
12. Wilson M. Getting a fix on good governance. J Med Ethics 2004; 30: 232.
13. Sen B. Research governance: implications for health library and information professionals. Health Info Libr J 2003; 20: 3-14.
14. Taylor M. Research governance. Health Soc Care Community 2002; 10: 6-9.
15. United States Department of Health and Human Services. Code of Federal Regulations. Part 46 — Protection of human subjects. Washington: USDHHS, 2005. http://www.hhs.gov/ohrp/humansubjects/guidance/45cfr46.htm (accessed Aug 2006).
16. Ferris LE, Singer PA, Naylor CD. The Olivieri Symposium. Better governance in academic health science centres: moving beyond the Olivieri/Apotex Affair in Toronto. J Med Ethics 2004; 30: 25-29.
17. UK Department of Health. Research governance framework for health and social care. 2nd ed. London: Department of Health, 2005. http://www.dh.gov.uk/assetRoot/04/12/24/27/04122427.pdf (accessed Dec 2005).
18. Canadian Institutes of Health/Natural Sciences and Engineering Research Council of Canada/Social Sciences and Humanities Research Council of Canada. Tri-Council Policy Statement. Ethical conduct for research involving humans. Ottawa: Interagency Secretariat on Research Ethics, 2005. http://www.pre.ethics.gc.ca/english/pdf/TCPS%20October%202005_E.pdf (accessed Nov 2006).
19. Interagency Advisory Panel on Research Ethics, Canada. Process and principles for developing a Canadian governance system for the ethical conduct of research involving humans. Ottawa: Government of Canada, 2002. http://www.pre.ethics.gc.ca/english/policyinitiatives/governance01.cfm (accessed Aug 2006).
20. OECD Principles of corporate governance. Paris: OECD, 2004. http://www.oecd.org/dataoecd/32/18/31557724.pdf (accessed Dec 2005).
21. Monash University Department of Epidemiology and Preventive Medicine. Annual reports. Melbourne: Monash University. http://www.med.monash.edu.au/epidemiology/publications/index.html#brochures (accessed Dec 2005).
22. Monash University Department of Epidemiology and Preventive Medicine. A guide to good research practice. Melbourne: Monash University, 2003. http://www.med.monash.edu.au/epidemiology/publications/grpg2003.pdf (accessed Dec 2005).
23. Risk management. AS/NZS 4360:2004. 3rd ed. Sydney and Wellington: Standards Australia and Standards New Zealand, 2004.
24. Alreck P, Settle R. The survey research handbook. Guidelines and strategies for conducting a survey. 2nd ed. Chicago: Irwin, 1995.
25. Grimes DA, Hubacher D, Nanda K, et al. The good clinical practice guideline: a bronze standard for clinical research. Lancet 2005; 366: 172-174.
26. Sather MR, Raisch DW, Haakenson CM, et al. Promoting good clinical practices in the conduct of clinical trials: experiences in the Department of Veterans Affairs Cooperative Studies Program. Control Clin Trials 2003; 24: 570-584.

Online responses are no longer available. Please refer to our instructions for authors page for more information.

Implementing a research governance framework for clinical and public health research

Topics

Abstract

Author

Comment

Do you have any competing interests to declare? *