The new curriculum framework and assessment practices: current challenges for postgraduate years 1 and 2

The launch of a national curriculum framework for junior doctors is a significant and necessary stride in the development of medical education in Australia.1 However, it is not sufficient to create a workable curriculum for junior doctors. The definition of core content alone will only go part of the way to establishing “the curriculum” that will drive learning for our junior doctors. Michael Eraut, Professor of Education at the University of Sussex in the United Kingdom, has identified any curriculum as requiring at least four interacting components, of which content is only one (Box).2 Additionally, a consistent and coherent set of aims and objectives related to that content, a teaching and learning strategy, and an assessment program focused on monitoring the outcomes are required. For example, in a functional medical curriculum (as opposed to a syllabus), the teaching and learning activities support the learners’ transformation of content into useable clinical expertise. The assessment processes underpin and make this personal development accountable. Without all these dynamic attributes of the curriculum, trainees or students would have to invent their own learning activities to comply with the demands that the curriculum makes upon them — they would have to “fill in the gaps” in the curriculum structure. These gaps — the “hidden curriculum” — can have deleterious effects on trainees’ development, especially if their ability to provide the missing components is constrained by the context in which they are working.3 In this article, I discuss the need to underpin the curriculum framework with other educational strategies, mainly focusing on the requirement for trainee-friendly but reliable and valid assessment, and I suggest appropriate tools. I conclude with a brief discussion and challenge about how the assessment continuum should align with the continuum of training.

Trends in medical education, including the new prevocational curriculum, usually arise from complex political, social, scientific and educational interactions. Such developments are rarely completely evidence-based.3-5 Changes to (or creation of) curricula need to fit the context in which they will be used. For a curriculum to be maximally effective, all its components need to be explicitly aligned in the intended direction. These components also need to be monitored to ensure they symbiotically enhance progress towards the program’s goals.

The new prevocational curriculum must operate in a social and clinical context in which relationships between trainees and supervisors can be challenging. For example, in the following interview segment from a study I conducted as part of a Master of Education thesis (Sussex University) in 1989, a new clinical student describes his impressions of the ward-based learning environment:

There are numerous other similar examples in the literature on learning in clinical contexts over a 25-year period.6-8 Although this culture has changed, some values persist. This example may have arisen because of:

• the “person culture”9 of hospital environments allowing senior medical personnel considerable control;

• the widespread use of the Socratic method; and/or

• the perceived need to withhold some medical information from the patient, mainly by the use of technical jargon.

With increasing patient empowerment, this last cause has virtually disappeared, but the culture persists. Moreover, it is a collusive culture. Senior doctors and trainees will collaborate over bureaucratic workplace requirements to concentrate on the needs of the patient, and on their own. So a curriculum for the workplace needs to recognise this. All elements of the model in the Box will need careful deliberation, discussion, consultation and implementation to preserve the integrity of the curriculum without antagonising or being weakened by the clinical context.

For example, a salient phenomenon in the clinical learning environment when choosing educational and assessment strategies is the relatively poor relationship between volume of clinical experience and performance or competence.10,11 With more experienced trainees, there is a larger correlation between the volume of experience on procedures generally and their day-to-day competence with specific procedures.12 Nevertheless, even for a group of surgical trainees, the best predictor of competence on a particular procedure is the volume of experience on that procedure, and this is significantly larger than the correlations between general surgical experience, time expired or other indices of experience, and competence.12,13

This finding has several implications:

• hours of experience or numbers of clinical encounters are not the best measures for assessment;

• examination-style assessment may not reflect real differences between levels of clinical experience and competence obtained by trainees or students; and

• clinical skills are not necessarily generic — the best way to get good at a clinical skill or procedure is to practise that particular skill.

Trainees trying to respond to the curriculum framework will vary enormously in the amount of clinical experience they obtain. This is true across the undergraduate/postgraduate divide, and both within and between attachments and trainees.10,12 It also seems that clinical and educational supervisors consistently overestimate how much feedback they give,14-16 and cannot accurately predict how much experience, guidance and feedback trainees can obtain.

Because of these issues, curricula require infrastructure. However, previous analyses of the Australian vocational training context have suggested that important elements of infrastructure that need to be in place to deliver a vibrant curriculum may be missing. Essentially, there is:

• little cohesion across health and education sections, and national and state jurisdictions inhibit effective national policies in medical education and training;

• lack of coordination in Australia for medical education and training, and programs are under-resourced and under pressure;

• need for a coordinated national approach to assessment;

• need for improved links between prevocational and vocational training; and

• need for workforce and training agencies to work collaboratively.16,17

Formal supervision arrangements, the training of supervisors, and the definition of and reward for their roles are all underdeveloped in Australia. Part of the solution lies in making the work of junior staff the focus of assessment, and integrating assessment methods into the training environment in a seamless fashion.

There has been increasing interest in the development of work-based learning and assessment activities in the medical profession over the past two decades.18,19 Innovations have occurred across a wide front, from new methods such as the miniCEX (a clinical evaluation exercise)20 to enhancements of older techniques such as “chart-stimulated recall”,21 used to investigate the practice of poorly performing doctors.22 In the UK, a comprehensive approach to assessment of the new Foundation years has been attempted,23 but such an approach in Australia will not be sustainable until both infrastructure is improved and there is general acceptance of and development around the notion of a training continuum from undergraduate to specialty levels.16 Any system will require extensive planning and discussion with stakeholders. Nevertheless, it is likely that effective assessment at prevocational level (either formative or summative) will include one or more of the validated and now increasingly accepted methods described below. The four methods share the following features:

• clinical work-based stimuli with which the trainee interacts — a patient, case records, or direct involvement in clinical work with the assessor;

• direct observation and rating of performance by the people most involved — peers, supervisors, or sometimes patients; and

• judgements captured on a rating form, across a number of different dimensions of the interaction — such as ethics, management, history, examination, and clinical decisions.

The separate jurisdictions in Australia have contributed to the fragmentation of medical training and the compartmentalisation of its regulation. Recently, a university consortium made overtures, subsequently muted, about the consortium becoming engaged in specialty training.35 This was met with some concern by Colleges. For example, a report of the Royal Australasian College of Surgeons Council stated:

However, there are good arguments for involving the nation’s clinical academics in a phase of training that many see as the natural target of most of those who enter medical school. For example, it would help to make the continuum of training a reality. It also makes little sense, from a regulatory perspective, for the continuum of assessment to be apparently suspended during internship and again at specialty accreditation. This is particularly so when Australia has recently experienced challenging events, concerning individual and collective responsibility for poor clinical performance, without the concepts of revalidation or relicensing even being raised in the public and professional consciousness.37 Work-based and 360-degree assessment might have avoided some of these problems. While the rest of the world grapples with these major issues,38 Australia is largely silent on patient involvement in assessment of doctors, and on revalidation. To draw a thoroughly Australian metaphor, it is helpful to view such assessment as professional back-burning — a preventive measure that comes with initial discomfort, but might avert much bigger disasters. You can never tell when the next bushfire is going to threaten communities, but you can do everything possible to prevent them happening.

Eraut’s model of the essential elements of a curriculum2