The quantity and quality of teaching and learning methods experienced by prevocational doctors has received less attention than those experienced by medical students or doctors pursuing vocational training.
In the United Kingdom in the early 1990s, senior house officers (the equivalent of prevocational years two and three in Australia) were described as a “lost tribe”.1 There was a perceived lack of structure for their education and guidance. Prevocational positions were characterised by poor job structure, poorly planned training, heavy workloads, inadequate supervision, minimal assessment and a lack of career advice. Appointment procedures were frequently not based on competencies.2-4 In response to these concerns, in August 2004, a 2-year Foundation Programme was introduced in the UK. This is based on a national curriculum and includes regular assessment of clinical competencies.5
At present, there is no curriculum for prevocational trainees in Australian hospitals, and the problems identified in the UK may apply in Australian hospitals. Recently announced increases in medical school places6 will lead to increased numbers of prevocational positions in Australia, and it is timely to investigate the quantity and quality of teaching and learning methods provided to Australian trainees. As prevocational doctors are adult learners, asking them about their learning needs and the training they would like is likely to provide information to assist with planning future training.7
identify perceptions of prevocational hospital doctors of their preparedness for day-to-day tasks;
assess prevocational doctors’ perceptions of the quantity, quality and usefulness of current teaching and learning methods;
determine desired future exposure to learning and teaching methods; and
identify subgroups of prevocational doctors with specific needs, including international medical graduates (IMGs) and those working in provincial hospitals or specialty rotations.
Prevocational doctors were defined as those in postgraduate years one (PGY1) and two (PGY2) and medical staff in subsequent years who were not enrolled in a specialty training program (PGY3+).
A questionnaire was developed using iterative feedback from a group of prevocational doctors. Demographics requested were sex, age, university of graduation, type of medical course (undergraduate or graduate entry), year of graduation, and hospital employment history. Respondents were asked to rate their general preparedness (using Likert-type scales) for 13 specific tasks required in their current role; to list any tasks or rotations for which they felt inadequately prepared; and to respond to ordinal multicategory scales on 14 educational methods they may have experienced, how useful they perceived each method, and whether they would like more or less of each one. The survey concluded by asking respondents to identify barriers to their continuing medical education (CME) and any rotations where they perceived CME to be inadequate, and to make free text general comments on their prevocational education.
The questionnaire was piloted from August 2003 to October 2003 at two tertiary care hospitals in Melbourne, and then distributed to 19 other Victorian hospitals between 23 September 2003 and 6 January 2004. Hospitals were classified A1 (tertiary), metropolitan or provincial. To sample other Australian states and territories, we asked Postgraduate Medical Councils to nominate an A1, a metropolitan and a provincial hospital. In Tasmania, the Northern Territory and the Australian Capital Territory, all public hospitals were surveyed (Box 1). Questionnaires to these hospitals were distributed from May to October 2004.
We sent 2601 questionnaires (corresponding to the number of prevocational doctors at all sites, as advised by the relevant Postgraduate Medical Council) to medical staff managers or medical education officers at 36 health services for secondary distribution to doctors. Apart from a stamped addressed envelope, there was no inducement or coercion to complete the questionnaire, a task requiring about 20 minutes. The actual number of questionnaires reaching the targeted doctors is unknown.
Respondents were analysed in four subgroups: PGY1, PGY2, PGY3+ (if not enrolled in a specialty training program), and prevocational IMG regardless of their postgraduate year.
Data were entered into Microsoft Access with analysis by Microsoft Excel, SPSS for Windows, version 13.0 (SPSS Inc, Chicago, Ill, USA) and SAS, version 9 (SAS, Cary, NC, USA). Variables were analysed with non-parametric and correlation statistics as appropriate (Mann–Whitney U test, and Spearman rank-order correlation). Yate’s correction for continuity was used for Pearson’s χ2 test of independence involving 2 ×2 contingency tables.
Ethics committee approval was obtained from 36 Australian health services and Monash University.
Eighteen per cent (470/2601) of valid questionnaires were returned: 216 PGY1, 126 PGY2, 59 PGY3+ and 69 IMG. As not all questions were completed by every respondent, the total number of responses analysed varies for each item and percentages expressed are based on the valid responses only, excluding “not applicable” or items not answered. Data from the pilot survey were included in the final analysis, as the questionnaire and methodology were identical (Box 1).
Two hundred and ninety-nine of 467 respondents (64%) felt well or very well prepared in general for their prevocational role (Box 2) and 26 (5.5%) felt unprepared. Perceived preparedness improved marginally with each postgraduate year: PGY1, 120/216 (55%) well or very well prepared; PGY2, 88/122 (72%); and PGY3+, 49/56 (87%) (Spearman correlation coefficient r = 0.28; P = 0.001). Forty of 67 (60%) IMGs felt well or very well prepared.
Prevocational doctors felt well or very well prepared for interacting with patients (90.6%; 425/469), patients’ relatives (69.5%; 325/467) and nursing staff (69.3%; 325/469), but less so for their medicolegal obligations (22.6%; 106/468), resuscitation and emergency skills (31.1%; 146/469), or making career choices (40.1%; 188/468).
There was no difference in the perceived general preparedness of doctors from graduate-entry courses (99 respondents) and undergraduate courses (370). However, doctors from graduate-entry courses felt better prepared for their medicolegal role (P = 0.049), using information technology (IT) to access the evidence base for medical practice (P = 0.005), and making appropriate career choices (P = 0.038) (Mann–Whitney U tests, primary data not displayed). Sex, age, state of origin or university did not affect perceptions of preparedness.
Nearly 90% (418/465) of prevocational doctors reported adequate informal contact with their registrar, the most commonly experienced learning method (Box 3). Fifty-six per cent (261/466) reported adequate informal contact with consultants.
PGY1 reported adequate exposure to hospital meetings (61%; 131/216), more often than other prevocational doctors (34%; 73/237) (χ2 = 44.7; df = 1; P < 0.001). Sixteen per cent (27/171) of PGY1 reported adequate exposure to high-fidelity simulation (using a life-sized computerised mannequin in a realistic environment to simulate complex clinical events), and 60% (102/171) had no exposure. Seven per cent (13/196) (χ2 = 8.3; df = 1; P = 0.004) of other doctors had adequate exposure to simulation and 69% (135/196) had none (χ2 = 4.5; df = 1; P = 0.033). Eighteen per cent (37/212) of PGY1 reported no exposure to clinical skills training compared with 42% (98/235) of other doctors (χ2 = 31.5; df = 1; P < 0.001). IMG reported adequate informal exposure to registrars less often (IMG, 61%; 49/65; others, 92%; 361/393) (χ2 = 4.5; df = 1; P = 0.034) and fewer hospital meetings than other doctors (34%; 21/61 v 85%; 183/216) (χ2 = 5.4; df = 1; P < 0.020), but more formal contact with consultants (37%; 23/63 v 18%; 70/390) (χ2 = 5.5; df = 1; P < 0.020).
None of the 33 responding rural hospital doctors reported adequate exposure to high-fidelity simulation, yet 11% (38/356) of metropolitan and A1 hospital doctors had adequate exposure (P < 0.005). Doctors in A1 and metropolitan hospitals reported adequate exposure to clinical or unit meetings (62%; 246/394) more often than provincial hospital doctors (38%; 15/40) (χ2 = 7.3; df = 1; P = 0.007).
Informal contact with registrars (433/463), college tutorials (199/251), high-fidelity simulation centres (179/216) and clinical skills sessions (299/370) were all perceived as useful or very useful by more than 75% of responding prevocational doctors (Box 4). Less than 20% considered grand rounds (71/414), computer programs (53/283), unit meetings (72/434) and videoconferencing (34/225) to be “very useful”. Of the 14 educational methods, grand rounds ranked 12th in usefulness.
PGY3 reported formal contact with registrars, clinical skills sessions, and hospital and unit meetings as being less useful teaching methods than did other doctors (P < 0.05 for each teaching method). There was no difference in perceived usefulness of learning methods between prevocational doctors at A1, metropolitan or provincial hospitals.
Most prevocational doctors expressed a desire for increased exposure to all teaching and learning methods except informal contact with registrars, and unit meetings, both with high existing exposure (Box 5). More than 80% of respondents would like more formal teaching from their registrars (383/456) and consultants (362/447), more high-fidelity simulation (265/316) and more access to professional college tutorials (283/350). More than 70% (303/399) would like more preparation for their role as a teacher and more clinical skills sessions, and nearly 60% (272/460) want more consultant feedback. Prevocational doctors want the same teaching and learning methods regardless of workplace location.
Doctors were asked to list up to three rotations for which they felt inadequately prepared. Of the 470 respondents, 195 listed at least one, and 18 listed three rotations. Most commonly listed rotations were emergency medicine (55 felt unprepared), intensive care unit (21), specialty medicine (21), general surgery (20), paediatrics (19), obstetrics and gynaecology (16), and rural terms (15).
Respondents were also asked to list rotations where they perceived the CME was limited or inadequate. Twenty-one listed one, and 56 listed two rotations. Surgery, orthopaedics, psychiatry, obstetrics and gynaecology, covering shifts and nights were the most commonly cited. However, the proportion of respondents who had worked in these areas is not known.
As the response rate was low, we compared hospitals with a response rate of more than 20% to those with less than 20% for all 56 quantitative responses (Mann–Whitney U test). Only five of the 56 items showed a significant difference (preparedness in theoretical knowledge, desire for more exposure to videoconferencing, usefulness of clinical skills sessions, exposure to and usefulness of simulation). The magnitude of all differences was small and order of preferences did not change for any set of responses.
Respondents were asked to make general comments on their educational needs or experiences. These comments generally reinforced themes identified earlier in the survey. Some comments included:
“Registrar teaching is undervalued.”
“Registrars should have training in their role as teachers; since they actually have much more opportunity/contact than consultants.”
“. . . need much more undergraduate and postgraduate training in advanced life support and management of emergencies.”
“The one major thing lacking in the medical course . . . is management of the sick patient. This should surely be of utmost importance before releasing interns into the wards.”
“I would like more input into career choices and how to get into various specialities.”
“Early exposure to career orientation would be of great benefit; even targeting during student years perhaps.”
We report the findings of a survey of the range, quality and perceived usefulness of educational programs provided to Australian prevocational doctors. Although few believe they are poorly prepared in general for their duties, there are several areas where they lack confidence, especially in management of emergencies, choosing a career and meeting medicolegal obligations.
To our knowledge, this survey is the first systematic attempt to describe the educational opportunities available to Australian prevocational doctors or to assess their perceptions of the teaching and learning methods in their hospitals.
The main threat to validity of this study is the apparent low response rate of 18.1%. This was disappointing, but not unexpected for an unsolicited, anonymous questionnaire. Nevertheless, with 470 responses, our survey is the largest study of Australian prevocational doctors’ educational activities. There is considerable support in the medical literature for the validity of surveys with similarly low response rates.8,9
We believe the survey accurately reflects the views of Australian prevocational trainees for two reasons. First, the true response rate is almost certainly higher than 18.1%, as the number of questionnaires reaching doctors was very likely much less than the 2601 sent to hospitals. For ethical reasons, we could not directly contact individual doctors, but had to rely on third parties to deliver the questionnaire.
Second, comparison of hospitals with response rates of more than 20% with those with response rates less than 20% showed internal consistency of responses.
A second potential weakness is that responses from Victoria (September 2003 to January 2004) and other states (May 2004 through October 2004) were obtained in different years and at different times of the year. This could introduce bias as the national and Victorian respondents did not graduate in the same year and because trainees’ confidence might be greater later in the year. However, there were no significant differences between the responses from Victoria and other states.
Our findings have practical implications for the design of training programs for prevocational doctors in Australian hospitals. The first is the need to recognise and develop the role of registrars, who are perceived as an important and reliable educational resource by the overwhelming majority of prevocational doctors. Seventy-five per cent of prevocational doctors want more preparation for their future role as a teacher. Lack and Cartmill have emphasised the importance of “good” registrars — registrars who make time to review patients with interns, explain decisions and plans to interns, and are effective and willing teachers.10 Australian registrars currently receive little or no training in teaching. Our findings provide strong support for wider dissemination of professional development programs such as “Teaching on the Run” and the professional development pilot project recently conducted by the Postgraduate Medical Council of Victoria.11,12
Second, teaching and learning methods that are not perceived as useful should be reviewed. Grand rounds, unit meetings, computer programs and videoconferencing (proposed as solutions to poor educational exposure on night duty and country rotations) are not highly regarded. Training programs should mainly comprise more popular learning methods such as registrar and consultant teaching, college tutorials, clinical skills teaching (particularly high-fidelity simulation), and instruction in teaching skills. The biggest difference between what trainees are getting (less than 10%) and what they want (more than 80%) is experience of high-fidelity simulation centres, confirming the findings of a recent Victorian study.13 It is disturbing that only 56% of trainees report adequate contact with their consultants and less than half received adequate feedback from their consultants.
Third, training programs should address prevocational doctors’ lack of confidence in their ability to manage emergencies, choose a career path, meet their medicolegal obligations or perform procedures. They should include more critical care skills, procedural skills and instruction in medicolegal issues. Career advice should be more accessible, particularly for trainees required to make a decision a few months into their medical career, such as those entering surgical training programs. More contact with consultants and more time to make a decision may help prevocational trainees feel more confident about their choice of career.
Educational programs should be informed by an understanding of current teaching and learning methods, the perceptions of the strengths and weaknesses of these educational activities, expressed learning needs and preferred learning styles. This study is the first step towards a more rational approach to designing educational programs for Australian prevocational doctors.
1 Response rates by state or territory and hospital type
* Each hospital was given questionnaires equal to the number of prevocational doctors, as advised by the relevant Postgraduate Medical Council. A1 = tertiary hospital. |
|||||||||||||||
2 Perceptions of preparedness for tasks and skills
* Likert-type scale where 1 is “very unprepared” and 5 is “very well prepared”. IT = information technology. |
|||||||||||||||
3 Perceptions of exposure to learning and teaching methods reported by Australian prevocational doctors
Received 21 June 2005, accepted 13 March 2006
- Andrew W Dent1,2
- Brendan Crotty3,4
- Helen L Cuddihy5
- Glenn C Duns1
- Joan Benjamin2
- Carol A Jordon4
- Jacqueline F Satchell4
- Stephen Farish2
- Tracey J Weiland1,2
- Brian C Jolly5
- 1 St Vincent’s Health, Melbourne, VIC.
- 2 Department of Medicine, University of Melbourne, Melbourne, VIC.
- 3 University of Melbourne Clinical School, Austin Health, Melbourne, VIC.
- 4 Postgraduate Medical Council of Victoria, St Vincent’s Hospital, Melbourne, VIC.
- 5 Monash University, Melbourne, VIC.
This study was supported by a grant from the Medical Training Review Panel of the Australian Government Department of Health and Ageing. The department had no input in the design of the study, analysis and interpretation of the data, or writing the manuscript. We thank the medical education officers and hospital medical officer managers who distributed the survey and the prevocational doctors who returned completed surveys.
None identified.
- 1. Dillner L. Senior House Officers: the lost tribes. BMJ 1993; 307: 1549-1551.
- 2. Paice E. Is the new deal compatible with good training? A survey of senior house officers. Hosp Med 1998; 59: 72-74.
- 3. Hurley PA, Paterson-Brown BS. Senior house officer training: some myths exposed. J R Coll Surg Edinb 1999; 44: 324-327.
- 4. Davies J, Tan K, Jenkins HR. The current status of senior house officer postgraduate education in a single region. Med Educ 2000; 34: 367-370.
- 5. Beard J, Strachan A, Davies H, et al. Developing an education and assessment framework for the Foundation Programme. Med Educ 2005; 39: 841-851.
- 6. Council of Australian Governments (COAG). Council of Australian Governments’ Meeting, 10 February 2006. Communiqué. Available at: http://www.coag.gov.au/meetings/100206/index.htm (accessed Mar 2006).
- 7. Beckert L, Wilkinson TJ, Sainsbury R. A needs-based study and examination skills course improves students' performance. Med Educ 2003; 37: 424-428.
- 8. Krosnick JA. Survey research. Annu Rev Psychol 1999; 50: 537-567.
- 9. Visser PS, Krosnick JA, Marquette J, Curtin M. Mail surveys for election forecasting? An evaluation of the Columbus Dispatch poll. Public Opin Q 1996; 60: 181-227.
- 10. Lack SL, Cartmill J. Working with registrars: a qualitative study of interns’ perceptions and experiences. Med J Aust 2005; 182: 70-72. <MJA full text>
- 11. Lake FR. Teaching on the run tips: doctors as teachers. Med J Aust 2004; 180: 415-416. <MJA full text>
- 12. McGrath BM, Singh J. Professional development of registrars stage 2: final project progress report. Melbourne: Postgraduate Medical Council of Victoria, 2006.
- 13. Postgraduate Medical Council of Victoria. Clinical skills requirements of the health professions in Victoria, a report commissioned by the Department of Human Services. Melbourne: Postgraduate Medical Council of Victoria, 2003.
Abstract
Objective: To survey prevocational doctors working in Australian hospitals on aspects of postgraduate learning.
Participants and setting: 470 prevocational doctors in 36 health services in Australia, August 2003 to October 2004.
Design: Cross-sectional cohort survey with a mix of ordinal multicategory questions and free text.
Main outcome measures: Perceived preparedness for aspects of clinical practice; perceptions of the quantity and usefulness of current teaching and learning methods and desired future exposure to learning methods.
Results: 64% (299/467) of responding doctors felt generally prepared for their job, 91% (425/469) felt prepared for dealing with patients, and 70% (325/467) for dealing with relatives. A minority felt prepared for medicolegal problems (23%, 106/468), clinical emergencies (31%, 146/469), choosing a career (40%, 188/468), or performing procedures (45%, 213/469). Adequate contact with registrars was reported by 90% (418/465) and adequate contact with consultants by 56% (257/466); 20% (94/467) reported exposure to clinical skills training and 11% (38/356) to high-fidelity simulation. Informal registrar contact was described as useful or very useful by 94% (433/463), and high-fidelity simulation by 83% (179/216). Most prevocational doctors would prefer more formal instruction from their registrars (84%, 383/456) and consultants (81%, 362/447); 84% (265/316) want increased exposure to high-fidelity simulation and 81% (283/350) to professional college tutorials.
Conclusion: Our findings should assist planning and development of training programs for prevocational doctors in Australian hospitals.