Abstract
Objective: To report, and determine reasons for, a change in the gender ratio observed among enrolled medical students after removal of the interview from the selection process.
Design, setting and participants: Cross-sectional study of 4051 students admitted to the medical program at the University of Queensland between 2004 and 2012. Students are enrolled either directly as graduates or via a school-leaver pathway.
Main outcome measures: Change in proportions of male and female students over time, and gender-specific scores in the three sections of the GAMSAT (Graduate Medical School Admissions Test).
Results: Between 2004 and 2008 (when an interview was part of the selection process), 891 enrolled students (51.4%) were male, whereas between 2009 and 2012 (no interview), 1134 (57.7%; P < 0.001) were male. This change in gender ratio was limited to domestic direct graduate-entry students, and the male proportion in this group rose from 50.9% (705 students) before the interview was removed to 64.0% (514 students; P < 0.001) after removal of the interview (reaching 73.8% in 2012). Between 2004 and 2012, male students consistently performed better than female students on GAMSAT section III (mean score, 71.5 v 68.5; P < 0.001).
Conclusion: The proportion of males enrolled in the medical program at this university increased markedly after removal of the interview from the selection process. This change is limited to domestic direct graduate-entry students, and seems to be due to higher scores by male students in section III of the GAMSAT. The interview may play an important role in ensuring gender equity in selection, and medical schools should carefully monitor the consequences of changes to selection policy.
Selection processes for entry into medical school are continually under scrutiny, not only for predictive validity, but also for equity and fairness.1-3 The importance of equitable selection criteria that can identify candidates who will succeed academically and negotiate the transition into professional practice is widely acknowledged.3,4 Simultaneously, broadening the admissions base is an important part of social accountability, and selection processes at some schools include mechanisms to select students based on background, social status and personal qualities, as well as academic performance.5,6 Mechanisms employed to do this include quotas, personal statements, portfolios and interviews.1,4,7
Internationally, debate continues about who to admit to medical study, and how to admit them.3 Equity issues identified include gender, age, socioeconomic status, rurality and Indigenous status.8,9 Prior academic performance has the highest predictive validity for performance in medical school but has itself been shown to be influenced by gender, educational background and social class.1,3,9,10 Despite their widespread use, evidence suggests that interviews have poor predictive validity and limited practicality and are subject to bias, including gender bias.11-14
Australian medical schools use various combinations of factors in making selection decisions, including prior academic performance (at high school or university), performance on standardised tests including UMAT (Undergraduate Medicine and Health Sciences Admissions Test) or GAMSAT (Graduate Medical School Admissions Test), a written statement or portfolio and an interview.15,16 All medical schools admitting graduates make use of the GAMSAT.15 Although the criterion validity of the GAMSAT is continually reviewed, questions remain about its predictive value and gender bias.15-18 Reports have shown a modest but statistically significant gender bias in favour of male students.17
In 2009, after analysis of the format, validity and utility of the interview, this component of the selection process was discontinued at the University of Queensland (UQ).18 Here, we report a consequent change in the gender ratio among enrolled students and analyse the reasons for this shift.
domestic students: about 300 places are available each year, as determined by the federal government; and
international students: about 120 places for students who spend all 4 years in Australia, and (in 2012) about 85 places for students from the United States who spend the first 2 years of their program in Australia and the second 2 years in the US.
In addition, there are two distinct entry pathways:
graduate-entry pathway (domestic): about half of all domestic places are available to students with a prior degree who apply for direct entry into the MB BS program; and
school leaver pathway, which is also called a dual degree (domestic): about half of all domestic places are available to students provisionally accepted directly from high school who first study a non-medical undergraduate degree before commencing their MB BS.
Admissions criteria are found on the UQ website (http://www.uq.edu.au). Before 2009, selection to the domestic graduate-entry pathway was based on exceeding a defined GPA (grade point average) in a prior degree, and then on ranking based on a composite score of performance in the GAMSAT and the interview. Entry to the domestic school leaver pathway was based on attaining the highest level of high school score (called an Overall Position 1 or equivalent), and then ranking based on the UMAT total score. (There has never been an interview for this subcohort.) From 2009 onwards, the interview was not used for graduate entry, but other criteria remained the same. International students are selected in a similar manner, but may take the North American MCAT (Medical College Admission Test) instead of the GAMSAT and the ISAT (International Student Admissions Test) instead of the UMAT. There have never been interviews for international students.
The GAMSAT has been used since 1996 in Australia and is a written aptitude test developed and administered by the Australian Council for Educational Research (ACER) on behalf of the consortium of graduate-entry medical schools (GAMSAT Consortium Universities).19,20 The GAMSAT is divided into three sections: section I, reasoning in humanities and social sciences; section II, written communication; and section III, reasoning in biological and physical sciences. The GAMSAT overall score is computed as a weighted average of the three section scores in the ratios 1:1:2, respectively.19 Each university determines its own cut-off value, and at UQ (at the time of this analysis) a minimum score of 50 in each of the three sections of the GAMSAT test was required to be eligible for entry, with ranking done on overall score and then by GPA within the GAMSAT score band.20
Statistical analysis was performed using SPSS version 21 (SPSS Inc). Descriptive statistics were calculated and Pearson χ2 tests performed to test for differences in gender proportion for all entry years combined, for each year and between the interview and no-interview periods. Results are reported with 95% CI, and a P value < 0.05 is regarded as significant. Preliminary Pearson correlations were run to determine the associations between GAMSAT sections (I–III) and the overall score (weighted average) as well as an unweighted mean score. A two-way, multivariate analysis of variance (ANOVA) was conducted to test for gender differences in GAMSAT section scores and to compare differences between gender over the years of the study period (Bonferroni-adjusted P = 0.017). Gender differences between the interview and no-interview periods in GAMSAT overall scores were examined using two-way ANOVA. Mean scores with 95% CI, standard errors, F, P and effect sizes are reported. Preliminary analyses were performed to test for violations of the assumptions of normality, linearity and homoscedasticity; where necessary, significance levels were reported at a more stringent level of P < 0.01.21
In all, 4051 students were admitted to study medicine between 2004 and 2012 (3698 students at the School of Medicine and 353 school leavers provisionally accepted into medicine but still completing their first degree) (Box 1). The annual intake into the School of Medicine increased substantially between 2004 (307 students) and 2012 (543 students). The median age of enrolled students was 23 years (interquartile range, 3 years), with no observed differences in mean age between the interview and no-interview periods (24.4 years and 24.5 years, respectively). The proportion of international students increased from 9.8% (30 students) in 2004 to 44.0% (239 students) in 2012. The proportion of domestic students enrolled as direct graduates fell from 100% in 2004–2007 to 71.1% in 2008–2012 (and reached 40.1% in 2012).
Between 2004 and 2008, 891 enrolled students (51.4%) were male, whereas between 2009 and 2012, 1134 (57.7%; χ2 = 15.01; df = 1; P < 0.001) were male. There was a notable shift in the gender ratio (in favour of males) in 2009, when the proportion of male students jumped from 52.2% in 2008 to 57.7%. This proportion has remained stable within 1% since 2009 (Box 1). In the international subcohort, the proportion of enrolled males has been consistently higher than that of females (by about 5%–10%) across the entire study period, especially since 2007, when the number of international students increased, but this difference is not statistically significant (χ2 = 2.41; df = 8; P = 0.97). There was no significant difference in gender ratio within the international student subcohort between the interview and no-interview periods (χ2 = 0.59; df = 1; P = 0.45). As shown in Box 1, the gender ratio for the domestic school leaver pathway between 2008 and 2012 is almost balanced. However, for the domestic direct graduate-entry pathway, the male proportion rose from 50.9% in 2004–2008 to 64.0% in 2009–2012 (χ2= 35.19; df = 1; P < 0.001). In 2011 the proportion of male students enrolled through the domestic direct graduate-entry pathway was 73.1%, and in 2012 it was 73.8% (Box 2).
Of the 2187 domestic students enrolled by the direct graduate pathway, 2047 GAMSAT scores were available (special admissions were excluded). Mean GAMSAT section scores are reported in Box 3, by gender (for 1151 male and 896 female students). The difference in mean scores in sections I and II between male and female students is small (less than 1 unit), while the difference in mean scores for section III is more than 3 units (71.5 for males v 68.5 for females). The Pearson correlation calculation showed that section III is more strongly correlated with GAMSAT overall scores (r[2045] = 0.85; P < 0.001) than sections I–II (r[2045] = 0.48 for section I, and 0.31 for section II; P < 0.001). By comparison, the correlations between sections (I–III) and an unweighted mean score were closer (r[2045] = 0.64 for section I, 0.57 for section II and 0.62 for section III; P < 0.001).
A two-way, between-group, multivariate ANOVA of GAMSAT section scores, applying Bonferroni adjustments, showed significant gender differences in section III (Box 3). The test showed a significant difference between male and female students on the combined GAMSAT sections I, II and III (F3,2027 = 25.18; P < 0.001; Wilk l = 0.96; partial eta-squared = 0.036), but when the section scores were considered separately, the only difference in gender was in section III (P < 0.001). There was no significant interaction between gender and year of entry on the combined three sections (P = 0.12) or on any of the sections when considered separately (P = 0.19 for section I, P = 0.16 for section II and P = 0.26 for section III), indicating that the influence of gender on differences in scores remained consistent across all years studied. A two-way ANOVA confirmed a significant difference between gender in GAMSAT overall scores (F1,2043 = 34.03; P < 0.001; partial eta = 0.016) but no difference in overall scores by gender between the interview and no-interview periods (P = 0.55).
Our observations are in stark contrast to the general trend in Australia where between 2009 and 2012, 51.7% of all enrolments in Australian medical schools were female students.22 Further, using data supplied by ACER, we found no evidence of an increased proportion of males sitting the GAMSAT or applying for entry to medical school between 2007 and 2011 (Appendix 1). Indeed, among the 12 graduate-entry medical programs, between 2007 and 2011, in only two cases was the proportion of males applying higher than 50% (and the highest of these was 54.1%) (Appendix 2).
The reason for the shift at UQ seems to be the influence of higher scores achieved by male applicants in section III of the GAMSAT. Higher scores on section III have been shown to be associated with higher GAMSAT overall scores, on which candidates are ultimately ranked for selection to medical school.17,20 The results from our analysis of GAMSAT scores are consistent with prior reports of a significant (but relatively small) gender effect favouring males on both section III and overall scores.17 We also reported strong correlations between section III and the overall score but when a mean score (using unweighted sections) was calculated for the UQ cohort, this effect was noticeably moderated.
Alternatively, the strong influence of academic background on GAMSAT performance is well documented.9,16,17 For example, graduates from disciplines such as engineering, where students are predominantly male, perform better on section III.17 While it is important to acknowledge the substantial effect of prior learning, the academic backgrounds of GAMSAT candidates have remained relatively stable over the study period.16 Further, the academic selection criteria at UQ have not changed. The one factor that has changed has been the discontinuation of the admissions interview at UQ.
Our findings have significant policy implications. Currently, all GAMSAT Consortium medical schools in Australia rank applicants using the weighted overall score calculated by ACER. The exception is the University of Melbourne, which uses an average (unweighted) GAMSAT score.20 GAMSAT section III focuses on reasoning in biological and physical sciences, and we find it difficult to rationalise the continued weighting of scores in this section if this practice leads to significant gender bias, as it seems to have done here. Our view is that cohorts of admitted medical students should be representative of the communities from which they are drawn, and which they will later serve.
There is consensus that selection processes aimed at broadening access to medical school have not been extensively studied.3 There are limited studies focusing on gender and the interview process; however, the literature does indicate a shift in attitudes from an earlier bias favouring males.13,14 We were not able to present data on gender differences in interview scores at UQ, but our study does lend support to the more recent studies suggesting that personal interviews favour female applicants.14 Apparently, without the moderating effect of the interview, gender bias in GAMSAT performance has led to a marked increase in the proportion of male medical students enrolled at UQ.
1 Proportions of male and female students, by subcohort, in each year between 2004 and 2012
International students (n = 1072)† |
|||||||||||||||
School leaver pathway group§ (n = 439) |
|||||||||||||||
2 Proportions of male and female medical students enrolled through the direct graduate-entry pathway
Received 21 January 2013, accepted 3 July 2013
- David Wilkinson1
- Mavourneen G Casey2
- Diann S Eley2
- 1 Macquarie University, Sydney, NSW.
- 2 School of Medicine, University of Queensland, Brisbane, QLD.
We would like to thank Michelle Doughan at UQ Central, Management Information Systems, for assistance with data extraction from UQ student records.
No relevant disclosures.
- 1. Lumsden MA, Bore M, Millar K, et al. Assessment of personal qualities in relation to admission to medical school. Med Educ 2005; 39: 258-265.
- 2. Bore M, Munro D, Powis D. A comprehensive model for the selection of medical students. Med Teach 2009; 31: 1066-1072.
- 3. Prideaux D, Roberts C, Eva K, et al. Assessment for selection for the health care professions and specialty training: consensus statement and recommendations from the Ottawa 2010 Conference. Med Teach 2011; 33: 215-223.
- 4. McManus C, Powis D. Testing medical school selection tests. Med J Aust 2007; 186: 118-119. <MJA full text>
- 5. James D, Yates J, Nicholson S. Comparison of A level and UKCAT performance in students applying to UK medical and dental schools in 2006: cohort study. BMJ 2010; 340: c478. doi: 10.1136/bmj.c478.
- 6. Hughes P. Can we improve on how we select medical students? J R Soc Med 2002; 95: 18-22.
- 7. Story M, Mercer A. Selection of medical students: an Australian perspective. Intern Med J 2005; 35: 647-649.
- 8. Wilkinson D, Birks J, Davies L, et al. Preliminary evidence from Queensland that rural clinical schools have a positive impact on rural intern choices. Rural Remote Health 2004; 4: 340.
- 9. Blackman I, Darmawan I. Graduate-entry medical student variables that predict academic and clinical achievement. Int Educ J 2004; 4: 30-41.
- 10. Ferguson E, James D, Madeley L. Factors associated with success in medical school: systematic review of the literature. BMJ 2002; 324: 952-957.
- 11. Goho J, Blackman A. The effectiveness of academic admission interviews: an exploratory meta-analysis. Med Teach 2006; 28: 335-340.
- 12. Groves MA, Gordon J, Ryan G. Entry tests for graduate medical programs: is it time to re-think? Med J Aust 2007; 186: 120-123. <MJA full text>
- 13. Marquart JA, Franco KN, Carroll BT. The influence of applicants’ gender on medical school interviews. Acad Med 1990; 65: 410-411.
- 14. Salvatori P. Reliability and validity of admissions tools used to select students for the health professions. Adv Health Sci Educ Theory Pract 2001; 6: 159-175.
- 15. Coates H. Establishing the criterion validity of the Graduate Medical School Admissions Test (GAMSAT). Med Educ 2008; 42: 999-1006.
- 16. Wilson IG, Roberts C, Flynn EM, Griffin B. Only the best: medical student selection in Australia. Med J Aust 2012; 196: 357. <MJA full text>
- 17. Pywell S, Hunt M, Edwards J, et al. Report on Graduate Australian Medical School Admissions Test 2012. Camberwell, Vic: Australian Council for Educational Research (ACER), 2012.
- 18. Wilkinson D, Zhang J, Byrne GJ, et al. Medical school selection criteria and the prediction of academic performance. Med J Aust 2008; 188: 349-354. <MJA full text>
- 19. Australian Council for Educational Research (ACER). Graduate Australian Medical School Admissions Test (GAMSAT) Information Booklet 2012. http://gamsat.acer.edu.au/gamsat-australia/info-booklet (accessed Dec 2012).
- 20. Gradauate Entry Medical Schools Admissions System (GEMSAS). Graduate medical school admisions guide 2014-2013. http://www.gemsas.edu.au/wp-content/uploads/2013/05/2014-Medicine-GEMSAS-Admissions-Guide-v1-5.pdf (accessed May 2013).
- 21. Parra-Frutos I. The behaviour of the modified Levene’s test when data are not normally distributed. Comput Stat 2009; 24: 671-693.
- 22. Medical Deans Australia and New Zealand (MDANZ). Annual tables. Medical student statistics. http://www.medicaldeans.org.au/statistics/annualtables (accessed Dec 2012).
Competing Interests:
Glen KerrLaslett
Competing Interests: No relevant disclosures
Mr Glen KerrLaslett
University of Sydne