A CHANGE in one of the ways to assess undergraduate students for acceptance into medical school appears to have increased the gender gap between men and women, and reduced the efficacy of professional coaching, according to research published today by the Medical Journal of Australia.
The Undergraduate Medical and Health Sciences Admissions Test (UMAT) was used to assess medical school applicants from 1991 until it was replaced by the University Clinical Aptitude Test (UCAT) used by applicants for the 2020 undergraduate medical school programs in Australia and New Zealand.
The UCAT is a timed (2 hours), computer-based test comprising five subsets of multiple choice questions. Four subsets assess cognitive ability: verbal reasoning, decision making, quantitative reasoning, and abstract reasoning. The fifth, a situational judgement test, assesses the “capacity to understand real world situations and to identify critical factors and appropriate behaviour in dealing with them”.
Professor Barbara Griffin from Macquarie University, and colleagues assessed the scores of applicants for the three NSW undergraduate medical programs for entry in 2019 (4114 applicants) or 2020 (4270); 703 people applied for both intakes. Applicants selected for interview were surveyed about whether they had received professional coaching for the selection test.
Griffin and colleagues found four significant results:
- Mean scores for UMAT and for all four UCAT cognitive subtests were higher for men than for women; mean UMAT 2 and UCAT situational judgement test scores were higher for women than men; all differences remained statistically significant after adjusting for age, socio-economic status, age, socio-economic status, remoteness, and ATAR ranking;
- Mean UMAT and UCAT subtest scores were all higher for people living in higher socio-economic status areas than for those from lower status areas;
- All mean UMAT and UCAT subtest scores for applicants with home addresses in rural areas were lower than for those from major cities;
- For the 2020 intake, a larger proportion of applicants from higher socio-economic status areas (225 of 449, 50%) were coached than of those from lower socio-economic status areas; coaching had no significant on any UCAT subtest scores, but its effect was significant for UMAT 2 and 3 scores.
“Our findings indicate that the difference in overall test scores between male and female applicants is greater with the UCAT than it was with the UMAT, while the influence of socio-economic status areas and remoteness of residential address were similar for the two tests,” Griffin and colleagues concluded.
“We found that women performed less well than men in all UCAT cognitive tests, but better in the situational judgement test.
“The proportion of women selected for interview following the UCAT was significantly lower than that of applicants (47% v 56%), and the difference was particularly large for applicants from large cities (40% v 53%).”
The authors also found that “professional coaching did not markedly improve performance in UCAT among those accepted for interview”.
“In view of the apparent sex-related differences we have identified, medical programs should consider incorporating situational judgement test scores into assessments; several institutions are considering this for future intakes,” Griffin and colleagues concluded.
“The inclusion of interviews (in which women typically perform better than men) as well as cognitive tests in selection processes also appears to reduce the risk of sex-related disadvantage in selecting candidates for the medical workforce.”
All MJA media releases are open access and can be found at: https://www.mja.com.au/journal/media
- Cate Swannell