The known: The UMAT, the test for selecting interviewees for undergraduate medical degrees, was replaced by the UCAT for the 2020 intake.
The new: Mean UCAT scores for women were lower than for men. Mean scores for applicants from lower socio‐economic status areas and from outside major cities were also lower. The influence of sex on total score was greater with the UCAT than the UMAT.
The implications: UCAT subtest weightings for selection decisions should be reviewed, and interview‐based assessments incorporated to increase the selection of women. Pathways and quotas for applicants from rural and disadvantaged areas should be retained.
Methods
We analysed data for 7691 domestic applicants for the three undergraduate medical training programs in New South Wales (University of Newcastle/University of New England, University of New South Wales, Western Sydney University) who sat the UMAT (2019 intake; 4114 applicants) or UCAT (2020 intake; 4270 applicants), including 703 people who applied in both years.
Applicants were selected for admission interview according to scores on the three UMAT sections (2019 intake) or the four UCAT cognitive subtests (2020 intake). Certain applicants (eg, those with rural backgrounds) were ranked separately to meet participation targets and government quotas. For the 2019 intake, 927 applicants were interviewed by at least one university, and 1074 were interviewed for the 2020 intake. After being interviewed, interviewees were invited to complete a survey (on paper or online) about their motivation for studying medicine and prior interview and testing experience; only the responses regarding test coaching item were included in this study.
Applicant information
Demographic data (age, sex, residential location) and test data (UMAT/UCAT, Australian Tertiary Admission Rank [ATAR] ranking) were provided for the study by the university admissions offices. Socio‐economic status was determined by residential address, using the Australian Bureau of Statistics Index of Relative Socioeconomic Advantage and Disadvantage (IRSAD),10 based on residents’ income, home ownership, and unemployment rates; we defined deciles 1–5 as lower and deciles 6–10 as higher socio‐economic status. Remoteness was also determined by residential address, using the 2016 Australian Bureau of Statistics remoteness structure;11 we classified remoteness as major cities or rural (inner regional, outer regional, remote, very remote area; combined because the individual category numbers were low).
For the 2019 intake, we analysed scores on the three UMAT subtests: UMAT 1 (problem solving), UMAT 2 (understanding people), and UMAT 3 (non‐verbal reasoning). For the 2020 intake, standardised UCAT scores (range, 300–900) were available for the four cognitive subtests and scaled scores for the situational judgement test.
Interviewee information
ATAR ranking was available only for interviewees (88% in 2019, 53% in 2020). Coaching attendance was assessed with the survey item, “Did you attend UMAT/UCAT coaching?”
Statistical analysis
All analyses were conducted in SPSS 25 (IBM). We assessed the influence of socio‐demographic factors and coaching on UMAT and UCAT scores in four analyses:
- We compared mean scores for men and women, for people from lower and higher socio‐economic status areas, for people from major cities and rural areas, and for applicants who were or were not coached. For these comparisons, we employed independent t tests, with P < 0.001 deemed statistically significant (Bonferroni correction for multiple comparisons).
- Total UMAT and UCAT scores and subtest scores (for all applicants) were separately assessed in multiple regression analyses adjusted for age (as it was negatively correlated with most subtest scores), sex, socio‐economic status, and remoteness. For multiple regression analyses of interviewee data, age, sex, socio‐economic status, remoteness, and ATAR ranking were included as variables; for multiple regression analyses of survey respondent data, age, sex, ATAR ranking, and coaching were included. For these comparisons, P < 0.05 was deemed statistically significant.
- The effect sizes (Cohen’s d) for t test results were calculated for the total UMAT and UCAT cognitive subtest scores, to allow comparison of effect sizes in the two tests. Effect sizes were calculated separately for all applicants and for those who completed both the UMAT in 2019 and the UCAT in 2020. Effect sizes of 0.10 are interpreted as very small, 0.20 as small, 0.50 as medium, 0.80 as large, and 1.20 as very large.
- Differences in the proportions of applicants and interviewees, by sex, socio‐economic status, and remoteness, were assessed in χ2 tests.
Ethics approval
The Macquarie University Human Research Ethics Committee approved our study, including our merging selection data (including demographic information), UMAT/UCAT scores (all applicants), and ATAR ranking (interviewees), as well as survey responses from consenting interviewees (reference, 5201836024731).
Results
The demographic characteristics and educational profiles of applicants are summarised in Box 1. The coaching question in the survey was completed by 636 of 927 interviewees (69%) for the 2019 intake and 624 of 1074 interviewees (58%) for the 2020 intake.
Sex
Mean scores for UMAT 1 and 3 and for all four UCAT cognitive subtests were higher for men than for women; mean UMAT 2 and UCAT situational judgement test scores were higher for women than men (Box 2, Box 3). All differences remained statistically significant after adjusting for age, socio‐economic status, and remoteness (all applicants; Supporting Information, table 1) or age, socio‐economic status, remoteness, and ATAR ranking (interviewees; Supporting Information, table 2).
The effect size for sex was 0.24 (95% CI, 0.18–0.30) for UMAT total score and 0.38 (95% CI, 0.32–0.44) for UCAT total score (all applicants). For applicants who sat both tests, the effect size was 0.24 (95% CI, 0.09–0.39) for UMAT total score and 0.41 (95% CI, 0.26–0.56) for UCAT total score. With the UMAT, 2279 of 4080 applicants (55.9%) and 438 of 926 interviewees (47.3%) were women (P < 0.001). With the UCAT, 2303 of 4270 applicants (53.9%) and 476 of 1074 interviewees (44.3%) were women (P < 0.001). Among applicants from major cities for the 2020 intake, the difference was particularly marked: 1889 of 3583 applicants (52.7%) and 300 of 749 interviewees (40.1%) were women.
Socio‐economic status
Mean UMAT and UCAT subtest scores were all higher for people living in higher socio‐economic status areas (IRSAD deciles 6–10) than for those from lower status areas (deciles 1–5) (Box 2, Box 3). All differences remained statistically significant after adjusting for age, sex, and remoteness (all applicants: Supporting Information, table 1). After adjusting interviewee scores for ATAR ranking, the influence of IRSAD was still significant for UMAT 1 and 2, UCAT verbal reasoning and decision making, and both total scores (Supporting Information, table 2).
The effect size for socio‐economic status was 0.47 (95% CI, 0.39–0.54) for UMAT total score and 0.43 (95% CI, 0.35–0.50) for UCAT total score (all applicants); for those who sat both tests, it was 0.40 (95% CI, 0.22–0.57) for UMAT total score and 0.24 (95% CI, 0.06–0.41) for UCAT total score. For the 2019 intake, 890 of 3829 applicants (23.2%) and 164 of 899 interviewees (18.2%) were from lower socio‐economic status areas (P < 0.001); for the 2020 intake, 1031 of 4066 applicants (25.4%) and 299 of 1052 interviewees (28.4%) were from lower socio‐economic status areas (P = 0.005). For applicants from major cities in the 2020 intake, 762 of 3574 applicants (21.3%) and 132 of 749 interviewees (17.7%) were from were from lower socio‐economic status areas (P = 0.006).
Remoteness (rurality)
All mean UMAT and UCAT subtest scores for applicants with home addresses in rural areas were lower than for those from major cities (Box 2, Box 3). All differences remained statistically significant after adjustment for sex, age, and IRSAD (all applicants: Supporting Information, table 1). After adjustment of interviewee scores for ATAR ranking, the influence of IRSAD was still significant for UMAT 1 and 3, UCAT quantitative reasoning and abstract reasoning, and both total scores (Supporting Information, table 2).
The effect size for remoteness was 0.54 (95% CI, 0.45–0.63) for UMAT total score and 0.48 (95% CI, 0.39–0.58) for UCAT total score (all applicants); for applicants who sat both tests, it was 0.71 (95% CI, 0.47–0.95) for UMAT total score and 0.59 (95% CI, 0.35–0.83) for UCAT total score.
Professional coaching (survey respondents only)
A total of 303 of 636 surveyed interviewees for the 2019 intake (47.6%) attended coaching before the UMAT exam; 291 of 624 surveyed interviewees (46.6%) for the 2020 intake attended coaching for the UCAT. In each year, similar proportions of men and women (2019: 52% v 48%, P = 0.93; 2020: 46% v 49%; P = 0.45) and of applicants from major cities and rural areas (2019: 49% v 38%; P = 0.07; 2020, 48% v 47%; P = 0.84) were coached. For the 2020 intake, a larger proportion of applicants from higher socio‐economic status areas (225 of 449, 50%) were coached than of those from lower socio‐economic status areas (59 of 147, 40%; P = 0.036).
For the UMAT, the only significant difference between coached and non‐coached applicants was for mean UMAT 3 score (Box 2); there were no significant differences for UCAT scores (Box 3). The effect size for coaching on total UMAT score was 0.25 (95% CI, 0.09–0.40) and 0.13 for total UCAT score (95% CI, –0.03 to 0.29). After adjusting for age, sex, and ATAR ranking, coaching had no significant on any UCAT subtest scores, but its effect was significant for UMAT 2 and 3 scores (Supporting Information, table 3).
Discussion
We report the first assessment of the impact of the change from the UMAT to the UCAT for selecting students for undergraduate medical programs in Australia and New Zealand. Our findings indicate that the difference in overall test scores between male and female applicants is greater with the UCAT than it was with the UMAT, while the influence of socio‐economic status areas and remoteness of residential address were similar for the two tests. Professional coaching did not markedly improve performance in UCAT among those accepted for interview.
Avoiding selection bias is critical for equity in high stake tests. It had previously been reported that women performed less well than men on the UMAT,12,13 and this difference was also evident in the 2019 intake UMAT data. We found that women performed less well than men in all UCAT cognitive tests, and that the differences appeared larger than for the UKCAT,1 but they performed better than men in the UCAT situational judgement test. Further, the size of the effect of sex on UCAT total cognitive test score was greater than for the UMAT, although still small (0.38 v 0.24). However, the proportion of women selected for interview following the UCAT was significantly lower than that of applicants (47% v 56%), and the difference was particularly large for applicants from large cities (40% v 53%).
A subgroup of applicants consistently receiving significantly lower scores than other applicants may reflect genuine differences in ability. Although men generally perform better than women on mathematical tests, women typically perform better in tests of verbal ability (especially those in high performing groups).14 Moreover, the academic performance of girls in high school is equivalent to or better than that of boys,15 and they typically achieve higher academic grades than men in medical school.16 Nevertheless, after adjusting for high school performance (ATAR ranking), women’s scores on UCAT cognitive ability tests were lower than those of men; the reasons for this discrepancy require further research. The UCAT tests may include indirect or unintended mathematical requirements, or the time‐limited, online tests may influence results. In view of the apparent sex‐related differences we have identified, medical programs should incorporate situational judgement test scores into assessments; several institutions are considering this for future intakes. The inclusion of interviews (in which women typically perform better than men) as well as cognitive tests in selection processes may also reduce the risk of sex‐related disadvantage in selecting candidates for the medical workforce.17
Our findings also suggest that applicants from rural or lower socio‐economic status areas performed less well on both the UMAT and UCAT. The small to medium effect sizes were similar for the two tests, and larger than for sex. The UCAT may limit the selection of applicants from outside major cities and from low socio‐economic status areas unless additional opportunities are provided by specific pathways and quotas. Quotas for rural students have been justified by the argument that they are more likely to later practise in underserviced rural areas.18,19 As women are more likely to choose careers in underserviced areas and in specialities with workforce shortages, such as general practice,20,21,22 a sex quota may also be appropriate.23
In Australia, large numbers of applicants for undergraduate medical programs believe they need commercial coaching to be competitive.24 The rate of coaching attendance in a 2008 University of Western Sydney study (51%)9 was similar to those for interviewees for the 2019 (48%) and 2020 intakes (47%) in our study. However, the effect sizes for coaching on overall score were small for the UMAT and statistically non‐significant for the UCAT. Even after adjusting for ATAR ranking and sex, UCAT scores were no higher for coached than uncoached interviewees. The novelty of the UCAT may have contributed to the low effectiveness of coaching, but our finding should assure applicants that the considerable financial and indirect costs of coaching, including lost opportunities for academic study not relevant to selection, is unnecessary.
Limitations
Although we included applicants for all three undergraduate medical programs in the most populous Australian state, our findings may not be generalisable to other parts of Australia or New Zealand with more rural applicants and fewer from academically selective schools. Other limitations were the broad measures of socio‐economic status and remoteness. The IRSAD allows area‐based socio‐economic comparisons, but areas are likely to include people of a range of wealth and advantage. We based remoteness on residence at the time of application, as we did not have access to more detailed information, such as time lived in rural areas. Although some interviewees may have not reported their attendance at coaching, published coaching rates have been fairly consistent in analyses of data collected before and after selection for interview, and the reported effectiveness of UMAT coaching has also been consistent.9,25
Box 1 – Demographic information and education profile of applicants for the 2019 and 2020 intakes into Australian and New Zealand undergraduate medical school programs
|
|
2019 intake (UMAT) |
2020 intake (UCAT) |
Applied in both years (UMAT and UCAT)* |
||||||||||||
|
|
|||||||||||||||
|
Number of applicants |
4114 |
4270 |
703 |
||||||||||||
|
Women |
2279 (55.4%) |
2303 (53.9%) |
367 (52.2%) |
||||||||||||
|
Men |
1801 (43.8%) |
1967 (46.1%) |
336 (47.8%) |
||||||||||||
|
Missing data |
34 (0.8%) |
0 |
0 |
||||||||||||
|
Age at first application (years), mean (SD) |
19.1 (3.5) |
19.6 (4.0) |
19.4 (4.0) |
||||||||||||
|
New South Wales residents |
2421 (58.8%) |
2615 (61.2%) |
523 (74.4%) |
||||||||||||
|
Socio‐economic status (IRSAD) |
|
|
|
||||||||||||
|
Deciles 1–5 (low) |
890 (21.6%) |
1031 (24.1%) |
174 (24.8%) |
||||||||||||
|
Deciles 6–10 (high) |
2939 (71.4%) |
3035 (71.1%) |
499 (71.0%) |
||||||||||||
|
Missing data† |
285 (6.9%) |
204 (4.8%) |
30 (4.3%) |
||||||||||||
|
Remoteness |
|
|
|
||||||||||||
|
Major cities |
3382 (82.2%) |
3583 (83.9%) |
591 (84.1%) |
||||||||||||
|
Rural |
525 (12.8%) |
497 (11.6%) |
81 (11.5%) |
||||||||||||
|
Missing data† |
207 (5.0%) |
190 (4.4%) |
31 (4.4%) |
||||||||||||
|
High school education |
|
|
|
||||||||||||
|
Government school |
1791 (43.5%) |
2044 (47.9%) |
368 (52.3%) |
||||||||||||
|
Independent school |
1169 (28.4%) |
1258 (29.5%) |
182 (25.9%) |
||||||||||||
|
Catholic school |
528 (12.8%) |
647 (15.2%) |
106 (15.1%) |
||||||||||||
|
Overseas school |
199 (4.8%) |
255 (6.0%) |
42 (6.0%) |
||||||||||||
|
Missing data† |
427 (10.4%) |
66 (1.5%) |
5 (0.7%) |
||||||||||||
|
|
|||||||||||||||
|
IRSAD = Index of Relative Socioeconomic Advantage and Disadvantage; SD = standard deviation; UMAT = Undergraduate Medical and Health Sciences Admissions Test; UCAT = University Clinical Aptitude Test. * Also included in the two individual intake years. † Applicant supplied no or ambiguous information that did not permit classification; eg, postcode or “Sydney” for residential location, or “university” for education. |
|||||||||||||||
Box 2 – Undergraduate Medical and Health Sciences Admissions Test (UMAT) scores (2019 intake), by sex, socio‐economic status and remoteness of residence, and coaching (interviewees only)
|
|
|
UMAT 1: problem solving |
UMAT 2: understanding people |
UMAT 3: non‐verbal reasoning |
UMAT total score |
||||||||||
|
Applicant category |
Number of applicants |
Mean (SD) |
Mean difference (95% CI) |
Mean (SD) |
Mean difference (95% CI) |
Mean (SD) |
Mean difference (95% CI) |
Mean (SD) |
Mean difference (95% CI) |
Cohen’s d (95% CI) |
|||||
|
|
|||||||||||||||
|
Sex |
|
|
|
|
|
|
|
|
|
|
|||||
|
Men |
1801 |
57.3 (10.1) |
5.1* |
52.4 (8.7) |
–2.6* |
57.2 (9.6) |
3.1* |
166.8 (22.8) |
5.5* |
0.24 |
|||||
|
Women |
2279 |
52.2 (9.1) |
55.0 (9.4) |
54.1 (9.5) |
161.4 (22.8) |
||||||||||
|
Socio‐economic status† |
|
|
|
|
|
|
|
|
|
|
|||||
|
Higher |
2939 |
55.4 (9.7) |
3.8* |
54.7 (8.9) |
3.0* |
56.4 (9.5) |
3.6* |
166.5 (22.0) |
10.5* |
0.47 |
|||||
|
Lower |
890 |
51.7 (10.0) |
51.7 (9.8) |
52.8 (10.1) |
156.0 (24.4) |
||||||||||
|
Remoteness |
|
|
|
|
|
|
|
|
|
|
|||||
|
Major cities |
3382 |
55.2 (9.8) |
4.8* |
54.3 (9.1) |
2.3* |
56.3 (9.6) |
5.2* |
165.6 (22.7) |
12.2* |
0.54 |
|||||
|
Rural |
525 |
50.4 (9.2) |
52.0 (9.7) |
51.1 (9.3) |
153.4 (12.2) |
||||||||||
|
Interviewees |
|
|
|
|
|
|
|
|
|
|
|||||
|
Coached |
303 |
64.6 (7.9) |
0.0 |
61.1 (7.8) |
0.7 |
63.5 (8.3) |
3.0* |
189.3 (15.2) |
3.6 |
0.25 |
|||||
|
Not coached |
333 |
64.6 (8.0) |
60.4 (7.6) |
60.5 (8.7) |
185.7 (14.1) |
||||||||||
|
|
|||||||||||||||
|
CI = confidence interval; SD = standard deviation. * P < 0.001. † Australian Bureau of Statistics Index of Relative Socioeconomic Advantage and Disadvantage (IRSAD):10 deciles 1–5 = lower, deciles 6–10 = higher socio-economic status. |
|||||||||||||||
Box 3 – University Clinical Aptitude Test (UCAT) scores (2020 intake), by sex, socio‐economic status and remoteness of residence, and coaching (interviewees only)
|
|
|
Verbal reasoning |
Decision making |
Quantitative reasoning |
Abstract reasoning |
UCAT total score (cognitive tests) |
Situational judgment |
||||||||
|
Applicant category |
Number of applicants |
Mean (SD) |
Mean difference (95% CI) |
Mean (SD) |
Mean difference (95% CI) |
Mean (SD) |
Mean difference (95% CI) |
Mean (SD) |
Mean difference (95% CI) |
Mean (SD) |
Mean difference (95% CI) |
Cohen’s d (95% CI) |
Mean (SD) |
Mean difference (95% CI) |
|
|
|
|||||||||||||||
|
Sex |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Men |
1967 |
609.4 |
14.8* |
662.4 |
15.5* |
721.2 |
38.9* |
686.8 |
27.4* |
2680 (259.8) |
96.5* |
0.38 |
618.1 (60.88) |
–6.2* |
|
|
Women |
2302 |
594.5 |
646.9 |
682.3 |
659.5 |
2583 (255.0) |
624.3 |
||||||||
|
Socio‐economic status† |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Higher |
3035 |
607.4 |
24.2* |
660.8 |
28.0* |
707.6 |
29.9* |
679.0 |
27.6* |
2655 (253.9) |
110* |
0.43 |
626.2 |
17.6* |
|
|
Lower |
1031 |
583.2 |
632.9 |
677.7 |
651.4 |
2545 (271.8) |
608.6 |
||||||||
|
Remoteness |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Major cities |
3583 |
604.1 |
22.6* |
656.5 |
21.6* |
705.4 |
43.7* |
676.6 |
36.3* |
2643 (263.2) |
124* |
0.48 |
623.2 |
12.2* |
|
|
Rural |
497 |
581.5 |
634.9 |
661.7 |
640.3 |
2518 (234.3) |
610.9 |
||||||||
|
Interviewees |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Coached |
291 |
671.3 |
–6.6 |
725.2 |
0.6 |
794.8 |
11.7 |
757.9 |
19.8 |
2949 (184.3) |
25.5 |
0.13 |
653.0 |
6.0 |
|
|
Not coached |
333 |
677.8 |
724.6 |
783.2 |
738.1 |
2924 (197.6) |
647.1 |
||||||||
|
|
|||||||||||||||
|
CI = confidence interval; SD = standard deviation. *P < 0.001. † Australian Bureau of Statistics Index of Relative Socioeconomic Advantage and Disadvantage (IRSAD):10 deciles 1–5 = lower, deciles 6–10 = higher socio-economic status. |
|||||||||||||||
Received 24 April 2020, accepted 11 August 2020
- Barbara Griffin1
- Graeme L Horton2
- Lisa Lampe2
- Boaz Shulruf3,4
- Wendy Hu5
- 1 Macquarie University, Sydney, NSW
- 2 The University of Newcastle, Newcastle, NSW
- 3 University of New South Wales, Sydney, NSW
- 4 Centre for Medical and Health Sciences Education, University of Auckland, Auckland, New Zealand
- 5 Western Sydney University, Sydney, NSW
No relevant disclosures.
- 1. James D, Yates J, Nicholson S. Comparison of A level and UKCAT performance in students applying to UK medical and dental schools in 2006: cohort study. BMJ 2010; 349: c478.
- 2. Lambe P, Waters C, Bristow D. The UK Clinical Aptitude Test: is it a fair test for selecting medical students? Med Teach 2012; 34: 557–565.
- 3. Wright SR, Bradley PM. Has the UK Clinical Aptitude Test improved medical student selection? Med Educ 2010; 44: 1069–1076.
- 4. The UCAT ANZ Consortium. UCAT: University Clinical Aptitude test ANZ. 2020. ucat.edu.au/ucat-anz/test-format (viewed Apr 2020).
- 5. Zedeck S. Adverse impact: history and evolution. In: Outtz JL, editor. Adverse impact: implications for organizational staffing and high stakes selection. New York: Taylor and Francis, 2010; pp. 3–28.
- 6. Garlick PB, Brown G. Widening participation in medicine. BMJ 2008; 336: 1111–1113.
- 7. Puddey IB, Mercer A, Carr S, Louden W. Potential influence of selection criteria on the demographic composition of students in an Australian medical school. BMC Med Educ 2011; 11: 97.
- 8. Griffin B. Coaching issues. In: Patterson F, Zibarras I, editors. Selection and recruitment in the healthcare professions: research, theory and practice. Cham: Palgrave Macmillan, 2018; pp. 223–248.
- 9. Griffin B, Harding DW, Wilson IG, Yeomans ND. Does practice make perfect? The effect of coaching and retesting on selection tests used for admission to an Australian medical school. Med J Aust 2008; 189: 270–273. https://www.mja.com.au/journal/2008/189/5/does-practice-make-perfect-effect-coaching-and-retesting-selection-tests-used
- 10. Australian Bureau of Statistics. 2033.0.55.001. Census of population and housing: Socio‐economic indexes for areas (SEIFA), Australia, 2016. Mar 2018. https://www.abs.gov.au/AUSSTATS/abs@.nsf/DetailsPage/2033.0.55.0012016?OpenDocument (viewed Apr 2020).
- 11. Australian Bureau of Statistics. 1270.0.55.005. Australian Statistical Geography Standard (ASGS): volume 5. Remoteness structure, July 2016. Mar 2018. https://www.abs.gov.au/AUSSTATS/abs@.nsf/DetailsPage/1270.0.55.005July%202016?OpenDocument (viewed Apr 2020).
- 12. Griffin B, Hu W. The interaction of socio‐economic status and gender in widening participation in medicine. Med Educ 2015; 49: 103–113.
- 13. Puddey IB, Mercer A. Socio‐economic predictors of performance in the Undergraduate Medicine and Health Sciences Admission Test (UMAT). BMC Med Educ 2013; 13: 155.
- 14. Wai J, Hodges J, Makel MC. Sex differences in ability tilt in the right tail of cognitive abilities: a 35‐year examination. Intelligence 2018; 67: 76–83.
- 15. Universities Admissions Centre. Report of the scaling of the 2018 NSW Higher School Certificate. Sydney: UAC, 2019. https://www.uac.edu.au/assets/documents/scaling-reports/Scaling-Report-2018-NSW-HSC.pdf (viewed Apr 2020).
- 16. Mercer A, Hay M, Hodgson WC, et al. The relative predictive value of undergraduate versus graduate selection tools in two Australian medical schools. Med Teach 2018; 40: 1183–1190.
- 17. Wilkinson D, Casey MG, Eley DS. Removing the interview for medical school selection is associated with gender bias among enrolled students. Med J Aust 2014; 200: 96–99. https://www.mja.com.au/journal/2014/200/2/removing-interview-medical-school-selection-associated-gender-bias-among
- 18. Laven G, Wilkinson D. Rural doctors and rural backgrounds: how strong is the evidence? A systematic review. Aust J Rural Hlth 2003; 11: 277–284.
- 19. Ogden J, Preston S, Partanen RL, et al. Recruiting and retaining general practitioners in rural practice: systematic review and meta‐analysis of rural pipeline effects. Med J Aust 2020; 213: 228–236. https://www.mja.com.au/journal/2020/213/5/recruiting-and-retaining-general-practitioners-rural-practice-systematic-review
- 20. Ward AM, Kamien M, Lopez DG. Medical career choice and practice location: early factors predicting course completion, career choice and practice location. Med Educ 2004; 38: 239–248.
- 21. White C. General practice wins over hospital medicine in doctors’ future career choice. BMJ 2016; 352: i1239.
- 22. Griffin B, Porfeli E, Hu W. Who do you think you are? Medical student socioeconomic status and intention to work in underserved areas. Adv Health Sci Edu 2017; 22: 491–504.
- 23. Teede HJ. Advancing women in medical leadership. Med J Aust 2019; 211: 392–394. https://www.mja.com.au/journal/2019/211/9/advancing-women-medical-leadership
- 24. Wong CX, Roberts‐Thomson RL. Does practice make perfect? The effect of coaching and retesting on selection tests used for admission to an Australian medical school. Med J Aust 2009; 190: 101–102. https://www.mja.com.au/journal/2008/189/5/does-practice-make-perfect-effect-coaching-and-retesting-selection-tests-used
- 25. Griffin B, Bayl‐Smith P, Hu W. Predicting patterns of change and stability in student performance across a medical degree. Med Educ 2018; 52: 438–446.
Abstract
Objectives: To assess whether the change from the Undergraduate Medical and Health Sciences Admissions Test (UMAT; 1991–2019) to the University Clinical Aptitude Test (UCAT) for the 2020 New South Wales undergraduate medical degree intake was associated with changes in the impact of sex, socio‐economic status and remoteness of residence, and professional coaching upon selection for interview.
Design, setting, participants: Cross‐sectional study of applicants for the three NSW undergraduate medical programs for entry in 2019 (4114 applicants) or 2020 (4270); 703 people applied for both intakes. Applicants selected for interview were surveyed about whether they had received professional coaching for the selection test.
Main outcome measures: Scores on the three sections of the UMAT (2019 entry cohort) and the five subtests of the UCAT (2020 entry); total UMAT and UCAT scores.
Results: Mean scores for UMAT 1 and 3 and for all four UCAT cognitive subtests were higher for men than women; the differences were statistically significant after adjusting for age, socio‐economic status, and remoteness. The effect size for sex was 0.24 (95% CI, 0.18–0.30) for UMAT total score, 0.38 (95% CI, 0.32–0.44) for UCAT total score. For the 2020 intake, 2303 of 4270 applicants (53.9%) and 476 of 1074 interviewees (44.3%) were women. The effect size for socio‐economic status was 0.47 (95% CI, 0.39–0.54) for UMAT, 0.43 (95% CI, 0.35–0.50) for UCAT total score; the effect size for remoteness was 0.54 (95% CI, 0.45–0.63) for UMAT, 0.48 (95% CI, 0.39–0.58) for UCAT total score. The impact of professional coaching on UCAT performance was not statistically significant among those accepted for interview.
Conclusions: Women and people from areas outside major cities or of lower socio‐economic status perform less well on the UCAT than other applicants. Reviewing the test and applicant quotas may be needed to achieve selection equity.