Diabetes, particularly type 2 diabetes, is a global epidemic.1 In Australia, the prevalence of diabetes more than doubled during the past two decades2 and the number of people with diabetes is projected to reach 2 million in 2025.3
Progression to manifest type 2 diabetes in people with impaired glucose tolerance or impaired fasting glucose can be prevented or delayed by lifestyle and pharmaceutical interventions.4 However, using the oral glucose tolerance test (OGTT) to identify high-risk individuals is impractical at the population level. Furthermore, nearly 40% of incident diabetes arises in people who had normal glucose tolerance 3–5 years earlier.5 Hence, a simple approach to identifying people who are asymptomatic but at risk of developing diabetes would be an advantage.
A number of risk scores for predicting incident diabetes based on self-assessed information have been derived from cohorts in Europe and Asia.6-10 However, the validity and applicability of these tools to the Australian population is questionable as they were derived from circumscribed populations with different risk-factor profiles and ethnicities.
Our aim was to use data from the 5-year follow-up of the Australian Diabetes, Obesity and Lifestyle study (AusDiab) to develop and validate a simple risk score to predict incident diabetes based on demographic, lifestyle and simple anthropometric information. Here, we describe this process.
Since its initial development, the Australian Type 2 Diabetes Risk Assessment Tool (AUSDRISK) has been translated into a “patient-friendly” version by the Baker IDI Heart and Diabetes Institute on behalf of the Australian, state and territory governments as part of the Council of Australian Governments Reducing the Risk of Type 2 Diabetes initiative.11 The tool was introduced for use in July 2008 and attracts a Medicare rebate.
The AusDiab study has been described in detail elsewhere.12,13 The baseline study (1999–2000) was a cross-sectional, national, population-based survey of 11 247 adults aged 25 years or older from 42 randomly selected census collection districts, six in each state and the Northern Territory. This represented 55% of those who completed an initial household interview. The number of participants in each state or territory was 1848 (Tasmania), 1796 (South Australia), 1634 (Queensland), 1561 (Western Australia), 1515 (New South Wales), 1459 (Northern Territory) and 1434 (Victoria). More than 85% of participants were born in Australia, New Zealand or the United Kingdom.
In 2004–2005, of 10 788 surviving AusDiab participants who were eligible for follow-up, 6537 (61%) returned for examination, and another 2261 (21%) completed a telephone questionnaire. The incidence of self-reported diabetes, after adjusting for age and sex, was the same in the 6537 attendees as in those who completed only the telephone questionnaire.13 However, the 6537 attendees were less likely to have a lower level of education or to be smokers, and had a smaller waist circumference at baseline.13 Among the 6537 attendees, we excluded 229 with physician-diagnosed diabetes at baseline, 246 whose diabetes status was not classifiable either at baseline or at follow-up, one with inadequate fasting, and a woman who was pregnant at baseline, leaving 6060 (2757 men, 3303 women) for the analysis.
The study was approved by the Ethics Committee of the International Diabetes Institute and the Standing Committee on Ethics in Research involving Humans, Monash University, Melbourne, Victoria. Written informed consent was obtained from all participants.
Data on demographic characteristics, dietary and alcohol consumption, smoking status, physical activity, television viewing time, parental history of diabetes and medical history were collected by an interviewer-administered questionnaire.12 Ethnicity was classified using country of birth, and Aboriginal and Torres Strait Islander heritage was ascertained by a separate question. Self-reported occupation was coded into occupation groups using the Australian Standard Classification of Occupations.14 Participants were coded as physically inactive if the total time engaged in walking (if continuous and ≥ 10 minutes) or in moderate or vigorous activity was less than 150 minutes per week.15 Participants were asked if they had ever been told that they had high blood glucose levels (including during pregnancy for women).
Incident diabetes at follow-up was defined by treatment with insulin or oral hypoglycaemic agents, fasting plasma glucose level ≥ 7.0 mmol/L, or 2-hour plasma glucose level in an OGTT ≥ 11.1 mmol/L.16
Nineteen potential risk factors were considered and converted into categorical variables (Box 1). Different categories of waist circumference were applied to Aboriginal and Torres Strait Islander and Asian people.17 Baseline characteristics of those with and without incident diabetes at the 5-year follow-up were compared by Pearson’s χ2 test. Models for predicting incident diabetes were developed using multiple logistic regression, with all variables with a P < 0.20 after age and sex adjustment included in the initial model, and then variables without statistical or clinical significance excluded in a stepwise manner. Data were missing for key variables for 445 individuals, who were thus excluded from the relevant analysis.
As the number of Aboriginal and Torres Strait Islander people who returned for follow-up was too small (n = 42) to accurately determine the risk of diabetes, the age- and sex-adjusted odds ratio for diabetes in a cross-sectional analysis of the full baseline population (n = 11 247) was used to allocate a risk status. As there were even fewer Pacific Islanders (n = 16), and there are abundant data on the very high diabetes risk in Pacific Islanders,1 we applied the same risk as was calculated for the Aboriginal and Torres Strait Islander group.
Different models for predicting incident diabetes were compared, using the area under the receiver operating characteristic (ROC) curve (AROC),18 the Hosmer–Lemeshow (HL) χ2 statistic (a measure of agreement between predicted and observed events — an HL χ2 statistic < 20 represents good calibration with a P ≥ 0.01),19 the Akaike information criterion (AIC)20 and the Bayesian information criterion (BIC),21 to assess goodness of fit.
A simple scoring system (AUSDRISK) was derived by dividing the β coefficient for each variable in the final model by the lowest β coefficient, then multiplying by 2 and rounding to whole numbers.22
The performance of the AUSDRISK was evaluated with the AROC and HL χ2 statistic in the data from the Blue Mountains Eye Study (BMES)23 and the North West Adelaide Health Study (NWAHS).24 In these studies, blood glucose measurements were available at both baseline and 5-year follow-up examinations.
As the BMES and NWAHS did not include an OGTT, incident diabetes was defined by pharmacological treatment for diabetes or by fasting plasma glucose levels ≥ 7.0 mmol/L at 5-year follow-up. As the NWAHS did not collect data on use of antihypertensive medications, we assumed no participants were taking antihypertensive medications. Similarly, as the BMES did not collect data on a history of high blood glucose level, we assumed that no participants had such a history. The BMES measured body mass index (BMI) but not waist circumference, and so β coefficients for BMI were substituted for waist circumference. These coefficients were derived from the AusDiab cohort, which used BMI instead of waist circumference (as the measure of obesity) and all other risk factors included in the final model. To provide useful information as to how well the AUSDRISK performed in the validation cohorts compared with the AusDiab cohort, we also applied the above restrictions to calculating scores for AusDiab participants.
Analyses were conducted using Stata statistical software, version 10.0 (StataCorp, College Station, Tex, USA).
Among the 6060 participants, 362 (194 men, 168 women) developed incident diabetes over the 5-year follow-up period (Box 1).
People with a southern European or Asian background had similar, elevated, age- and sex-adjusted odds ratios for incident diabetes in comparison with people of Australian or New Zealand origin: 2.38 (95% CI, 1.53–3.70) and 2.13 (95% CI, 1.35–3.36), respectively. The age- and sex-adjusted odds ratio of diabetes (from the baseline cross-sectional data) for those of Aboriginal and Torres Strait Islander origin compared with those of other Australian or New Zealand origin was 3.81 (95% CI, 1.85–7.85). Therefore, people of southern European, Asian, Aboriginal and Torres Strait Islander and Pacific Islander background were combined into a single, high-risk ethnic group.
Thirteen variables were found to be independent predictors of incident diabetes (Box 2, Model 1). Four of these variables — occupation, education, lipid-lowering treatment and BMI — were removed in successive steps between Model 2 and Model 5 (Box 2). The removal of the first three did not greatly affect the goodness of fit or the discriminative ability of the model (Model 4 compared with Model 1, Box 2). AIC and BIC for Model 4 were close to and lower than those for Model 1, respectively.
The removal of BMI from Model 4 resulted in a decrease in AROC of only 0.003, with excellent calibration (Model 5, HL χ2 statistic 4.1) (Box 2). The substitution of BMI for waist circumference resulted in a model (Model 6) with lower discrimination than the final model with waist circumference (Model 5).
Box 3 shows the β coefficients for the final model and the points allocated to each risk factor category. The AROC for the final model, using only waist circumference as the measure of obesity, was 0.783 (95%CI, 0.759 to 0.806). The AUSDRISK score varied from 0 to 35. A score of 12 or higher corresponded to the point on the ROC curve at which sensitivity (74.0%) plus specificity (67.7%) were maximised, with a positive predictive value (PPV) of 12.7% (Box 4).
Of the 10 106 participants without diagnosed diabetes at baseline, 445 (4.4%) were classified at baseline as having undiagnosed diabetes based on OGTT results. The AROC of the AUSDRISK for identifying undiagnosed diabetes in this context was 0.781 (95% CI, 0.762–0.801).
Of the 1993 BMES participants with data available for validation of AUSDRISK, 109 had incident diabetes after 5 years of follow-up (5.5%). The modified AUSDRISK (with BMI substituted for waist circumference as the measure of obesity) was well calibrated in the BMES cohort (HL χ2 statistic, 9.2; P = 0.32). However, it had less discrimination in predicting incident diabetes, with an AROC of 0.66 (95% CI, 0.60–0.71), lower than the AROC of 0.75 (95% CI, 0.72–0.78) in the AusDiab cohort when the same definition of incident diabetes and the same modified AUSDRISK were applied.
Of the 1465 NWAHS participants, 41 had incident diabetes within 5 years of follow-up (2.8%). The AROC for the AUSDRISK in the NWAHS cohort was similar to the AROC for the AusDiab cohort when the same definition for incident diabetes and the same modified AUSDRISK were applied (0.79 [95% CI, 0.72–0.86] v 0.79 [95% CI, 0.76–0.82]), but calibration was poor (HL χ2 statistic, 29.4; P < 0.001).
We have developed an Australian type 2 diabetes risk assessment tool (AUSDRISK) for predicting 5-year risk of diabetes based on nine risk factors that are either known or easily self-assessed — age, sex, ethnicity, parental history of diabetes, history of high blood glucose level, use of antihypertensive medications, smoking, physical inactivity and waist circumference. This tool is unique with respect to the inclusion of ethnicity, and ethnic- and sex-specific cut-points for waist circumference. This not only captures the variation in risk of diabetes caused by ethnic differences but also suggests that AUSDRISK has broader application for different Australian ethnic populations. Validation analysis showed good discrimination and acceptable calibration properties.
Ethnicity has been previously found to be a significant predictor for incident diabetes and thus was included in the models from the Atherosclerosis Risk in Communities study25 and the San Antonio Heart Study26 in which African Americans and Mexican Americans were assigned extra risk score points, respectively.
In the diagnostic criteria of the metabolic syndrome proposed by the International Diabetes Federation, ethnic- and sex-specific thresholds for waist circumference are recommended to define central adiposity.17 This approach was adopted in our analysis to reflect the fact that the risk of metabolic disorders in Aboriginal and Torres Strait Islander or Asian people is associated with a smaller waist circumference than in people of Europid origin.
As we aimed to provide a simple diabetes risk score, several variables with statistical significance were excluded. Occupation was removed despite one category (home duties) showing a statistically significant relationship with incident diabetes; this was an unexpected finding, for which there is no support in previously published literature, and probably reflects an artefact of the AusDiab data rather than a real phenomenon. Education was excluded as it was of borderline significance and did not contribute greatly to the model. Lipid-lowering treatment was then removed because only a small percentage of AusDiab participants reported using lipid-lowering therapy at baseline, which is unlikely to reflect current clinical practice.
We also considered removing BMI from the model. First, as the obese category of BMI (30 – < 35 kg/m2) had only borderline statistical significance, and only the morbidly obese category of BMI (≥ 35 kg/m2) clearly entered the model, we felt its inclusion added unnecessary complexity. Secondly, the inclusion of only obese or morbidly obese categories of BMI could send an inappropriate public health message, suggesting that those who are overweight (25 – < 30 kg/m2) are not at elevated risk of diabetes. Measuring waist circumference is increasingly promoted as a public health tool, but it may be difficult for individuals to accurately measure their own waist circumference. Therefore, we tested the substitution of BMI for waist circumference but found a loss of discriminative power. Further, although weight and height seem more straightforward, accurate scales and measures are not routinely available to the general public, many people are likely to rely on remembering an earlier measurement, and the conversion of weight and height into BMI (even with appropriate tables) is likely to lead to errors.
The discriminative ability we found for the AUSDRISK in the derivation cohort was higher than that reported for some other published risk scores that use similar information, including the European Prospective Investigation into Cancer and Nutrition (EPIC)-Norfolk (AROC, 0.76),6 the Thailand study (AROC, 0.75)7 and the Data from an Epidemiological Study on the Insulin Resistance Syndrome (DESIR) study in men (AROC, 0.71).8 However, it was lower than that reported in the Finnish FINDRISC study (AROC, 0.85),9 the EPIC-Potsdam study (AROC, 0.84)10 and the DESIR study in women (AROC, 0.83).8
If an AUSDRISK score of 12, the point at which sensitivity and specificity were maximised on the ROC curve, was selected as the threshold for increased risk of incident diabetes, then the probability of developing incident diabetes during 5-year follow-up was 12.7%. This is close to the PPV of the FINDRISC score (13.0%).9 However, the choice of a threshold at which to intervene with a diabetes prevention program depends not only on sensitivity and specificity, but also on the cost and feasibility of the program. Therefore, we provided data on the performance of the AUSDRISK at different cut-points.
In addition to its predictive value, the AUSDRISK performed well in discriminating between those who had and those who did not have undiagnosed diabetes in the cross-sectional baseline AusDiab study.
Finally, validation studies showed that the discriminative ability of the AUSDRISK was very good in the NWAHS but only moderate in the BMES. This may be due, in part, to the limited age range of participants in the BMES, which recruited only people older than 49 years. A score developed specifically for a population with a limited age range is likely to give less weight to age categories than a score derived from a population with a wider age range. The AUSDRISK was well calibrated in the BMES cohort but was only reasonable for the NWAHS. The less satisfactory calibration might be related to the lower incidence of diabetes in the NWAHS (2.8%).
This study had some limitations. As it included only people aged over 25 years at baseline, the use of the AUSDRISK in younger age groups would probably overestimate their risk of diabetes. Secondly, as too few individuals of Aboriginal and Torres Strait Islander and Pacific Islander origin were followed up to accurately determine diabetes risk for this group, they were combined with southern European and Asian people to generate a high-risk ethnic group. This approach might not be sufficiently accurate to capture the excess risk of diabetes in these two populations. Finally, the AUSDRISK was developed for predicting incident diabetes, not impaired glucose tolerance or impaired fasting glucose. Examining its ability to predict these two conditions is beyond the scope of the current analysis.
As noted in the introduction, the AUSDRISK has been converted into a points-based, patient-friendly questionnaire11 (available at http://www.bakeridi.edu.au/aus_diabetes_risk) and an online interactive risk assessment tool (available at http://health.gov.au/internet/main/publishing.nsf/Content/diabetesriskassessmenttool). These versions include a risk factor pertaining to fruit and vegetable consumption, which was not a significant predictor of diabetes in the final model but was added for its value as a public health message; one point is allocated for those who consume less than one serve of fruit or vegetable per day. The AUSDRISK was adopted for use by the Australian Government Department of Health and Ageing in July 2008 and attracts a Medicare rebate (Medicare Benefits Schedule item 713) for its application in people aged 40–49 years.
In conclusion, the AUSDRISK provides a valid and reliable method to estimate the risk of developing type 2 diabetes and also to identify asymptomatic individuals who are likely to have undiagnosed diabetes in cross-sectional settings.
1 Baseline characteristics (potential risk factors for developing diabetes) of the AusDiab cohort according to diabetes status at follow-up, and age- and sex-adjusted odds ratios for incident diabetes
2 Performance of models for predicting incident diabetes in the 5-year follow-up of the AusDiab study
3 Beta coefficients from the multiple logistic regression final model predicting incident diabetes, and points allocated to each component of the AUSDRISK score
Received 21 January 2009, accepted 9 September 2009
- Lei Chen1
- Dianna J Magliano2
- Beverley Balkau2,3
- Stephen Colagiuri4
- Paul Z Zimmet2
- Andrew M Tonkin1
- Paul Mitchell5
- Patrick J Phillips6
- Jonathan E Shaw2
- 1 Department of Epidemiology and Preventive Medicine, Monash University, Melbourne, VIC.
- 2 Baker IDI Heart and Diabetes Institute, Melbourne, VIC.
- 3 Centre for Research in Epidemiology and Public Health, Institut National de la Santé et de la Recherche Médicale, University Paris-Sud, Villejuif, France.
- 4 Institute of Obesity, Nutrition and Exercise, University of Sydney, Sydney, NSW.
- 5 Centre for Vision Research, Westmead Millennium Institute, University of Sydney, Sydney, NSW.
- 6 Department of Endocrinology, Queen Elizabeth Hospital, Adelaide, SA.
The AUSDRISK was developed by the Baker IDI Heart and Diabetes Institute on behalf of the Australian, state and territory governments as part of the Council of Australian Governments (COAG) Reducing the Risk of Type 2 Diabetes initiative.
Funding for the AUSDRISK instrument was provided by the COAG as part of the Reducing the Risk of Type 2 Diabetes initiative. Oversight of this work was provided by a management committee comprising diabetes expertise and jurisdictional representation. Other supportive funding for the AusDiab study is listed in the Acknowledgements.
- 1. Zimmet P, Alberti KGMM, Shaw J. Global and societal implications of the diabetes epidemic. Nature 2001; 414: 782-787.
- 2. Dunstan DW, Zimmet PZ, Welborn TA, et al. The rising prevalence of diabetes and impaired glucose tolerance: the Australian Diabetes, Obesity and Lifestyle Study. Diabetes Care 2002; 25: 829-834.
- 3. Magliano DJ, Shaw JE, Shortreed SM, et al. Lifetime risk and projected population prevalence of diabetes. Diabetologia 2008; 51: 2179-2186.
- 4. Hussain A, Claussen B, Ramachandran A, Williams R. Prevention of type 2 diabetes: a review. Diabetes Res Clin Pract 2007; 76: 317-326.
- 5. Unwin N, Shaw J, Zimmet P, Alberti KG. Impaired glucose tolerance and impaired fasting glycaemia: the current status on definition and intervention. Diabet Med 2002; 19: 708-723.
- 6. Simmons RK, Harding AH, Wareham NJ, Griffin SJ. Do simple questions about diet and physical activity help to identify those at risk of type 2 diabetes? Diabet Med 2007; 24: 830-835.
- 7. Aekplakorn W, Bunnag P, Woodward M, et al. A risk score for predicting incident diabetes in the Thai population. Diabetes Care 2006; 29: 1872-1877.
- 8. Balkau B, Lange C, Fezeu L, et al. Predicting diabetes: clinical, biological, and genetic approaches: data from the Epidemiological Study on the Insulin Resistance Syndrome (DESIR). Diabetes Care 2008; 31: 2056-2061.
- 9. Lindstrom J, Tuomilehto J. The diabetes risk score: a practical tool to predict type 2 diabetes risk. Diabetes Care 2003; 26: 725-731.
- 10. Schulze MB, Hoffmann K, Boeing H, et al. An accurate risk score based on anthropometric, dietary, and lifestyle factors to predict the development of type 2 diabetes. Diabetes Care 2007; 30: 510-515.
- 11. Australian Government Department of Health and Ageing. The Australian Type 2 Diabetes Risk Assessment Tool. Canberra: DoHA, 2008. http://www.health.gov.au/internet/main/publishing.nsf/Content/62703F3AFCA6268 BCA25769400199781/$File/austool.pdf (accessed Jan 2010).
- 12. Dunstan DW, Zimmet PZ, Welborn TA, et al. The Australian Diabetes, Obesity and Lifestyle Study (AusDiab) — methods and response rates. Diabetes Res Clin Pract 2002; 57: 119-129.
- 13. Magliano DJ, Barr EL, Zimmet PZ, et al. Glucose indices, health behaviors, and incidence of diabetes in Australia: the Australian Diabetes, Obesity and Lifestyle Study. Diabetes Care 2008; 31: 267-272.
- 14. Australian Bureau of Statistics. Australian Standard Classification of Occupations (ASCO). 2nd ed. Canberra: ABS, 1997. (ABS Cat. No. 1220.0.) http://www.abs.gov.au/Ausstats/abs@.nsf/0/B70D48AB5ACC56FECA25692600 235A67?Open (accessed Dec 2008).
- 15. Australian Institute of Health and Welfare. The Active Australia Survey: a guide and manual for implementation, analysis and reporting. Canberra: AIHW, 2003. (AIHW Cat. No. CVD 22.) http://www.aihw.gov.au/publications/index.cfm/title/8559 (accessed Dec 2008).
- 16. World Health Organization Consultation. Definition and diagnosis of diabetes mellitus and intermediate hyperglycemia: report of a WHO/IDF consultation. Geneva: WHO, 2006.
- 17. International Diabetes Federation. The IDF consensus worldwide definition of the metabolic syndrome. Brussels: International Diabetes Federation, 2006.
- 18. Hanley JA, McNeil BJ. The meaning and use of the area under a receiver operating characteristic (ROC) curve. Radiology 1982; 143: 29-36.
- 19. Hosmer DW, Hjort NL. Goodness-of-fit processes for logistic regression: simulation results. Stat Med 2002; 21: 2723-2738.
- 20. Tobias A, Campbell MJ. Akaike’s information criterion and Schwarz’s criterion. Stata Tech Bull 1998; 45: 23-25.
- 21. Raftery AE. Bayesian model selection in social research. In: Marsden PV, editor. Sociological methodology. Oxford: Blackwell, 1996: 111-163.
- 22. Sullivan LM, Massaro JM, D’Agostino RB. Presentation of multivariate data for clinical use: the Framingham Study risk score functions. Stat Med 2004; 23: 1631-1660.
- 23. Cugati S, Wang JJ, Rochtchina E, Mitchell P. Ten-year incidence of diabetes in older Australians: the Blue Mountains Eye Study. Med J Aust 2007; 186: 131-135. <MJA full text>
- 24. Grant JF, Chittleborough CR, Taylor AW, et al. The North West Adelaide Health Study: detailed methods and baseline segmentation of a cohort for selected chronic diseases. Epidemiol Perspect Innov 2006; 3: 4.
- 25. Schmidt MI, Duncan BB, Bang H, et al. Identifying individuals at high risk for diabetes: the Atherosclerosis Risk in Communities study. Diabetes Care 2005; 28: 2013-2018.
- 26. Stern MP, Williams K, Haffner SM. Identification of persons at high risk for type 2 diabetes mellitus: do we need the oral glucose tolerance test? Ann Intern Med 2002; 136: 575-581.
Abstract
Objective: To develop and validate a diabetes risk assessment tool for Australia based on demographic, lifestyle and simple anthropometric measures.
Design and setting: 5-year follow-up (2004–2005) of the Australian Diabetes, Obesity and Lifestyle study (AusDiab, 1999–2000).
Participants: 6060 AusDiab participants aged 25 years or older who did not have diagnosed diabetes at baseline.
Main outcome measures: Incident diabetes at follow-up was defined by treatment with insulin or oral hypoglycaemic agents or by fasting plasma glucose level ≥ 7.0 mmol/L or 2-hour plasma glucose level in an oral glucose tolerance test ≥ 11.1 mmol/L. The risk prediction model was developed using logistic regression and converted to a simple score, which was then validated in two independent Australian cohorts (the Blue Mountains Eye Study and the North West Adelaide Health Study) using the area under the receiver operating characteristic curve (AROC) and the Hosmer–Lemeshow (HL) χ2 statistic.
Results: 362 people developed diabetes. Age, sex, ethnicity, parental history of diabetes, history of high blood glucose level, use of antihypertensive medications, smoking, physical inactivity and waist circumference were included in the final prediction model. The AROC of the diabetes risk tool was 0.78 (95% CI, 0.76–0.81) and HL χ2 statistic was 4.1 (P = 0.85). Using a score ≥ 12 (maximum, 35), the sensitivity, specificity and positive predictive value for identifying incident diabetes were 74.0%, 67.7% and 12.7%, respectively. The AROC and HL χ2 statistic in the two independent validation cohorts were 0.66 (95% CI, 0.60–0.71) and 9.2 (P = 0.32), and 0.79 (95% CI, 0.72–0.86) and 29.4 (P < 0.001), respectively.
Conclusions: This diabetes risk assessment tool provides a simple, non-invasive method to identify Australian adults at high risk of type 2 diabetes who might benefit from interventions to prevent or delay its onset.