Getting it Right: validating a culturally specific screening tool for depression (aPHQ‐9) in Aboriginal and Torres Strait Islander Australians

The burden of disease for mental and substance use disorders, in terms of disability‐adjusted life years (DALYs), was the third highest of all diseases in Australia in 2011.1 Major depression, a chronic and relapsing disorder, impairs cognitive and emotional functioning, has substantial social and economic impacts, and increases the risk of premature death.2 Evidence‐based management of people with depression in primary care is beneficial for their health,3 but the rates of detection, diagnosis and effective intervention are inadequate.4 High quality primary care investigations of this problem have been undertaken in the United Kingdom and the United States,3 but detection of depression in Aboriginal and Torres Strait Islander people (Indigenous Australians) in primary care has been little investigated.

A recent systematic review of diagnostic psychiatric instruments found that none had been formally validated for Indigenous Australians.5 To rectify the paucity of Indigenous Australian‐specific depression research, a culturally adapted depression screening tool validated in multiple Australian states and territories is needed. The nine‐item Patient Health Questionnaire (PHQ‐9)6 has been used for nearly two decades as a screening tool for depression and for assessing symptom severity in a wide range of cultural settings, but lacked face validity for use in Indigenous Australian communities.7 The PHQ‐9 text has been re‐worded in “Aboriginal English”, and the adapted instrument (aPHQ‐9) was found to be internally consistent in a study with a community sample of 78 Aboriginal men (Cronbach α = 0.776) and women (α = 0.767) from central Australia.8

The objective of the Getting it Right study was to determine the validity of the aPHQ‐9 as a tool for screening Indigenous people attending primary health care services for depression, comparing it with the standard tool, the MINI International Neuropsychiatric Interview (MINI) 6.0.0.9

While adapting the aPHQ‐9 for use with people from five Aboriginal language groups, seven key features of depression in Indigenous Australian men not covered by the aPHQ‐9 were identified: anger, weakened spirit, homesickness, irritability, excessive worry, rumination, and drug or alcohol use.8 Additional questions were developed for assessing these features; we will report our findings regarding these questions in a separate article.

Methods

Study design and participants

Getting it Right was a prospective, observational diagnostic accuracy study undertaken in ten Indigenous primary health care services in the Australian Capital Territory, New South Wales (four sites), the Northern Territory (two sites), Queensland, South Australia, and Western Australia. The protocol10 was conceived and designed in accordance with the principles of reciprocity, respect, equality, responsibility, survival and protection, and spirit and integrity.11 The study was coordinated by the George Institute for Global Health in Sydney.

Participants were recruited between 25 March 2015 and 2 November 2016. People were eligible for the study if, at the time of their presentation to a participating health service or health service event, they were at least 18 years of age, identified as Indigenous Australians, were able to communicate sufficiently to respond to the questionnaire and interview questions, and gave informed consent. People with a diagnosis of psychosis or bipolar disorder were excluded. Trained staff members at each service were asked to screen all people attending the service on recruitment days and to record written or verbal informed consent for those who agreed to participate. At two services, staff members did not always recruit consecutive patients, sometimes selecting as potential candidates people they had met previously and believed were more likely to participate.

Study outcomes

We assessed the criterion validity of the aPHQ‐9. The reference criterion standard was a diagnosis of depression with the MINI 6.0.0,9 a structured interview for the major Diagnostic and Statistical Manual of Mental Disorders (DSM‐IV) Axis I psychiatric disorders; we removed the bereavement exclusion criterion for major depression, as foreshadowed for DSM‐5. The MINI, which can be modularised and administered by clinicians and lay interviewers after appropriate training, is the most widely used structured psychiatric diagnostic interview instrument, having been validated in more than 100 countries. The interview and algorithm provide the dichotomous categories “current major depressive episode” and “no current major depressive episode.”

Procedures

In the first assessment, a trained (as outlined in the protocol10), culturally competent staff member from the primary health care service interviewed each participant, using a printed or electronic questionnaire during a face‐to‐face interview (or, if necessary, by telephone). At the discretion of the interviewer and participant, participants either directly answered the eleven aPHQ‐9 questions (numbered 1–4, 5a, 5b, 6, 7, 8a, 8b, 9; response options: not at all, several days, more than half the days, nearly every day), seven additional questions, questions about the acceptability and ease of use of the aPHQ‐9, and questions on demographic details, or the questionnaire was administered by the interviewer in English or the appropriate Aboriginal or Torres Strait Island language. All data were entered into a secure online study database.

Within seven days of the first assessment, a local, trained member of staff who had not participated in and was blind to the results of the initial assessment administered the major depressive episode/disorder (current or recurrent), generalised anxiety disorder (past 6 months), and post‐traumatic stress disorder (past month) modules of the MINI in face‐to‐face interviews (or, if necessary, by telephone).

Each primary health care service had protocols for the follow‐up and care of study participants presenting with depression, deliberate self‐harm, or suicidal ideation or intent. If a participant had a psychiatric disorder, their general practitioner was encouraged to arrange for re‐assessment, treatment, or formal referral according to their clinical judgement.10

Statistical methods

Sample size: All analyses were conducted in R 3.3.2 (R Project), and required sample sizes were calculated with the package samplingbook (https://CRAN.R-project.org/package=samplingbook). Assuming a prevalence of major depressive episode (as assessed with the MINI) of 10% and a true sensitivity of 0.85, a sample size of 500 participants was required to achieve a precision of 0.1 for the sensitivity 95% confidence interval (CI). Assuming a prevalence of 10% and a true specificity of 0.75, 500 participants were similarly required to achieve a precision of 0.04 for the specificity CI.

Data analysis: Categorical data were summarised as frequencies and percentages, continuous variables as means and standard deviations (SDs) or medians and interquartile ranges (IQRs); proportions were compared in χ² tests, means in t tests. We computed the area under the receiver operating characteristic (ROC) curve to estimate the discrimination of the aPHQ‐9. Sensitivities and specificities using different aPHQ‐9 thresholds were computed with a generalised estimation equation (GEE), using a logit link and exchangeable working covariance matrix to account for clustering of participants by centre. P < 0.05 was deemed statistically significant.

Primary analysis: The validity of the aPHQ‐9 (compared with the MINI) was assessed with two common criteria for a major depressive episode:

• the algorithm scoring method, aligned with DSM‐IV diagnostic criteria; major depressive episode was detected if the responses to questions 1 or 2 and five or more of questions 1‐4, 5a or 5b, 6, 7, 8a or 8b, and 9 were at least “more than half the days” (for question 9: at least “several days”)7,12,13 and
• a total score of 10 points or more, similar to the cut‐point for the original PHQ‐9 as a screening tool.13,14

The original PHQ‐9 scoring method was used, except that each of the two split questions (questions 5 and 8 in the original PHQ‐9) were scored once only and the higher score retained. The properties of other cut‐points were explored by constructing ROC curves. Sensitivity and specificity were computed for subgroups (eg, people with a chronic disease) by logistic regression, allowing adjustment for demographic differences.

Missing data: Three participants each missed single aPHQ‐9 questions (none was the question about suicidal ideation or intent). We computed a partial score for these participants by summing scores for the answered questions, and multiplied it by 9/8 to derive their global scores.

Ethics approval

The study was approved by the University of Sydney Human Research Ethics Committee (HREC) (reference, 2014/361), the Aboriginal Health and Medical Research Council of NSW HREC (reference, 1044/14), the ACT Health HREC (reference, ETH.8.14.207), the Queensland Health Metro South HREC (reference, HREC/14/QPAH/503), the Central Australian HREC (reference, HREC‐15‐287), the Menzies School of Health Research HREC (reference, 2014‐2289), the Aboriginal Health Council of South Australia Aboriginal Health Research Ethics Committee (reference, 04‐15‐622), and the Western Australian Aboriginal Health Ethics Committee (reference, 607). Each participating health service also approved the conduct of Getting it Right at their service.

Results

Ten of the 34 primary health care services invited to participate in Getting it Right agreed to do so. Reasons for non‐participation included insufficient staff capacity, having other research interests, or failure to respond to multiple contact attempts. Initial decisions about participation were made by staff in the chief executive office, the social and emotional wellbeing team, general practitioners, research staff, or clinical managers.

Between 25 March 2015 and 2 November 2016, 913 people were screened for eligibility, of whom 533 provided informed consent; 530 participants completed the aPHQ‐9, of whom 500 also completed the clinical MINI interview (Box 1). There were no differences in baseline characteristics between these participants and the 30 who did not complete the PHQ‐9 and MINI (data not shown).

Most participants (485, 97%) identified as Aboriginal Australians; ten (2%) identified as Torres Strait Islanders and five (1%) as both. The mean age of participants was 43 years (SD, 15 years; range, 18–80 years), 267 were women (53%) and 300 were the main income earners in their households (60%) (Box 2). A previous diagnosis of depression was reported by 216 (45%) and anxiety by 160 participants (33%) (Box 3). Most participants (347, 69%) had been told at some point by a doctor or other health professional that they had at least one of the pre‐specified chronic health conditions; 74 (15%) reported four or more pre‐specified chronic conditions, while 105 participants (21%) reported a health problem that restricted activities of daily living in the two months before the study (Box 2).

The prevalence of a current major depressive episode according to the MINI criterion was 22% (95% CI, 18–25%), of generalised anxiety disorder 21% (95% CI, 18–25%), and of post‐traumatic stress disorder 11% (95% CI, 8–14%). No MINI diagnosis was made for 347 participants (69%), while 27 participants (5%) met diagnostic criteria for all three conditions (Box 4). There were statistically significant associations between having a current major depressive episode and arthritis, asthma, obstructive sleep apnoea, having an illness that restricted activities of daily living in the preceding two months, and having been previously diagnosed with depression or anxiety (Box 2, Box 3).

The internal consistency of the aPHQ‐9 questions was very good (Cronbach α = 0.88). Problems with sleeping were the most frequently reported aPHQ‐9 item; 189 respondents (38%) found it hard to sleep at night or had other problems with sleeping at least “more than half the days”. Thoughts of self‐harm or killing oneself (a little bit, most of the time, or all the time) were reported by 78 participants (16%), including two who felt this way all the time. The reporting of other symptoms, most or all the time, ranged from 19% to 31% (data not shown).

The sensitivity of the aPHQ‐9 DSM‐IV algorithm method (criterion I) for diagnosing a current major depressive episode was 54% (95% CI, 40–68%), its specificity 91% (95% CI, 88–94%), and the positive predictive value (PPV) 64%.

For screening for a current major depressive episode (criterion II), the area under the ROC curve was 0.88 (95% CI, 0.85–0.92). The sensitivity with a cut‐point of 10 was 84% (95% CI, 74–91%) and the specificity 77% (95% CI, 71–83%); with a cut‐point of 9, the sensitivity was 87% (95% CI, 78–93%) and the specificity 72% (95% CI, 66–77%), and with a cut‐point of 11 the sensitivity was 81% (95% CI, 79–89%) and the specificity 82% (95% CI, 77–87%) (Box 5, Box 6). The estimates were nearly identical if the three incomplete aPHQ‐9 questionnaires were excluded from the analysis (data not shown).

Feedback from participants about the acceptability of the aPHQ‐9 was predominantly positive, but 65 respondents (13%) felt that some or all questions were too personal (Box 7).

Discussion

In a heterogeneous primary health care population of Indigenous Australian adults across six Australian states and territories, we found that the performance of the aPHQ‐9 for screening for depression, with a cut‐point of 10 points, was good; in primary care validation studies of the standard PHQ‐9, 10 points was also considered the optimal cut‐point.14 The best positive predictive value for detecting a major depressive episode (64%) was obtained when using the DSM‐based diagnostic scoring algorithm, although sensitivity was low (54%), consistent with other reports on the algorithm approach.13

The 22% point prevalence of a major depressive episode in our primary health care‐based study is similar to that reported for other Australian general practice populations15 and higher than that reported in similar studies of Indigenous primary care patients,16,17 suggesting our recruitment method did not cause selection bias. The generalisability of our findings is strengthened by the participation of ten heterogeneous primary health care services across Australia; the participants were not involved in the adaptation of the PHQ‐9, and they regarded the aPHQ‐9 as being acceptable. Two earlier validation studies of culturally adapted depression screening tools for Indigenous Australians were conducted in the same communities in which the original screening tools had been modified, which may have limited the generalisability of their results, given the cultural and linguistic diversity of Indigenous Australian communities.16,17 No alternative culturally specific screening or assessment tools for assessing depression in Indigenous Australians were identified in a recent systematic review.18

We completed structured training for site staff, achieved high rates of interview completion, recruited an adequate number of participants with a MINI major depressive episode diagnosis to enable subgroup analyses, and complied with the National Health and Medical Research Council guidelines for Indigenous health research.11 Ideally, our criterion standard would have been a semi‐structured, culturally valid psychiatric interview, but such a diagnostic assessment is not available.5 However, the interviews were conducted by local, culturally aware clinicians.

Neither the Royal Australian College of General Practitioners (RACGP) national guide for preventive health assessment of Aboriginal and Torres Strait Islander people19 nor the Royal Australian and New Zealand College of Psychiatrists clinical practice guideline for mood disorders20 recommends universal screening for depression of people attending primary care services, as stand‐alone screening programs have little or no benefit for improving the detection and management of depression.21 Similar concerns were expressed when Google included a link to the original PHQ‐9 for people who searched with “am I depressed?” or related questions.22

The aPHQ‐9 screening specificity of 77% (95% CI, 71–83%) and negative predictive value of 95% indicate that it reliably differentiates between people who require further assessment of their social and emotional wellbeing and people unlikely to have depression. The aPHQ‐9 is a free, easy to administer, and culturally acceptable tool for initiating discussions with Indigenous people about their mood, consistent with recommendations by the RACGP national guideline and the Central Australian Rural Practitioners Association (CARPA) manual23 to employ the aPHQ‐9 for screening Indigenous people at high risk of depression when culturally competent, locally knowledgeable practitioners have the resources for providing further evaluation and guideline‐based treatment.

The aPHQ‐9 cannot replace careful assessment and diagnosis, nor should it be used to determine the need for treatment. Even at the highest positive predictive value in our study, one‐third of people identified with the aPHQ‐9 as having a major depressive episode would not have major depression according to assessment with the MINI, and, conversely, we would miss some people who had major depression. Determining the consistency (test–retest reliability) and inter‐rater reliability of the aPHQ‐9 are the next steps for ensuring that the aPHQ‐9 provides consistent results, regardless of who administers the test.

Apart from screening and diagnosis, assessments for depression may be used in epidemiology studies, treatment monitoring, and outcome assessment. We do not yet know the responsiveness of the aPHQ‐9 scores to treatment of patients. As the evidence base for screening for depression increases, we must develop culturally appropriate, cost‐effective interventions for preventing, treating and managing depression in Indigenous Australians.

Box 1 – Flow of participants through the Getting it Right study

---

aPHQ‐9 = adapted Patient Health Questionnaire. * Positive result (possible major depressive episode): responses to questions 1 or 2 and five or more of questions 1‐4, 5a or 5b, 6, 7, 8a or 8b, and 9 were at least “more than half the days”’ (for question 9: at least “several days”). † Determined with the current major depressive episode module of the MINI International Neuropsychiatric Interview (MINI). ◆

Box 4 – Proportions of the 500 participants in the Getting it Right study diagnosed with major depressive episode (MDE), generalised anxiety disorder (GAD) or post‐traumatic stress disorder (PTSD) with the Mini‐International Neuropsychiatric Interview (MINI)

---

 

Box 6 – Receiver operating characteristic (ROC) curve for the aPHQ‐9 score

---

aPHQ‐9 = adapted Patient Health Questionnaire. The shaded region represents the 95% confidence region for the curve. The scoring cut‐points (8, 9, 10, 11, 12) are indicated with their respective cross‐type 95% confidence intervals (CIs). Area under the ROC curve: 88.3% (95% CI, 84.8–91.7%). ◆