Changes in psychological distress and psychosocial functioning in young people visiting headspace centres for mental health problems

Improving the mental health and wellbeing of adolescents and young adults is receiving increasing attention throughout the world.1 The Australian Government was the first to invest significant funds in a practical and systematic response to this challenge, initiating a national reform process that created new service platforms for young people through its founding of headspace, the National Youth Mental Health Foundation.2

The initiative commenced in 2006, establishing an initial 10 centres and is set to increase to a network of 100 centres across Australia by 2016. headspace centres are one-stop entry points offering a mix of the services that young people need most. Centres provide early intervention by responding to early presentations of mental health problems and by assisting young people at greater risk of developing mental disorders. Being youth-friendly and non-stigmatising are priorities, and centre activities are founded on youth participation and engagement at all levels.3

From the beginning, the headspace initiative has evaluated its activities, despite the significant challenges inherent in determining the outcomes of such a complex, long-term, real-world, system-wide intervention. A preliminary external evaluation in 2009 showed that young people approved the approach used by the initial centres.4 At that time, however, it was still too early, in terms of implementation of the headspace initiative, to assess outcomes for the clients.

To facilitate investigation of the impact of the headspace centres, an innovative routine data capture system was introduced in 2013. This system collects information each time a young person accesses a headspace centre for service, and attempts to follow them up after they have finished engaging with the centre. Analysis of the dataset has shown that young people presenting to headspace centres have a wide range of mental health concerns, and are typically in the early stages of the development of a mental disorder.5 Further analyses have explored the types of service young people receive at the centres. In the companion paper to this article, we report that most of the young people seeking help at headspace centres present with mental health concerns, that they generally receive a timely response, and receive assessment and mental health care services. We also found that the initiative is primarily supported by funding from the headspace grant and by the Australian Government Medical Benefits Schedule.6

The current study reports the main clinical outcomes for young people who had presented to headspace centres for mental health concerns. The primary aim was to determine the extent to which psychological distress was reduced and psychosocial functioning improved in headspace clients.

Methods

Participants and procedure

Participants were all clients who had commenced an episode of care at a headspace centre for mental health reasons between 1 April 2013 and 31 March 2014. Young people who initially visited headspace for other reasons (situational, physical or sexual health, alcohol or other drug, or vocational reasons) were excluded from analyses. This selection was made because young people presenting with mental health concerns comprise the vast majority of those who seek help at headspace centres and definitely use their mental health care services; young people primarily presenting for other reasons may not have used mental health care services (see the companion paper to this article6). Analyses were limited to a young person's first episode of care during the 12-month data collection period.

The procedure for the routine collection of data provided by the young people and service providers to the headspace Minimum Data Set is described elsewhere.5 Data related to psychological distress were collected from young people immediately before their first, third, sixth, 10th and 15th visits, as well as at follow-up. Measures of psychosocial functioning were recorded by service providers at each occasion of service.

Young people were invited to consent to being followed up when they first attended headspace. They provided an email address, and data were solicited after a 90-day pause in service provision by sending an email with a link to the follow-up questions. Young people could choose to answer these questions electronically, and responses were uploaded into the headspace data warehouse. Ethics approval for the follow-up was obtained from Melbourne Health Quality Assurance Review.

Measures

• The primary presenting concern was categorised according to the clinical presentation features as determined by clinicians. These did not comprise diagnoses, but were rather the main symptoms evident at the initial presentation that were indicative of mental health problems.
• Treatment services were recorded by clinicians, and were categorised as: cognitive behaviour therapy (CBT), interpersonal therapy, acceptance and commitment therapy, psychoeducation (including skills training and relaxation strategies), general and supportive counselling, mindfulness-based therapies, motivational interviewing, problem-solving therapy, and other interventions.
• Client outcomes that were assessed were:
  • the level of psychological distress, based on self-reports according to the 10-item Kessler Psychological Distress Scale (K10);7 and
  • overall psychosocial functioning, assessed by service providers using the Social and Occupational Functioning Assessment Scale (SOFAS).8

Appendix 1 presents the number of clients for whom data were available at key time points.

Statistical analyses

IBM SPSS Statistics 21 was used for statistical analyses. Frequencies of each primary presenting concern were calculated, and age group and sex differences were assessed by χ2 analyses with Bonferroni correction for multiple comparisons.

Changes in each of the outcome measures over time were analysed in two ways.9 First, mixed-model repeated measures analysis of variance (ANOVA) was used to assess aggregate changes over time in K10 and SOFAS scores according to time point, number of service sessions, age group and sex. The statistical relationship between K10 and SOFAS scores was expressed as a Pearson product-moment correlation coefficient (r). Differences between the characteristics of clients who provided follow-up data and those who did not were analysed by logistic regression.

Second, significant change, reliable change and clinically significant change scores were calculated for the K10 and SOFAS data, as increasingly conditional indicators of change. The criterion for significant change was a moderate effect size (0.5) or greater for the degree of change.10 The reliable change index (RCI) (indicating reliable improvement or decline) and clinically significant change index (CSI) (cut-off point at which the person is more likely to belong to a non-clinical rather than a clinical population) were determined using Jacobson and Truax's method.11

For the K10 scores, the RCI was estimated as a 6.73-point change (rounded to 7 points) using reliability coefficients reported for an Australian normative group (age group, 16–24 years) in the 2007 National Survey of Mental Health and Wellbeing.12 Using the same norms, the CSI cut-off was estimated to be 22.56 points (rounded to 23 points). For the SOFAS data, an RCI score of 10 was used; this was based on comparable outpatient psychiatric services data using the Global Assessment of Functioning scale as an equivalent. The CSI for the same comparison group was a score of 69 (Söderberg and Tungström [2006], cited by Falkenström13).

Results

The participants were 24 034 clients from the 55 headspace centres fully operational during the study period. Almost two-thirds of clients were female (62.7%), 36.9% were male and 0.4% were intersex or transgender. The mean age was 17.8 years (SD, 3.3), with 16.7% aged 12–14 years, 35.0% aged 15–17 years, 25.7% aged 18–20 years, and 22.6% aged 21–25 years.

Follow-up data were collected between June 2013 and August 2014. Of the total sample, 20 903 clients (87.0%) were eligible to provide follow-up data; the remaining 13.0% were still receiving headspace services or had not yet had a 90-day service-free period. Only 3.1% of eligible young people (651 clients) responded to the follow-up survey.

Presenting concern and treatment services

The most common mental health problems at initial presentation were depressive symptoms and anxiety, which together accounted for more than two-thirds of presentations. These were the most common presenting reasons for all age/sex groups, with the exception of 12–14-year-old boys, who presented most frequently with anxiety and anger problems and less frequently for depressive symptoms (Appendix 2).

Age and sex differences among those presenting with mental health concerns were indicated by χ2 analysis (χ2 [70] = 3300.57, P < 0.001). The proportions of younger males (12–14 years of age) presenting for anger or behavioural problems was greater than for other age/sex groups. Younger females (12–14 years of age) had higher presentation rates for deliberate self-harm than other groups (Appendix 2).

The most common treatment provided for all primary presenting concerns was CBT; for example, 43.6% of service provided to clients presenting with depressive symptoms involved CBT. A similar pattern of treatments was evident for all primary presenting concerns, with the second most common treatment being supportive counselling (except for borderline personality trait presentations). Psychoeducation was ranked third for most mental health problems (Box 1).

Mean changes in outcomes over time

Changes in the two outcome scores over time are depicted in Box 2 and Box 3. These plot the mean scores at each session that they were recorded, according to the total number of sessions attended. The sample sizes for each point declined as the number of sessions attended increased (Appendix 3). The follow-up data analyses were based on a particularly small sample size; further, no clinician-rated measures were available at this point, as the follow-up was based solely on self-report.

For the change in K10 between initial presentation and last recorded assessment, the factor with the greatest effect size was time, which explained 10.8% of the variance (Appendix 4, ANOVA 1; Box 2). Including the 3-month follow-up in the analysis showed that the time effect remained significant and explained 12.5% of the variance (Appendix 4, ANOVA 2). On average, there was a 3-point improvement in K10 scores from first to last assessment, and a further 3-point improvement from last service to follow-up for the small proportion of young people who provided follow-up data.

It is, however, important to note that the group of clients who provided follow-up data was significantly different from the much larger group of those who did not (χ2 [17] = 153.43, P < 0.001, Nagelkerke R2 = 0.062). Those who provided follow-up data were more likely to be female (odds ratio [OR], 1.63; 95% CI, 1.27–2.11), older (OR, 1.07; 95% CI, 1.04–1.11), have attended a greater number of service sessions (OR, 1.59; 95% CI, 1.39–1.82) and had better psychosocial functioning at exit (OR, 1.03; 95% CI, 1.02–1.05).

For change in SOFAS scores, time was again the strongest factor, but explained only 4.5% of the variance in this outcome measure Appendix 3, ANOVA 3; Box 3).

Significant, reliable and clinically significant change

The percentages of young people showing significant, reliable and clinically significant change between their first and last recorded assessments (not including follow-up) are presented in Box 4. Of the young people for whom data were available, psychological distress was significantly reduced in 36%, was reliably improved in 26%, and clinically significantly improved (by crossing the threshold distinguishing a clinical from a non-clinical population) in 21%. In 13% of clients, K10 scores significantly worsened, and in 8% they reliably deteriorated. According to clinician ratings of psychosocial functioning, significant and clinically significant improvement were each evident for 37% of the assessed clients, while 31% reliably improved. In contrast, function significantly declined in almost a fifth of clients, and reliably declined in 15%.

For 9957 clients, both K10 and SOFAS change data were available. Of these, 59.9% significantly improved and 49.2% reliably improved on at least one of the two scales, while 40.4% of those in the clinical group showed clinically significantly improvement on one or both of the scales.

It is important to note that the K10 and SOFAS scales measure different aspects of mental health, and that psychological distress (K10) was self-reported by young people, while social and occupational functioning (SOFAS) was assessed by a clinician. K10 and SOFAS scores were weakly correlated at presentation (r = − 0.19, P < 0.001) and at final assessment (r = − 0.23, P < 0.001).

There were statistically significant differences between those who improved and those who did not (significant improvement on at least one measure: χ2 [15] = 1168.48, P < 0.001, Nagelkerke R2 = 0.153). Improvement was predicted by greater distress (OR, 1.03; 95% CI, 1.02–1.04) and lower psychosocial functioning (OR, 0.94; 95% CI, 0.94–0.95) at service entry, and by attending a greater number of service sessions (OR, 1.16; 95% CI, 1.10–1.22). Age, sex and primary presenting concern did not predict improvement.

Discussion

This article reports the first outcome data for young people who have accessed the national headspace centre network for mental health problems. The analyses focused on the two key clinical outcomes, psychological distress and psychosocial functioning. The results show that psychological distress was significantly reduced in more than one-third of clients for whom data were available, and psychosocial functioning improved in a similar proportion. If improvement in either measure is considered, 60% of clients experienced significant change. Improvements in young people with greater distress and poorer functioning at service entry were noted in those who engaged well with the service (ie, attended more health care sessions). The findings are consistent with those reported from a single Sydney-based headspace service that found both symptomatic and functional improvements in its clients.14

Comparative data that would help determine whether these outcomes are acceptable are difficult to find. headspace clients present for a wide range of reasons and attend for varying numbers of sessions; although only outcomes for mental health clients were examined here, these young people still constitute a diverse group.6 Comparisons with outcomes from highly controlled clinical studies are therefore inappropriate. A study of psychotherapeutic outcomes in similarly aged young people attending a mental health clinic in the Netherlands, where the clients also presented with a variety of mental health concerns and received varying amounts of service, found that psychosocial functioning reliably improved in 19% of clients.13 This compares with the considerably higher rate of 31% that we have reported.

Comparative Australian data are scarce. Public tertiary mental health services use age bands of 0–17 and 18–64 years in their outcomes reports, and these are not comparable with either the age range of clients in these analyses or with the enhanced primary care service model of headspace. The most recent report from the National Outcomes and Casemix Collection (NOCC), which used the Health of the Nation Outcome Scales (HoNOS) family of outcome measures, showed that 37% of those aged 0–17 years and 24% of those aged 18–64 years using community-based public mental health services reported a significant improvement between the first and last occasions of service.15 The outcomes in young people reported here are similar to the child and adolescent results of the NOCC report, but much better than its findings for adults. However, the degree to which HoNOS outcomes are comparable with K10 and SOFAS scores is unclear, and the lack of directly comparable age groups makes interpretation difficult.

Drawing conclusions from the current study is restricted by several limitations. Primarily, the absence of a control group and other limitations inherent to observational studies means that the changes in scores reported cannot be attributed to headspace care.16 Further, most of the outcome data were derived from the last recorded assessment point for each client, but for many young people this was not at the completion of treatment. Our results are therefore likely to underestimate psychological and psychosocial gains in the course of treatment.

The follow-up rate was disappointing, although wholly expected, and highlights the considerable challenges in persuading young people to provide follow-up information after they have stopped attending for service. Without committing substantial resources to maintaining contact with people after leaving a health service, obtaining longer-term outcomes from real-world interventions will always be a major hurdle. Nevertheless, the headspace initiative has developed a process that attempts to routinely follow up young people after the end of service, and this may be unique in service delivery outside a well resourced prospective clinical trial. Over time, this follow-up database will grow and yield a rich source of information, even though there will be inevitable bias in those who provide follow-up data.

Another limitation is that the data cannot clearly determine the extent to which headspace clients received sufficient and appropriately matched “doses” of evidence-based therapies for different presenting problems and diagnoses, although it is evident that most clients did receive evidence-based therapies. headspace centres differ considerably in both their priorities and their capacity as a result of the diverse community and workforce contexts in which they are embedded,17 although all centres pursue a common vision of youth-focused, evidence-based, early intervention.3 The complexity and severity of young people's presenting concerns also varies, with a substantial subset of young people who need, but are unable to gain, access to specialised tertiary services,18 which may have an impact on average improvement scores for the total client group.

Nevertheless, this article demonstrates that headspace is committed to examining and reporting outcomes for young people using its services, and that the headspace centre initiative is associated with improved mental health outcomes for a large number of young people assisted by this network across Australia.

2 Mean psychological distress scores (K10) at different time points

3 Mean psychosocial functioning scores (SOFAS) at different time points