Targeting mental health care attributes by diagnosis and clinical stage: the views of youth mental health clinicians

Recent and previous approaches to planning Australian mental health care suggest targeting services mainly on the basis of the severity of full-threshold disorder.1-3 Such an approach incorporates information that is both clinically relevant and regularly collected in epidemiological surveys and health service administrative data. However, illness severity does not adequately account for the natural history of mental illnesses — a consideration that should be central to strategies to pre-empt, delay or avert progression from earlier stages of mental ill health (which might involve severe but transient levels of psychological distress) to conditions that are established, enduring and disabling. In practical terms, full-threshold disorder is an inappropriately high bar for access to care for many young people with emerging mental health problems,4-6 and this approach may promote potentially harmful over-reliance on “watchful waiting” strategies.7

An alternative approach is the clinical staging concept, which has been progressively developed in mental health for more than a decade.5,8-17 Clinical staging is increasingly being used in clinical services and mental health research and has recently been recommended as a basis for an alternative stepped model of mental health care.7 The clinical staging model uses a combination of help-seeking, symptoms and functioning information to categorise a mental health problem into one of six stages (Box 1).

Although clinical staging is a transdiagnostic model, it is currently better defined for certain disorder pathways, such as psychosis. However, each year in Australia’s headspace centres, data on clinical stage of illness are routinely collected from tens of thousands of young people with a wide range of diagnoses. Clients attending headspace centres predominantly have Stage 1a (help-seeking with some symptoms) or Stage 1b (attenuated syndromes) disorders, with a smaller but still sizeable number of young people at Stage 2 (first episode of severe disorder).5 Many young people with Stage 1b or 2 disorders will need more specialised supports than those typically provided in primary care. For example, specialist early psychosis services mainly target Stages 1b and 2 psychotic disorders, as well as providing relapse and treatment resistance responses for currently admitted clients who have progressed to Stage 3 (recurrent illness) or Stage 4 (persistent and unremitting).

For young people in earlier stages of illness (Stages 1a and 1b), the type of mental health treatments prescribed and the level of service use are significantly differentiated by stage of illness.4 However, good mental health care often requires more than just provision of psychological and pharmacological therapies, and it is less clear how the need for other attributes of mental health care, such as family engagement or neurocognitive assessment and symptom tools, might vary by clinical stage. Also unclear is the extent to which clinical staging should complement or replace disorder-based diagnostic criteria as the basis for planning mental health service responses.

In this study, we aimed to explore the potential utility of clinical stage and mental disorder categories as a basis for determining when, and to whom, different attributes of youth mental health care would most beneficially be provided, by eliciting the opinions of youth mental health clinicians. Specifically, we explored the targeting of attributes of mental health care, other than individual psychological or pharmacological treatments, for Stage 1b and Stage 2 alcohol use disorder, anorexia nervosa, borderline personality disorder (BPD), generalised anxiety disorder (GAD), major depressive disorder (MDD) and psychotic disorders. The rationale for our focus on Stages 1b and 2 is that these stages represent a period of potential overlap and transition between primary care and specialist services in which, at least for non-psychotic disorders, there is a need for more clearly elaborated and evidence-based descriptions of optimal service provision.

Methods

We undertook an anonymous online survey of youth mental health clinicians in which we asked them to provide quantitative estimates of the potential benefit of each of 12 attributes of mental health care for young people with mental illness (Box 2). These attributes were selected from a longer list identified by a literature review.18 We based our decisions about which of the attributes identified in that review to include in the final survey instrument on our subjective judgements regarding where clinician opinion had the most potential to add to existing evidence. For example, acute inpatient care was one of the options identified in the review for implementing crisis support, but we did not explore this in our survey as we felt that health service use data on presentations to acute services would likely provide a better guide to the level of need for these services.

Ethics approval for the study was provided by the Centre of Youth Mental Health Human Ethics Advisory Group of the University of Melbourne (ethics ID 1647464).

Survey design and administration

Survey design was shaped by consultation with subject experts, including young people with a declared interest in mental health research. Consultation activities included a facilitated discussion of invited experts at the International Conference on Youth Mental Health in Montreal in 2015; a presentation to, and discussion with, members of the Orygen Youth Research Council (a national advisory group of young people); and piloting of versions of the survey with youth mental health clinicians and researchers. We made changes based on the feedback from the pilot surveys, mainly to shorten the survey by removing questions relating to the capabilities of youth mental health workers and computer algorithms and by merging other questions (eg, asking questions about “assessment and care” collectively rather than assessing each separately).

The survey was administered using the online survey tool Qualtrics. Information about the survey was distributed to individuals we believed were youth mental health clinicians and to mailing lists to which we believed a high proportion of youth mental health clinicians would be subscribed. Most of these mailing lists were third-party distribution lists and not under our direct control. The survey was open for 4 weeks in June 2017. The information we distributed directed potential participants to a filter webpage, where they were asked to confirm that they were current or past youth mental health clinicians with experience treating one of six named mental disorders. Those declaring they did not meet the inclusion criteria were provided with background information about the survey. Those declaring they did meet the inclusion criteria were supplied with a link to the consent form. Those who then consented were provided with a link to the online survey and the password required to access it.

The survey consisted of two parts (see Survey Questions in online Appendix). Part 1 asked participants for background information about themselves, including current role types, clinical discipline, role in research, years of experience, the health care setting in which they had the most experience, the country in which they worked and which of six mental disorders was most relevant to their experience.

Part 2 presented participants with two vignettes of young people with mental health problems consistent with the mental disorder selected by the participants in Part 1. We used a vignette approach because clinical staging is a relatively novel concept in mental health, and understanding of it is likely to vary considerably among youth mental health clinicians. For each of the six selected mental disorders, vignettes were written to represent Stage 1b and Stage 2 of the disorder, but they did not refer to clinical staging concepts (Box 3). Clinicians with expertise in each of the six disorders either wrote the vignettes or reviewed and amended our drafts. Vignettes were created to accord with a previously published schema of clinical staging in mental health,10 and the descriptions were kept relatively brief to increase generalisability to young people with the same disorder and stage of illness. Vignettes were counterbalanced for sex (for each participant, a random draw determined if the characters in Stage 1b and Stage 2 vignettes were female and male, or male and female, respectively). The names of the characters in the vignettes were selected by an online random name generator.

For each of the two vignettes they were shown, participants were asked to consider their current or previous caseload and to estimate the proportion of young people with mental health problems very similar to those described in the vignette that they thought would clinically benefit from each of the 12 mental health care attributes.

Statistical analysis

Only survey respondents who attempted at least one question relating to the two vignettes were included in the main analysis. We analysed results first as independent samples based on the six disorders, using standard statistical tests of significance. We then analysed results as a stratified sample, using a mixed-effects modelling approach. Kruskal–Wallis and χ² tests explored whether the disorder-based groups were significantly different in terms of the background of participants. Due to the non-normally distributed primary outcome of interest (estimated proportion of young people who would clinically benefit), we used the Wilcoxon–Mann–Whitney test to explore the potential impact of the sex of the vignette characters on the results, and whether results differed significantly based on stage of illness.

A mixed-effects modelling approach was used to account for both clustering of participants by disorder and the covariance arising from each participant answering multiple questions. The respondents’ numerical estimates of the proportion of young people who would benefit were transformed into quintiles of the range of potential response values (from 0 to 100%) and a series of ordered logit mixed-effects models were estimated. A preferred model was selected using the Bayesian information criterion, which trades off model explanatory power and parsimony. As a secondary analysis, we simulated 20 dichotomous outcomes (indicated or not indicated) for each response, using the proportion estimated by each respondent as a probability of indication, and then estimated a generalised mixed model using a logit link and binomial family. Simulating outcomes allowed us to analyse results as if they were true proportions (a successes from b trials) rather than bounded estimates, and to use probability rather than membership of a categorical group as a response variable. All data were analysed using Stata/IC 14.2 (StataCorp).

Results

Of 370 individuals we believed were youth mental health clinicians who received the targeted email, 59 opened the email and 11 clicked through to the survey information link. Of 8307 subscribers to a general youth mental health newsletter who were sent the survey information, 1973 opened the email and 58 clicked through. These figures suggest that the bulk of our respondents were reached by survey promotion emails sent on our behalf by third parties, for which we did not have distribution or response data, so we could not determine the overall response rate. A total of 530 people consented, 467 started the survey and 412 attempted at least one question in Part 2. Some of the drop-off (12%) between consent and survey attempt may have been caused by the password requirement for the survey. Some potential participants reported that they had clicked through to the survey without realising they had been provided with a password.

Participant characteristics

People who started the survey but did not attempt Part 2 (n = 55) differed from those who completed both parts. Those who did not attempt Part 2 were more likely to work in service development or training, to be a non-researcher or have an “other” current role description, to be nurses or registered psychologists, to mainly work in schools or “other” settings, and to be from Australia and New Zealand or Europe.

The key characteristics of the survey participants are summarised in Box 4 (see also Figures 1–3 in the Appendix). More than a third of the total sample answered questions relating to psychosis. The anorexia nervosa participant group was small (n = 18), with the remaining disorder-based participant groups ranging in size from 38 (alcohol use disorder) to 82 (GAD). Participants were reasonably experienced, with a mean of 12.6 years of clinical mental health work with young people and almost 16 years of working clinically in mental health with any age group. Those in the GAD group were slightly less experienced.

Just over 90% of survey participants reported that they were currently working as a clinician. About 21% had a current role as a researcher, 26% worked in positions of clinical services leadership, 13% were in service development and training roles and 2% worked in policy roles. The proportions who were currently clinicians or in policy or service development and training roles did not vary significantly across the six disorder-based groups. The proportion currently working in research was higher for those in the psychosis and BPD groups, while clinical leadership positions were less common among the GAD and MDD groups.

Participants represented a broad range of disciplines, with psychiatry (19%), clinical psychology (19%), other psychology, including counselling (18%), social work (16%) and nursing (15%) accounting for nearly 90% of those who named their discipline. Only one participant reported being a general practitioner. There was a higher proportion of psychiatrists in the psychosis group compared with the non-psychotic disorder groups, which had relatively higher numbers of social workers and clinicians from psychology backgrounds other than clinical psychology.

Participants were asked to name the setting in which they had the most expertise, and 44% nominated outpatient specialist care. The next most common settings were primary and secondary care (both 18%). Only one participant reported that his or her experience was mostly in online settings. We did not ask whether participants had any experience in settings other than their nominated main setting. The proportions of nominated settings varied by disorder-based group, with higher proportions of those in the psychosis and BPD groups reporting specialist outpatient experience.

The 412 participants worked in 32 countries. Most participants said they worked in Australia or New Zealand (61%), with North America (16%) and Europe (15%) the next most common regions. Participants in the psychosis group were more likely to be from regions other than Australia and New Zealand.

Participant responses

There was no strong evidence to suggest that the sex of the characters in the vignettes influenced the responses of participants. When tests were applied to the whole sample, none of the 24 means of the participants’ estimated proportions (of young people likely to benefit from each attribute) significantly varied by sex of the vignette character. We applied the same tests within each disorder-based group and, of the 6 × 24 means we tested, the only results that significantly differed based on sex of the vignette character were those for Stage 2 GAD (family engagement and participation), Stage 1b BPD (neurocognitive assessment and symptom tools) and both Stage 1b and Stage 2 alcohol use disorder (face-to-face delivery in the home). These few significant results may be an artefact of the large number of tests deployed.

There were wide interquartile ranges for individual estimated proportions (Box 5). Psychosis was a significant differentiator of estimated clinical benefit for 11 of the 12 attributes, followed by anorexia nervosa (10), GAD (nine), MDD (eight), BPD (five) and alcohol use disorder (one) (Appendix, Figure 4). Clinician-perceived benefit of attributes varied by the clinical stage of psychosis (for eight attributes), MDD (seven) and GAD (six) (Appendix, Figure 5). The evidence was less strong for anorexia nervosa and alcohol use disorder (only one and two, respectively, of the 12 attributes differed by clinical stage), while for BPD, there was no evidence of clinical stage predicting perceived benefit of any mental health care attribute.

The attributes for which perceived clinical benefit most frequently varied by disorder were family engagement and participation, and collection and processing of biological samples, followed by use of neurocognitive assessment and symptom tools, information exchange with other services, face-to-face consultations in a young person’s home, and case management with a multidisciplinary team. The attributes for which perceived clinical benefit most frequently varied by clinical stage were the use of neurocognitive assessment and symptom tools, and delivery of care in a subacute residential setting, followed by the collection and processing of biological samples, and case management with a multidisciplinary team. The estimated clinical benefit of family engagement and participation was not differentiated by clinical stage.

These within-group results are consistent with the model we fitted for the whole sample (Box 6), which, of five candidate models, best met our selection criteria (Appendix, Table 1). The model based on simulated outcomes, which estimated a probability rather than membership of a categorical group, is shown in Table 2 in the Appendix.

The model results suggest that clinical stage, disorder and interactions between stage and disorder were significant predictors of clinician-perceived benefit of mental health care attributes. Although individual clinician characteristics, such as discipline and setting, were significant predictors in the univariate analysis, none remained significant in the models we estimated.

Discussion

We aimed to identify the extent to which clinical stage and mental disorder categories might be used as a basis for providing the right attributes of mental health care services to the right young people at the right time. The estimated clinical benefit of the 12 attributes was often high across disorders and clinical stages. This suggests that there is a good case for including these attributes in services for young people with Stage 1b or 2 mental disorders. Furthermore, for five of the six disorders (the exception being alcohol use disorder), the person in the Stage 1b vignette was described as having a subthreshold disorder. The high clinician-estimated benefit of a wide range of mental health care attributes for young people with similar mental health problems is further evidence against taking a minimalist approach to service provision for young people with subthreshold disorders. Our results for BPD are consistent with evidence that BPD criteria are a marker of severity of psychological dysfunction in general.19 Work to integrate BPD into the staging model of mental disorders14 might consider whether the presence of any BPD criteria is indicative of a Stage 2 disorder.

Existing research literature does not allow strong conclusions to be drawn about how the clinical utility of various attributes of youth mental health care — other than pharmacological or psychological therapies — varies by subgroups of young people. The patchy availability of many of the attributes assessed in this study also hinders using existing health care service use data to draw conclusions about the potential demand for or benefit of these attributes. Our study partially addressed this gap by using a survey design that examined the clinician-perceived benefit of a set of care attributes across a range of disorder and clinical stage combinations.

There was substantial between-clinician variance in their assessments of the proportions of young people who would clinically benefit from the attributes explored in the survey. These large variances emphasise the uncertainty attached to clinician opinion-based estimates of need and may reflect the variable real-world availability of these attributes in mental health care services (and, by extension, clinician familiarity with them). The best way of overcoming this uncertainty is through trials of service models that provide these attributes. A further interpretation of these results is that, even when such attributes are provided, service use patterns might vary by referring clinician, potentially warranting clinician education programs to accompany the roll-out of novel services.

Our study builds on existing evidence about the utility of clinical staging in differentiating treatment selection4 and recent arguments for a stepped model of care based on clinical staging.7 The results of the survey suggest that clinical stage information might be usefully combined with disorder information to ensure that attributes of mental health care are provided to the young people with the greatest capacity to benefit from them. This finding may be useful in informing future development of the stepped model of care currently recommended by the Australian Government,2 potentially enhancing the utility of that framework for planning early intervention services. Ensuring youth mental health services routinely capture clinical stage data, and that youth mental health clinicians are trained and supported to record this information consistently, will further strengthen the evidence base for using clinical staging in mental health services planning.

The main limitation of our study is that it used a self-selected online sample. As survey participants are not a probability sample, the estimates we report cannot be interpreted as representative of clinician opinion in general. Other limitations include the effective non-participation by GPs (who comprise a major component of the youth mental health workforce in Australia) and the much smaller sample size of anorexia nervosa clinicians than for the other five disorders. Future surveys of youth mental health clinicians may need to deploy differentiated strategies to better engage GPs. In our study, the potential impact of greater GP participation on model coefficients is unclear, although neither the professional discipline nor the care setting of respondents was a significant predictor in the models we estimated. Finally, our study provides no information about how clinicians would prioritise the relative importance of mental health care attributes for each target group — a deficit that could be dealt with by using discrete choice experiment designs. As many observations and parameters were involved in estimating the models, it is possible that our preferred model is over-fitted.

Given the substantial uncertainty among youth mental health clinicians about when to provide different attributes of mental health care, and to whom, we found that a combination of clinical stage and disorder information might be an appropriate basis for targeting early intervention mental health services. High-fidelity collection of clinical stage data should be a goal of youth mental health services. Service planning frameworks may need to better account for the care requirements of young people with subthreshold disorders. Additional research, including trials of novel service models and preferences research, can help improve targeting of mental health services.

Box 1 – Clinical stages in mental illness