Mental health policy — stumbling in the dark?

Australia has a proud recent history of developing mental health plans — broad, collaborative documents involving all state, territory and federal governments. In 1992, Australian health ministers agreed to a National Mental Health Plan with the aim to:

• promote the mental health of the Australian community and, where possible, prevent the development of mental health problems and mental disorders;

• reduce the impact of mental health disorders on individuals, families and the community; and

• assure the rights of people with a mental illness.2

These overall aims have changed very little in the past 15 years.3

Bringing together different governments and stakeholders to agree to a national approach to mental health is quite an achievement. People outside of the national policy processes may not be aware of how much time and energy such strategies and plans require. Their formulation is a costly exercise, and the achievements of such plans should be reported publicly.

In addition to the National Mental Health Strategy, other mental health plans include the Council of Australian Governments (COAG) National Action Plan on Mental Health (2006) and a series of associated strategies, primarily focused on increasing access to mental health treatment through the Medicare Benefits Schedule.4

These plans and strategies all devote some attention to the issue of monitoring and evaluation measures to report on achievements against the aims of the policy. There are now 10 national reports summarising information about levels of mental health service provision, shifts in funding allocations, and efforts to engage the community, carers and consumers. Unfortunately, the available information has significant limitations. In practice, the existing monitoring and reporting tells us little about mental health and mental illness in Australia.5,6

We do not currently monitor the development of mental health disorders; their impact on individuals, families and the community; and the rights of people with a mental illness. The aims of our mental health strategies and plans remain largely unmeasured. We have some useful one-off reports and collections of anecdotal stories,7,8 but no ongoing systemic collection or documentation of what happens to Australians who experience a significant mental illness.

Existing mental health services already have invoicing systems in place to receive government funding for the services they provide. Monitoring the existing activity of funded services is therefore a straightforward process, particularly compared with trying to identify outcome indicators. Providing this kind of program input and output data requires minimal investment, yet, even in this domain of monitoring and reporting on levels of service activity, there are significant gaps in our knowledge. A report of the Information Strategy Committee of the Australian Health Ministers’ Advisory Council National Mental Health Working Group found that less than a third of the key performance measures required for full accountability of public mental health services were available.9

There have been several attempts to identify the necessary datasets for more effective monitoring,10,11 but the barriers to collecting these data within the current information systems are overwhelming because current systems rely on data extracted from a disparate range of mental health service systems across Australia. The authenticity of information collection as part of service system delivery is questionable. The information often serves a range of purposes, the comparability of data varies, and it takes considerable time to collect, validate, approve and publish these data. The most recent report on mental health services in Australia released in 2008 summarised data up until 2005.6 The usefulness of data from over 3 years ago is questionable, particularly in the context of seeking to monitor the impact of reform. How do we know whether changes in the past 3 years require a shift in priority or investment? In practice, Australia’s capacity to prepare mental health plans and strategies that everyone can agree to is not matched by a history of monitoring the impact of these plans and strategies on mental health and wellbeing within our community.

The concern about this lack of outcome data and effective evaluation is the potential perpetuation of failure. If the measures are not in place to monitor outcomes, service systems have a tendency to continue providing the same services to the same patients.

In the current context, there is limited information about the extent to which people experience and recover from mental illness; what role any interaction with mental health services plays; and the degree to which mental health treatment results in people being able to return to their homes, families, workplaces and the broader community. This lack of real outcome data not only impedes effective reporting, policy development and planning, it also enables perpetuation of community stereotypes about the nature of mental illness, reinforcing broader discrimination and stigma.

In the past 6 months, the Rudd Labor Government has released three new documents that offer a challenge to the way mental health strategies have been maintained and monitored for over a decade. Perhaps the most important of these is Beyond the blame game, by the National Health and Hospitals Reform Commission (NHHRC).12 The role of the NHHRC is critical in terms of health policy and practice in Australia. In the health discussions at the Australia 2020 Summit, the Prime Minister and the Minister for Health publicly reiterated that health reform and the structure of the relationship between the federal and state and territory governments would be informed by the work of the NHHRC.

The NHHRC report clearly emphasises individuals, rather than the system. Consequently, of the 50 or so key measures proposed to measure health system performance in Australia, several encompass new measures to document the experiences of people within the health system, as well as the level of linkages to service systems beyond mainstream health. Proposed measures include:

• Waiting time for admission to a supported mental health place in the community . . .

• Waiting time for admission to a supported drug and alcohol place in the community . . .

• Waiting time for mental health emergency community support . . [and]

• Patient experience with being treated with dignity.12

No previous government report discusses measuring patient experience as a key priority in health system accountability.

Similarly, the Australian Government Department of Families, Housing, Community Services, Indigenous Affairs 2008 “green paper” on homelessness, Which way home?, places significant emphasis on measuring the level of homelessness.13 The report also makes some challenging statements, suggesting that existing homelessness services may not actually reduce homelessness. Questions for consultation on the homelessness paper include:

• What goals should we set to reduce homelessness?

• What targets will best help us to reach our goals?

• What are the best ways to measure the targets we set?

• What are the three research priorities for a national homelessness research agenda?

The consultation on the proposed new National Mental Health and Disability Employment Strategy14 puts the same issue of outcome measures at the forefront of its call for public submissions:

• How would you know that these ideas were successful in achieving the goal?

• How would you measure the success of these ideas and actions in achieving the goal?

• What key performance indicators or milestones could be used to measure the success of these ideas and actions in achieving the goal?

• Are there benchmarks that you would like to see put in place? If so, what are they?14

It is novel to see a series of national policy consultations where measurement of the consequences of government policies and programs has been given such a high priority. This emphasis on outcome measurement is the hallmark of a reforming government. Measurement of outcomes not only allows governments to say whether they have achieved real change, but also creates a legacy of monitoring and reporting that will ensure ongoing action towards the goal. This approach is in stark contrast to the more process-oriented reporting reflected in over 10 years of partial reporting from governments on their progress towards reform of mental health systems in Australia.

Although the current Australian Government may be shifting its approach to measurement across programs, and specifically within the mental health sector, there are still some blockages to this approach, including the reluctance of existing policymakers and system managers to provide increased transparency in their role. Relying on state and territory governments to provide outcome data has proved a fruitless investment. The Australian Government has provided over $40 million for this specific purpose in the past 10 years, but, as noted earlier, there is still no national reporting of mental health outcome data.6

This failure highlights the key barriers to effective outcome measurement. It also demonstrates the need to have such data collected and reported on by dedicated research bodies rather than relying on governments pooling their separate data. There are several exceptional mental health research bodies in Australia, but the research effort is not strongly focused on program evaluation and monitoring. Most mental health research in Australia appears to be the product of individual one-off National Health and Medical Research Council (NHMRC) research funding.15 There is no critical mass of mental health researchers working together, as there is in many other areas of health. There is also a lack of government investment in core infrastructure support for mental health research. This is despite a history of significant, ongoing federal government investment in the research capacity of allied areas, such as alcohol and other drug research.16

The importance of outcome measurement cannot be overstated. What gets measured gets done. Until now, mental health policymakers have emphasised collecting and reporting data that reflect service system inputs and outputs. There has been very limited effort to focus on real measures of mental health or the experience of mental health patients and their carers.

Given the $2 billion increased investment in mental health through COAG and other initiatives,4 it would be foolish to perpetuate over a decade of inaction in measuring the real impact of national mental health policy. The investment required for effective outcome measurement is miniscule compared with the ongoing investment in mental health programs, plans, strategies and systems that are all largely outcome-blind.

Until governments invest in systematically measuring the experiences of mental health patients and their carers beyond the service system, policymakers will continue to stumble in the dark, often taking what appears to be the path with the fewest obstacles.