Health consumers are increasingly turning to the web to find health information and to get direct assistance for their health needs. The Pew Internet & American Life Project found that 75% of internet users have searched for health information online.1 More recent polling, in 2008, found that 23% of internet users had searched for information about depression, anxiety, stress or mental health conditions.2 Polling also indicates that consumers living with a disability or chronic disease are more frequent internet users, and more likely to state that they made health decisions based on information found on the web.2
Consumers are also seeking direct help through web-based interventions and therapy services.3 Web interventions offer therapeutic assistance for a range of mental health conditions. Some offer automated self-help on open access websites, while others provide interactive screening, guided assistance through helplines, supervision by a health professional, or a full virtual clinic experience.4-6
A major problem with the proliferation of web intervention sites is that the quality of services provided by or through the website may not be high. Assessing the quality of health information on the internet has long been recognised as a challenge by researchers and policymakers,7 prompting a range of responses including quality labels based on compliance with codes of conduct (such as the Health On the Net Foundation’s HONCode; http://www.hon.ch); rating tools designed for consumer use;8 and automated quality indicators9 and portals that provide “gateways” to high-quality information.10 These quality indicators aim to assist consumers to identify high-quality passive health information. However, consumers also need assistance in identifying high-quality health interventions on the web, as these become more common. One response to this growing need is to provide an accessible list of current web interventions and services, along with information about their availability and evidence-based quality.
Here, we describe the development and features of a web portal named Beacon (http://beacon.anu.edu.au), which provides information about a range of web interventions used in the prevention or treatment of health disorders. This report focuses on Beacon’s health applications for generalised anxiety disorder, social anxiety disorder, panic disorder, post-traumatic stress disorder and depression.
Beacon was designed to list all therapy and prevention sites, as well as health information or psychoeducation sites for which there is evidence of efficacy or effectiveness. Websites are categorised into diagnostic or health categories, and the scientific evidence on which they are based is evaluated. Beacon has the capacity to collect demographic and mental health information from its users, to publish user feedback about the websites it lists and to examine associations between users’ mental health symptoms and web behaviour.
Our search strategy was threefold. In February 2009, we first identified sites for which there was scientific evidence on efficacy or effectiveness, by reviewing all systematic reviews of the efficacy or effectiveness of mental health websites, including a comprehensive book by Marks and colleagues that reviewed the literature available to May 2006.11 We re-ran this book’s literature search terms for the period May 2006 to February 2009 to identify further sites. Details of this search are available on the Beacon website under “FAQ” (frequently asked questions). A website that lists the titles of literature on e-health research (http://construct.haifa.ac.il/~azy/refindx.htm) was also searched to locate relevant websites.
Second, we identified potentially relevant sites and then searched for relevant supporting research literature. In particular, we undertook an internet search of relevant Open Directory Project (ODP; http://www.dmoz.org/about.html) categories corresponding to International Classification of Diseases medical diagnoses (eg, depression, asthma). The ODP is a web directory edited by volunteers and claims to be the largest human-edited directory of the web. All links within these sites were also searched for links to other potentially relevant sites, and additional information was sought from site owners.
Finally, we sought details of web applications from all members of the International Society for Research on Internet Interventions (http://www.isrii.org) through the society’s virtual network.
To be included in Beacon, sites were required to offer interventions (treatment or prevention) to reduce symptoms or improve health functioning, quality of life or help-seeking capacity. Health conditions included mental disorders (including depression, generalised anxiety disorder, social anxiety disorder, panic disorder), physical illnesses, and sleep and behavioural problems (including eating behaviour, drug and alcohol use, and smoking). Sites were included regardless of whether they were open access or access was contingent on enrolment in a research trial or a health service. Intervention sites were included regardless of whether or not there was evidence for their efficacy or effectiveness. In contrast, as the pool of sites providing mental health information or psychoeducation is large and increasing rapidly, these were only included if there was evidence that they were efficacious.
All identified sites were described in terms of the site structure (eg, website, game), target condition (eg, panic disorder), target age group (eg, adult, child, youth), program content (eg, cognitive behaviour therapy, interpersonal psychotherapy), program length (eg, long — more than five modules), intervention type (eg, automated, supported), support provided (eg, access to trained support workers through a helpline), language, and supporting scientific evidence.
All sites were rated on a 7-point scale from − 1 to 5 (Box 1), accompanied by an explanation for the rating that comprised a brief comment and, in some categories, a link to additional information. Quality ratings were initially made by one researcher (K M) and verified later by another (A L C). Discrepancies in ratings were resolved through discussion between the two raters or with input from a third researcher (H C).
The Beacon website tracks and records the characteristics and behaviour of its visitors. Beacon users register on the site, consent to the collection of data about their symptoms, and complete introductory surveys that capture data on their age, remoteness of residence, education levels, current levels of depression and anxiety, symptoms of panic, levels of disability and social problems. Depression and anxiety are measured using the Goldberg Depression and Anxiety Scales.12 The web behaviour of the visitor is tracked to permit an examination of the relationship between symptom levels and on-site behaviour. Beacon users are sent an email 2 weeks after initial registration, which repeats introductory questions and invites feedback about their experience with the websites listed on Beacon.
The Beacon site has an administrator interface that allows new information to be uploaded, and to track the status of the evaluation of the website. Web users are able to post information about their satisfaction with and experience of each of the websites listed. Website developers of individual sites can provide information about their sites to the Beacon administrator using online forms and can apply directly for inclusion of their sites in the Beacon database. The administrator interface allows the administrator to approve consumer information sent to Beacon, to answer emails, and to liaise with website owners.
By March 2010, 183 websites had been identified for inclusion in Beacon, of which 122 focused on physical health or wellbeing, 40 focused on anxiety and 23 focused on depression. Box 2 lists those mental health websites published on Beacon with ratings of 2 or above.
The 40 sites identified for anxiety disorders all offered interventions rather than information or assessment. Eight offered interventions for generalised anxiety disorder: one achieved a rating of 3, three achieved a rating of 2, and four sites had either no evidence or were under review. Three social anxiety disorder sites were identified: one with a rating of 4, one with a rating of 3, and one with no research evidence. Ten panic disorder sites were identified: one achieved a rating of 4, one a rating of 3 and one a rating of 2. Five post-traumatic stress disorder sites were identified: ratings of 3 and 2 were achieved by the two sites in Box 2. One site achieved a rating of 1 and the two other sites were without evidence. Other sites were identified for test anxiety, public speaking, and claustrophobia (one site for each), obsessive compulsive disorder (three sites) and stress (eight sites). These categories were either too small for further analysis or did not correspond to a psychiatric diagnosis.
Twenty-three depression sites were identified. All were intervention sites except one that provided health information and online assessment (BluePages). One site achieved a rating of 3, three sites achieved a rating of 2, six sites achieved a rating of 1, and 12 sites were without evidence. The remaining depression site was identified through our searches and did have experimental support, but it was no longer available on the internet. A further three sites focused on bipolar disorder and one on suicide ideation.
Beacon provides information on a variety of interventions for anxiety and depression on an open access website that can be reached by the public and health professionals. There is emerging evidence that high-quality psychoeducation can improve the public’s knowledge of effective treatments and some evidence that it can also reduce levels of symptoms.22 Websites can be accessed anonymously, at convenient times, and in rural and remote areas. For clinicians, Beacon provides information on sites that may be useful for their patients or clients, either as self-help tools or as programs for use in low-intensity practitioner-facilitated interventions. High-quality sites may also be of use to community organisations, such as Lifeline, who wish to offer practical help. The Beacon website, through its compilation of new web interventions, identifies gaps in services and may prevent the duplication of multiple website services targeting the same population groups. It also identifies sites that may be of use for citizens of countries with developing economies who have no other access to psychological help.
Other potential criticisms of Beacon concern the search criteria used to find sites, as these may have excluded some evidence-based materials, and the criteria that are used to measure quality and efficacy or effectiveness. There may also be debate about the rating scale. Sites are rated on the basis of whether research trials indicate they demonstrate positive effects over control conditions (regardless of whether these conditions are wait-list or attention placebo). However, all sites are included regardless of evidence of quality. The source of the rating is clearly indicated in the individual entries, for the information of clinicians and consumers.
Finally, although our aim is to provide a useful service to the community, we do not yet have outcome data to determine whether users of the portal value the service, use it in the way that it is intended to be used, or benefit from exposure to it. Further, we do not know whether the service is used by those for whom it is designed (eg, people with high levels of anxiety and depression symptoms with an unmet need for help). However, Beacon is designed to answer these questions through the capture of online information and user behaviour. Symptom levels of major mental disorders are measured at registration. We are able to examine whether consumers at risk for particular disorders self-navigate to relevant sites, and we follow up all registrants (with their permission) 2 weeks after their first visit to investigate whether their symptom levels have lowered. With sufficient web visitors, we will be able to report the results of these investigations, and begin to estimate the value of the site to the community.
1 Beacon ratings of scientific quality of health websites
No research evidence is currently available or a null result in one trial |
|
2 Quality websites for mental health, based on evaluations from Beacon*
Abstract
Objective: To describe the Beacon web portal, which lists and rates quality health websites, collects user characteristics and publishes user feedback; and to report summary data on Beacon’s highest-rated (best evidence-based) sites for mental health.
Data sources: A systematic search was undertaken in February 2009 for potentially relevant websites through a review of research papers and a recently published book, an internet search of Open Directory Project medical categories, a review of material on a high-profile mental health portal, and a survey of international researchers.
Selection criteria: All sites were ranked on a 7-point scale from − 1 to 5, with negative scores indicating evidence of no effect and scores of 2 or more indicating evidence of efficacy based on reports in the scientific literature.
Results: By March 2010, 183 sites had been identified, of which 122 focused on physical health or wellbeing, 40 targeted anxiety, and 23 targeted depression. Of the eight generalised anxiety disorder sites identified, four achieved ratings of 2 or above. Two social anxiety disorder sites achieved scores higher than 2. Ten panic disorder sites were identified, with three achieving ratings of 2 or above; and five post-traumatic stress disorder sites were identified, with two achieving ratings of 2 or above. Of the 23 identified depression sites, four achieved a rating of 2 or above.
Conclusions: There are a number of high-quality mental health websites on the internet, and Beacon provides a portal to enable the wide dissemination of these resources.