Whither Divisions of General Practice? An empirical and policy analysis of the impact of Divisions within the Australian health care system

A strong primary health care sector is an essential part of an efficient and equitable 21st century health system.1 Across countries, primary care organisations vary in their roles, function, and level of integration with the health system.2-4 Divisions of General Practice were introduced in 1992 and covered all of Australia by 1998.5 They are legal entities with boards of directors. In 2005, there were 120 geographically based Divisions, eight state-based organisations, and one national organisation, the Australian General Practice Network (AGPN). In financial year 2004–05, Divisions reported membership of 94.2% of all general practitioners.6 Box 1 summarises some key characteristics of Divisions.

In 2005, a national performance framework linked to Division funding was introduced.7-9 The Divisions network, including the AGPN and the state-based organisations, received $157.3 million in financial year 2004–05.6 Divisions received $140.6 million, of which $66.5 million was “core” funding from the Australian Government Department of Health and Ageing, with the remainder from state health departments, specific government programs, and other sources. This is equivalent to $8 per Australian, compared with $4319 in total health spending per Australian and $1098 per Australian on public hospitals. Division spending was equivalent to $7342 per GP, or $1.75 per GP service provided.

Is the investment in Divisions worthwhile? Are there better uses for these resources? Little empirical evidence exists to answer these questions. Divisions are national in scope, limiting the opportunity to experiment and have control groups. The interventions and mechanisms that Divisions use to effect change vary, partly because of a lack of central direction, but also because of a focus on local circumstances. Furthermore, it can be difficult to distinguish the effect of Divisions as it depends on effective interaction with other parts of the system. One study found that Divisions’ activities in diabetes were not associated with the number of diabetes Service Incentive Payments (SIPs).10 The evaluation of the Better Outcomes in Mental Health Care program found that only some types of service delivery models offered by Divisions influenced health outcomes.11

Our aim was to examine the association between Divisions’ activities and measurable aspects of primary care performance. Each Division comprises inputs (Board of Directors, employed staff, and their skills and abilities), which are used to make decisions about what activities the Division should engage in. These activities are the outputs of Divisions, and include mechanisms or interventions they can use to influence the structure, process and outcomes of general practices and other parts of the health system. Outputs include “in-kind” support and resources provided to general practices, such as the transfer of knowledge and information through education and training, support for information technology, coordination of after-hours provision, employment of nurses and allied health professionals who provide support in specific disease areas, and facilitating connections with other parts of the health system and other primary care providers. These outputs are used by general practices to help reduce the cost to the practices of providing services. The practices can then provide higher levels of service and develop infrastructure that is related to improving quality of care. Divisions can provide a higher level of output and different mix of outputs through the receipt of funding to provide government programs such as the GP Immunisation Incentives Scheme, the Better Outcomes in Mental Health Care initiative, the Aged Care GP Panels Initiative, the More Allied Health Services Program, and the Nursing in General Practice scheme. Each of these has specific objectives, and Divisions have discretion as to the mechanisms they use to meet these objectives.9

Our analysis sought to isolate the effect of Divisions on measurable areas of primary care performance, while controlling for other factors influencing performance, such as remoteness, population, and GP and general practice characteristics. There are two parts to the analysis. The first part estimates an overall “Division effect” on each indicator of primary care performance. This quantifies the proportion of the variation in each indicator that can be attributed to Divisions, while controlling for population and general practice characteristics. The second part examines the association between each indicator and specific Division outputs, while controlling for Division, population and practice characteristics. Separate regression models were estimated for each measure of performance.

The overall Division effect is based on the following linear regression model:

yit = α + βxit + μi + εit

where yit is the dependent variable measuring the indicator of primary care performance of Division i in time t, α is a constant term representing the average level of performance, xit is a group of independent variables influencing primary care performance, and β denotes the regression coefficients on the independent variables. The unexplained variation in the model is composed of μi and εit. Because each Division has an observation for each period in the data, the correlation of observations over time (the random effect μi) is used as our measure of the overall Division effect, after controlling for other factors.12 This is a random effects regression model, similar to a multilevel or hierarchical model, but the unit of observation is grouped over time, rather than, say, within geographical areas. The random Division effect is interpreted as containing unobserved variables that are associated with primary care performance but that are specific to Divisions and constant over the period of the analysis, such as the managerial and leadership style and culture of a Division and general practices, relationships with other parts of the health system, the overall level of resources in the Division, and the level of integration between practices. Other factors that are constant over the period of the analysis include the supply of GPs, the health needs and demographic characteristics of the population (which influence demand for GP services), and remoteness and rurality (which influence access to health care). All these are included as control variables in xit.13 The main hypothesis tested is that if Divisions have no effect on performance, then the variance of μi will be zero and/or will not be significant for each performance measure. The proportion of the variation in performance that can be attributed to Divisions, μi, is interpreted as the average effect of Divisions over the period.

Quantification of the overall Division effect is useful in informing whether Divisions have had an overall impact, but tells us little about the specific Division outputs that influence primary care performance. Therefore, the second set of analyses extends the model to include variables on Divisions’ outputs, zit:

yit = α + βxit + δzit + μi + εit

A fixed effects panel data regression model is used.12 This allows the observed Division outputs, zit (as well as xit), to be correlated with the Division effect. Using a fixed effects model allows us to control for all unobserved factors associated with Divisions and their geographical areas that are constant over time and influence primary care performance.12 As Division outputs zit are also likely to correlate with observed Division characteristics that vary over time, Division characteristics are also included in xit. The regression coefficient of zit, δ, is the association, and not the causal effect of Division outputs on primary care performance. There may be reverse causality, such that general practices with interests in an area of performance influence the Division to provide support. Therefore, the association between Division outputs and primary care performance reflects the net effect of both the effect of Divisions on GPs and the effect of GPs on Divisions. The longitudinal nature of the data can be used to remove the reverse causality, as we would expect Divisional activities in the previous period to influence primary care activity in the current period, but not the other way around. This can be modelled by including the value of the variable from the previous time period (zit-1) rather than the current period (zit). Although this removes the bias due to reverse causality, the results must still be interpreted as associations, as there may be unobservable factors that influence both the specific activity and the performance indicator.

All data were measured at Division level and over time (Box 2). Fourteen of a possible 28 indicators of primary care performance were used. Some measures did not cover a long enough period, or the data were censored for confidentiality reasons. These data were obtained from the Medicare Australia website by quarter, and cover a number of the domains of the National Quality and Performance Framework for Divisions.14 The performance measures were continuous and defined in terms of either proportions in each Division or the natural logarithm of dollar value of Medicare benefits paid in each Division. Most are for payments made under the Practice Incentives Program (PIP) and SIPs, which are part of the blended payment system for GPs in Australia. Independent variables included annual data on remoteness, GP and population characteristics obtained from the Department of Health and Ageing, the Primary Health Care Research and Information Service at Flinders University, and the Public Health Information Development Unit at the University of Adelaide. All these variables were included in all regression models. Data on Divisions’ outputs and characteristics were obtained from the annual surveys of Divisions. Only Division outputs thought to directly influence the specific performance indicator were included in each regression. All data were merged into one dataset and all variables were available over 12 quarters (3 calendar years) between 2002 and 2004 (see Scott and Coote15 for more detail). State and quarterly dummies were also included in all models.

The levels and median growth in performance between February 2002 and May 2004 are shown in Box 3. The highest areas of growth included care plans and health assessments. The provision of asthma SIPs and PIP claims for after-hours provision declined over the period. The asthma SIP claims fell because of problems in being able to claim the incentive payments.16 The PIP claims fell because of growth in after-hours services provided by GPs outside of the practice. There was substantial variation across Divisions in rates of growth, as indicated by the interquartile ranges.

Box 4 shows the results derived from the first set of 14 regressions. As the focus of the study is on Division effects, the coefficients for the control variables are not reported here, but are available.15 Between 11% and 64% of the total variation in each performance indicator can be attributed to Divisions, while controlling for all GP and population characteristics in Box 1. The variance of the Division effect (µi) for each performance indicator was significantly different from zero (F test, P < 0.05 in all models). Overall, Divisions had more effect on aspects of performance related to practice infrastructure than on performance related to clinical activities conducted by GPs. For example, Divisions had a relatively large role in explaining the variation in the proportion of PIP practices in a Division, the proportion of PIP practices claiming sign-on payments, and in after-hours care. They had a relatively smaller role in explaining the variation in GP clinical behaviour such as SIP benefits or health assessments.

The second analysis provides further evidence of the effect of Divisions. Results from the 14 regression models are not shown here (see Scott and Coote15), but broadly confirm the first set of results that Division activities in the area of practice infrastructure were more likely to be associated with performance than Division activities supporting specific disease areas.

Divisions that provided support in electronic communication and electronic transfer of data were associated with a 0.56 percentage point increase in the proportion of PIP practices (95% CI, 2 0.04 to 1.2; P = 0.07), a 0.73 percentage point increase in the proportion of PIP practices with electronic prescribing software (95% CI, 2 0.09 to 1.5; P = 0.08) and a 0.66 percentage point increase in the proportion of PIP practices with a modem (95% CI, 0.05 to 1.3; P = 0.03). Divisions providing activities with an asthma focus were associated with a 0.84 percentage point increase in the proportion of PIP practices receiving the asthma sign-on payment (95% CI, 0.02 to 1.5; P = 0.01). There were no significant effects of Division outputs on the dollar value of asthma, diabetes, or cervical SIP claims per PIP practice. There were also some significant but counterintuitive results for some variables, suggesting the existence of unobservable factors influencing both Division activity and primary care performance.15

Divisions of General Practice influenced up to 64% of the variation in some indicators of primary care performance. The largest effects of Divisions were on general practice infrastructure. Divisions were less likely to have influenced performance in clinical areas. This finding was apparent across different regression models, adding to the robustness of this result.

An important addition to the literature was our use of longitudinal data that enabled estimation of overall Division random effects, the ability to control for unobserved Division-level factors that were constant over time, and the ability to remove reverse causality.

Our study had some limitations. Only a relatively narrow range of Divisions’ outputs and primary care performance was examined, because of a lack of data. For example, it was not possible to examine integration with the rest of the health care system.17 Although Divisions explained variation in the provision of after-hours care and asthma SIPs, these all declined over the study period (Box 3). Divisions may have contributed to the decline by focusing on other activities that were more likely to be successful, particularly the coordination of after-hours care, which reduced the need for GPs to provide it themselves. Few specific Division outputs were associated with primary care performance, and for those that were, the effects were small. Data on Division outputs were self-reported, which may have introduced measurement error, and were measured in terms of whether the Division undertook the activity, rather than what mechanisms the Division used to try to influence general practices (eg, GP education).

The analysis was conducted during a period (2002–2004) when the early effect of Divisions had perhaps already been realised, and was a period of funding uncertainty for Divisions.8 There was a GP workforce shortage that led to falling bulk-billing rates, and GPs were finding it difficult to comply with the administrative requirements of government programs.18,19 The results show that Divisions still influenced primary care performance while operating in an unusually difficult environment.

We could not examine the effects of Divisions’ more recent evolution into service providers. The measurement and rigorous evaluation of these activities is necessary to ensure that Divisions are using the most cost-effective mechanisms to achieve these outcomes. Careful management will be required to balance the potential trade-off that existed in primary care organisations in the United Kingdom and New Zealand between Divisions’ roles in supporting general practice and as regional agents and fundholders for government-funded programs.3 The future of Divisions depends as much on the health care system in which they operate as on what they do. The message of this research is that Divisions have made some difference by improving infrastructure and other support to GPs and general practices. This role should not be lost, given the pivotal role that primary care plays in modern health care systems.

4 Percentage of total variation in performance explained by Divisions

PIP = Practice Incentives Program. SIP = Service Incentive Payment.