The research output of Australian general practice lags behind that of other disciplines.1 Practice-based research networks (PBRNs) are a key enabler of research in general practice and primary care internationally.2-4 PBRNs have thus been proposed as a vital element in expanding Australia’s primary care research capacity and output.5,6 PBRNs are a means of providing research infrastructure in the geographically dispersed environment of primary care. Definitions of PBRNs vary (as do structures and functions) but a key feature is a formal administrative structure that transcends individual studies.7
An extensive mix of regional and national research networks has been developed in the United Kingdom.8 Similarly, PBRNs have proven to be a cornerstone of primary care research in a number of countries, particularly the Netherlands,4,8 Canada4,9 and the United States.4,8,10
In Australia to date, progress in PBRN development has been very modest. An audit in 2010 documented six geographically defined PBRNs.11 These networks do not receive dedicated funding. Currently PBRNs are largely supported as one of the many activities undertaken by the local Primary Health Care Research, Evaluation and Development (PHCRED) programs12 based in university departments of general practice and rural health.11 This program and funding will cease at the end of 2011. If PBRNs are to be considered for dedicated funding, it is timely to review their relevance and, in particular, the generalisability of findings from current PBRNs to contemporary Australian practice.
A principal rationale for promoting research carried out in primary care is that, compared with research carried out in secondary/tertiary care, the primary care-derived data will be more generalisable to primary care settings.13 An early criticism of PBRNs was that research carried out within these networks may not be generalisable — as a volunteer sample, PBRN clinicians may be systematically different from other clinicians. The characteristics that motivate participation by research network members may make these individuals different from the average practitioner, possibly biasing results towards higher standards of care.13
Generalisability of study results conducted in primary care may be affected by the representativeness of the participating clinicians and/or of the practices’ patients. Our study addressed the representativeness of GP members of two Australian PBRNs.
This was a questionnaire-based cross-sectional study.
Our sample was a convenience sample of two New South Wales practice-based networks: the Network of Research General Practices (NRGP), which covers Central Coast, Hunter and New England, and the Primary Healthcare Research Network-General Practice (PHReNet-GP), which encompasses South Western Sydney, Southern and South Eastern Sydney, and the Illawarra and Shoalhaven. NRGP and PHReNet-GP are supported by the PHCRED programs of the University of Newcastle and University of New South Wales, respectively. NRGP comprised 133 GPs from 16 practices and PHReNet-GP, 50 GPs from 47 practices. NRGP membership is at the whole-practice level. In PHReNet-GP, membership is at the individual GP level.
In August 2010, GPs in both networks were invited to participate via information packs containing an information sheet and an anonymous questionnaire. PHReNet-GP GPs were individually posted information packs. Packs were delivered to individual NRGP GPs by their respective practice managers. There was a single blanket repeat distribution 4 weeks later.
GPs returned completed questionnaires by reply-paid post or fax.
The questionnaire elicited demographic information for the GP and his or her practice. Country of qualification, years of GP experience, fellowship of the Royal Australian College of General Practitioners (RACGP) and Australian College of Rural and Remote Medicine (ACRRM), use of languages other than English in consultations, employment of practice nurses in the practice, and previous participation in research were also elicited. Practice postcode defined the Australian Standard Geographical Classification-Remoteness Area (ASGC-RA) classification14 and Socio-Economic Indexes for Areas (SEIFA) for disadvantage15 of participants’ place of practice. SEIFA was recoded into quintiles, with Category 1 being the most disadvantaged area.
Univariate analyses of demographic factors were by independent t test, Mann–Whitney U test, χ2 test or Fisher exact test as appropriate. Statistical significance was assumed at P < 0.05 (two-sided). Adjustment for multiple comparisons was not performed as each outcome variable was independent of the others. Analyses were performed using PASW Statistics, version 18.0.3 (SPSS Inc, Chicago, Ill, USA) and Minitab, version 16 (Minitab Inc, State College, Pa, USA).
Of 183 members of the two PBRNs, 140 GPs (77%) returned questionnaires. Of these, 101 were NRGP members (response rate, 76%) and 39 were PHReNet-GP members (response rate, 78%).
Comparisons of results for each of the practitioner and practice characteristics for the two networks are presented in the Box. NRGP practices were significantly more likely to employ a practice nurse, were larger, and were more likely to be located outside a major city. NRGP GPs worked in practices with a different spectrum of socioeconomic disadvantage, and were significantly less likely to have been involved in research. Furthermore, significantly fewer NRGP GPs consulted in languages other than English.
Combined results of the two PBRNs are presented in the Box, along with national-level figures for comparison, where available. Comparison of individual PBRN and/or combined NRGP-PHReNet demographics with national figures suggests potentially important differences, notably in practice size, consultations in non-English language, employment of practice nurses, and rurality or urbanicity of practice.
The response rates of the two networks were similar, despite NRGP membership being at the practice level, and PHReNet-GP at the individual clinician level.
Significant differences in practice size, rurality, socioeconomic status, employment of practice nurses and frequency of non-English language consultations raise the question of possible differences on these parameters between PBRN GPs and other Australian GPs. Consideration of national GP characteristic data seems to support this contention. This calls into question the generalisability to the wider Australian general practice environment of results from studies conducted in such networks.
A number of factors, however, mitigate this conclusion.
First, the extent to which these differences compromise the external validity of network studies depends on the research question.13 In studies in which patients, rather than GPs, are the unit of analysis, selection bias in the composition of networks (with respect to GP characteristics) may not be a major impediment to the generalisability of results.13 This is the case for some NRGP and PHReNet-GP studies. Further, many projects conducted in research networks are not dependent on a representative study sample. Both PBRNs have conducted qualitative studies21-23 and pilot studies that did not require representative sampling.
Second, research projects may use PBRN members while sampling more widely if the PBRN cannot fully support the study recruitment itself.24 This raises the possibility of individual networks collaborating with other geographically based networks if there is a fit of a particular project with the combination of networks. The obvious example is where sample size considerations require expansion of the study sample frame.
Another scenario is where a combination of networks will provide a more generalisable sample when representativeness is desirable. As illustrated in the Box, for some parameters, a combination of GPs from the two PBRNs provides a demographic profile that more closely approximates the national profile. There may be an opportunity to “mix and match” potentially collaborating, geographically based PBRNs within Australia in order to find the appropriate settings for particular research projects.
GPs in our PBRNs are much more likely to have been involved in research than GPs in a previous Queensland study (47.1% v 14.4%).25 This greater engagement may well represent effect rather than, or as well as, cause. Rationale for PBRNs includes bottom-up capacity building as well as top-down recruitment functions. In addition to providing a means of recruitment for researchers (top down), PBRNs provide an opportunity for practitioners to become involved in research and acquire research literacy and experience (bottom up).6
A further pragmatic consideration is that although PBRN GPs may not be comparable to the national GP population for some important attributes, such as size of practice and ASGC-RA classification, this is also likely to be the case for any sample of randomly recruited GPs. Response rates are often poor in this setting (unlike our response rate of 77%) and the responders may be systematically different from the reference population of GPs in ways that are similar to those of PBRN GPs. A German study found characteristics of network and non-network GPs recruited to a regional study to differ from national reference data, and to differ in similar ways.26 This is in the context of a higher participation rate in the study by network as opposed to non-network GPs (66% v 23%). The interplay of response rate (optimal via PBRN recruitment) and theoretical generalisability (via random probability sampling) may produce similar representativeness of study samples, regardless of which recruitment strategy is pursued. In another Australian study, compared with ours, GPs’ mean age more closely approximated (but sex distribution of GPs less closely approximated) national statistics.27 Furthermore, PBRN-based recruitment is more efficient (recruitment is targeted via a smaller sample frame and efficient intra-PBRN communication). No matter what the recruitment strategy, “careful analysis of participants based on publicly available data is therefore crucial for the assessment of generalizability”.26
PBRN GPs in our study were 2.7 years older than GPs nationally and more likely to be women (38.8% v 37.2%). A German study found that, compared with national reference data, its participating PBRN GPs were 0.7 years younger and more likely to be men (76.4% v 66.5%).26 A UK study also found PBRN GPs to be younger than national comparisons.28 As in our study, this PBRN contained relatively few small practices. Similarly, another UK study found network practice size to be larger than that of other local practices.29
PBRN-member participants in US studies, compared with non-PBRN participants, were older and, unlike our study’s NRGP-members, more likely to be in urban practice.30
Thus, despite differences in PBRN member and non-member demographics in several countries, there is no consistent pattern in these differences.
National-level coordination of PBRNs will facilitate collaboration of regionally based PBRNs. Maintenance of autonomy and geographic integrity of individual networks is still vital to continuing engagement of local practitioners and bottom-up capacity building, but central facilitation of collaborations is also required.
Further research should examine the composition of patient populations recruited to studies via PBRNs.
Demographics and characteristics of general practitioners and practices for two practice-based research networks in New South Wales, with formal assessment of group differences
- Parker J Magin1
- Melanie J Marshall2
- Susan M Goode2
- Georgina L Cotter1
- C Dimity Pond1
- Nicholas A Zwar3
- 1 Discipline of General Practice, University of Newcastle, Newcastle, NSW.
- 2 Primary Health Care Research, Evaluation and Development Program, Research Centre for Primary Health Care and Equity, University of New South Wales, Sydney, NSW.
- 3 School of Public Health and Community Medicine, University of New South Wales, Sydney, NSW.
We thank the PHReNet-GP and NRGP members who participated in this study.
None relevant to this article declared (ICMJE disclosure forms completed).
- 1. Askew DA, Schluter PJ, Gunn JM. Research productivity in Australian general practice: what has changed since the 1990s? Med J Aust 2008; 189: 103-104. <MJA full text>
- 2. van Weel C. General practice research networks: gateway to primary care evidence [editorial]. Med J Aust 2002; 177: 62-63. <MJA full text>
- 3. Fagnan LJ, Davis M, Deyo RA, et al. Linking practice-based research networks and Clinical and Translational Science Awards: new opportunities for community engagement by academic health centers. Acad Med 2010; 85: 476-483.
- 4. Green LA, Hickner J. A short history of primary care practice-based research networks: from concept to essential research laboratories. J Am Board Fam Med 2006; 19: 1-10.
- 5. Gunn JM. Should Australia develop primary care research networks? Med J Aust 2002; 177: 63-66. <MJA full text>
- 6. Zwar NA, Weller DP, McCloughan L, Traynor VJ. Supporting research in primary care: are practice-based research networks the missing link? Med J Aust 2006; 185: 110-113. <MJA full text>
- 7. Tierney WM, Oppenheimer CC, Hudson BL, et al. A national survey of primary care practice-based research networks. Ann Fam Med 2007; 5: 242-250.
- 8. van Weel C, Smith H, Beasley J. Family practice research networks. Experiences from 3 countries. J Fam Pract 2000; 49: 938-943.
- 9. Birtwhistle R, Keshavjee K, Lambert-Lanning A, et al. Building a pan-Canadian primary care sentinel surveillance network: initial development and moving forward. J Am Board Fam Med 2009; 22: 412-422.
- 10. Graham DG, Spano MS, Stewart TV, et al. Strategies for planning and launching PBRN research studies: a project of the Academy of Family Physicians National Research Network (AAFP NRN). J Am Board Fam Med 2007; 20: 220-228.
- 11. Soós M, Temple-Smith M, Gunn J, et al. Establishing the Victorian Primary Care Practice Based Research Network. Aust Fam Physician 2010; 39: 857-862.
- 12. Department of Health and Ageing. Research Capacity Building Initiative: a strategic plan for 2006–2009. Canberra: Commonwealth of Australia, 2005. http://www.phcris.org.au/phcred/RCBI%20Strategic%20Plan%202006-2009.pdf (accessed Apr 2006).
- 13. Stange KC. Practice-based research networks. Their current level of validity, generalizability, and potential for wider application. Arch Fam Med 1993; 2: 921-923.
- 14. Australian Bureau of Statistics. ASGC Remoteness classification: purpose and use. Canberra: Commonwealth of Australia, 2003. (Census Paper No. 03/01.)
- 15. Australian Bureau of Statistics. Census of population and housing: Socio-Economic Indexes for Areas (SEIFA), Australia — data only, 2006. Canberra: ABS, 2006. (ABS Cat. No. 2033.0.55.001.)
- 16. Australian Institute of Health and Welfare. Health and community services labour force 2006. Canberra: AIHW, 2009. (AIHW Cat. No. HWL 43; National Health Labour Force Series No. 42.) http://www.aihw.gov.au/publication-detail/?id=6442468220&libID=6442468 218&tab=3 (accessed Apr 2011).
- 17. Primary Health Care Research and Information Service. Division and SBO characteristics. http://www.phcris.org.au/products/asd/keycharacteristic/ (accessed Apr 2011).
- 18. Britt H, Miller GC, Charles J, et al. General practice activity in Australia 2000–01 to 2009–10: 10 year data tables. Canberra: AIHW, 2010. (AIHW Cat. No. GEP 27; General Practice Series No. 27.)
- 19. Australian Government Department of Health and Ageing. General practice in Australia: 2004. Canberra: DoHA, 2005.
- 20. Australian Bureau of Statistics. Selected health occupations: Australia, 2006. Canberra: ABS, 2008. (ABS Cat. No. 4819.0.) http://www.abs.gov.au/ausstats/abs@.nsf/mf/4819.0 (accessed Apr 2011).
- 21. Dennis SM, Zwar NA, Marks GB. Diagnosing asthma in adults in primary care: a qualitative study of Australian GPs’ experiences. Prim Care Respir J 2010; 19: 52-56.
- 22. Magin P, Joyce T, Adams J, et al. Receptionists’ experiences of occupational violence in general practice: a qualitative study. Br J Gen Pract 2009; 59: 578-583.
- 23. Magin P, Joyce T, Adams J, et al. General practice as a fortress: a qualitative study of occupational violence in general practice receptionists. Aust Fam Physician 2010; 39: 854-856.
- 24. Sylvester S, Magin P, Sweeney K, et al. Procedural skills in general practice vocational training What should be taught? Aust Fam Physician 2011; 40: 50-54.
- 25. Askew DA, Clavarino AM, Glasziou PP, Del Mar CB. General practice research: attitudes and involvement of Queensland general practitioners. Med J Aust 2002; 177: 74-77. <MJA full text>
- 26. Wetzel D, Himmel W, Heidenreich R, et al. Participation in a quality of care study and consequences for generalizability of general practice research. Fam Pract 2005; 22: 458-464.
- 27. Reid CM, Ryan P, Nelson M, et al. General practitioner participation in the second Australian National Blood Pressure Study (ANBP2). Clin Exp Pharmacol Physiol 2001; 28: 663-667.
- 28. Fleming DM, Miles J. The representativeness of sentinel practice networks. J Public Health (Oxf) 2009; 32: 90-96.
- 29. McManus RJ, Ryan R, Jones M, et al. How representative of primary care are research active practices? Cross-sectional survey. Fam Pract 2008; 25: 56-62.
- 30. Galliher JM, Bonham AJ, Dickinson LM, et al. Representativeness of PBRN physician practice patterns and related beliefs: the case of the AAFP National Research Network. Ann Fam Med 2009; 7: 547-554.
Abstract
Objective: To compare the demographics of general practitioners in two practice-based research networks (PBRNs) and to explore the generalisability of research findings from these PBRNs.
Design, setting and participants: Cross-sectional questionnaire-based study of two geographically-based PBRNs — Hunter New England Central Coast Network of Research General Practices (NRGP) and Primary Healthcare Research Network-General Practice (PHReNet-GP) — during August–September 2010. All 183 GP members of both PBRNs were invited to participate; of these, 140 (77%) participated.
Main outcome measures: GPs’ demographics, use of languages other than English in consultations, and previous participation in research. Practices’ use of practice nurses. Socioeconomic status and rurality or urbanicity of practice location.
Results: Compared with PHReNet-GP GPs, NRGP GPs were more likely to work in a practice employing a practice nurse (100% v 53.8%; 95% CI for difference, 30.5%–61.8%; P < 0.001), worked in larger practices (2.9 more full-time-equivalent GPs per practice; 95% CI, 2.1–3.6; P < 0.001), and were less likely to work in a major city (33.7% v 89.7%; 95% CI for difference, 42.8%–69.3%; P < 0.001). NRGP GPs also worked in practices with a different spectrum of socioeconomic disadvantage, and were less likely to have been involved in research as a researcher (35.4% v 76.9%; 95% CI for difference, 25.3%–57.8%; P < 0.001). Fewer NRGP GPs consulted in languages other than English (8.9% v 64.1%; 95% CI for difference, 39.1%–71.2%; P < 0.001). There were also differences between these and national general practice statistics.
Conclusions: These results suggest possible lack of generalisability of findings from some types of studies conducted in single PBRNs. In such circumstances, collaboration of PBRNs may produce more generalisable results.