Health service access and quality of care provided by the Western NSW Local Health District Virtual Rural Generalist Service: a retrospective analysis of linked administrative data

Virtual health care, the use of telecommunication technologies to enable provision of medical care at a distance, has been proposed as a solution for both addressing rural health workforce challenges and improving patient access to medical care.1 Workforce challenges include the recruitment and retention of health care workers, and distribution of workers across vast geographic distances. The coronavirus disease 2019 (COVID‐19) pandemic highlighted the fragility and susceptibility of sustained rural health care provision; border closures and travel restrictions prevented the movement of rurally based health care staff and the fly‐in/fly‐out and drive‐in/drive‐out workforces that are common in rural settings.2 The Virtual Rural Generalist Service (VRGS) was developed as a solution to rural medical workforce challenges — specifically, the lack of medical staff in rural hospitals and the need for medical coverage and relief.3

The VRGS provides 24‐hour 7‐days‐a‐week medical support, both virtually and in person, to small rural hospitals and multipurpose services in the Western NSW Local Health District. The VRGS is not a replacement service for in‐person doctors; rather, it provides support when local doctors need relief or are unavailable. When VRGS doctors support a facility in person (at least 25% of shifts), they review patients in a manner consistent with traditional in‐person medical care.

The VRGS is predominantly for lower acuity emergency department (ED) presentations, daily ward rounds for inpatients admitted under a VRGS medical officer and ad‐hoc inpatient medical reviews. When VRGS doctors are consulting virtually, consultation requests are registered by onsite nurses in the electronic medical record (EMR). Proactive ward rounds are automatically scheduled (eg, without an EMR order) at sites where there are no local onsite doctors. When the onsite nurse and VRGS doctor are ready for a patient consultation, nurses bring a mobile cart equipped with wireless videoconferencing and peripheral examination devices to the patient's bedside.

The COVID‐19 pandemic prompted a rapid expansion of virtual models of care in emergency and tertiary care settings.4 ED models use synchronous video telehealth for a range purposes: tele‐triage in the community,5,6 and tele‐triage, tele‐screening and tele‐intake in ED settings;7,8,9 facilitation of remote attending supervision;10 provision of ED care to lower acuity patients;11,12 and in‐home remote monitoring following an ED visit.13 Evidence on the impact that virtual care provided in ED settings has on time spent in an ED is inconsistent, but there appear to be trends towards shorter lengths of stay and lower rates of patients leaving without being seen.14

While inpatient‐level care provided outside of hospital settings (such as in virtual hospitals, virtual wards and hospital‐in‐the‐home models) is well described,15 there are few descriptions of virtual models providing generalist medical inpatient care. The COVID‐19 pandemic prompted the rapid development of innovations to address inpatient surges, the need to quarantine staff, and the need to protect staff and patients from the virus. Models such as virtual team rounding programs and virtual hospitalist programs were established to address these challenges.16,17 However, evidence regarding such models for inpatient care, beyond program service descriptions, is scant.

The objective of this evaluation was to compare service access and quality‐of‐care measures between patients receiving care from the VRGS and those receiving usual care, in emergency department and inpatient hospital settings in Western NSW Local Health District hospitals during the period 2021–22.

Methods

Study design and patients

We analysed routinely collected administrative data for all people who, between 1 July 2021 and 30 June 2022, presented to or were admitted to any of the 29 hospitals in Western NSW Local Health District that were supported by the VRGS. For admitted patients, a completed episode of care was an inclusion criterion. Patients admitted for haemodialysis who had no other diagnosis or treatment and patients cared for by the hospital‐in‐the‐home service were excluded from analyses.

Data sources

NSW Health datasets that routinely collect quality of care measures and are external to the VRGS (ie, the NSW Emergency Department Data Collection and NSW Admitted Patient Data Collection) have no specific data flags to denote VRGS involvement in care and facilitate categorisation of patients into cohorts relating to VRGS involvement. We therefore used administrative datasets internal to the VRGS for deterministic matching to identify VRGS involvement for each patient presentation and episode. The Area Unique Identifier, an identifier generated for a patient within a local health district, was common across all data collections and was used for data linkage undertaken by one of us (AJA).

PC039 entry log and local VRGS shift roster

In EMRs, the PC039 entry log details each time a health professional updates a NSW Health patient record, with the name of the person who updated the record and the date and time of the entry. As study patients could present to an ED or be admitted to a hospital on more than one occasion, and could do so at different facilities, the dates, times and facility names recorded on the two patient data collections and the PC039 entry log were used for linkage, in addition to the Area Unique Identifier. The names of all the doctors who had interacted with a patient during each episode of care could thus be determined and cross‐referenced against the VRGS shift roster (with details on shift location, date and time). Linking the PC039 entry log with patient data enabled us to categorise episodes of patient care into four mutually exclusive doctor cohorts: only VRGS doctor(s) (referred to as VRGS); both VRGS and non‐VRGS doctors (referred to as combined care); only non‐VRGS doctors (referred to as non‐VRGS); and no doctor recorded.

NSW Emergency Department Data Collection

The NSW Emergency Department Data Collection includes records on presentations to public hospital emergency departments. We extracted data on patients’ demographics (age, sex, residential postcode), patients’ clinical characteristics (triage category, referral source, mode of arrival, ED visit type, urgency related group major diagnostic block category), and quality of care received by patients (did not wait, left at own risk, transferred to another hospital, admitted, departed after treatment was completed, died in ED, re‐presented within 48 hours, date and time of ED departure) (details are in the Supporting Information, table 1).

NSW Admitted Patient Data Collection

The NSW Admitted Patient Data Collection contains records on admitted patient services in hospitals. We extracted data on patients’ demographics (age, sex, residential postcode), patients’ clinical characteristics (referral source, admission status, day‐only stay from overnight separation, length of stay, potentially preventable hospitalisation, care type, clinical complexity score [cumulative effect of diagnostic complexity levels], major diagnostic category [based on the International statistical classification of diseases and related health problems, 10th revision, Australian modification]), and quality of care received by patients (hospital‐acquired complication, transferred to another hospital, discharged at own risk, died in hospital, re‐admitted within 28 days [unplanned]) (details are in the Supporting Information, table 2).

Derived variables

Length of stay was calculated from the start of each inpatient episode. For inpatients who first presented to an ED, episode start time was calculated from time of arrival at the ED. Length of stay was categorised as long stay outlier (referred to as a DRG‐based length‐of‐stay outlier) if the total length of stay exceeded the upper bound for the DRG assigned to that hospital admission based on version 10.0 of the Australian Refined Diagnosis Related Groups classification.18 Each facility was assigned a remoteness category per the Australian Statistical Geography Standard Remoteness Areas Structure, which classifies a location in terms of its physical distance to the nearest urban centre.19 Patient indices of socio‐economic disadvantage were assigned by linking residential postcode with the corresponding Index of Relative Socio‐economic Disadvantage, collapsed into quintiles from 1 (most disadvantaged) to 5 (least disadvantaged) (details are in the Supporting Information, tables 1 and 2).20

Data analysis

Descriptive statistics are reported for ED presentations by receipt of VRGS (any VRGS care v non‐VRGS care) and for admitted patients by three treating doctor cohorts (VRGS v combined care v non‐VRGS). The ED patient cohort was dichotomised since there were too few patients receiving combined care for inclusion in analyses as a separate patient group. Data were summarised as counts and proportions or medians with interquartile ranges (IQRs).

We explored the relationship between ED or inpatient activity (number of presentations or admitted patients per facility) and proportion of presentations or admissions in which patients received care from the VRGS using Spearman's rho (ρ) statistic.

As all outcome measures (quality‐of‐care measures) were binary, and to account for potential clustering of patient outcomes at a facility level, mixed effects logistic regression was used with random intercept at the facility level. Separate models were built for each outcome using complete case analysis. Adjusted odds ratios comparing doctor cohorts, with the reference cohort being non‐VRGS, were estimated for each outcome measure by adjusting for the potential confounders of patient demographics, presentation‐specific clinical factors and admission‐specific clinical factors (details are in the Supporting Information, tables 1 and 2). Backwards elimination of potential confounders until all remaining variables were statistically significant (P < 0.05) was used for the final selection of adjustment variables.

All analyses were conducted in SAS 9.4, and the study is reported in accordance with Strengthening the Reporting of Observational Studies in Epidemiology guidelines.21

Ethics approval

This study was approved by the Greater Western Human Research Ethics Committee (2021/ETH01379 and 2021/ETH01355) and the Human Research Ethics Committee of the Aboriginal Health and Medical Research Council of NSW (2010/22).

Results

ED presentations

From 1 July 2021 to 30 June 2022, there were 56 278 NSW Emergency Department Data Collection records that met the inclusion criteria, of which 56 164 (99.8%) were successfully linked to EMR entry log records. Of these 56 164 records, 39 701 (70.7%) could be classified into a treating doctor cohort as they had doctor name(s) recorded in the EMR entry log. Of these 39 701 records, 13 660 (34.4%) showed treatment by a VRGS doctor (12 179 [30.7%] exclusively by a VRGS doctor and 1481 [3.7%] by both a VRGS doctor and a non‐VRGS doctor). VRGS activity was more common in hospital sites with fewer ED presentations (Spearman's ρ, ‐0.67; P < 0.001), and in remote (40.9% [4366/10 669]) and very remote (39.4% [596/1513]) facilities (Supporting Information, tables 3 and 4).

Patient characteristics

Patients presenting to an ED who were attended by a VRGS doctor were younger than those in the non‐VRGS care group (median age, 37 years [IQR, 18–62 years] v median age, 43 years [IQR, 20–65 years]). In the any VRGS care group, patients were less likely to be from the most socio‐economically disadvantaged regions (6547 [48.7%] v 16 552 [64.6%]) (Box 1). In the any VRGS care group, patients were more likely to be classified as lower acuity on presentation; for example, a lower proportion of these patients were triaged as emergency presentations (918 [6.7%]) compared with patients in the non‐VRGS care group (4078 [15.7%]). Patients in the any VRGS care group were less likely to be treated for single site major injuries and circulatory system illness than those in the non‐VRGS care group, but more likely to be treated for digestive system and ear, nose and throat illnesses (Box 1).

Quality of care indicators

Although only 294 ED presentations (0.7%) were classified as did not wait, after adjusting for patient characteristics and facility, the odds for this outcome were higher for those receiving care from the VRGS (adjusted odds ratio [aOR], 3.69; 95% CI, 2.79–4.89) (Supporting Information, table 5). Similarly, although only 851 ED presentations (2.1%) were classified as left at own risk, patients in the any VRGS care group had a higher likelihood of this outcome than those in the non‐VRGS care group (aOR, 1.90; 95% CI, 1.62–2.22).

Patients receiving care from the VRGS were less likely to be transferred to another hospital than those in the non‐VRGS care group (aOR, 0.66; 95% CI, 0.60–0.72), and the likelihood of departing from the ED within 4 hours was similar in these two patient groups (aOR 0.92; 95% CI, 0.86–0.98) (Box 2; Supporting Information, table 5). Time from arrival at ED to departure from ED was longer by 5 minutes for patients seen by a VRGS doctor (median, 113 minutes; IQR, 71–192 minutes) than for those in the non‐VRGS care group (median, 108 minutes; IQR, 60–193 minutes). A total of 30 patients (< 0.1%) died in an ED, with equivalent odds between doctor cohorts.

Inpatient admissions

From 1 July 2021 to 30 June 2022, there were 6744 NSW Admitted Patient Data Collection patient records that met the inclusion criteria, of which 6732 (99.8%) were successfully linked to EMR entry log records. Of these 6732 records, 6328 (94.0%) were classified into a treating doctor cohort. Of these 6328 records, 2531 (40.0%) showed treatment by a VRGS doctor (681 [10.8%] exclusively by a VRGS doctor and 1853 [29.3%] by both a VRGS doctor and a non‐VRGS doctor). VRGS activity was more common in hospital sites with fewer admissions (Spearman's ρ, –0.56, P < 0.001), and in outer regional facilities (14.2% [454/3188] VRGS, 33.4% [1064/3188] combined care) (Supporting Information, tables 6 and 7).

Patient characteristics

Inpatients in the combined care group were slightly older than those in the VRGS and non‐VRGS cohorts (median age, 73 years [IQR, 56–82 years] v median age, 71 years [IQR, 55–81 years v median age, 71 years [IQR, 55–81 years], respectively). The combined care inpatients also had greater clinical complexity levels, with 627 (33.8%) classified in the most complex category, compared with only 161 (23.6%) in the VRGS cohort and 914 (24.1%) in the non‐VRGS cohort. The proportion of inpatients with a same‐day stay was lower for the combined care cohort than for the other cohorts (Box 3).

Quality of care indicators

The proportion of inpatients who discharged at their own risk was higher for the VRGS cohort (4.3% [29/681]) than for the non‐VRGS cohort (1.6% [62/3794]). After adjusting for patient socio‐demographic and clinical factors, patients exclusively receiving VRGS care were more likely to discharge at their own risk than non‐VRGS patients (aOR, 3.33; 95% CI, 1.98–5.61) and less likely to be a long stay outlier (aOR 0.51; 95% CI, 0.35–0.74) (Box 4; Supporting Information, table 8). The likelihoods of transfer to another hospital were similar for VRGS and non‐VRGS inpatients (aOR, 0.80; 95% CI, 0.60–1.01). A total of 275 patients died while in hospital, of whom 98 (35.6%) were receiving palliative care. The odds of mortality were equivalent for inpatients receiving VRGS and non‐VRGS care (aOR, 0.78; 95% CI, 0.48–1.28).

Inpatients in the combined care cohort were more likely to be transferred to another hospital (aOR 1.41; 95% CI, 1.21–1.65) and more likely to be a long stay outlier (aOR 2.10; 95% CI, 1.74–2.53) than those in the non‐VRGS cohort, even after adjusting for complexity levels (Box 4; Supporting Information, table 9). Other differences between inpatients in the combined care and non‐VRGS cohorts were not statistically significant, including the odds of in‐hospital death (aOR 1.21; 95% CI, 0.91–1.61).

Proportions of inpatients with unplanned readmissions within 28 days to any facility within the Western NSW Local Health District were similar for all three the patient cohorts. However, the proportion with unplanned readmissions within 28 days to the same facility was highest for admitted patients seeing a VRGS doctor only (10.6% [72/681]) and lowest for those in the combined care group (6.6% [123/1853]).

Discussion

The VRGS provided hybrid virtual medical care for 34% of ED presentations and 40% of inpatient admissions across 29 Western NSW Local Health District hospitals during 2021–22. The VRGS was more active in facilities with fewer ED presentations and hospital admissions, and the provision of care exclusively by a VRGS doctor was more common for ED presentations than inpatient admissions. Our analysis of more than 39 000 ED presentations and more than 6000 inpatient admissions showed an overall similar quality of care for VRGS and usual care recipients. Mortality rates were similar, regardless of care received and setting.

The rates at which patients leave without being seen or leave at their own risk and rates of unplanned re‐attendance are common and important quality‐of‐care metrics in acute care settings.22 Among patients who presented to an ED, those who received care via the VRGS were more likely to leave without being seen and leave at their own risk, but the rates of each of these outcomes were low (< 2% and < 4%, respectively). Rates of treatment completion were equivalent. One study from the United States reported reduced rates of leaving without being seen (ie, increased ED input) following the introduction of a tele‐intake model in an urban setting, but no changes in the rates of leaving at own risk or treatment completion.23 In contrast, other studies have observed slightly increased rates of leaving without being seen with tele‐screening7 and significant increases in leaving at own risk with tele‐consultation.24 Factors affecting ED throughput and patient centredness of care are complex. In the VRGS, wait lists were not visible to local staff, meaning that the staff were unable to provide an estimated wait time to patients, and VRGS patients reported this as a challenge.25

Evidence on the impact of virtual care on ED length of stay is mixed; it has been shown to be associated with both significantly shorter and longer ED lengths of stay.12,23 In our study, rates of departure from an ED within 4 hours were similar for those who received care via the VRGS and those seen only by a non‐VRGS doctor. However, patient disposition varied between groups; patients in the VRGS group were less likely to be transferred to another hospital, but more likely to be admitted and to re‐present within 48 hours, even after adjusting for acuity. Data on patient disposition for virtual models providing tele‐consultation are scarce, but rates of 72‐hour returns were reported as equivalent to those for usual care in an urban US study of low acuity presentations,12 as were transfer rates in a rural US study.12

Quality of care indicators differed for inpatients seen exclusively by VRGS doctors versus those treated by both VRGS and non‐VRGS doctors. Inpatients in the combined care group were older and had greater clinical complexity. However, after adjusting for these factors, inpatients in the combined care group were more likely to be transferred to another hospital compared with those in the non‐VRGS group. In contrast, inpatients exclusively treated by VRGS doctors were less likely to be transferred compared with those in the non‐VRGS group, consistent with results from other studies which indicate that telemedicine reduces patient transfer through provision of remote access to medical expertise.26 Decision making relating to interhospital transfer in rural settings is complex; factors that need to be considered include capability at the presenting hospital, bed availability at the receiving hospital, patient preference and risks of transfer.27,28 These factors may have contributed to the different transfer rates for VRGS inpatients versus and combined care inpatients.

Similar to what we saw for ED presentations, the rate of discharge at own risk was higher for VRGS inpatients than for non‐VRGS inpatients (4.3% v 1.6%). Unplanned readmissions within 28 days, to any facility within Western NSW Local Health District or to the same facility, were largely unaffected by service type. The exception was a higher rate of unplanned readmissions to the same facility for the VRGS group. Length of stay was significantly shorter for the VRGS group and significantly longer for the combined care group. It is important to note that none of these differences were substantial, so they are unlikely to be of clinical significance. The reasons for these differences in quality‐of‐care indicators between the VRGS group and the combined care group are unclear, but they highlight the need for more research on safety when using virtual care models in rural tertiary settings with consideration of contextual factors.

Our analysis had several limitations. The lack of a specific data flag to indicate receipt of care from the VRGS necessitated reliance on PC039 logs of EMR entries and linking of doctor names with VRGS rosters. Not only was this method of patient cohorting reliant on the accuracy of data entry in these systems, but also a substantial proportion of ED presentations (29.3%) were not associated with a doctor name on the PC039 logs. However, a post hoc review of presenting problems in ED presentation records not associated with a doctor name (data not shown) indicated that the majority of these presentations were for reasons that would not typically require doctor involvement, including suture removal, investigation requests and wound care. In addition, we only analysed data that are routinely collected, so the measures of quality of care are quite high level, but these are standard measures that are used across the Australian health care system. Furthermore, data on incidents were collected from the NSW Health's incident management system (ims+), but there were too few data points for meaningful analysis of safety of care. Finally, the VRGS is a hybrid virtual model, predominantly providing virtual medical care, but our analyses did not distinguish between virtually provided VRGS care and in‐person VRGS care.

In conclusion, evidence on the quality of medical care provided by virtual services in emergency department and hospital settings is inconsistent, and for rural settings it is limited. Many evaluations of virtual ED services have focused on tele‐screening in urban settings for low acuity presentations and, to our knowledge, research on virtual medical care services for general inpatient care in rural settings is largely absent. In contrast, the VRGS is a unique hybrid virtual rural generalist service that has been shown to provide similar quality of care to existing ED and inpatient services, and has the potential to address rural workforce issues by complementing existing services. More research on the safety and utility of virtual generalist medical services is required, particularly in rural and remote settings where workforce challenges, poorer health and limited access to health care services make this a priority.

Box 2 – Associations between quality of care measures for ED presentations in which patients received care from the VRGS, New South Wales, 2021–22: adjusted multivariable analyses

---

ED = emergency department; WNSWLHD = Western NSW Local Health District. * 12 179 treated exclusively by a VRGS doctor, 1481 by both a VRGS doctor and a non‐VRGS doctor. † Data for those who re‐presented within 48 hours to any ED within WNSWLHD includes those who re‐presented to the same ED. ‡ Missing data: any VRGS care n = 19, non‐VRGS care n = 38. § Adjusted odds ratios were adjusted for patient socio‐demographic and clinical characteristics and for facility (details are in the Supporting Information, table 5).