Connect
MJA
MJA

Defining a core set of research and development priorities for virtual care in the post‐pandemic environment: a call to action

Kavisha Shah, Nicki Newton, Emma Charlston, Miranda Shaw, Jagdev Singh, Adam Johnston, Owen Hutchings, Chenyao Yu, Pearl Wang, Aaron Jones, Angus Ritchie, Rebecca Davis, Fiona Robinson, Jennifer A Alison, Melissa T Baysari, Meredith Makeham, Sarah Norris, Liliana Laranjo, Emma Nicholls, Clara K Chow and Tim Shaw
Med J Aust 2024; 221 (11): S49-S56. || doi: 10.5694/mja2.52524
Published online: 9 December 2024

Abstract

Objectives: To identify research and development priorities for virtual care following the coronavirus disease 2019 pandemic from the perspective of key stakeholders (patients, clinicians, informaticians and academics).

Design: Qualitative study using a modified nominal group technique.

Setting: Online semi‐structured interviews and workshops held in November 2022 and February 2023.

Participants: Health workers involved in delivering virtual care in two metropolitan local health districts and one specialty statewide network, and people who had received care from these sites, were recruited using passive snowball sampling. Research and academic staff from a tertiary institution were also invited to participate.

Main outcome measures: Priorities to support a translational research agenda for virtual care.

Results: Twenty‐five individuals participated including 18 innovation deliverers, two innovation recipients and five implementation facilitators. Stakeholders identified several key priorities for developing virtual care models and for sustaining and scaling virtual care services. These included demonstrating the economic and societal value of virtual care, developing a common framework to support evaluation and comparison of virtual care services, ensuring virtual care services integrate acute and primary care, and defining which models of care are most appropriate for virtual care delivery.

Conclusion: As the health system recalibrates with the return of in‐person care, there is a growing need to demonstrate the value of virtual care models to patients, the health system, and society at large. Demonstrating this value while also demonstrating improvements to health outcomes will future‐proof virtual care, enabling it to be used to address broader challenges of health care delivery. In addition, sustaining virtual care will depend on robust operational structures and workforce training and education. As services evolve, research and development priorities must be revisited to ensure that translational research aligns with stakeholder interests.

The known: Virtual care was successfully introduced during the coronavirus disease 2019 pandemic as an emergency solution to reduce spread of infection and support social distancing.

The new: Developing an evidence‐based rationale for the retention and/or expansion of virtual models of care is a key priority for stakeholders. In addition, the sustainability of virtual care will depend on the de‐implementation of low value virtual care as determined by consumers and the health workforce.

The implications: Evaluations of virtual care services should be prioritised. This should include development of industry standards for evaluating and monitoring virtual care services that can be applied and scaled across multiple settings and populations.

The coronavirus disease 2019 (COVID‐19) pandemic saw the rapid rise of virtual care and telehealth,1 enabling rapid outbreak responses and minimising care disruptions.2 The benefits continue to be realised as digital technologies now support continuity of care and health information sharing.3 While the affordability of modern digital technologies generates the potential for cost savings,4 inequity continues to influence quality of care for some populations.5 The resumption of in‐person care also creates a further challenge as health systems grapple with sustaining both virtual and in‐person modalities of care without the unifying imperative of a global pandemic. Research priorities must evolve accordingly to determine how virtual care models can be successfully integrated.

Evidence to support decision making about the role and scope of virtual care in the post‐pandemic environment is lacking. Building an evidence base that is relevant and acceptable to health services starts with reaching a consensus among key stakeholders about the values which should guide decision making.6 Similar priority‐setting processes have been used to provide strategic direction on resource allocation and maximise research investment.7 To our knowledge, few studies have explored research and development priorities for virtual care from a stakeholder perspective or, more specifically, how to seamlessly incorporate virtual services into standard practice. Defining such priorities can also inform implementation strategies relating to design, delivery and uptake of virtual care, thereby minimising gaps between research outputs and clinical practice (ie, evidence–practice gaps).8

In this study, we explored the research and development priorities for virtual care from the perspectives of stakeholders in collaboration with four virtual care services and an academic institution. This priority‐setting exercise was guided by the five domains of the Consolidated Framework for Implementation Research (CFIR) to understand innovation, relational and contextual factors that influence the implementation of research priorities within the health system.9

Methods

Setting

This study was conducted as part of wider research translation work being undertaken by Sydney Health Partners’ Virtual Care Clinical Academic Group. Sydney Health Partners is an accredited research translation centre that supports the implementation of evidence into practice and service delivery through partnership with academia and health services.10 The partnership includes representation from Northern Sydney Local Health District, Sydney Local Health District, the Sydney Children's Hospital Network, Western Sydney Local Health District and the University of Sydney.

Research team

The investigator team included an experienced implementation scientist and senior academic (TS), a project manager (EC), and two early career researchers (KS, NN) with experience in qualitative research. This investigator team was overseen by the leadership committee of the Virtual Care Clinical Academic Group, which includes most of the authorship team (MS, JS, AJ, OH, CY, AJ, MM, MB, JA, AJ, AR, RD, SN, LL, CC, TS). This committee included a consumer representative, managerial and clinical leads from partnering virtual care services and senior academics with expertise in qualitative methods, health economics, human factors and implementation science.

Participant recruitment

Individuals were eligible to take part in the study if they were involved in the delivery of virtual care services at partnering sites (innovation deliverers) or had received any part of their treatment or care virtually (innovation recipients).9 Individuals were also eligible to take part if they had specialised knowledge or expertise in an area relevant to virtual care and translational research, including digital health, health economics, human factors and implementation science (implementation facilitators). These participant groups were selected as they would be able to use their lived experience and subject matter expertise to inform priority setting and to lead, support or inform implementation.9

Purposive sampling was used to recruit innovation deliverers and implementation facilitators. We did this by approaching individuals in our professional networks, because potential participants were unlikely to have publicly available contact information, and those of us who are academic and virtual care leaders could identify relevant potential participants. Passive snowball sampling was used to recruit innovation recipients using a short summary of the research study, accompanied by a link to a participant information statement and consent form, which was distributed to consumer advisory boards at partnering sites. Participants were asked to forward the summary to their personal networks after completing an interview and/or workshop to minimise risks to privacy.

Recruitment for interviews continued until data saturation was reached and no new themes emerged. Recruitment for workshops continued until no new participants agreed to take part over a three‐month period. All three groups of innovation deliverers, innovation recipients and implementation facilitators were invited to participate in an interview.11

Design

This qualitative study was undertaken online in two stages using a modified nominal group technique consisting of online semi‐structured interviews followed by workshops to identify evidence–practice gaps and seek feedback on research and development priorities. Detailed information on study design, participant demographics, data collection and analysis for semi‐structured qualitative interviews is reported elsewhere in this Supplement.11

Two of us (NN, TS) analysed interview transcripts using a grounded theory approach of open coding as a first step to identify evidence–practice gaps in virtual care.12 These gaps were framed by investigators as research questions which were then iteratively grouped via axial coding into subdomains and domains based on similarities, differences and relationships between priority areas to create a priority‐setting matrix (Supporting Information, appendix 1).12 The matrix was reviewed by two of us (KS, MS) for consensus and clarity. It was then presented to workshop participants for feedback and identification of key research and development priorities. TS facilitated two 2‐hour online workshops in November 2022 and February 2023 using the Microsoft Teams videoconferencing platform.

Workshop participants were first asked to simultaneously review and provide feedback on the matrix in Google Docs (an online document editor); they could add research questions, subdomains and domains based on their experiences. This was followed by a group discussion. New themes, priorities and/or feedback raised were contemporaneously added by the research team in a second online document. Participants were then asked to anonymously rank the five evidence–practice gaps that were of most importance to them in the same online document. This was followed by a group discussion on the top ranked priorities (Box 1). Finally, in a third online document, participants were asked to list research solutions that they thought could address ranked priorities by closing evidence–practice gaps (Box 2).

Workshops were transcribed verbatim by a third‐party transcription service provider (Rev.com). Workshop transcripts were analysed thematically and iteratively by one of us (KS) and feedback, rankings, research outputs and translational projects created in the online documents were analysed descriptively by one of us (KS). In addition, one of us (NN) reviewed the workshop transcripts to confirm the themes.

Four of us (TS, NN, EC, KS) developed a research translation agenda for virtual care after the first workshop, and reviewed and iteratively refined this after the second workshop. Three of us (TS, NN, KS) used constant comparative analysis to document changes in research and development priorities over the three months between the first and second workshop.12 The three of us then selectively coded the top ranked research and development priorities against the 39 constructs from the CFIR,12 to inform development of a translational research agenda for virtual care,9 and presented this agenda to the other members of the authorship team for review and endorsement.

Ethics approval and reporting

This study was approved by the University of Sydney Human Research Ethics Committee (2022/213). It is reported in alignment with the Standards for Reporting Qualitative Research checklist (Supporting Information, appendix 2).13

Results

Participants

Newton and colleagues11 detail the interview process in the MJA supplement on the VRGS. Of 34 individuals who were sent recruitment materials through professional networks, 24 (71%) responded and took part in one of two the online workshops, which were held in November 2022 and February 2023. One recipient agreed to take part immediately after the second workshop and returned asynchronous feedback on the final online document detailing the rankings, research solutions and translational output reviewed by attendees of this workshop. Each workshop had representatives from all three participant groups: innovation deliverers, innovation recipients and implementation facilitators (Box 3).

A total of 66 priorities were identified across six domains of virtual care, and nine top ranked research and development priorities were identified. Only nine CFIR constructs from the first three domains (innovation, inner setting and outer setting) emerged as innovation and contextual determinants of the long term sustainability of virtual care (Box 4). Participants repeatedly raised service and process mapping as a vital first step to identify facilitators and barriers to service delivery, with repeated reference to the need for qualitative scoping studies, literature reviews and document analyses as possible research solutions. Such audits were seen as building blocks for robust and standardised economic evaluations that can demonstrate effectiveness and improve service delivery. Key quotations and research solutions for top ranked priorities, mapped against CFIR constructs, are shown in Box 5.

Innovation

Innovation design: defining virtual care

Participants were concerned about the overly narrow definition of virtual care, limiting possible scope to the immediate patient–provider relationship. Establishing an expansive definition has been complicated by a historical association of virtual care with telehealth delivered during the pandemic. One participant suggested shifting the focus to digitally enabled care that combines in‐person services with technology to improve patient experiences of care. It was considered important to extend understanding of virtual care to advice and specialist support between health care providers and patients being managed in the community to reflect existing and emerging virtual models of care.

Innovation adaptability: patient experiences

Better understanding of consumer preferences for virtual care and the impact of social determinants of health was believed to be the first step to improving quality and safety of virtual care and closing the gap in population inequities. Furthermore, in written feedback, most participants highlighted novel challenges faced by culturally and linguistically diverse populations when engaging with virtual care. To this effect, several participants advocated for a new domain of virtual care in the priority‐setting matrix that is focused on consumers, to support targeted research on the experiences and preferences of innovation recipients.

Innovation evidence base and relative advantage: evaluation and monitoring

Developing specific evaluation datasets was deemed important to enable virtual care services to demonstrate overall value to the health system with respect to safety and effectiveness. Most of the discussion about this focused on collecting data that are complete, accurate and representative of the patient population, outcomes of interest and complexities surrounding models of care and related workflows. Informaticians noted that data collected by the health system, such as patient‐reported experiences and outcome measures, must be tailored to the needs of relevant stakeholders. Research solutions included conducting a qualitative study of available datapoints to ensure representativeness and relevance to existing models of care and workflows. Likewise, other participants stressed the importance of identifying the type of modelling to be used for a broader economic evaluation of virtual care, while other participants highlighted the need to quantify benefits of virtual care unique to the patient such as improving work–life flexibility and time saved travelling to access care. Ease of assessment was also identified as a facilitator, with participants advocating for services to ask the right questions before collecting any data to minimise burden on patients and providers alike.

Outer setting

Financing

Most participants highlighted significant variation in organisational structure and lack of investment in virtual care across the health system. Insufficient resources reduce service capacity and capability, particularly the ability of virtual services to deliver seamless and integrated care. Discussions focused on the unsuitability of current activity‐based funding models for virtual care, where health services are paid for the number of discrete services delivered to patients. Many virtual care models are delivered by nurses or members of allied health teams — health professionals who cannot be reimbursed for their time under the current activity‐based models. In addition, virtual models of care encompass a range of activities, delivered in person and online, in a range of settings, and this complicates reimbursement for the entire patient journey. Participants supported the idea of a permanent transition to the use of block funding, under which each health service is allocated a grant to fulfil a specified purpose or combination of funding models.

External pressure: performance‐measurement pressure

Participants felt pressured to demonstrate the value of virtual care when seeking investment from the health system as activity levels and engagement with virtual care have dropped following the pandemic. Participants disagreed about which outcomes to prioritise in evaluations; some innovation deliverers focused on avoidable presentations, while others argued for patient‐reported measures. Compared with the first workshop, participants in the second workshop were more focused on the potential for funding to be removed from virtual care services as decision makers begin reprioritising in‐person care.

Inner setting

Structural characteristics: information technology infrastructure

Some participants discussed population‐specific issues with the transition to virtual care. Paediatric patients regularly encountered functionality issues as digital platforms are not designed with consent laws for children and young people in mind. User testing remained of the utmost importance to ensure that digital infrastructure supporting virtual care services is uniform across settings but adaptable to the differing needs of diverse patient populations. Leveraging user‐tested technologies across virtual care services was identified as a key translational output in written feedback.

Structural characteristics: work infrastructure

Workforce sustainability was considered a key priority, with most participants advocating for rotation of clinical staff across in‐person and virtual wards to maintain clinical competencies. Other participants recommended better integration of clinical, virtual and informatics staff to optimise hybrid models of care and enable staff to deliver a modality of their choosing. Participants also emphasised that programs of study focusing on virtual care and digital health technologies will be critical for resolving workforce shortages and embedding virtual care in the patient pathway as business as usual.

Relational connections: primary and tertiary care integration

Several participants raised the issue of lack of interoperability between acute and primary care services that deliver in‐person and virtual care, and how virtual care could bridge the gap. It was noted that these health siloes interrupt continuity of care as health care providers cannot communicate effectively, and because primary services cannot access patient information stored on acute care systems and vice versa. Different funding arrangements for primary and tertiary care, across federal and state jurisdictions, are a barrier here. Possible solutions include employing primary care providers as honorary staff members in acute services or monetarily incentivising the integration of systems.

Discussion

To our knowledge, this novel study is one of the first priority‐setting exercises to be conducted to develop a translational research agenda for virtual care in collaboration with key stakeholders. From both workshops, we identified emerging challenges for virtual care to be: innovation design, workforce sustainability, and building a robust evidence base to support implementation of virtual care models. Underlying discussions at both workshops was the importance of virtual care services learning from each other, reducing duplication of research investment, and setting industry standards collaboratively for virtual care. From these findings, we propose two agenda items and one standing item as research and development priorities for virtual care in the post‐pandemic environment.

The first agenda item we propose is demonstration of relative advantage. In our study, we documented a shift in priorities for virtual care — from integrating virtual models of care as business as usual to demonstrating safety and effectiveness of virtual care services — and this revealed a gap in translational research designed to support virtual care. While service‐related outcomes of interest (eg, emergency department avoidance and transport costs) have been routinely measured, studies that have measured these have been isolated to specific models of care, patient populations and clinical settings.14,15,16 Some authors have speculated that cost savings may be generalisable to the broader health system,14 but studies have fallen short of identifying the relational characteristics, communication networks and cultural constructs that promote change.9 Our study affirms the growing need to quantify non‐tangible costs and outcomes of interest to participants, such as social determinants of health and increased patient burden associated with care delivery that depends on their personal digital infrastructure.15 A key recommendation for policy makers is to redirect research investment towards developing a standardised template for economic evaluation of virtual care models and services. This template should enable interservice comparisons and be framed from a societal perspective to detect shifting costs outside of health care.17 To our knowledge, no such comparison or model has been developed or shared to date.

The second agenda item we propose is work infrastructure. In our study, stakeholders had major concerns about workforce sustainability if and when virtual care is promoted as a long term solution for increasing care burden and health care costs. Recent research has started to address this evidence–practice gap with the development of educational frameworks and curriculums focusing on skills of interest to study participants, including interprofessional collaboration and environmental considerations.18,19 Part of this training, as highlighted in our study, is enabling clinical staff to identify safe and appropriate technologies and tailor their use for a diverse range of patients.18,19 A key recommendation for policy makers is to rapidly adopt and accredit virtual care training and educational programs to ensure that these frameworks have their intended effect and attract skilled workers to virtual services.

As a standing item for research, we propose keeping our “fingers on the pulse”. The thematic differences between our workshops, which were about three months apart, illustrate the evolving needs of stakeholders in rapidly evolving fields like virtual care. To our knowledge, our study is the first to use a modified nominal group technique to review and refine recommendations. This priority‐setting exercise could be replicated over a protracted timeframe to capture the dynamism of health services decision making and ensure that translational research agendas remain relevant to end users — a key element of successful priority setting.6 Simultaneously, building partnerships between academic institutions and clinical services could enable real‐time translation of evidence into practice as implementation facilitators, implementation recipients and innovation deliverers could co‐design and implement research solutions that are acceptable to stakeholders and adapted to local contexts.19,20,21

A limitation of our study, which was verbally reported to investigators, was that the priority‐setting matrix was difficult to understand for individuals with limited expertise in health systems and familiarity with medical jargon. This comprehension barrier was perceived to affect consumer participation in the workshop and, as such, top ranked priorities may not be representative of most innovation recipients. It was recommended that an alternative matrix be prepared that is suitable for people with varying levels of health literacy and that consumer‐only workshops be held.

In conclusion, as we move beyond the COVID‐19 pandemic, there is a paucity of evidence to guide whether and how virtual care models should be integrated into the health system post‐pandemic. Of immediate concern to virtual care services is providing an evidence‐based rationale for the effectiveness of virtual care models to support retention of existing services. The sustainability of virtual care appears to depend on standardised education and training programs for the health workforce and recalibration of models around consumer preferences and priorities. As such, there is a growing need for multidisciplinary translational research to help sustain the delivery of high quality, evidence‐based virtual care.

Box 1 – Online document used for voting on priorities for translational research

Key questions

Detailed questions


Example: How do we demonstrate that virtual care services are effective?

Example: Are virtual care services and models as safe and effective as traditional face‐to‐face models?
VOTE: XXXX, Anonymous


 

Box 2 – Online document used to collect feedback on translational research and outputs*

Key questions

Detailed questions

What research could we do to address this?

What solutions or outputs would you like to see?


Example: How do we demonstrate that virtual care services are effective?

Example: Are virtual care services and virtual care models as safe and effective as traditional face‐to‐face models?

Example: Identify key indicators and outcome measures.

Example: Site where clinicians or consumers could easily access the research/evidence‐based guidelines to make informed choices.


* Text in the first two columns of this document was pre‐filled from the matrix, and participants added their responses in the last two columns.

Box 3 – Professional role and position of participants at each workshop

Role and position

First workshop (November 2022)

Second workshop (February 2023)


Innovation deliverers

13

5

 Executive/leadership

3

2

 Nursing

1

1

 Medicine

3

1

 Informaticians

2

1

 Non‐clinical/other

4

0

Innovation recipients

1

1*

Implementation facilitators

4

1

 Primary care

2

0

 Allied health

1

0

 Health economics

0

1

 Implementation science

2

0


* Participant did not take part in the workshop and shared their feedback asynchronously after the second workshop. † Two implementation facilitators had dual specialties in implementation science and primary care and allied health (one participant was implementation science/primary care, the other was implementation science/allied health).

Box 4 – Schematic representation of relationships between key CFIR constructs that are likely to influence research and development priorities and sustainability of virtual care*


CFIR = Consolidated Framework for Implementation Research. * The schematic shows the key levers that are anticipated to influence research and development priorities and/or implementation of research solutions within three domains: innovation (innovation deliverers and recipients, implementation facilitators), inner setting (mid‐level leaders) and outer setting (high level leaders).

Box 5 – Quotations and research solutions from participants of both workshops regarding the top ranked priorities, mapped against CFIR constructs

Top ranked priorities

CFIR construct

Quotations

Research solutions


What models of care are suitable for virtual care services? Which aren't and how do we define those?

  • Innovation design
  • “… identify which key components are common, effective, safe, innovating, replicable and translatable at scale while preserving limited resources.” (Anonymous, Workshop 1)
  • Audit existing virtual models of care to define what is and what is not virtual care
  • Delphi study with virtual care leaders to define scope of virtual care
  • Environmental scan or document review of internationally acknowledged frameworks to guide application in clinical practice; if none are suitable, create frameworks and guidelines for virtual care to drive change
  • Define accreditation processes for virtual care services
  • Define lines of responsibilities for different staff members, clinical or otherwise

How can we structure virtual care services to best support general practitioners and ambulance services? How do we promote virtual care services with primary, community and ambulance services? How do we enhance the sharing of information and data between acute, primary, community and ambulance care in virtual care services?

  • Relational connections
  • “… there is a complete chasm between the two. And when I think about the virtual hospital, we're trying to fill that chasm. These are the kinds of things we talk about all the time, but what's the role of the [general practitioner] in these models?” (Deliverer, Workshop 1)
  • “And if we could understand what the gaps we might have, we could plug with virtual care technologies.” (Facilitator, Workshop 1)
  • Qualitative interviews or focus groups with general practitioners and ambulance service staff to understand:
    1. ‣ their roles in virtual care
    2. ‣ how primary and community services deliver virtual care
    3. ‣ knowledge of virtual care services and pathways delivered by acute services
    4. ‣ attitudes towards virtual care
    5. ‣ preferences for engagement with virtual care services in acute care settings
  • Review interoperability of acute and primary care services

How do we introduce and/or improve e‐prescribing in the context of virtual care services?

  • Structural characteristics — information technology infrastructure
  • “Public health systems actually moving in time with the rest of health in Australia, and we'll improve that communication with [general practitioners], at least at the prescribing level, of being able to access what's been prescribed for their patients by other health services.” (Deliverer, Workshop 1)
  • User testing to modify technologies for paediatric populations
  • Qualitative interviews to understand stakeholder perspectives on implementation and sustainability of digital innovations, particularly e‐prescribing
  • Co‐design of best practice guidelines for digital health technologies
  • System mapping to understand national digital infrastructure

What factors such as culture, socio‐economic, age, location, disability, sexual identity and health and digital literacy impact on equity of access to virtual care services?

  • Innovation adaptability
  • “… we're sitting on a system where we have huge inequity in the availability of virtual care services, and the delivery of virtual care services. How do we improve that? How do we share better across the districts? … but we've got a long way to go to share more equitably across the system.” (Deliverer, Workshop 1).
  • “We have an issue with my virtual care that we use, because there's some functionalities that you have to be signed in as the patient, rather than the parent. And if you've got a 2‐week‐old baby, then the natural process is to, actually, sign in as a parent, because the child is not going to be talking. I think we do need to remember those things, because that's what really disables virtual care when you get to, actually, physically, using the devices.” (Facilitator, Workshop 1)
  • Replicate prioritisation study with a representative sample of consumers and carers
  • Qualitative interviews with priority populations to determine needs and preferences for virtual care, and how priority populations could be better supported
  • Co‐design a culturally safe model of care for priority populations such as culturally and linguistically diverse patients

Are virtual care services and virtual care models as safe and effective as traditional face‐to‐face models?

  • Innovation evidence base
  • Innovation relative advantage
  • “… a qualitative study of the data, and [making] sure the data are actually … representing the workflow level of care.” (Deliverer, Workshop 1)
  • “… if we're going to evaluate things, what data can be generally available to enable easy evaluation without onerous … where the patient can be consented when they enter the environment.” (Facilitator, Workshop 1)
  • “I think when we talk about things like avoidance of ED presentations and reductions in length of stay, it immediately reduces everything down to cost–benefit analysis. It's all about the dollars. Whereas what we really want to also demonstrate is value for money. And the way to do that is to have some outcome in the denominator that we are interested in.” (Facilitator, Workshop 2)
  • Economic evaluation of safety, effectiveness and cost‐effectiveness of virtual care compared with usual care from a health system and societal perspective; for example:
    1. ‣ audit existing models of care to identify common features
  • Identify comparators, key indicators and outcome measures, focusing on safety and avoidance of harm, by:
    1. ‣ running workshops with key stakeholders to understand data that should be collected for evaluation
    2. ‣ collecting data on patient‐reported experiences and outcome measures for good evaluation and monitoring
  • Policy review of funding models for virtual care at state and federal levels
  • Analyse a cross‐section of virtual models of care to identify which key components are common, effective, safe, innovative, replicable and translatable at scale while preserving limited resources

 

How do we work with state and federal funding agencies to ensure ongoing funding including block funding?

  • Financing
  • External pressure — performance‐measurement pressure
  • “… hampered by the ability to, even, bill for consulting, whereas [general practitioners] and primary health are not. There's just a difference between how … hospitals are funded to support [virtual care]. And I think that really influences how each of us are individually set up.” (Deliverer, Workshop 1)
  • “I find that we are constantly having to justify virtual care, prove virtual care. It's proven that it's added value that we can reduce the burden of cost in all our organisations. But we're still struggling with seeking investment.” (Deliverer, Workshop 1)
  • “… more we can do around PREMs in this space to try and quantify and qualify benefits of virtual care.” (Deliverer, Workshop 2)
  • “… people were very keen to see what we could deliver, and we're excited about the innovation, but when it comes to the crunch and everyone's looking at their budgets and we're looking at staffing, less of a priority and the support is dwindling.” (Deliverer, Workshop 2)
  • Economic evaluation of safety, effectiveness and cost‐effectiveness of virtual care compared with usual care from a health system and societal perspective; for example:
    1. ‣ audit existing models of care to identify common features
  • Identify comparators, key indicators and outcome measures, focusing on safety and avoidance of harm, by:
    1. ‣ running workshops with key stakeholders to understand data that should be collected for evaluation
    2. ‣ collecting data on patient‐reported experiences and outcome measures for good evaluation and monitoring
  • Policy review of funding models for virtual care at state and federal levels
  • Analyse a cross‐section of virtual models of care to identify which key components are common, effective, safe, innovative, replicable and translatable at scale while preserving limited resources

How can we develop the workforce skills required for effective delivery of virtual care? For example, communication, recognising deterioration, technical skills

  • Structural characteristics — work infrastructure
  • “… do a combination of virtual and direct patient care so that they are able to cross‐contaminate between the two and have the discussions with the clinicians in the acute area, but also go back to their virtual care and maintain their clinical skills.” (Deliverer, Workshop 2)
  • “… advertise the benefits of virtual care and try to get [virtual and face‐to‐face care] more integrated with each other.” (Deliverer, Workshop 2)
  • Observational studies of patient–clinician interactions to identify key workforce capabilities and competencies
  • Qualitative interviews with clinicians and patients about essential capabilities and competencies for virtual care staff from health professionals to administrators
  • Develop educational modules that cover virtual care competencies
  • Literature review of virtual care models used in Australia and overseas to understand operational requirements
  • Review of existing health care employment awards, with a view to adapting awards or developing new employment awards specific to virtual care

CFIR = Consolidated Framework for Implementation Research; ED = emergency department; PREM = patient‐reported experience measure.

Received 10 July 2024, accepted 19 August 2024

  • Kavisha Shah1
  • Nicki Newton1
  • Emma Charlston2
  • Miranda Shaw3
  • Jagdev Singh4
  • Adam Johnston5
  • Owen Hutchings3
  • Chenyao Yu5
  • Pearl Wang5
  • Aaron Jones6
  • Angus Ritchie7
  • Rebecca Davis7
  • Fiona Robinson5
  • Jennifer A Alison1,6
  • Melissa T Baysari1
  • Meredith Makeham1
  • Sarah Norris8
  • Liliana Laranjo9
  • Emma Nicholls1
  • Clara K Chow9
  • Tim Shaw1,9

  • 1 University of Sydney, Sydney, NSW
  • 2 Menzies School of Health Research, Darwin, NT
  • 3 RPA Virtual Hospital, Sydney Local Health District, Sydney, NSW
  • 4 virtualKIDS Urgent Care Service, Sydney Children's Hospital Network, Sydney, NSW
  • 5 Northern Sydney Local Health District, Sydney, NSW
  • 6 Sydney Local Health District, Sydney, NSW
  • 7 Royal Prince Alfred Hospital, Sydney, NSW
  • 8 Leeder Centre for Health Policy, Economics and Data, University of Sydney, Sydney, NSW
  • 9 Westmead Applied Research Centre, University of Sydney, Sydney, NSW


Correspondence: kavisha.shah@sydney.edu.au


Data Sharing:

The deidentified data we analysed are not publicly available; requests for the data will be considered on a case‐by‐case basis.


Acknowledgements: 

This research was supported by funding received from Sydney Health Partners for the Virtual Care Clinical Academic Group. The funding provided salary support for Kavisha Shah, and research and administrative support to deliver the workshop (eg, to cover printing costs). The funder had no direct involvement with the research.

Competing interests:

No relevant disclosures.

  • 1. Demaerschalk BM, Hollander JE, Krupinski E, et al. Quality frameworks for virtual care: expert panel recommendations. Mayo Clin Proc Innov Qual Outcomes 2023; 7: 31‐44.
  • 2. Bhatia RS, Chu C, Pang A, el al. Virtual care use before and during the COVID‐19 pandemic: a repeated cross‐sectional study. CMAJ Open 2021; 9: E107‐E114.
  • 3. Hardcastle L, Ogbogu U. Virtual care: enhancing access or harming care? Healthc Manage Forum 2020; 33: 288‐292.
  • 4. Walter RJ, Schwab SD, Wilkes M, et al. Financial and clinical impact of virtual care during the COVID‐19 pandemic: difference‐in‐differences analysis. J Med Internet Res 2023; 25: e44121.
  • 5. Schwamm LH, Estrada J, Erskine A, et al. Virtual care: new models of caring for our patients and workforce. Lancet Digit Health 2020; 2: e282‐e285.
  • 6. Sibbald SL, Singer PA, Upshur R, et al. Priority setting: what constitutes success? A conceptual framework for successful priority setting. BMC Health Serv Res 2009; 9: 43.
  • 7. Centre for Epidemiology and Evidence. Setting research priorities: a guide. Sydney: NSW Ministry of Health, 2023.
  • 8. Ivers NM, Grimshaw JM. Reducing research waste with implementation laboratories. Lancet 2016; 388: 547‐548.
  • 9. Damschroder LJ, Reardon CM, Opra Widerquist MA, et al. The updated Consolidated Framework for Implementation Research based on user feedback. Imp Sci 2022; 17: 75.
  • 10. Sydney Health Partners. Translating research into better health outcomes. https://sydneyhealthpartners.org.au (viewed Apr 2024).
  • 11. Newton N, Shah K, Shaw M, et al. Barriers, facilitators and next steps for sustaining and scaling virtual hospital services in Australia: a qualitative study. Med J Aust 2024; 221 (Suppl): S37‐S48.
  • 12. Akkaya B. Grounded theory approaches: a comprehensive examination of systematic design data coding. Int J Contemp Educ Res 2023; 10: 89‐103.
  • 13. O'Brien BC, Harris IB, Beckman TJ, et al. Standards for reporting qualitative research: a synthesis of recommendations. Acad Med 2014; 89: 1245‐1251.
  • 14. Lovell T, Albritton J, Dalto J, et al. Virtual vs traditional care settings for low‐acuity urgent conditions: an economic analysis of cost and utilisation using claims data. J Telemed Telecare 2019; 27: 59‐65.
  • 15. Jacobs JC, Hu J, Slightam C, et al. Virtual savings: patient‐reported time and money savings from a VA national telehealth tablet initiative. Telemed J E Health 2020; 26: 1178‐1183.
  • 16. Patel KB, Turner K, Tabriz AA, et al. Estimated indirect cost savings of using telehealth among nonelderly patients with cancer. JAMA Netw Open 2023; 6: e2250211.
  • 17. Byford S, Raftery J. Perspectives in economic evaluation. BMJ 1998; 316: 1529‐1530.
  • 18. Health Education and Training Institute. Curriculum interventions and pedagogical approaches for virtual care: a literature review. Sydney: HETI, 2022. https://www.heti.nsw.gov.au/__data/assets/pdf_file/0004/805756/NSW‐Health‐Virtual‐Care‐Literature‐Review_Full‐Report_single‐pages.pdf (viewed Apr 2024).
  • 19. Curran V, Hollett A, Peddle E. Training for virtual care: what do the experts think? Digit Health 2023; 9: 20552076231179028.
  • 20. Westwood G, Richardson A, Latter S, et al. Building clinical academic leadership capacity: sustainability through partnership. J Res Nurs 2018; 23: 346‐357.
  • 21. Albert NM, Chipps E, Olson AC, et al. Fostering academic‐clinical research partnerships. J Nurs Adm 2019; 49: 234‐241.

Author

remove_circle_outline Delete Author
add_circle_outline Add Author

Comment
Do you have any competing interests to declare? *

I/we agree to assign copyright to the Medical Journal of Australia and agree to the Conditions of publication *
I/we agree to the Terms of use of the Medical Journal of Australia *
Email me when people comment on this article

Online responses are no longer available. Please refer to our instructions for authors page for more information.