Exploring the introduction of remote external clinical teaching visits

The COVID-19 pandemic has changed the way that education is delivered. Change has happened in schools and universities, and in the spaces of general practice training. Although a move to digital platforms began prior to 2019, COVID-19 amplified the speed of that move.

The external clinical teaching visit (ECTV) was developed in Australia in the mid-1980s to augment in-practice teaching. Most general practice registrars have five ECTVs during their training. The visit has both an assessment and educational function. It is conducted by a trained medical educator or general practitioner (GP) supervisor from another practice, who directly observes the general practice trainee (registrar) during four to five consultations with consenting patients. Each consultation, with subsequent feedback and discussion, takes around 30 minutes. The visitor then meets with the registrar’s supervisor to discuss the registrar’s progress.

Although ECTVs have been conducted in Australia for the past 40 years, there is a paucity of research exploring participants’ experiences. Of the studies undertaken, these have focused on registrars’ satisfaction with ECTVs,¹ the use of different teaching and assessment tools within the visit^2,3 and the uptake of ECTVs regionally.⁴ A questionnaire-based study is planned to look at the utility of different modalities within the ECTV.⁵

The COVID-19 pandemic meant that all ECTVs had to be undertaken remotely from early 2020, raising questions about feasibility and effectiveness, including cost-effectiveness. It might be that the remote ECTV could totally or partially replace the face-to-face visit, but it would be useful to know which elements of the ECTV are equivalent, or perhaps even better, remotely, and for which registrars, based on situated research.

Remote supervision has been a part of GP education in Australia for many years, mostly due to the tyranny of distance.⁶ We found only one earlier study that had piloted remote ECTVs.⁷ That study reported that the technology was relatively new, and there were several technological challenges. Apart from these challenges, the authors noted positive aspects of remote supervision, including flexibility, reduced travel time and reduced intrusiveness.⁷ Negative aspects included the loss of the overall ‘feel’, a reduction in direct personal engagement and issues around confidentiality.⁷

Studies investigating the experiences of remotely supervised GP trainees in Australia and Canada have noted that the effectiveness depended on registrar factors, such as attitude, experience and personality, as well as factors external to the registrar, including the relationship to the supervisor.^6,8,9 In a study of the experience of supervisors in the Remote Vocational Training Scheme (RVTS), Wearne found that supervisors felt that this type of remote supervision was effective given the right circumstances, particularly if it was ‘triangulated’ with face-to-face visits, including ECTVs and other data, such as case notes and letters.⁸ Wearne et al discussed the advantages and disadvantages of remote supervision, including the relative speed in learning because of the autonomy, but noted the probability of missing ‘blind spots’ and the potential for bad habits to become ingrained.⁹

A review of the experience of doctors regarding clinical supervision and educational support via videoconferencing by Cameron et al¹⁰ looked at research articles between 1990 and 2013. The authors noted positive aspects of videoconferencing, including increased education and clinical practice support, as well as autonomy for rural areas. Challenging aspects related to the application of the technology itself, including poorer interaction and decreased rapport building.¹⁰ The authors also noted a paucity of literature regarding the perceptions of videoconference-based supervision of junior doctors in rural areas, and that no studies had considered measures of cost-effectiveness.¹⁰ A subsequent review on telesupervision by Martin et al identified ways to be most effective in using technology in this context.¹¹ This included the need for good communication, technological support, the importance of the supervisory relationship and a need for extra time, flexibility and ongoing review of the process.¹¹

Research on remote supervision suggests that conducting the ECTV remotely might be feasible and effective, and have some potential advantages as well as disadvantages. In a post-COVID-19 world, research that provides evidence on the effectiveness and appropriateness of remote ECTVs for assessment and learning would inform not only the rollout of ECTVs, but also potentially other remote learning and supervision practice.

Aim and research questions

This research aimed to explore the experiences of the different participants in remote ECTVs to try to answer the following questions:

1. What are the learner, visitor and practice personnel experiences of the remote ECTV process, including acceptability, logistics and effectiveness for learning?
2. What are the strengths and weaknesses of the remote ECTV perceived to be in observing and recording aspects of the clinical consultation?
3. What perceptions do registrars, visitors and the regional training organisation (RTO) have of the remote ECTV as a learning and assessment tool?
4. What are the perceived advantages and disadvantages of remote compared to face-to-face ECTVs, and how does this inform future practice?
5. Is it possible to identify when registrars might benefit from a face-to-face ECTV as opposed to a remote ECTV?

Methods

Because we were exploring and attempting to understand human experience, we used an interpretive qualitative approach, gathering the different participants’ interpretations of their experience.¹² We were not measuring outcomes, but hearing the stories of the participants.

Using a descriptive research design, we began with an organisation-wide survey of personnel (registrars, supervisors, ECT visitors and practice managers) involved in remote ECTVs within Murray City Country Coast (MCCC), one of the two GP training organisations in Victoria. Data from the surveys informed the next stage, which involved in-depth semistructured interviews with ECT visitors, supervisors and practice managers, and focus group discussions with registrars.

A stratified block sampling method was used for the selection of registrars for focus group discussions. Practice managers, GP supervisors and ECT visitors were recruited for semistructured interviews through purposive sampling.

Data from focus group discussions and semistructured interviews were recorded, transcribed and stored in a password-protected database accessible only by the research team. The data were analysed thematically by the research team using a constant comparison approach.¹³ This was undertaken individually by team members; then, through a series of team meetings, a consensus was reached on major themes and subthemes.

Results

Surveys

Results from the surveys indicated that a large majority of all participants found the remote ECTV to be acceptable, feasible and effective in fulfilling its purpose. Additional comments suggested that there were advantages of doing the visit remotely, as well as disadvantages.

Survey results identified themes that were explored through the focus group discussions and semistructured interviews. Details of the participants involved in the interviews and focus groups are presented in Table 1

Interviews and focus groups

Thematic analysis of the interview and focus group data identified four broad themes, each of which contained several subthemes. We developed a visual representation of these themes, as shown in Figure 1.

Figure 1. Interactive elements of the remote external clinical teaching visit (ECTV).

The arrows and circular nature of the figure aim to illustrate something of the complexity and interactive nature of the ECTV. Each organising category informs and is informed by all others; for example, assessment requires communication, which depends on good technology, and then informs the feedback and learning that takes place, and so on.

Themes

Findings are reported under the following themes: Technology; Assessment; Feedback and learning; and Communication and interaction. Quotes from participants are provided, with participants identified as V (visitor), R (registrar) or PM (practice manager).

Technology

It is obvious that good-quality picture and sound are crucial to the success of a remote ECTV:

The main issue we sometimes had was sound quality … So depending on what device they were using and where the microphone was, sometimes I found it difficult to hear everything. They were all wearing masks most of the time, so that sometimes made it harder too. (V)

What else has been challenging? I think sometimes trying to angle the camera. So sometimes a registrar can focus it on themselves or focus it on the patient, but it’s really hard to get both of them in the same frame. (V)

The second ECTV didn’t go well because we didn’t have technology … the internet was not clear, was not good … my audio was all breaking up. So he couldn’t hear my audio properly. He could only really see my face because the camera was also not ideal. (R)

Most participants indicated that the technology used was appropriate and that familiarity and adaptability to the technology improved with time and use:

To start with, it was clunky, but now that the practices are used to the processes … it’s actually relatively easy. (V)

From the perspective of the visitor, it was identified that technology was more challenging if the registrar is undertaking a telehealth consultation:

I think with face-to-face consults they work better, with telehealth consults less so because sometimes there’s issues with sound quality, video quality. So it can actually be quite hard to follow what’s going on if the registrar is doing telehealth. (V)

The use of the shared screen was helpful because the visitor could see how the trainee used online resources and the medical record, and wrote clinical notes:

Share screen for looking at the notes is really awesome. (V)

Connectivity was raised as an important aspect:

Connectivity issues out in the regional areas. That’s probably at the top of this list of challenges. While NBN promises to be a wonderful thing for all of us, it’s certainly not. IT in the regional areas is tricky, and sometimes it can be hard to fix long term. (PM)

The efficiencies inherent in a remote ECTV were identified by all cohorts: registrars, visitors, supervisors and practice managers. The efficiencies included time savings for visitors, with significantly less travel required; the cost savings for the RTO; the ease and flexibility of organising for the practice managers; and the greater pool of visitor experience from which to draw. Savings for the environment were also noted:

They’re saving money on our travel time. And the planet. Not driving and polluting. (V)

Assessment

All cohorts (registrars, practice managers, supervisors and visitors) recognised that remote ECTVs tended to reduce the level of anxiety for registrars and provide a more natural assessment where the registrar forgot about being observed by the assessor and was more likely to behave as they normally would:

It’s easier just to get into the motions of a normal consultation when there’s not a physical person sitting in the room. (R)

The registrars were often impressed by what the visitors could see and hear, including subtle interactions with patients. In contrast, visitors commented on the reduced ability to view subtle cues and body language.

I was very surprised that they … didn’t miss anything in the consult. (R)

I think we do miss things that we would see if we were there in the room … you often do miss subtle things in the doctor–patient interaction … we very rarely get to see physical examination. (V)

All participants noted the lack of ability to observe examinations and procedures:

I think the downfalls would be clinical examination is really hard to appreciate and get feedback on. (R)

Many visitors felt that there was a reduced ability to assess overall registrar experience, relationships within the practice and the overall practice ‘vibe’:

Being able to see the car park, the surrounds, what the patients put up with. How the reception staff treat people. How the practice nurse, the treatment room, how the camaraderie is, or the disconnect. The physical layout. If it’s structurally well set-up, ergonomically well set-up. The registrar doesn’t just have the broom closet. (V)

Feedback and learning

The feeling from most of the participants was that high-quality feedback was possible:

The remote process allows you to get good feedback on most things. Things like non-verbal communication and seeing how the patient reacts is tricky. (V)

They got everything and they gave me very good feedback. (R)

Several visitors noted the importance of a good rapport when delivering feedback and had concerns about this, particularly if they had not met the registrar previously face to face:

You’re missing a lot of those subtle cues that might help decide where you’re heading with your feedback. (V)

Others felt that registrars who take feedback well will do so regardless of mode of delivery, and registrars themselves can take into account the difficulties of being assessed remotely:

I think generally the registrars who are going to take feedback well are going to take it well regardless. (V)

The feedback that is – even if I … if I take it to heart and feel bad afterwards about how things went, if it’s remote, then it … I have a thicker skin for it [laughs]; you know what I mean? (R)

It was noted that learning and working generally through the time of COVID-19 has been difficult:

Everybody including registrars, are feeling quite brittle, and so I do find … that you’ve got to be much more gentle and cautious in the discussions I have … I have found. (V)

Communication and interaction

The ECTV involves a lot of communication, including RTO to practices, visitor to registrar, visitor to practice manager, visitor to supervisor and visitor to RTO. Concern was expressed about loss of human contact and that it was less personal doing the visit remotely:

I think just generally the humanity of it. I think you do get more of an impression of the registrar’s warmth and how they’re managing patients in that kind of humanistic sense. It’s very difficult to see that behind a video camera. (V)

For me, the only difference was the personal touch; you see a person there and you feel that you can have – you’re talking to someone – a real person … (R)

The difficulty in hearing came up again in this context, as did the missing out on registrar–staff interactions:

That’s probably an area where that’s one of the main advantages of face-to-face, to really see how the whole team works and meet the supervisor for the first time and the nurses and admin and see how the registrar interacts with the team. (V)

Pastoral care of the registrar might be more difficult remotely:

I think the pastoral care part of it is much harder. (V)

The arrows and circular nature shown in Figure 1 aim to illustrate something of the complexity and interactive nature of the ECTV. Each organising category informs and is informed by all others; for example, assessment requires communication, which depends on good technology and then informs the feedback and learning that takes place, and so on.

Future rollout of remote ECTVs

All cohorts were asked about how ECTVs should be used in a post-pandemic world. The majority thought that a combination of remote and face-to-face ECTVs would be the best approach, noting the advantages and disadvantages of each method (Table 2):

I think it’s got to be seen as something that is built on, rather than just a simple replacement. It’s an entirely different way of doing things … I wonder if it’s worth having a face-to-face teaching visit as a totally separate bucket to a Zoom ECTV. That they’re two separate things. (V)

For the very first one, in the initial assessment at the beginning of GPT1, I think that’s an important one to do face to face. (V)

I think there should be a choice, perhaps, in there, so that you can nominate what you would prefer, perhaps. So that way, people who might prefer face-to-face and would benefit more from it can do that, (because) we all have different learning styles. (R)

I would say we haven’t got one, but if we had registrars where English wasn’t a first language, or English wasn’t as strong, for either the registrar or the supervisor, I think that language barrier could make it a little bit more challenging being remote rather than face-to-face. (PM)

Discussion

Given the scarcity of direct research in the field, the currency of the topic and its relevance to medical education going forward, the findings of this study contribute significantly to the understanding of the introduction of remote ECTVs into GP training and provide guidance on how they might be used.

Our findings suggest that conducting ECTVs remotely is not only feasible and effective, but also has some advantages over the traditional face-to-face modality. The ‘authenticity’ of the observed interaction between registrar and patient, along with the reduced intrusiveness of not having an extra person sitting in, was mentioned by all participants. This has been noted in other studies on remote supervision.^6,7,14

Another advantage was the ability for the visitor to share the screen with the registrar, thus being able to observe the registrar’s note taking and follow their clinical reasoning and thinking. This has not been previously mentioned in the literature on remote supervision. The ability to do this depended on the technology used by practices and, in our study, many could not. With appropriate infrastructure support, this could become common practice.

The issue of technology was clearly important in this study and has been highlighted in the past by several studies.^7,11,15–18 We had thought that the issue of connectivity might have highlighted differences between rural and metropolitan practices, but in this study this was not the usual case. Connectivity issues were infrequent and no more common in rural areas. More frequent were issues of having the right hardware and software, quality camera(s), good camera positions and, most importantly, good audio.

The lack of consistency in the type and quality of equipment across practice contexts and broader geography points to a potential need for specific financial resourcing for teaching practices involving information and communications technology into the future. There is a need for equitable access to funding for hardware and software, as well as training to address discrepancies in user knowledge of computer systems, laptops, smartphones and cameras in synchronous and asynchronous teaching and learning situations. Interestingly, the technology issue was less frequently raised as a problem the further we went into the study. We interpreted this as the visitors and registrars, as well as the practices, becoming more used to it, and learning as they went. This points to the need for specific training for visitors and registrars on conducting remote ECTVs, and a handbook for practices on the running of a remote ECTV. Several visitors mentioned a desire for training, along with a wish to share their experiences with other visitors.

A disadvantage of the remote ECTV was a reduced ability of the visitor to observe and demonstrate clinical examination and procedural techniques. This has been noted in previous studies.^17,19 These examination and procedural skills are central to general practice, but it was noted that the registrar’s supervisor is already observing examinations and teaching procedural skills. For example, within the Australian College of Rural and Remote Medicine, registrars are required to undertake a number of mini clinical examination exercises, which includes the assessment by their supervisor of various types of physical examination. This deficiency in the remote ECTV can clearly be compensated for.

Another disadvantage that several visitors and registrars mentioned was that the remote ECTV was less accurate at observing the subtle nuances in registrar–patient interactions. This has also been noted previously.^6,7,14 Interestingly, several registrars volunteered that they were surprised at the skill of the visitor in being able to pick up subtle nuances of body language and interpersonal communication. It was also noted by a visitor that this difficulty in observing subtleties could be compensated for by the visitor asking the registrar to describe what happened, thus aiding not only in teasing out what went on during that interaction, but also contributing to the registrar’s reflection on the whole consultation. These findings add weight to the importance of training for visitors undertaking ECTVs, whether remote or face to face.

The importance of face-to-face human interaction for optimal learning was highlighted in a recent study by Smith et al¹⁸ exploring the experiences of registrars and medical educators in the change from face-to-face to online learning. The importance of this direct human contact also came out in our study. The assessment side of the ECTV is not only to evaluate the competence of the registrar to function as an independent practitioner, but also to consider them from a wellbeing perspective and check that the experience they are having in the practice is positive and fruitful. Most visitors mentioned that this overall holistic assessment was more difficult to do remotely.

Communication and relationship building between practices and the RTO was mentioned by several practice managers and visitors. Although the practice managers appreciated the communication from the RTO and were complimentary on the accuracy and efficiency of this, they also mentioned the loss of the ‘human face’ of the organisation.

There are differing ideas about the ability to establish good rapport during online interaction.^6,10 This came out in our study. Many participants, particularly visitors, noted the importance of meeting face-to-face to develop rapport, and that the visit was easier if they had already met the registrar face to face. The importance of meeting face to face was also mentioned when visitors were talking about giving feedback, which they felt was easier when there was a better rapport. Registrars were mostly happy with the online giving of feedback and felt that the learning experience within the remote ECTV was extremely valuable. That there might be differences in ideas about feedback between registrars and visitors fits in with a large study on feedback by Dawson et al, which found differences in the perspectives from staff and students on what makes for effective feedback.²⁰ The ease and speed of writing the report, which goes to the registrar and forms part of the feedback, was noted by visitors to be better remotely.

The findings of this study might help in deciding on the timing and mode of future ECTVs. Criteria to keep in mind when planning should include the stage of training, the ‘newness’ of practices (ie the practice is new to taking registrars), registrars who might be struggling, visitor availability and registrar choice.

Limitations

This study was undertaken with 37 participants at a particular time and locations across one of two RTOs in Victoria. Our findings might not translate to different contexts.

This study was an ethics-approved low-risk project that precluded the evaluation of the patient’s experience of the ECTV.

This study was not designed to measure the cost-effectiveness of doing the ECTV remotely, but there are clearly several efficiencies in doing it in this way.

Conclusion

In summary, this study has contributed to the literature on remote teaching and assessment by focusing on the phenomenon of doing ECTVs remotely. We have found the remote ECTV is an effective assessment and learning tool but is different to the face-to-face ECTV. Remote ECTVs cannot replace face-to-face ECTVs but can complement and fruitfully enhance them given the appropriate level of technology and training. Remote ECTVs could be improved by funding teaching practices to have consistent quality information and communications technology equipment and guidelines for their use, plus ECTV guidelines and training for visitors, registrars and practices. These findings might help in future rollout of ECTVs.