Evaluation of the Use of the Eyesi Virtual Reality Surgical Simulator byResidents and Medical Specialists in the Argentine Council of Ophthalmology

Purpose: To assess whether the use of virtual reality surgical simulators accelerates the learning curve in cataract surgery, decreases the anxiety and reduces the complication rate in a real patient. Methods: A Virtual Simulation Course of Cataract Surgery was carried out at the headquarters of the Argentine Council of Ophthalmology using the EyeSi simulator (VRmagic) between January to November 2018. At the end of the course resident physicians and specialists answered an anonymous survey to evaluate the objectives of this work. Results: In total 12 physicians completed course and answered survey, 2/3 were specialists and 1/3 residents or concurrents. Prior to course start, most surgeons felt very confidents in final steps of cataract surgery and were not confidents in initial steps. The majority considered that simulator improved their performance in steps in which they had more difficulty and felt less confident (capsulorhexis, phacofracture and removal of quadrants), but only a minority felt that improved their performance in complex cases (pupillary dilators use and anterior vitrectomy). The 100% of participants considered that virtual simulation served to accelerate their learning curve of different steps of cataract surgery and decrease their rate of complications in a real patient, while 91.7% answered that it will reduce anxiety during the surgical act. Conclusions: The use of virtual reality surgical simulators by resident physicians and specialists of a few years of professional practice is beneficial to shorten the learning curve of the different steps of cataract surgery, reduce stress and anxiety during the act surgical and decrease the rate of complications in real patients.


Introduction
Surgical training and the acquisition of skills in specific surgeries is increasingly difficult in medical specialties due to the shortage of patients willing to undergo surgery by surgeons in training and the medical-legal risks that this practice entails. Learning to perform cataract surgery with phacoemulsification is a very important part of ophthalmology residency programs, although the number of cases performed by a resident can vary between 50 to 300 [1]. Limitations to surgical training in ophthalmology residency programs include financial costs, human costs, and time constraints. One of the major benefits of surgical simulation is that there is no risk of harm to patients [2].
It has been shown that the rate of complications in surgery is affected by the experience and skill of the surgeon [3,4]. The use of surgical simulators in a residence offers the potential to improve results and decrease the rate of complications in surgeries performed by residents, given their ability to develop the coordination between the two hands and two feet required in surgery to remove the cataract [5]. Many authors proposed that the use of virtual reality simulators would be more successful if they were incorporated into a systematic training program [6]. Feudner EM, et al. [7] reported superior performance in the wet pig-eye laboratory in residents who trained with the EyeSi virtual simulator. This is very important since the need for new animal tissue and the time consuming preparation of pig eyes limits the possibility of performing specific surgical steps on multiple occasions [7].
The EyeSi simulator is a state-of-the-art equipment that allows practicing simple and complex cases, learning the use of surgical instruments and adding professional experience in an environment that does not pose risks to the health of patients [8,9]. Regarding the performance of cataract surgery in a real operating room, Belyea DA, et al. [5] demonstrated that simulator-trained surgeons had shorter phacoemulsification times and lower ultrasound powers [5]. Pokroy R, et al. [10] demonstrated that training in virtual reality surgical simulators shortens the learning curve in the first 50 cases of cataract surgery, and noted that less apt residents benefit more during training [10]. Another study found that the scores from the EyeSi simulator modules used early in the residence can predict a resident's future performance in actual surgery and may allow for the identification of residents who need supplemental training in cataract surgery [11].
Therefore, the objective of this work has been to evaluate the utility of the virtual reality EyeSi surgical simulator for cataract surgery in the training of an ophthalmologist.

Study design
A prospective, observational, descriptive cross-sectional study was carried out between the months of January to November 2018, which included the doctors who attended the Virtual Simulation Course of Cataract Surgery at the headquarters of the Argentine Council of Ophthalmology, in the City of Buenos Aires, where the EyeSi simulator virtual reality surgical simulator (VRmagic, Mannheim, Germany) is located, which is the most commonly used simulator in ophthalmic surgery.

Parameters to evaluate
A psychometric evaluation was carried out where the student had to answer at the end of the course, an anonymous survey designed specifically for this study, structured with closed questions with dichotomous, polytomous or multiple-choice answers (Annex 1).

Features of the EyeSi simulator
Its state-of-the-art technology provides a three-dimensional interactive experience with an exceptional degree of realism and without risk in patients. It has a virtual operating microscope that simulates stereoscopic images and manual probes that simulate virtual instruments when inserted into a virtual eye ( Figure 1). The monitor shows the phacoemulsification parameters, which can be modified by the tutor or the practitioner at any time, just as it happens in real surgery ( Figure 2). Allows for repeated performance and measurement of instructor-defined surgical steps. Its hardware and software permanently record measurements that provide feedback in 4 main categories: surgeon efficiency, surgical goal achievement, surgeon injury or error, and formative education during a task [8,9].

Course content and modality
The course consisted of 6 modules: 2 virtual (4 hours) and 4 face-toface (20 hours). The virtual course included a module of introduction to the cataract and one of introduction to the use of the simulator. Both must have been completed and approved at the time of the first faceto-face session. The face-to-face course was divided into 10 sessions of 2 hours each. The maximum time for face-to-face study was 3 months from the date of the first session. The student had to complete tasks and obtain certain amounts of points to advance to the next level of complexity. A trained surgeon (instructor) supervised and advised the student in person, personalized and constantly. MODULE 1 was theoretical, from Introduction to Cataract was taken online, mandatory and prior to the use of the simulator with duration of 3 hours and was divided into eight classes: Phacodynamics, Phaco techniques, Complications and difficult cases, Injection of IOL, Continuous circular capsulorhexis, Incisions, Cortical mass and viscoelastic aspirations and Hydrodissection. The module was carried out through the CAO Virtual Platform. MODULE 2 of Guidance on Simulator Management was taken online, mandatory and prior to using the simulator. It lasted 1 hour and consisted of an orientation class on the handling of the simulation equipment. The module was made through the VR Magic Platform.

MODULE 3 of Introduction (CAT-A) covered four parts: Anterior
Chamber Navigation, Intracapsular Navigation, Two-hand Navigation and Instruments. All tasks had to be completed in sequential order and required 50 points (to be achieved by completing the task correctly three times in a row) to advance to the next step ( Figure 3).

MODULE 4 of Cataract Beginners (CAT-B) covered five parts:
Navigation and Instruments, Capsulorhexis, Intracapsular Tissue, Stop and Chop and IOL Insertion. All tasks had to be completed in sequential order and required 60 points (to be achieved by completing the task correctly three times in a row) to advance to the next step ( Figure 4). MODULE 5 of Cataracts Intermediate (CAT-C) covered four parts: Capsulorhexis, Divide and Conquer, Choping and Irrigation and aspiration. All tasks had to be completed in sequential order and required 70 points (to be achieved by completing the task correctly three times in a row) to advance to the next step.

Results
In total 12 doctors completed the Virtual Simulation Course of Cataract Surgery at the headquarters of the Argentine Council of Ophthalmology, approving the 2 virtual modules and the 4 face-toface modules and also answered the anonymous survey at the end of it. The total of the responses to the survey is found in table 1. Likewise, they will be described in more detail below.
The minority (four) was resident doctors in training (33.3%), of them 3 were residents or concurrent of 3 years and 1 resident or MODULE 6 of Advanced Cataract (CAT-D) comprised five parts: Erratic Capsulorhexis Tear, Weak Capsule and Zonula, Variations in Capsulorhexis, White Cataracts and Capsule Plaques. All assignments had to be completed in sequential order and required 80 points (to be achieved by completing the assignment correctly three times in a row) to pass the module.

Evaluation
Each task was graded and documented. Thus, the student was able to know in detail their strengths and areas to improve. At the end of the course, a detailed report of the student's performance was delivered, with a total score and total training time. The student also received a course approval diploma and, in order to assess the learning and   4 Journal of Surgery: Open Access Open Access Journal concurrent of 2 years, with no residents or concurrent of 1 year. The remaining 8 doctors were specialists in ophthalmology (66.6%), of whom 4 had between 5 to 10 years of profession, 3 less than 5 years of profession and 1 more than 10 years of profession.
The age of the participants varied between 26 and 48 years, with an average of 35.8 years and a median of 36.5 years. There was a slight predominance of men (7 or 58.3%) over women (5 or 41.7%). A slight majority were Argentines (7 or 58.3%), 5 of them were from the interior of the country and 2 from the Autonomous City of Buenos Aires. A slight minority were foreigners (5 or 41.7%), 3 of them were from Uruguay, 1 from Paraguay and 1 from Bolivia.
The majority (eleven) of the doctors had performed cataract surgeries in humans prior to the start of this course (91.7%), with the resident or concurrent of 2 years (the one with the least training) being the only interviewee who had not performed them. Of the 11 physicians who had previously performed cataract surgeries on patients ( Figure  5), approximately half of the respondents who affirmatively had performed less than 50 cataract surgeries on humans, that is, they were in the early stages of the learning curve and corresponded to doctors in training and some specialists with less than 5 years of profession. The other half corresponded to specialist doctors with more years of profession and with more experience in cataract surgeries.
In figure 6, it can be seen that most of the surgeons responded that, prior to the start of the virtual simulation course, they felt very confident in the final steps of cataract surgery (90.9% in viscoelastic aspiration and 72.7% in intraocular lens insertion), moderately confident in hydrodissection and hydrodelamination (63.6%) and little confidence in key and initial steps in cataract surgery (27.3% in capsulorhexis, 18.2% in the facofracture and 18.2% in the removal of quadrants), which can lead to major complications in the course of other surgical steps. Neither was confident in steps that require more experience to resolve complex cases such as the use of pupillary dilators or performing an anterior vitrectomy. These confidence levels were not related to the degree of professional training or the number of surgeries previously performed.
The most difficult surgical steps for the respondents coincided with the steps in which they felt least confident (72.7% in lens fracture, 54.5% in capsulorhexis and 36.4% in quadrant removal) or in those who did not have any confidence (anterior vitrectomy or use of pupillary dilators), while the easiest surgical steps for the assistants coincided with those who felt the most confidence (viscoelastic aspiration, intraocular lens insertion, hydrodissection, and hydrodelamination). Capsulorhexis was the most difficult surgical step for practicing physicians (100% of cases). Facofracture was the most difficult step for specialist doctors (87.5%), followed by anterior vitrectomy and the use of pupillary dilators (50%). Information represented in figure 7.
The majority of respondents (63.6%) answered that they had a qualified surgical instructor while performing cataract surgeries on humans during their training period. The presence or absence of the surgical instructor was not related to the degree of training of the surgeons. All course participants had prior knowledge in the management of phacoemulsification fluidics. All the course participants considered that the virtual course allowed them to improve their general performance in the different steps of a cataract surgery in a real patient. Figure 8 shows the results of the survey, about which the majority of the participants considered that their performance improved in the steps in which they had more difficulty and felt less confident (91.7% improved their performance in the capsulorhexis and 66.7% in facofracture and quadrant removal), while a little more than a third improved in steps that felt more confident and had less difficulty (41.7% in hydrodissection and hydrodelamination and in the insertion of the intraocular lens and 33.3% in the viscoelastic aspiration). Only a minority considered that their performance improved in cases that require more experience to resolve complex cases (16.7% in the use of pupillary dilators and 8.3% in the anterior vitrectomy). There were no significant differences between the degree of professional training and the improvement in specific steps of cataract surgery. The improvement in the performance of capsulorhexis was present in doctors in training and in specialist doctors, except for the doctor with more experience, of more than 10 years of profession, who only reported an improvement in the hydrodissection and hydrodelamination.
All the participants considered that the virtual simulation course served to accelerate their learning curve of the different steps of cataract surgery in a real patient. Most of the respondents (91.7%) answered that the course will serve to reduce their anxiety before cataract surgery in a real patient. This answer was not related to the degree of professional training. All the participants considered that     what was learned in the virtual course will serve to reduce their complication rate in a real patient.

Discussion
Part of this discussion was previously published in the OCE magazine of the Argentine Council of Ophthalmology in spanish [12].
Most of the course participants were ophthalmology physicians who had been in the profession for up to 10 years and had already performed cataract surgeries on humans, half of whom had performed fewer than 50 surgeries. This demonstrates the importance of using the simulator for continuing medical education not only in the medical residence, but also in specialist doctors during their professional practice. We can also infer that the traditional methodology of teaching cataract surgery in the ophthalmic medical residence does not cover all the needs or expectations for a modern practice of the specialty.
In a systematic review published in March 2020 to evaluate all simulation models for ophthalmology technical and non-technical skills training, a wide range of models have been described but only the EyeSi has undergone comprehensive investigation. Fifty-three studies were based on virtual reality tools, 47 on wet-lab models, 26 on dry-lab models and 5 on e-learning. Models with the strongest validity evidence were the Eyesi Surgical, Eyesi Direct Ophthalmoscope and Eye Surgical Skills Assessment Test. Effectiveness ratings for simulator models were mostly limited to level 2 (contained effects) with the exception of the Sophocle vitreoretinal surgery simulator, which was shown at level 3 (downstream effects), and the Eyesi at level 5 (target effects) for cataract surgery [13]. That is why to date the EyeSi equipment is the most valid and effective tool for simulation-based surgical training in ophthalmology.
However, a work published in 2019 asks whether it is worth spending time and money on surgical simulators, which concludes that the discriminative power of EyeSI simulation between emerging surgeons and experts was found to reside only in the first training sessions and the EyeSI simulator in its current state of development seems to require further development before it can be used as a tool to select residents for surgical training and to re-certify more senior surgeons [14].
Most surgeons felt more confident and had less difficulty in the final steps of surgery, which are generally the easiest, and they felt less confident and had more difficulty in key steps of surgery such as lens management (capsulorhexis, facofracture and removal of quadrants), which constitute the central nucleus of the surgery and which, if not resolved satisfactorily, can lead to greater complications in the following steps. Furthermore, they felt no confidence in key steps to resolve complex cases or surgical complications such as anterior vitrectomy or use of pupillary dilators. This confirms that the teaching of this surgery in the ophthalmic medical residence is not complete and that it is necessary to complement it with non-traditional teaching methods such as the use of virtual reality surgical simulators. This deficiency also manifests itself in specialist doctors, so it is necessary that the use of simulators is not limited to medical residency, but can also be used by specialists.
Nearly two-thirds of respondents had a qualified surgical instructor during their medical residency training, which is very good and necessary, but also implies that more than a third completed their surgical training without the supervision of a qualified instructor.
All the participants had previous knowledge of the fluidics in cataract surgery, which is very important for the management of the surgical teams and for understanding the situations that arise in normal surgery and complications.
All the participants answered that the virtual course allowed them to improve their general performance in cataract surgery in a real patient, which confirms the usefulness of the virtual simulator in medical training to train more effective, efficient and reliable surgeons with the consequent benefit for the patients. It is also reinforced by the following question, in which the majority of the participants answered that they improved their performance in the steps in which they had more difficulty and felt less confident.
This coincides with a retrospective study conducted at the University of Los Angeles Medical Center that found that training residents with the virtual reality simulator reduced the range of errant capsulorhexis by 3.2 times in real surgery [15].
This result also coincides with a work sponsored by the Royal College of Ophthalmology in which there was a 38% reduction in the first and second year surgeon's unadjusted Posterior Capsule Rupture (PCR) rates from 4.2% in 2009 to 2.6% in 2015 for surgeons with access to an EyeSi, and a 3% reduction from 2.9% to 2.8% for surgeons without access to an EyeSi. The decrease in the with-access to an EyeSi group PCR rate was similar for surgeons with access to an EyeSi 'on site' or 'off site' [16].  The responses obtained in this course also coincide with a multicenter prospective work from the University of Copenhagen that showed that novice surgeons and those with up to 75 cataract surgeries significantly improved their performance in a real operating room (32% and 38% respectively) after training in a virtual simulator, while more experienced surgeons did not benefit from this virtual training [17].
A Mexican study showed that previous training of the lens phacoemulsification graduates with the EYESi® surgical simulator reduced surgical time, the incidence of prolapse or loss of vitreous by almost 50% and the learning curve in the first 50 real surgeries, compared to the graduates who did not receive it [18].
However, a major problem still persists in complex cases or surgical complications, since only a minority considered that their performance improved in these cases (16.7% in the use of pupillary dilators and 8.3% in the anterior vitrectomy). This problem could be addressed by dedicating more simulation time to these complex cases or doubling the virtual simulation course at an initial level and at an advanced level and allowing participants to carry out one or the other.
All participants considered that the virtual simulation course served to accelerate their learning curve of cataract surgery and will serve to reduce their rate of complications, while the majority answered that the course will reduce their anxiety before cataract surgery in one real patient. All this confirms the usefulness of virtual simulation in the surgical training of specialist doctors, improving surgical results and benefiting patients. This coincides with works found in the literature that residents who trained with a virtual reality simulator had shorter ultrasound times, fewer intraoperative complications, and a shorter learning curve [5,9,[19][20][21][22].
Among the strengths of the work we have the descriptive observational cross-sectional design that allows us to quickly and directly study the relationship between the variables, is inexpensive, has no ethical problems when using a virtual reality simulator, and is easily reproducible. Another strength is that it encompasses the entire population under study without the need to sample it, which implies working with complete information. The use of a structured questionnaire with closed questions is also positive, which allows obtaining reliable information quickly, directly, at a low cost and easy to systematize, without the bias of the interviewer.
Among the weaknesses of the work we have the relatively small population size, which can lead to errors in the reliability and validity of the results. To increase the population size, certain barriers that hinder the access of residents and medical specialists to the use of the virtual simulator should be overcome, such as economic (reduce the cost of the course or establish a scholarship system), geographic (establish a schedule in advance use of the simulator in different parts of the country or at different conferences), information (increase publicity about the advantages of virtual simulation in the learning curve of real surgery and in the reduction of complications) or others.
The cross-sectional design could also imply a weakness in itself, since the answers to the research questions are influenced by the subjectivity of the interviewee. To objectively assess whether the use of virtual reality surgical simulators shortens the learning curve, reduces stress and anxiety, and reduces the rate of complications in cataract surgeries in real patients, a prospective, observational, analytical and longitudinal study with two cohorts of residents and specialists, one that uses the virtual reality simulator and another that does not use it, and then in a certain period of time evaluate these 3 aspects by more than one independent observer and masked with a performance tool previously validated by the bibliography. An example of this tool is GRASIS (Global Rating Assessment of Skills in Intraocular Surgery) that uses different aspects of eye surgery such as instrument management, operation development, surgical time, treatment of eye structures and other tissues, use of the nondominant hand and average performance are scored individually for a total score and allows one surgeon to be compared to another [11].
Another tool that can be used is the Objective Structured Assessment of Cataract Surgical Skill (OSACSS), which consists of a scale of global indices and specific task items scoring from 1 (inadequately performed) to 5 (well performed) (Figure 9). Then all points are added together and a total value is obtained that allows comparing the performance between one surgeon and another and between one surgery and another [23].
This tool was validated in a danish multicenter prospective work in 2019 that demonstrated there was a statistically significant correlation between the simulator performance score and the mean OSACSS score across all cataract surgeons with different experience levels in real-life, with a Pearson correlation of 0.65 [24].
In order to generate strong evidence and demonstrate that simulation should be an integral part of ophthalmology training, a global collaboration program was created, the International Forum of Ophthalmic Simulation (IFOS), which has proposed a structured, sequential and supervised program, and which postulates that it can offer a significant transfer of skills, especially in the earliest stages of surgical training in ophthalmology [25].

Conclusions
According to the results of this study, and in accordance with the existing bibliography, the use of virtual reality surgical simulators for the training of an ophthalmologist, either in the Ophthalmology residence or in specialist doctors with only a few years of professional practice, seems be beneficial, potentially shortening the learning curve of the different steps of cataract surgery, indirectly reducing stress and anxiety during surgery, and possibly decreasing the rate of complications in real patients. These data should be confirmed with another study design, as commented in the discussion.

•
To the authorities of the Argentine Council of Ophthalmology in the person of its president Dr. Javier Casiraghi.
• To the directors of the Virtual Simulation Course of Cataract Surgery Dr. Daniel Badoza and Dra. Mariana Palavecino.
• To the secretary of the educational area of the Argentine Council of Ophthalmology Mariana Schiffer.