Collaborative Networked Virtual Surgical Simulator

Virtual reality supporting teamwork training and remote mentoring in surgery.

Nowadays, the way in which surgical training is performed in medicine schools and hospitals is changing around world, mainly due of two reasons: (i) A change to a new training paradigm which highlight the role of the mentor during the training process and the importance of training not only basic surgical skills but also collaborative work in a surgical room, and (ii) the low number of expert surgeons located on regions distant from big cities. The use of tools based on virtual reality allowing the training of teamwork and remote mentoring is starting to be implemented in this new way of training. One of these tools is collaborative networked virtual surgical simulators (CNVSS).

However, to provide a successful training, collaborative virtual surgical simulators need to address challenges related to surgical simulators and collaborative virtual environments (CVE) such as: real time collision detection, deformable objects simulation, haptic rendering, heterogeneity of the users’ computing resources, variable network conditions, among others. Some research works have addressed some of these challenges in CNVSS, but none, as far as we know, has focused on handling of heterogeneity in CNVSS. Heterogeneity in CVE is defined as the differences that exist among machine capabilities, network conditions, preferences and role of the users taking part of a collaborative task. Therefore, handling heterogeneity in CNVSS is important because not all remotely located users have homogeneous Internet connections, same interaction devices and displays, same computational capabilities, same user preferences and same user roles, among other factors. Additionally, if this heterogeneity is not handled properly, it will have an adverse impact on the performance of each of the users during the collaborative session.

In this work we propose the development of a context-aware model for CNVSS in order to handle the heterogeneity factors of the collaborative system. The proposed model will involve mathematical modeling of these factors to infer the decisions required to handle the heterogeneity of the CVE. This decision have to maximize the collaboration performance between users, understanding collaboration performance as decreasing the number of errors and the task completion time. The output variables that will be modified in our simulation depending of the heterogeneity factors state will be the resolution and algorithms applied for collision detection, visual and haptic rendering and deformable computation of the anatomical structures in each user machine.

Pictures

Videos

Next video shows a simulation of a procedure called cholecystectomy. On each monitor screen, the point of view that each of the users connected remotely have can be observed. The user interaction with the simulation scenario is performed by using a Phantom Omni device.

Video demonstration of a simulation with parameters and qualities modified considering the heterogeneity conditions of machine and network. In the left screen, a simulation adapted to a low performance machine and in the right screen a simulation adapted to a high performance machine can be observed, in general the connectivity between the two machines is not appropiate to guarantee a good collaboration. Modifications are performed on the visualization, deformation and collision data structures of the simulation.

Next, a real-time adaptation of the simulation parameters performed by the simulator is observed.

The following video shows how heterogenous network conditions can affect a training session performed in a collaborative manner between two users. In the video each monitor screen is connected to independent machines and between the two machines there is a network emulator that applies controlled network impairments such as: reduced bandwidth, packet loss, jitter and latency which are not appropriate for a good collaboration.

In this video, how the developed surgical simulator modifies several parameters of the simulation in real time in order to adapt to the conditions of heterogeneity of the context of the simulation is observed. The video shows changes in the resolution of the data deformation, collision and visualization structures. Also in the visualization algorithms.

To handle network bandwidth limitations, the surgical simulator can use different deformation resolutions for each user. This reduces the amount of data transmitted between the two machines and therefore the required bandwidth, maintaining a similar view of the anatomical structures deformation from both users view points.

The architecture implemented for the simulator calculates the deformation locally in one of the machines and transmits it to the other. In the following video, how the collaborative surgical simulator simulates the touch of the deformable structures with the instrument, done by a local user and by a remote user to the deformation computation is observed.

Person in charge

Christian Diaz