Augmented reality in the operating theatre

by Gabriel Salg, Resident Physician

What many people think they only know from science fiction films and which is particularly popular in the gaming sector is increasingly developing into an experimental field for technology companies in the medical sector. This involves augmented reality in the operating theatre, realised through the use of 3D glasses known as holo-lenses. This technology, also known as extended reality, comprises augmented reality, virtual reality and mixed reality. In the medical context, this is referred to as ‘holomedicine’

An area of IT and medical technology that offers doctors support in planning operations, during surgery and in education and training. This is done by providing doctors with three-dimensional images of the patient before and during the operation; a pioneering technology that generates an anatomical image from up to 500 images of individual layers of body regions; it is like operating with an X-ray view.

The doctor digitally transfers the medical data, for example from computer tomography or magnetic resonance imaging (CT/MRI) examinations, to the software. This data is then transformed so that it can later be projected over the patient's real structures in the operating theatre with millimetre precision. ‘Through the glasses, we can see the CT image in 3D in exactly the right place. The image is immediately projected onto the relevant areas of the patient on the operating table. In future, we will no longer have to take our eyes off the patient to look at the screen with the images, as everything will be visible right in front of us.’ The devices are controlled using gestures, eye movements or voice commands.

This technology is ideal for preparing operations and for training doctors. ‘Patients' illnesses have led to deviations from normal anatomy. This applies in particular to complex tumour operations in visceral surgery. Thanks to holography, we can plan the procedure in detail and optimise the operation in advance.’

This aspect in particular also supports the explanatory discussions with patients. ‘In complicated procedures or when several organs are affected, patients often find it difficult to visualise the treatment. With this new method, we will be able to explain everything clearly to patients in future using 3D images. This promotes a completely different understanding and reduces anxiety.’