Improving liver surgery in Europe

The past four years, from November 2016 to April 2021, scientists from the Hipernav project researched how to improve surgical navigation by combining articifial intelligence, augemented reality, computer vision and mathematical modeling. This european digital health project aimed at changing liver cancer surgery, while focusing on training the next generation of young researchers and doctors, writes Inria in a press release. .

Liver surgery, a challenge

Despite not being the most common cancer, liver cancer is one of the deadliest, with about 900,000 deaths in 2020. Different therapeutical options can increase the survival chances, one of them being the surgical removal of the tumor. But cutting through a soft organ and making sure the tumor is entirely resected while sparing as much healthy tissue as possible is no small challenge.

By bringing together augmented reality, image segmentation and enhancement, high performance computing, physically accurate biomechanical modeling and much more scientific fields, the European project Hipernav wanted to tackle several of the bottlenecks in surgical navigation. It has been coordinated by Ole Jakob Elle, manager of the medical cybernetics and image processing group at Oslo University Hospital. This Norway’s central hospital handles all the transplants and congenital heart diseases treatments in the country, as well as a large part of its biomedical research.

The state of the art

To this day, most liver tumor resections are done via open surgery, and even so the tumors can be hard to find, reach or remove. Surgeons rely on different pre-operative images to identify the regions of the organ that need to be removed  and the ones that should not be compromised, in particular the liver veins and arteries. Usually, doctors use a combination of imaging techniques (Magnetic Resonance Imagery, Ultrasounds, CT-scan…) to get the most accurate image of the patient’s liver. These images allow surgeons to get a view of the organ, slice by slice. To help identify the different (anatomical) structures of the organ, such as the tumor(s) and vessels, these images are turned into a 3D model. This helps the surgeon plan the resection workflow. But the liver is a highly deformable organ. “On contrary to the brain, for example, that will move a little when touched but is constrained in the skull, the liver shape will change drastically when the patient switches position or when you touch the organ with surgical tools” stresses Ole Jakob Elle. Intra-operative data are then necessary to take this deformation into account and make sure the cuts are properly done.

More recently, laparoscopic surgery (also named keyhole surgery), where the surgeon can access the inside of the abdomen through small incisions rather than larger cuts, has proved to reduce the patient’s recovery time and improve their quality of life. But to perform this kind of surgery, you need to insufflate the abdomen with CO2, to create a larger surgical field. And that insufflation as well has an impact on the liver shape. Being able to see the organs only through the laparoscope is also challenging and demands even more accurate 3D models of the liver to make sure the procedure is performed safely.

The goal to be achieved

Hipernav, which stands for High-Performance soft tissue Navigation, was launched to explore what the most advanced 3D modeling and imaging techniques could bring to this type of surgery. Following up on previous work on augmented reality’s possible contributions to laparoscopy by Ole Jakob Elle and surgeon Bjorn Edwin, over the last 4 years, the 14 partners of the European research project aimed at making liver surgery possible, even for complex cases with unfavorable prognosis. The final objective is to make it possible sometime, in the future, for the surgeon to have access in real-time to an accurate, virtual liver model. This model will need to take into account pre-operative and intra-operative data of the patient and deform according to soft tissue biomechanical properties to help the team navigate into the organ throughout the operation workflow. Ultimately one could envision using tracking and navigation systems that would help clinicians navigate around the parts of the organ to keep and tumors to remove “a bit like your GPS is helping you navigate your car on a map” indicates Ole Jakob Elle. This could have a significant impact on the survival rate of patients. Indeed, by making it easier for surgeons to prepare and perform the operation, this type of system will allow for better resection and shorter operations. And therefore, a better recovery.

Hipernav network of young researchers

To reach the project ambitious goal, numerous disciplinary fields have to work together, from bioengineering to image processing via navigation systems, high performance computing, tracking techniques, surgery procedure analysis and planning… All of these key areas of expertise were brought together by a coordinated plan of individual research projects addressing specific bottlenecks. With the objective “to train early-stage researchers to acquire highly transferable skills and become international leaders in their field”, this Innovative Training Network (ITN) was funded by the European Union’s Horizon 2020 research and innovation program through a Marie Skłodowska-Curie grant. The 4 million euro budget made it possible to fund 16 PhDs working on the project working with 9 of the 14 partners of the network, according to the work packages needs. Inria’s research team MIMESIS, headed by Stephane Cotin, which focuses its research on real-time simulation for training and per-operative clinical guidance, was one of these hosting partners.

From Inria’s 3D model to tracking systems

Based in Strasbourg, Stephane Cotin’s team has developed several numerical models for clinical training in eye, liver, endovascular, and kidney surgery. One of their main achievement is a biomechanical model of the liver which considers the biophysical properties of soft tissues to accurately reflect the deformation of the organ when it is subjected to external forces. Using this model in a clinical setting by finely tuning it to the preoperative data of a patient and combining it with updated intraoperative data (for instance using ultrasound images) could make a huge difference in the operation risks. Indeed, having a model that is as close as possible to the actual organ the surgeon is operating on makes it possible to envision the safe use of navigation systems in soft tissues. By relying on cheaper imaging techniques rather than on intra-operative MRI or CT-scan, this technology could also reduce the operative cost.

Valuable new knowledge on which to build

Of course, not all steps leading to an augmented reality device that would allow the surgeon to visualize the organ in real-time have been completed in the four years of Hipernav. But significant results have been obtained in several key fields, from improvements to the biomechanical model to a better estimation of operation length based on workflow analysis or a drastic decrease in the computing time needed to update the model. Most of points to be improved having been identified, perhaps a next ITN will allow to go even further in making happen this liver surgery of the future?  

Also interesting: Tumor organoids to overcome therapy resistance of colorectal cancer