Objective
find a way to localize breast tumors during lumpectomy more precisely to reduce the reoperation rate caused by positive margins
find a way to localize breast tumors during lumpectomy more precisely to reduce the reoperation rate caused by positive margins
Outcome
an auditory display for breast cancer localization using multi-parameter sound mapping for simultaneous encoding of shape information (tumor margin) and point location (seed)
an auditory display for breast cancer localization using multi-parameter sound mapping for simultaneous encoding of shape information (tumor margin) and point location (seed)
Publication
Interactive Shape Sonification for Tumor Localization in Breast Cancer Surgery
L. Schütz, T. El Chemaly, E. Weber, A. Doan, J. Tsai, C. Leuze, B. Daniel, N. Navab
Proceedings of the 2024 CHI Conference on Human Factors in Computing Systems
Interactive Shape Sonification for Tumor Localization in Breast Cancer Surgery
L. Schütz, T. El Chemaly, E. Weber, A. Doan, J. Tsai, C. Leuze, B. Daniel, N. Navab
Proceedings of the 2024 CHI Conference on Human Factors in Computing Systems
About 20 percent of patients undergoing breast-conserving surgery require reoperation due to cancerous tissue remaining inside the breast. Breast cancer localization systems utilize auditory feedback to convey the distance between a localization probe and a small marker (seed) implanted into the breast tumor prior to surgery. However, no information on the location of the tumor margin is provided. To reduce the reoperation rate by improving the usability and accuracy of the surgical task, we developed an auditory display using shape sonification to assist with tumor margin localization. Accuracy and usability of the interactive shape sonification were determined on models of the female breast in three user studies with both breast surgeons and non-clinical participants. The comparative studies showed a significant increase in usability (p < 0.05) and localization accuracy (p < 0.001) of the shape sonification over the auditory feedback currently used in surgery.
Contributions
(1) Introduction of shape sonification into surgical guidance systems.
(2) An auditory display for breast cancer localization using multi-parameter sound mapping for simultaneous encoding of shape information (tumor margin) and point location (seed).
(3) Reporting of results from three user studies with four breast surgeons and 33 non-clinical participants comparing shape sonification to the current clinical sound feedback.
(4) Evidence that shape sonification has the potential to improve the usability and accuracy of surgical localization tasks.
(1) Introduction of shape sonification into surgical guidance systems.
(2) An auditory display for breast cancer localization using multi-parameter sound mapping for simultaneous encoding of shape information (tumor margin) and point location (seed).
(3) Reporting of results from three user studies with four breast surgeons and 33 non-clinical participants comparing shape sonification to the current clinical sound feedback.
(4) Evidence that shape sonification has the potential to improve the usability and accuracy of surgical localization tasks.
Sonifications
The distance to sound mapping was done in Wekinator using a multilayer perceptron Neural Network with two hidden layers per input parameter. The model was trained on 80 to 100 recordings of the probe position for each sonification model and the manually set desired sound parameter values. Two sonifications were created for the purpose of Study 2 and Study 3: one mimicking the clinical status quo (Beep) and a shape sonification (Sine).
The distance to sound mapping was done in Wekinator using a multilayer perceptron Neural Network with two hidden layers per input parameter. The model was trained on 80 to 100 recordings of the probe position for each sonification model and the manually set desired sound parameter values. Two sonifications were created for the purpose of Study 2 and Study 3: one mimicking the clinical status quo (Beep) and a shape sonification (Sine).
Beep (old): The distance between the tip of the probe and the location of a virtual seed (point) inside the tumor object was used as input to the Neural Network. Two output parameters were mapped to the volume and frequency of a beeping sound. The baseline beep sonification used a sine wave oscillator at 440 Hertz (Hz). As the distance between the probe and the tumor decreased, the frequency of the beeps increased. No information about the tumor shape or margin location was included in this condition.
Sine (new): a continuous sound is used to represent the shape and a beat to represent the seed location. This sonification separated the areas of interest into discrete zones. The Sine sonification model played a continuous synthesizer sound (musical note C4) when the probe was above the tumor area and no sound when the probe was outside the area. For the seed location, a ticking beat sound (ChucK ModalBar at 660 Hz) was synthesized to indicate the point of interest. Three output parameters from the Neural Network were mapped to the volume and timbre of the ticking seed sound and the volume of the continuous tumor sound.
We conducted three user studies to investigate whether shape sonification can increase the usability and accuracy of the tumor localization task in lumpectomy. Repeat testing helped us to incrementally
refine the shape sonification design. For the first study, the use context was abstracted to a 2D plane The second study evaluated a refined shape sonification design on a silicone model of the female breast, and the third study tested the final sonification design on 16 agar breast models with breast surgeons. Participants were tasked to mark the tumor margin, including the seed location on the paper sheet / the breast model. The location of the margin and seed were recorded by tracing the drawn markings with the tracked probe. These locations were stored in the Unity coordinate frame and later processed in Matlab. The dice coefficient, area ratio, and euclidean distance were computed from the ground truth and marked tumor area and seed location.
refine the shape sonification design. For the first study, the use context was abstracted to a 2D plane The second study evaluated a refined shape sonification design on a silicone model of the female breast, and the third study tested the final sonification design on 16 agar breast models with breast surgeons. Participants were tasked to mark the tumor margin, including the seed location on the paper sheet / the breast model. The location of the margin and seed were recorded by tracing the drawn markings with the tracked probe. These locations were stored in the Unity coordinate frame and later processed in Matlab. The dice coefficient, area ratio, and euclidean distance were computed from the ground truth and marked tumor area and seed location.
Results
Compared to current localization systems that only provide sound feedback on the location of a marker implanted inside the tumor, our approach provides more comprehensive guidance through additional sonification of tumor margins. In three user studies, we demonstrated that the proposed shape sonification significantly enhances localization accuracy. We were furthermore able to show that breast surgeons utilizing shape sonification experienced improved usability compared to the current clinical auditory feedback. We showed evidence of the accuracy of auditory displays, reinforcing their potential as a substitution or addition to conventional, visual user interfaces. Our work presents an exemplary use case for shape sonification in a surgical precision task and hopefully helps to promote broader application of auditory displays and multimodal, e.g., audio-visual and audio-haptic, interfaces in surgical applications and beyond.
Compared to current localization systems that only provide sound feedback on the location of a marker implanted inside the tumor, our approach provides more comprehensive guidance through additional sonification of tumor margins. In three user studies, we demonstrated that the proposed shape sonification significantly enhances localization accuracy. We were furthermore able to show that breast surgeons utilizing shape sonification experienced improved usability compared to the current clinical auditory feedback. We showed evidence of the accuracy of auditory displays, reinforcing their potential as a substitution or addition to conventional, visual user interfaces. Our work presents an exemplary use case for shape sonification in a surgical precision task and hopefully helps to promote broader application of auditory displays and multimodal, e.g., audio-visual and audio-haptic, interfaces in surgical applications and beyond.
Tools
Unity - Tracking & Registration
Wekinator - sound parameter mapping
ChucK - Audio programming
Electromagnetic tracking system (NDI trakSTAR EM 6 degrees-of freedom)
Savi Scout radar localization probe
3D printing
Unity - Tracking & Registration
Wekinator - sound parameter mapping
ChucK - Audio programming
Electromagnetic tracking system (NDI trakSTAR EM 6 degrees-of freedom)
Savi Scout radar localization probe
3D printing
