SciVis: Mid-Air Haptic & Thermo-Tactile Visualization

Developed at the Multimodal Interaction Lab (MI Lab), the SciVis Multi-Sensory Visualization Suite explored novel paradigms for perceiving complex multi-dimensional scientific data. By pairing immersive stereoscopic VR with mid-air ultrasound radiation pressure arrays and micro-thermal feedback actuators, researchers can literally feel volumetric scalar fields, temperature gradients, and turbulent flow boundaries without wearing physical gloves or holding mechanical controllers.

Mid-Air Acoustic Focal Point Modulation

The system leverages an Ultraleap phased transducer array, driving hundreds of ultrasonic speakers to construct focused acoustic pressure points in free space. By modulating carrier wave frequencies in real time:

• Users perceive tactile textures, distinct vibration frequencies, and surface boundaries hovering in mid-air.
• Hand-tracking cameras continuously map bare fingertips to virtual data isosurfaces, steering focal points dynamically to follow the user’s hand gestures.

Developed at the Multimodal Interaction Lab (MI Lab) in collaboration with Computer Science researchers, this project engineered a multi-sensory scientific visualization system allowing aerodynamicists to simultaneously see, navigate, and physically “feel” complex 3D wind tunnel vector fields.

Multi-Sensory Ultrasonic Haptics

Using an Ultraleap acoustic radiation pressure phased array, ultrasonic transducers focus acoustic focal points in mid-air. As researchers place their bare hands into the virtual flow volume in VR, ultrasound wave modulations produce tactile sensations corresponding directly to flow turbulence, velocity gradients, and aerodynamic vortices.

Thermo-Tactile Multi-Modal Fusion

To visualize multi-variate datasets (such as atmospheric heat flux or thermodynamics):

• Peltier thermoelectric elements were synchronized with the spatial rendering engine, applying rapid heating and cooling cycles alongside ultrasound vibrations.
• This multi-sensory feedback significantly reduced cognitive error rates when test subjects identified critical thresholds within dense 3D scalar fields.

Peer-Reviewed ACM Publications

• ACM VRST ‘21: TangibleData: Interactive Data Visualization with Mid-Air Haptics
  Proceedings of the 27th ACM Symposium on Virtual Reality Software and Technology.
  DOI: 10.1145/3489849.3489889

• ACM VRST ‘21: Mid-Air Thermo-Tactile Feedback using Ultrasound Haptic Display
  Proceedings of the 27th ACM Symposium on Virtual Reality Software and Technology.
  DOI: 10.1145/3489849.3489889