RESEARCH INITIATIVES/Mixed Reality & Healthcare Spatial Computing

Mixed Reality & Healthcare Spatial Computing● ACTIVE RESEARCH

MRConsultation: Asynchronous Mixed Reality Pre-Surgical Teleconsultation System

Project Summary

“An asymmetric asynchronous Mixed Reality teleconsultation platform bridging pre-surgical patient-clinician communication via intentional semantic abstraction, dynamic local-coordinate serialization, and interactive 6DOF organ models.”

RESEARCH OVERVIEW & DETAILS

Project Scope & Details

MRConsultation enables surgical specialists and patients to asynchronously inspect, annotate, and review volumetric 3D organ models. Grounded in Cognitive Fit Theory, it incorporates Intentional Semantic Abstraction to present calming geometries without diagnostic loss, validated through an N=27 clinical usability study (SUS 77.22, UEQ-S Pragmatic +2.15). Features sub-millimeter dynamic coordinate serialization running at 89.7 FPS on Meta Quest 3 and WebGL Three.js.

ARCHITECTURAL & SCIENTIFIC HIGHLIGHTS

01.

Intentional Semantic Abstraction

Calming pastel geometries and CVD-redundant multi-coding bridge specialist-patient communication without visceral anxiety.

02.

Sub-Millimeter Anchor Stability

Local-coordinate dynamic serialization engine maintains millimeter precision under 10x pinch scaling and rotation.

03.

Unity / Three.js Handedness Bridge

Seamlessly synchronizes 6DOF spatial coordinates and quaternions between Meta Quest 3 headsets and browser WebGL views.

04.

Taubin Volume-Preserving CT Smoothing

Automated medical imaging reconstruction pipeline yielding high-fidelity multi-organ models with sub-0.3mm Hausdorff error.

INTERACTIVE 3D SPATIAL MODEL & KINEMATIC TELEMETRY

Real-Time 3D Mesh & Spatial Coordinate Simulator

Explore an interactive WebGL OpenXR spatial asset preview. Rotate, orbit, and toggle mesh visualization modes to inspect simulated sub-millimeter 6DOF coordinate tracking.

PROJECT GALLERY [19 PHOTOS]

Click to expand

FIG. 01

System Architecture Pipeline

FIG. 02

Full Clinical Workflow Composite

FIG. 03

System Usability Scale (SUS)

FIG. 04

NASA-TLX Cognitive Workload Analysis

FIG. 05

Task Success Rates

FIG. 06

Telemetry Console

FIG. 07

3D Organ Rendering

FIG. 08

Surgical Inspection

FIG. 09

Anatomical Slicing

FIG. 010

Spatial Annotation

FIG. 011

Session Recording

FIG. 012

Clinical Interface

FIG. 013

Segmentation View

FIG. 014

Spatial Anchoring

FIG. 015

Trajectory Planning

FIG. 016

Patient Consultation

FIG. 017

Web Dashboard

FIG. 018

Performance Telemetry

FIG. 019

Pilot Evaluation

ACADEMIC PUBLICATIONS [2]

2027Under Review / ACM CHI 2027

Asymmetric Asynchronous Mixed Reality Teleconsultation: Empowering Pre-Operative Patient-Clinician Dialogue through Semantic Abstraction

Authors: Chaitanya Anand, Himangshu Sarma

2026Under Review / IEEE VR 2026 & TVCG

Local-Coordinate Serialization and Volumetric Smoothing for Low-Latency 6DOF Mixed Reality Consultation

Authors: Chaitanya Anand, Himangshu Sarma

RESEARCH TEAM

Chaitanya Anand
Dr. Himangshu Sarma
HCI Lab Research Team

EXTERNAL COLLABORATORS

▸ACM CHI 2027
▸IEEE VR 2026

TECHNOLOGIES

Mixed RealityMeta Quest 3Three.js WebGL6DOF Spatial AnchorsDynamic Coordinate SerializationIntentional Semantic AbstractionTaubin SmoothingHealthcare UXUnityC#

PREVIOUS PROJECTVISI - SCROLL: EOG-Based Hands-Free Object Selection System

NEXT PROJECTTirumala Darshan with AR