EgoScore: Importance-Weighted Retrieval for Egocentric Lifelogging via Multi-Signal Perceptual Scoring
“When users wear always-on egocentric cameras (such as Meta Ray-Ban glasses), they accumulate hours of video daily. EgoScore operationalizes moment importance across social significance, emotional salience, novelty, and memorability by fusing six perceptual signals with Ebbinghaus-inspired temporal decay.”
Project Scope & Details
EgoScore introduces importance-weighted retrieval for egocentric lifelogging. Drawing on Dual-Process Memory Theory (Tulving & Baddeley) and Event Segmentation Theory (Zacks et al.), the system indexes 2-minute egocentric video segments into a steerable topological memory graph.\n\nThe pipeline extracts six orthogonal perceptual signals—SigLIP visual novelty, YOLOv8/v11 social presence, optical flow activity intensity, Whisper audio affect, MediaPipe interaction density, and Mahalanobis contextual surprise—fused with an Ebbinghaus forgetting curve. In a within-subjects user study (N=16) searching personal lifelogs, importance-reranked retrieval achieved significantly higher satisfaction than semantic-only retrieval (p < .01), with 75% of participants preferring the importance-aware system for daily lifelogging and memory augmentation.
Poincaré Hyperbolic Graph Retrieval
Continuous Riemannian manifold embeddings supporting hierarchical episodic memory recall without geometric distortion.
Multi-Signal Perceptual Scoring
Fuses 6 orthogonal visual-acoustic signals (head fixation, audio energy, face engagement, zoom kinematics, novelty, emotion) into composite frame importance.
Ebbinghaus Temporal Retention
Cognitive exponential decay modeling that organically deprioritizes routine experiences while preserving key autobiographical anchors.
Meta Ray-Ban Wearables Integration
Native Android integration using Meta Wearables DAT for hands-free 1080p egocentric capture and spatial microphone streaming.
Real-Time 3D Mesh & Spatial Coordinate Simulator
Explore an interactive WebGL OpenXR spatial asset preview. Rotate, orbit, and toggle mesh visualization modes to inspect simulated sub-millimeter 6DOF coordinate tracking.
EgoGraph System Architecture
Perceptual Signal Extraction Flow
Poincaré Hyperbolic vs Euclidean Embedding
Assistive Memory Prosthetic Architecture
EgoScore: Importance-Weighted Retrieval for Egocentric Lifelogging via Multi-Signal Perceptual Scoring
Authors: Chaitanya Anand, Himangshu Sarma
- Chaitanya Anand
- Dr. Himangshu Sarma