GraphCLaRa: Decoupling Semantic Alignment from Reasoning Potential in Dense Retrieval via Causal Representation Partitioning
“Dense retrieval systems encode documents into a single semantic vector, coupling factual content with logical implication. GraphCLaRa introduces a causal partition mask over distilled reasoning LLMs, enforcing architecturally disjoint reasoning and semantic sub-representations within a single forward pass alongside differentiable Personalized PageRank graph routing.”
Project Scope & Details
Dense retrieval systems achieve strong recall by encoding documents into a single semantic vector, but this geometry couples factual content with logical implication into one undifferentiated space.
We present GraphCLaRa, a retrieval architecture that imposes a causal partition mask over a distilled reasoning LLM (DeepSeek-R1-Distill-Qwen), enforcing architecturally disjoint reasoning (M_r) and semantic (M_s) sub-representations within a single forward pass. Within an oracle-corpus evaluation setting (665,766-passage index), GraphCLaRa CPC embeddings achieve 81.2% NQ Recall@5—matching ColBERTv2 (83.7%) within the confidence interval—validating representation quality and ranking precision under controlled corpus conditions.
The project formulates a differentiable Personalized PageRank (PPR) routing layer using implicit differentiation alongside a Convex Residual Highway that bounds the multi-hop Lipschitz constant, characterizing the scaling limits of end-to-end differentiable routing on large document graphs.
Causal Partitioned Compressor (CPC)
Imposes architectural decoupling between semantic similarity and multi-hop reasoning sub-representations in a single forward pass.
Zero Jacobian Orthogonality
Enforces strict mathematical independence between routing and semantic embeddings, preventing catastrophic capacity collapse.
Differentiable Graph Routing
Dynamic topological propagation across knowledge subgraphs achieving superior passage recall on complex multi-hop queries.
Local Sparse Training Fix
Eliminates dense scaling walls by restricting backpropagation to active localized graph neighborhoods.
GraphCLaRa System Architecture
Causal Representation Partitioning Mask
Lipschitz Contraction Bounds
NQ Recall@5 Comparison
Decoupling Semantic Alignment from Reasoning Potential in Dense Retrieval via Causal Representation Partitioning
Authors: Chaitanya Anand, Himangshu Sarma
- Chaitanya Anand
- Dr. Himangshu Sarma