Customer-obsessed science
Research areas
-
November 11, 20265 min readStatic analysis of deployed networks helps reduce the risk that sensitive data leaks across partitions, supporting the security and reliability of global-scale networking.
-
-
-
-
October 20, 20263 min read
Featured news
-
2026Human annotation cost and time remain significant bottlenecks in Natural Language Processing (NLP), with test data annotation being particularly expensive due to the stringent requirement for low-error and high-quality labels necessary for reliable model evaluation. Traditional approaches require annotating entire test sets, leading to substantial resource requirements. Active Testing is a framework that
-
2026The LLM-as-a-judge (LLMaJ) framework offers a cost-effective and reproducible solution for automatic evaluation. However, current evaluation practices typically compare LLMaJ judgments against aggregated ground-truth labels, overlooking the valuable information contained in Human Label Variation (HLV). Inspired by an increasing line of work that proposes to leverage HLV, we systematically study LLMaJ performance
-
2026Learning from experience is critical for developing capable, self-improving large language model (LLM) agents. Existing methods typically extract knowledge from accumulated trajectories via reflection, memory, rules, or skills. However, agents in realistic environments continuously encounter novel tasks, often offering only a one-shot opportunity to improve. These executions yield rich but highly noisy
-
2026When we fine-tune an LLM on code in a new programming language, what actually changes inside the model? Does it learn new algorithms, new syntax, or does it develop the ability to faithfully implement known algorithms in an unfamiliar language? This question is hard to answer with existing languages because we cannot disentangle what the model learned from fine-tuning versus what it already knew from pretraining
-
2026Group relative policy optimization for reinforcement learning with verifiable rewards (RLVR) typically uses a fixed importance-sampling (IS) ratio clipping boundary across all rollouts. We identify a key limitation: rare correct rollouts on harder problems and abundant correct rollouts on easier problems are clipped at comparable rates, despite contributing very different learning signals. Rollouts with
Collaborations
View allWhether you're a faculty member or student, there are number of ways you can engage with Amazon.
View all