Customer-obsessed science
Research areas
-
October 29, 20265 min readTwo optimization techniques, when combined, can significantly improve a model’s speed and cost effectiveness without sacrificing quality.
-
-
October 20, 20263 min read
-
October 12, 202614 min read
-
Featured news
-
ACM SIGSPATIAL 20262026Accurate and timely identification of road closures is critical for large-scale navigation and logistics systems, yet remains challenging due to the need to jointly reason over visual evidence, spatial context, and road network topology. Existing approaches either rely on manual map updates or focus on detecting closure-related objects without resolving which road segment is affected. In this work, we introduce
-
2026Human annotation cost and time remain significant bottlenecks in Natural Language Processing (NLP), with test data annotation being particularly expensive due to the stringent requirement for low-error and high-quality labels necessary for reliable model evaluation. Traditional approaches require annotating entire test sets, leading to substantial resource requirements. Active Testing is a framework that
-
2026The LLM-as-a-judge (LLMaJ) framework offers a cost-effective and reproducible solution for automatic evaluation. However, current evaluation practices typically compare LLMaJ judgments against aggregated ground-truth labels, overlooking the valuable information contained in Human Label Variation (HLV). Inspired by an increasing line of work that proposes to leverage HLV, we systematically study LLMaJ performance
-
2026Learning from experience is critical for developing capable, self-improving large language model (LLM) agents. Existing methods typically extract knowledge from accumulated trajectories via reflection, memory, rules, or skills. However, agents in realistic environments continuously encounter novel tasks, often offering only a one-shot opportunity to improve. These executions yield rich but highly noisy
-
2026When we fine-tune an LLM on code in a new programming language, what actually changes inside the model? Does it learn new algorithms, new syntax, or does it develop the ability to faithfully implement known algorithms in an unfamiliar language? This question is hard to answer with existing languages because we cannot disentangle what the model learned from fine-tuning versus what it already knew from pretraining
Collaborations
View allWhether you're a faculty member or student, there are number of ways you can engage with Amazon.
View all