Customer-obsessed science
Research areas
-
October 20, 20263 min readHow Amazon partnered with AVERI to independently assess — and improve — our internal safety testing of Amazon models.
-
October 12, 202614 min read
-
-
-
August 21, 20269 min read
Featured news
-
NeurIPS 20262026Uncertainty estimation is essential not only for the trustworthy deployment of large language models (LLMs) but also as a foundation for self-refinement in LLM generation. However, existing approaches operate at suboptimal granularities: token-level scores lack semantic coherence, while sequence-level scores fail to localize errors. We formalize Span-Level Uncertainty Estimation (SLUE), a new task that
-
2026Evaluating enterprise agents on domain-specific benchmarks is critical, yet public benchmarks rarely evaluate whether agents can integrate business knowledge with analytical computation, and constructing such benchmarks manually is costly. We present DI-Bench, a pipeline for generating realistic benchmarks for data intelligence (DI), the practice of extracting insights from large volumes of enterprise data
-
2026Multi-agent debate improves LLM reasoning, yet agreement is not evidence of correctness. When agents converge on a wrong answer through social reinforcement, consensus stopping commits that error to automated action with no recourse. We introduce Conformal Social Choice, a post-hoc layer that converts debate outputs into calibrated act-versus-escalate decisions. Verbalized probability distributions from
-
2026Budget delivery in programmatic advertising requires controlling spend across advertising campaigns whose plant gains vary significantly across campaigns and evolve over time, rendering fixed-gain controllers inadequate. This paper proposes an adaptive spend control architecture comprising four components: an input reparameterization that transforms the nonlinear spend-to-control relationship into a near-linear
-
2026A neural network often settles on its answer long before it shows it. We make that precise with a training-free tool, the sign lens. It reads a concept straight from a hidden state using only the sign of each coordinate, positive or negative (Nalagatla, 2026): a concept is a set of coordinates that share a fixed sign pattern, detected by counting agreements. No probe, no dictionary, no training. With this
Collaborations
View allWhether you're a faculty member or student, there are number of ways you can engage with Amazon.
View all