Publications

Generative AI has unlocked new possibilities in content discovery and management. Through collaboration with the National Football League (NFL), we demonstrate how a generative-AI based workflow allows media researchers and analysts to query relevant historical plays using natural language, rather than using traditional filter and click-based interfaces. The agentic workflow takes a user query in natural

Large language models (LLMs) have demonstrated remarkable performance across a wide range of tasks in recent years. While prior work has explored leveraging LLMs to generate synthetic data for self-improvement, repeated iterations often suffer from diminishing returns due to the reliance on homogeneous reasoning patterns and limited exploration of alternative perspectives. In this paper, we introduce a

AI systems for software development are rapidly gaining prominence, yet significant challenges remain in ensuring their safety. To address this, Amazon launched the Trusted AI track of the Amazon Nova AI Challenge, a global competition among 10 university teams to drive advances in secure AI. In the challenge, five teams focus on developing automated red teaming bots, while the other five create safe AI

We introduce AlquistCoder, a code-generating system that effectively minimizes the risk of producing malicious content or vulnerable code while maintaining excellent Python coding and question answering standards across a wide range of tasks. The architecture of AlquistCoder employs a sophisticated input guardrail classifier that analyzes whether the user’s intention is benign, potentially harmful, or falls

WARNING: Contains harmful content that can be offensive in nature Large language models (LLMs) for code generation have enhanced developer productivity while introducing new misuse vectors, as these models can generate potentially harmful code. Existing evaluation methods fail to assess such misuse scenarios, and structured red teaming pipelines for code generation remain under-developed. This paper presents

Large Language Models (LLMs) for code generation (i.e., Code LLMs) have demonstrated impressive capabilities in AI-assisted software development and testing. However, recent studies have shown that these models are prone to gen-erating vulnerable or even malicious code under adversarial settings. Existing red-teaming approaches rely on extensive human effort, limiting their scalability and practicality,

While large language models (LLMs) achieve strong performance in code generation, persistent security vulnerabilities hinder their safe deployment. Starting from a pretrained CodeGen model without inherent safety mechanisms, we develop a systematic synthetic instruction-tuning workflow to progressively enhance model security. Our pipeline begins with taxonomy-guided synthetic data, capturing diverse attack

We present SecureLion, a trustworthy AI assistant designed to securely han-dle cybersecurity queries and generate vulnerability-free code. Compared to the base model, our system achieves a 76.6% relative reduction on insecure messages in an adversarial competition setting at the cost of negligible utility loss. The success of SecureLion stems primarily from: (1) pervasive security-focused reasoning integrated

AI coding assistants like GitHub Copilot are rapidly transforming software development, but their safety remains deeply uncertain—especially in high-stakes domains like cybersecurity. Current red-teaming tools often rely on fixed bench-marks or unrealistic prompts, missing many real-world vulnerabilities. We present ASTRA, an automated agent system designed to systematically uncover safety flaws in AI-driven

We introduce PurpCode, a novel post-training method that aligns coding assistants to perform safety reasoning to defend against malicious cyber activities and provide secure and functional code. Our approach trains a reasoning model in two stages:(i) Rule learning, which explicitly teaches the model to reference cyber safety rules to avoid facilitating malicious cyber events and to generate vulnerability-free

Warning: This report contains partially redacted content that may be offensive to the reader Large language models (LLMs) may assist users with malicious cybersecurity at-tacks or inadvertently generate code with critical security flaws. These failures stem from their broader inability to reliably identify safe data or generate safe outputs, despite advances in alignment research. We identify three potential

This paper presents the vision, scientific contributions, and technical details of RedTWIZ: an adaptive and diverse multi-turn red teaming framework, to audit the robustness of Large Language Models (LLMs) in AI-assisted software development. Our work is driven by three major research streams: (1) robust and systematic assessment of LLM conversational jailbreaks; (2) a diverse generative multi-turn attack

In this technical report, we present our automated red-teaming framework designed to induce jailbreaks in targeted code-generating Large Language Models (LLMs), prompting them to generate malicious and vulnerable code. As of May 13, according to the latest competition leaderboard, our solution has achieved top performance in the second tournament. Our solution consists of three primary modules. First, we

Recent smaller language models such Phi-3.5 and Phi-4 rely on synthetic data generated using larger Language models. Questions remain about leveraging synthetic data for other use cases, such as adapting LLMs to specific domains. A key limitation of synthetic data is low diversity, which negatively impacts its downstream applicability for improving other models. To address this, we propose MetaSynth, a

The emergence of AI-driven web automation through Large Language Models (LLMs) offers unprecedented opportunities for optimizing digital workflows. However, deploying such systems within industry's real-world environments presents four core challenges: (1) ensuring consistent execution, (2) accurately identifying critical HTML elements, (3) meeting human-like accuracy in order to automate operations at