Search results

AI systems for software development are rapidly gaining prominence, yet significant challenges remain in ensuring their safety. To address this, Amazon launched the Trusted AI track of the Amazon Nova AI Challenge, a global competition among 10 university teams to drive advances in secure AI. In the challenge, five teams focus on developing automated red teaming bots, while the other five create safe AI

We introduce AlquistCoder, a code-generating system that effectively minimizes the risk of producing malicious content or vulnerable code while maintaining excellent Python coding and question answering standards across a wide range of tasks. The architecture of AlquistCoder employs a sophisticated input guardrail classifier that analyzes whether the user’s intention is benign, potentially harmful, or falls

We present SecureLion, a trustworthy AI assistant designed to securely han-dle cybersecurity queries and generate vulnerability-free code. Compared to the base model, our system achieves a 76.6% relative reduction on insecure messages in an adversarial competition setting at the cost of negligible utility loss. The success of SecureLion stems primarily from: (1) pervasive security-focused reasoning integrated

WARNING: Contains harmful content that can be offensive in nature Large language models (LLMs) for code generation have enhanced developer productivity while introducing new misuse vectors, as these models can generate potentially harmful code. Existing evaluation methods fail to assess such misuse scenarios, and structured red teaming pipelines for code generation remain under-developed. This paper presents

Large Language Models (LLMs) for code generation (i.e., Code LLMs) have demonstrated impressive capabilities in AI-assisted software development and testing. However, recent studies have shown that these models are prone to gen-erating vulnerable or even malicious code under adversarial settings. Existing red-teaming approaches rely on extensive human effort, limiting their scalability and practicality,

While large language models (LLMs) achieve strong performance in code generation, persistent security vulnerabilities hinder their safe deployment. Starting from a pretrained CodeGen model without inherent safety mechanisms, we develop a systematic synthetic instruction-tuning workflow to progressively enhance model security. Our pipeline begins with taxonomy-guided synthetic data, capturing diverse attack

AI coding assistants like GitHub Copilot are rapidly transforming software development, but their safety remains deeply uncertain—especially in high-stakes domains like cybersecurity. Current red-teaming tools often rely on fixed bench-marks or unrealistic prompts, missing many real-world vulnerabilities. We present ASTRA, an automated agent system designed to systematically uncover safety flaws in AI-driven

We introduce PurpCode, a novel post-training method that aligns coding assistants to perform safety reasoning to defend against malicious cyber activities and provide secure and functional code. Our approach trains a reasoning model in two stages:(i) Rule learning, which explicitly teaches the model to reference cyber safety rules to avoid facilitating malicious cyber events and to generate vulnerability-free

This paper presents the vision, scientific contributions, and technical details of RedTWIZ: an adaptive and diverse multi-turn red teaming framework, to audit the robustness of Large Language Models (LLMs) in AI-assisted software development. Our work is driven by three major research streams: (1) robust and systematic assessment of LLM conversational jailbreaks; (2) a diverse generative multi-turn attack

Warning: This report contains partially redacted content that may be offensive to the reader Large language models (LLMs) may assist users with malicious cybersecurity at-tacks or inadvertently generate code with critical security flaws. These failures stem from their broader inability to reliably identify safe data or generate safe outputs, despite advances in alignment research. We identify three potential

In this technical report, we present our automated red-teaming framework designed to induce jailbreaks in targeted code-generating Large Language Models (LLMs), prompting them to generate malicious and vulnerable code. As of May 13, according to the latest competition leaderboard, our solution has achieved top performance in the second tournament. Our solution consists of three primary modules. First, we

Recent smaller language models such Phi-3.5 and Phi-4 rely on synthetic data generated using larger Language models. Questions remain about leveraging synthetic data for other use cases, such as adapting LLMs to specific domains. A key limitation of synthetic data is low diversity, which negatively impacts its downstream applicability for improving other models. To address this, we propose MetaSynth, a

The emergence of AI-driven web automation through Large Language Models (LLMs) offers unprecedented opportunities for optimizing digital workflows. However, deploying such systems within industry's real-world environments presents four core challenges: (1) ensuring consistent execution, (2) accurately identifying critical HTML elements, (3) meeting human-like accuracy in order to automate operations at

Unlike closed-vocabulary 3D instance segmentation that is often trained end-to-end, open-vocabulary 3D instance segmentation (OV-3DIS) often leverages vision-language models (VLMs) to generate 3D instance proposals and classify them. While various concepts have been proposed from existing research, we observe that these individual concepts are not mutually exclusive but complementary. In this paper, we

Many security- and performance-critical domains, such as cryptography, rely on low-level verification to minimize the trusted computing surface and allow code to be written directly in assembly. However, verifying assembly code against a realistic machine model is a challenging task. Furthermore, certain security properties—such as constant-time behavior—require relational reasoning that goes beyond traditional

Targeting long-form question-answering, chain-of-query (CoQ) has been studied, integrating chain-of-thought (CoT) with retrieval-augmented generation. CoQ breaks down complex questions into simpler subquestions (SQs), allowing relevant information to be retrieved step by step. By doing so, CoQ aims to improve the answer comprehensiveness and verifiability, at the expense of latency. Our first contribution

Customized video editing aims at substituting the object in a given source video with a target object from reference images (Fig. 1). Existing approaches often rely on fine-tuning pre-trained models by learning the appearance of the objects in the reference images, as well as the temporal information from the source video. These methods are however not scalable as fine-tuning is required for each source

Trained on millions of hours of data from Amazon fulfillment centers and sortation centers, Amazon’s new DeepFleet models predict future traffic patterns for fleets of mobile robots.

Large Language Models (LLMs) pruning seeks to remove unimportant weights for inference speedup with minimal accuracy impact. However, existing methods often suffer from accuracy degradation without full-model sparsity-aware fine-tuning. This paper presents Wanda++, a novel pruning framework that outperforms the state-of-the-art methods by utilizing decoder-block-level regional gradients. Specifically, Wanda

A new philosophy for developing LLM architectures reduces energy requirements, speeds up runtime, and preserves pretrained-model performance.