Modern LLM architectures have co-evolved within a single hardware family. The shapes of our attention mechanisms, the structure of our multilayer perceptrons (MLPs), the choice of numerical formats, and even the granularity of parallelism strategies have all been shaped by hardware constraints: warp sizes, tensor core geometries, memory hierarchies, and the kernel abstractions those chips expose. When the hardware changes, the efficient frontier of model architectures changes with it. Here we present an opportunity for academic and industry labs to explore this frontier in detail on AWS Trainium.
Purpose-built accelerators like AWS Trainium present a genuinely different design surface. More on-chip SRAM (SBUF), explicit software control over data movement and acceleration at the lowest levels, energy-efficient systolic matrix multiplication (matmuls), and a memory hierarchy designed for training and inference-scale data flows. The resulting TFLOPs-to-memory-bandwidth ratio shifts the performance bottleneck profile: key operations that are memory-bound on conventional accelerators may become compute-bound on Trainium, opening design space for architectures that trade additional computation for reduced memory traffic. These hardware differences mean the optimal attention patterns, MLP structures, and parallelism strategies may be fundamentally different.
The research question is open: What does an optimal model look like when the hardware constraints are fundamentally different? The AWS Trainium Frontier is a competition designed to answer this question empirically: participants train language models from scratch on Trainium, exploring the full design space from model architecture to custom kernels. The core task is training a language model from scratch, starting from a provided ~50M parameter baseline (nanochat-derived, GPT-style dense LLM with RMSNorm, rotary embeddings, and ReLU² MLP). Participants modify everything: architecture, optimizer, training loop, and optionally custom NKI kernels. The baseline is a starting point, not a ceiling.
The AWS Trainium Frontier competition rewards full-stack thinking under a fixed time and compute budget. Participants optimize the model architecture, the optimizer, the training loop, and, if they choose, custom hardware kernels. This enables innovation on a combination of objectives: within the allotted training budget, how low can you drive validation bits-per-byte, and how high can you drive downstream in-context learning capability? The fixed budget creates a direct tradeoff between model capacity (better architecture = fewer steps needed) and training throughput (faster kernels = more steps in the same time). The winning solution finds the balance: the most intelligent model trained most efficiently within the time constraint.
Final submissions find the optimal point on that frontier, and because Trainium's hardware benefits differ from those of existing accelerators, the optimal architectures will differ as well. Be among the first to discover what model architectures look like when designed for a purpose-built AI chip, contributing to a genuinely new area of machine learning research.
The Neuron Kernel Interface (NKI), native PyTorch support, and AI-assisted tooling (including Amazon Bedrock access) give you direct access to Trainium's unique hardware features: the SBUF scratchpad, TensorEngine tiling, and explicit DMA control that standard framework abstractions cannot expose. This is what enables genuinely hardware-native model designs. The entire NKI API surface fits in a weekend, making it equally accessible to both a human writing kernels by hand and an AI agent generating them under human direction.
The challenge: Exploring the full design space
Phase 1 gives every team a single Trainium2 chip and a 30-minute training budget, fast enough to test dozens of hypotheses in a single day. Phase 1 scores on a single number: validation bits-per-byte (val_bpb) after exactly 30 minutes of training on a single Trn2 chip. Lower is better. Any improvement that fits within that wall-clock budget counts, whether it comes from architecture, optimizer, kernel, or all three. Phase 2 promotes the top 10 teams to a full Trainium2 server with a four-hour budget, opening the door to distributed parallelism and communication-aware model shaping. Phase 2 adds a second axis — inference performance on CORE, an aggregate score across in-context learning tasks spanning reasoning, comprehension, and world knowledge. Your final score is a 50/50 composite: you need a model that trains efficiently and learns to reason. It's a research arc from, "Does my idea work?" to, "Does my idea scale?”.
Participants have flexibility in how they improve the model. A better learning rate schedule matters as much as a faster kernel. This rewards the full stack: a novel attention mechanism is only as fast as the kernel that runs it, and the fastest kernel only matters if the architecture knows how to use it. Model size is uncapped: the constraint isn't parameters, it's time on the chip.
You choose the model architecture that maximizes capability within a fixed training window. This inversion of the usual scaling paradigm is what makes this a more challenging research question, not just an engineering exercise, and it's where the most publishable insights will emerge.
What you get
- A complete nanochat-derived training pipeline with Muon +AdamW optimizer, ready to run on all NeuronCores
- A Trainium-optimized autoresearch framework for AI-assisted experimentation
- Full NKI documentation: programming guide, ISA reference, architecture docs, and example kernels
- Neuron Explorer for comprehensive profiling and performance debugging of NKI kernels
- The CORE evaluation harness for inference self-scoring during Phase 2
- Eligible academic teams can obtain AWS Promotional Credits covering Trainium compute and Amazon Bedrock access
- Native PyTorch for Neuron with no additional package installation required
Who should compete
- ML architecture and training researchers exploring model designs, optimizers, and training recipes. Familiarity with PyTorch and transformer training is expected; no hardware kernel experience is required to be competitive at the ML layer.
- ML systems researchers and performance engineers interested in hardware-aware optimization, custom kernels, and the interplay between model design and hardware. Familiarity with CUDA, Triton, or similar kernel programming transfers directly to NKI; no prior Trainium experience is required.
- Teams building with AI research agents, using LLMs and automation to run more experiments, write more kernels, and explore more architectures than any single person could. This competition rewards breadth of exploration, making agentic approaches a natural fit.
- Teams of one to four members are welcome. Strong submissions will likely combine multiple of these perspectives, either within a single team or via AI-assisted workflows that extend a team’s reach across the stack.
What's at stake
- Top three finalists present their work at an exclusive Annapurna Labs research event during NeurIPS 2026 in Sydney, Australia, sharing findings with the ML community and AWS AI Chips leadership. Travel and expenses are the finalists’ responsibility
- Top 10 team members receive exclusive Neuron team jackets and finalist swag packs.
- Top three finalists have the opportunity to co-publish findings with Annapurna Labs researchers, contributing to a seminal paper on hardware-native model design.
- Prize pool: $25,000 (first), $10,000 (second), $5,000 (third).
Key dates
- Aug. 31, 2026: Phase 1 opens; leaderboard goes live
- Sept. 30, 2026: Phase 1 closes; top 10 announced
- Oct. 7, 2026: Phase 2 opens on full Trn2 servers for top 10
- Nov. 4, 2026: Phase 2 closes
- Nov. 11, 2026: Finalists selected
- December 6–12, 2026: Finalist presentations at competition workshop in Sydney
Register by Sept 30, 2026. Other eligibility restrictions apply. See terms and conditions.
The competition is open to the first 100 teams to register. Participants must be 18 or older. AWS employees, interns, and scholars (2025–2026) and their immediate family members are ineligible. Residents of certain countries are excluded; see full competition terms for details. Team sizes can be one to four participants.Register today to secure your team’s spot and start building on genuinely new silicon. The frontier is open — come find out what’s on the other side.
