Customer-obsessed science
Research areas
-
October 12, 202614 min read“Priming” reuses pretrained Transformer weights to build models with more efficient architectures at less than 1% of the original training cost and with throughputs up to twice as high.
-
-
-
August 21, 20269 min read
-
July 30, 20268 min read
Featured news
-
NeurIPS 20222022We study semi-supervised learning (SSL) for vision transformers (ViT), an underexplored topic despite the wide adoption of the ViT architectures to different tasks. To tackle this problem, we use a SSL pipeline, consisting of first un/self-supervised pre-training, followed by supervised fine-tuning, and finally semi-supervised finetuning. At the semi-supervised fine-tuning stage, we adopt an exponential
-
AACL-IJCNLP 20222022Reasoning with preconditions such as “glass can be used for drinking water unless the glass is shattered” remains an open problem for language models. The main challenge lies in the scarcity of preconditions data and model’s lack of support for such reasoning. We present PInKS , Preconditioned Commonsense Inference with WeaK Supervision, an improved model for reasoning with preconditions through minimum
-
EMNLP 20222022Recent research has shown that large language models pretrained using unsupervised approaches can achieve significant performance improvement on many downstream tasks. Typically when adapting these language models to downstream tasks, like a classification or regression task, we employ a finetuning paradigm in which the sentence representation from the language model is input to a task-specific head; the
-
NeurIPS 20222022Conventionally, Earth system (e.g., weather and climate) forecasting relies on numerical simulation with complex physical models and hence is both expensive in computation and demanding on domain expertise. With the explosive growth of spatiotemporal Earth observation data in the past decade, data-driven models that apply Deep Learning (DL) are demonstrating impressive potential for various Earth system
-
NeurIPS 20222022In the context of personalized federated learning (FL), the critical challenge is to balance local model improvement and global model tuning when the personal and global objectives may not be exactly aligned. Inspired by Bayesian hierarchical models, we develop a self-aware personalized FL method where each client can automatically balance the training of its local personal model and the global model that
Collaborations
View allWhether you're a faculty member or student, there are number of ways you can engage with Amazon.
View all