-
2024This paper proposes the use of “multicalibration” to yield interpretable and reliable confidence scores for outputs generated by large language models (LLMs). Multicalibration asks for calibration not just marginally, but simultaneously across various intersecting groupings of the data. We show how to form groupings for prompt/completion pairs that are correlated with the probability of correctness via
-
2024Adapting large language models (LLMs) to unseen tasks with in-context training samples without fine-tuning remains an important research problem. To learn a robust LLM that adapts well to unseen tasks, multiple meta-training approaches have been proposed such as MetaICL and MetaICT, which involve meta-training pre-trained LLMs on a wide variety of diverse tasks. These meta-training approaches essentially
-
2024Following the introduction of Adam, several novel adaptive optimizers for deep learning have been proposed. These optimizers typically excel in some tasks but may not outperform Adam uniformly across all tasks. In this work, we introduce Meta-Adaptive Optimizers (MADA), a unified optimizer framework that can generalize several known optimizers and dynamically learn the most suitable one during training.
-
2024A key challenge in contrastive learning is to generate negative samples from a large sample set to contrast with positive samples, for learning better encoding of the data. These negative samples often follow a softmax distribution which are dynamically updated during the training process. However, sampling from this distribution is non-trivial due to the high computational costs in computing the partition
-
2024Selecting appropriate thresholds for anomaly detection in online, unsupervised settings is a challenging task, especially in the presence of data distribution shifts. Addressing these challenges is critical in many practical large scale systems, such as infrastructure monitoring and network intrusion detection. This paper proposes an algorithm that connects online thresholding with constructing confidence
Related content
-
August 10, 2022The author of Ethical Machines explains why companies pursuing ethical AI must ultimately place the responsibility with their senior leadership.
-
August 09, 2022A combination of cutting-edge hardware, sensor technology, and bespoke machine learning approaches can predict trajectories of vehicles, people, and even animals, as far as 8 seconds into the future.
-
August 08, 2022Thirteen new projects focus on ensuring fairness in AI algorithms and the systems that incorporate them.
-
August 04, 2022How Jared Wilber is using his skills as a storyteller and data scientist to help others learn about machine learning.
-
August 03, 2022Paper presents a criterion for halting the hyperparameter optimization process.
-
July 29, 2022Syne Tune supports multiple backends, single-fidelity and multi-fidelity (early-exit) optimization algorithms, and hyperparameter transfer learning.