arXiv research

Machine learning predicts failure in brittle materials with high accuracy.

Neural program analyzers are brittle and sensitive to input changes.

The paper highlights AI brittleness and the need for robust testing out-of-distribution performance.

Study identifies key parameters and input dimensions making LLMs and VLMs brittle.

In this paper, five different approaches for reduced-order modeling of brittle fracture in geomaterials, specifically concrete, are presented and compared. Four of the five methods rely on machine learning (ML) algorithms to approximate important aspects of the brittle fracture problem. In addition to the ML algorithms…

Improves efficiency of simulators that fail to return.

ARO overfits by making constraints dependent on uncertainty, leading to brittleness.

We propose a machine learning approach to address a key challenge in materials science: predicting how fractures propagate in brittle materials under stress, and how these materials ultimately fail. Our methods use deep learning and train on simulation data from high-fidelity models, emulating the results of these mode…

New method robust to semi-random sparse recovery, nearly-linear time.

Transformers simulate finite-state automata with fewer layers.

Recent reinforcement learning algorithms, though achieving impressive results in various fields, suffer from brittle training effects such as regression in results and high sensitivity to initialization and parameters. We claim that some of the brittleness stems from variance differences, i.e. when different environmen…

DAFNO learns surrogates for complex systems on irregular geometries.

Standard neural network architectures are non-linear only by virtue of a simple element-wise activation function, making them both brittle and excessively large. In this paper, we consider methods for making the feed-forward layer more flexible while preserving its basic structure. We develop simple drop-in replacement…

Standard stochastic optimization methods are brittle, sensitive to stepsize choices and other algorithmic parameters, and they exhibit instability outside of well-behaved families of objectives. To address these challenges, we investigate models for stochastic minimization and learning problems that exhibit better robu…

Unified framework for fair decision-making across diverse groups.

New method calibrates models under covariate shifts.

Novel Bayesian neural network method for robustness.

Introduces PCG for better counterfactual explanations in vision models.

Maximizes robustness in Bayesian experimental design under model uncertainty.

Optimizes structure topology for ductile and brittle fracture resistance.

Adversarial examples have attracted significant attention in machine learning, but the reasons for their existence and pervasiveness remain unclear. We demonstrate that adversarial examples can be directly attributed to the presence of non-robust features: features derived from patterns in the data distribution that ar…

A simple method treats heteroscedastic variance variatively, improving model calibration and sample quality.

The risks and perils of overfitting in machine learning are well known. However most of the treatment of this, including diagnostic tools and remedies, was developed for the supervised learning case. In this work, we aim to offer new perspectives on the characterization and prevention of overfitting in deep Reinforceme…

The DoD needs a robust process to evaluate AI/ML model performance and robustness.

Deep Reinforcement Learning (DRL) algorithms for continuous action spaces are known to be brittle toward hyperparameters as well as \cut{being}sample inefficient. Soft Actor Critic (SAC) proposes an off-policy deep actor critic algorithm within the maximum entropy RL framework which offers greater stability and empiric…

Machine learning is currently dominated by largely experimental work focused on improvements in a few key tasks. However, the impressive accuracy numbers of the best performing models are questionable because the same test sets have been used to select these models for multiple years now. To understand the danger of ov…

Variational Proximal Policy Optimization improves reinforcement learning from human feedback.

Machine learning promises methods that generalize well from finite labeled data. However, the brittleness of existing neural net approaches is revealed by notable failures, such as the existence of adversarial examples that are misclassified despite being nearly identical to a training example, or the inability of recu…

Stabilizes deep Bayesian neural networks with self-stabilizing priors.

In medicine, both ethical and monetary costs of incorrect predictions can be significant, and the complexity of the problems often necessitates increasingly complex models. Recent work has shown that changing just the random seed is enough for otherwise well-tuned deep neural networks to vary in their individual predic…

DSI improves tail-risk estimation in generative models by averaging checkpoints.

Modern deep neural networks are well known to be brittle in the face of unknown data instances and recognition of the latter remains a challenge. Although it is inevitable for continual-learning systems to encounter such unseen concepts, the corresponding literature appears to nonetheless focus primarily on alleviating…

SmoothLLM defends LLMs from jailbreaking attacks by randomly perturbing inputs.

Extends DRFGP to make GPs more robust and adaptive for dynamic, noisy data.

While current benchmark reinforcement learning (RL) tasks have been useful to drive progress in the field, they are in many ways poor substitutes for learning with real-world data. By testing increasingly complex RL algorithms on low-complexity simulation environments, we often end up with brittle RL policies that gene…

Deconfounding scores improve causal effect estimation with weak overlap.

Combining diffusion models with Langevin dynamics improves posterior sampling efficiency.

Framework uses LLMs to automate strategy finding in quantitative finance.

We study the problem of using i.i.d. samples from an unknown multivariate probability distribution $p$ to estimate the mutual information of $p$. This problem has recently received attention in two settings: (1) where $p$ is assumed to be Gaussian and (2) where $p$ is assumed only to lie in a large nonparametric smooth…

Simple tabular event prediction model outperforms existing methods.

Cold-start problems are long-standing challenges for practical recommendations. Most existing recommendation algorithms rely on extensive observed data and are brittle to recommendation scenarios with few interactions. This paper addresses such problems using few-shot learning and meta learning. Our approach is based o…

Recent work has shown that state-of-the-art classifiers are quite brittle, in the sense that a small adversarial change of an originally with high confidence correctly classified input leads to a wrong classification again with high confidence. This raises concerns that such classifiers are vulnerable to attacks and ca…

New method reduces over-pessimism in Bayesian control under parameter uncertainty.

Model patching closes subgroup performance gaps in skin cancer classification.

There is a widespread need for techniques that can discover structure from time series data. Recently introduced techniques such as Automatic Bayesian Covariance Discovery (ABCD) provide a way to find structure within a single time series by searching through a space of covariance kernels that is generated using a simp…

This work improves neural network robustness to symbol substitutions using formal verification.

New method stabilizes DEQ models by regularizing Jacobian of fixed-point equations.

Datamodels predict model outcomes from training data subsets.