arXiv research

Jointly tuning ensemble models improves performance and uncertainty calibration.

Proposes baselines for joint NAS and HPO optimization.

Deep neural networks require a large amount of labeled training data during supervised learning. However, collecting and labeling so much data might be infeasible in many cases. In this paper, we introduce a source-target selective joint fine-tuning scheme for improving the performance of deep learning tasks with insuf…

Deep structured-prediction energy-based models combine the expressive power of learned representations and the ability of embedding knowledge about the task at hand into the system. A common way to learn parameters of such models consists in a multistage procedure where different combinations of components are trained …

New method improves uncertainty quantification in latent variable models.

TensorGuide improves LoRA efficiency and expressivity through joint tensor-train optimization.

New framework improves LLM performance by avoiding forgetting during sequential training stages.

We analyze SGAs for statistical inference via asymptotics, improving tuning methods.

In this paper, we introduce the problem of jointly learning feed-forward neural networks across a set of relevant but diverse datasets. Compared to learning a separate network from each dataset in isolation, joint learning enables us to extract correlated information across multiple datasets to significantly improve th…

Objective: Joint analysis of multi-subject brain imaging datasets has wide applications in biomedical engineering. In these datasets, some sources belong to all subjects (joint), a subset of subjects (partially-joint), or a single subject (individual). In this paper, this source model is referred to as joint/partially-…

A tuning-free method recovers jointly sparse signals in MMV using implicit regularization.

New algorithms eliminate stepsize tuning for bilevel optimization problems.

While existing work on neural architecture search (NAS) tunes hyperparameters in a separate post-processing step, we demonstrate that architectural choices and other hyperparameter settings interact in a way that can render this separation suboptimal. Likewise, we demonstrate that the common practice of using very few …

A new method aligns source and target distributions by tuning their weights.

SCOPE estimator improves covariance and precision matrix estimation.

Fine-tuning neural networks is widely used to transfer valuable knowledge from high-resource to low-resource domains. In a standard fine-tuning scheme, source and target problems are trained using the same architecture. Although capable of adapting to new domains, pre-trained units struggle with learning uncommon targe…

A scalable algorithm for sampling and fine-tuning models using Tilt Matching.

CoDeQ simplifies joint model compression by integrating pruning and quantization.

New method disentangles style features from data augmentations.

Optimizes VAE hyperparameters for efficient training and manifold discovery.

Novel approach trains ASR models with less supervision using bilevel optimization.

Both feature selection and hyperparameter tuning are key tasks in machine learning. Hyperparameter tuning is often useful to increase model performance, while feature selection is undertaken to attain sparse models. Sparsity may yield better model interpretability and lower cost of data acquisition, data handling and m…

NGBoost boosts multivariate probabilistic regression.

Optimizing full likelihoods adapts loss scales and shapes for robust modeling.

VT-DIS improves sampling from Boltzmann distributions with minimal overhead.

We present a variational Bayesian method of joint image reconstruction and point spread function (PSF) estimation when the PSF of the imaging device is only partially known. To solve this semi-blind deconvolution problem, prior distributions are specified for the PSF and the 3D image. Joint image reconstruction and PSF…

Piecewise Linear-Quadratic (PLQ) penalties are widely used to develop models in statistical inference, signal processing, and machine learning. Common examples of PLQ penalties include least squares, Huber, Vapnik, 1-norm, and their asymmetric generalizations. Properties of these estimators depend on the choice of pena…

Residual networks with depthwise hyperparameter scaling transfer optimal hyperparameters across width and depth.

JEPA fails to improve language model performance when fine-tuning.

A new method prunes neural network channels based on operation characteristics.

We investigate the choice of tuning parameters for a Bayesian multi-level group lasso model developed for the joint analysis of neuroimaging and genetic data. The regression model we consider relates multivariate phenotypes consisting of brain summary measures (volumetric and cortical thickness values) to single nucleo…

LOV model calibrates European and American options with path-dependent volatility.

We propose generative neural network methods to generate DNA sequences and tune them to have desired properties. We present three approaches: creating synthetic DNA sequences using a generative adversarial network; a DNA-based variant of the activation maximization ("deep dream") design method; and a joint procedure wh…

FalconBC improves patient-specific cardiovascular modeling by estimating boundary conditions efficiently.

Paper presents a probabilistic model to improve LLM cascade performance.

This paper presents a non-trivial reconstruction of a previous joint topic-sentiment-preference review model TSPRA with stick-breaking representation under the framework of variational inference (VI) and stochastic variational inference (SVI). TSPRA is a Gibbs Sampling based model that solves topics, word sentiments an…

Bayesian priors improve neural network performance on weak signals.

The ever increasing computational cost of Deep Neural Networks (DNN) and the demand for energy efficient hardware for DNN acceleration has made accuracy and hardware cost co-optimization for DNNs tremendously important, especially for edge devices. Owing to the large parameter space and cost of evaluating each paramete…

Paper defines a new distance metric for comparing learning tasks.

A new filter adapts to heavy-tailed data without tuning, improving performance in challenging conditions.

We consider the problem of comparing probability densities between two groups. A new probabilistic tensor product smoothing spline framework is developed to model the joint density of two variables. Under such a framework, the probability density comparison is equivalent to testing the presence/absence of interactions.…

Unified study of stateful replay for streaming learning, reducing forgetting by 2-3x.

Robotic table tennis learns efficient policies to return balls at 100Hz.

New algorithm improves fairness and robustness in federated learning.

We design a self size-estimating feed-forward network (SSFN) using a joint optimization approach for estimation of number of layers, number of nodes and learning of weight matrices. The learning algorithm has a low computational complexity, preferably within few minutes using a laptop. In addition the algorithm has a l…

AgEBO-Tabular combines NAS and hyperparameter tuning for fast, high-performing tabular models.

Popular sparse estimation methods based on $\ell_1$-relaxation, such as the Lasso and the Dantzig selector, require the knowledge of the variance of the noise in order to properly tune the regularization parameter. This constitutes a major obstacle in applying these methods in several frameworks---such as time series, …

GenSBI offers JAX-native SBI methods for natural sciences.