arXiv research

This work fills the gap in understanding multi-objective learning generalization.

MOL-TS uses Thompson Sampling for multi-objective linear bandits with Pareto guarantees.

This paper presents a new mechanism for producing sanitized statistical summaries that achieve \emph{differential privacy}, called the \emph{K-Norm Gradient} Mechanism, or KNG. This new approach maintains the strong flexibility of the exponential mechanism, while achieving the powerful utility performance of objective …

Improves multi-objective learning by adapting to local subintervals.

YAHPO Gym introduces a new benchmark for evaluating hyperparameter optimization methods.

Recent policy optimization approaches (Schulman et al., 2015a; 2017) have achieved substantial empirical successes by constructing new proxy optimization objectives. These proxy objectives allow stable and low variance policy learning, but require small policy updates to ensure that the proxy objective remains an accur…

We introduce a rich model for multi-objective clustering with lexicographic ordering over objectives and a slack. The slack denotes the allowed multiplicative deviation from the optimal objective value of the higher priority objective to facilitate improvement in lower-priority objectives. We then propose an algorithm …

Generative adversarial networks (GANs) have enjoyed much success in learning high-dimensional distributions. Learning objectives approximately minimize an $f$-divergence ($f$-GANs) or an integral probability metric (Wasserstein GANs) between the model and the data distribution using a discriminator. Wasserstein GANs en…

SelMix fine-tunes pre-trained models to optimize non-decomposable objectives.

We consider the problem of consistently matching multiple sets of elements to each other, which is a common task in fields such as computer vision. To solve the underlying NP-hard objective, existing methods often relax or approximate it, but end up with unsatisfying empirical performance due to a misaligned objective.…

The paper explores fair machine learning policies for balancing competing objectives in noisy data.

Bayesian model for multi-environment prediction with latent variable changes.

Introduces BPEL for EL, enhancing flexibility and using MCMC for inference.

New algorithms identify Pareto optimal sets in multi-objective bandit problems.

Unified framework for aligning LLMs from human feedback.

New framework for optimizing machine learning risks.

A new method for faster multi-objective optimization by evaluating objectives separately.

We propose a new objective, the counterfactual objective, unifying existing objectives for off-policy policy gradient algorithms in the continuing reinforcement learning (RL) setting. Compared to the commonly used excursion objective, which can be misleading about the performance of the target policy when deployed, our…

Study shows bottlenecks improve image segmentation quality.

Optimizes bilevel empirical risk minimization with improved oracle calls.

Simple greedy algorithms can excel in multi-objective bandits with multiple good arms.

SAIL-RevKL improves SAIL's convergence by regularizing the objective function.

Slot Attention extracts object-centric representations from images.

Stochastic optimization of continuous objectives is at the heart of modern machine learning. However, many important problems are of discrete nature and often involve submodular objectives. We seek to unleash the power of stochastic continuous optimization, namely stochastic gradient descent and its variants, to such d…

OBSER framework infers sub-environments from objects, outperforming scene-based methods.

Improved GFlowNets learn more efficiently with trajectory balance.

Proves accuracy guarantees for self-supervised learning with correlated positive pairs.

Selecting appropriate regularization coefficients is critical to performance with respect to regularized empirical risk minimization problems. Existing theoretical approaches attempt to determine the coefficients in order for regularized empirical objectives to be upper-bounds of true objectives, uniformly over a hypot…

Generative models create indistinguishable adversarial objects for object detection.

We consider object recognition in the context of lifelong learning, where a robotic agent learns to discriminate between a growing number of object classes as it accumulates experience about the environment. We propose an incremental variant of the Regularized Least Squares for Classification (RLSC) algorithm, and expl…

New framework converts multi-objective to single-objective optimisation.

Develops a new method to improve performance in multi-objective learning problems.

New method ranks multivariate distributions in SMOOP using q-dominance.

Improved DP optimization for nonconvex, nonsmooth objectives with reduced sample complexity.

MWGraD solves multi-objective distributional optimization using particle-based gradient descent.

Study proves convergence of subgradients for optimal transport-based objectives.

VIB balances empirical and Bayesian approaches in predictive models.

Following widely used in visual recognition concept of relative attributes, the article establishes definition of the relative PCA attributes for a class of objects defined by vectors of their parameters. A new rating model (RELARM) is built using relative PCA attribute ranking functions for rating object description a…

When working with three-dimensional data, choice of representation is key. We explore voxel-based models, and present evidence for the viability of voxellated representations in applications including shape modeling and object classification. Our key contributions are methods for training voxel-based variational autoen…

New research suggests continual learning should focus on both optimization objective and optimization trajectory.

To investigate objects without a describable notion of distance, one can gather ordinal information by asking triplet comparisons of the form "Is object $x$ closer to $y$ or is $x$ closer to $z$?" In order to learn from such data, the objects are typically embedded in a Euclidean space while satisfying as many triplet …

Theoretical analysis of vision transformers' performance with MAE and CL objectives.

We present novel minibatch stochastic optimization methods for empirical risk minimization problems, the methods efficiently leverage variance reduced first-order and sub-sampled higher-order information to accelerate the convergence speed. For quadratic objectives, we prove improved iteration complexity over state-of-…

Online TERM improves robustness and fairness in streaming data.

Generative adversarial networks (GAN) approximate a target data distribution by jointly optimizing an objective function through a "two-player game" between a generator and a discriminator. Despite their empirical success, however, two very basic questions on how well they can approximate the target distribution remain…

Study inverse problems with measure samples, improving estimator calibration and recovery.

This work analyzes the gap between off-policy and on-policy policy gradient methods and provides conditions to reduce this gap.

Bayesian framework learns latent preference archetypes for many-objective optimization.