A new method estimates corporate bond defaults in financial networks efficiently.
problem Challenges in valuing corporate bonds in interconnected financial systems.
method Bi-Level Importance Sampling with Splitting
result The method efficiently estimates rare default events in financial networks.
New method uses bi-level optimization to learn useful representations for imitation learning.
problem Learning useful representations for multiple tasks in imitation learning settings.
method Formulates representation learning as a bi-level optimization problem.
result Bi-level optimization framework provides sample complexity benefits for imitation learning.
BiDVL improves EBLVMs for visual tasks by optimizing two variational distributions.
problem Training EBLVMs is challenging due to intractable distributions.
method Bi-level doubly variational learning with two tractable distributions.
result BiDVL achieves impressive image generation and reconstruction performance.
New algorithm stabilizes bi-level hyperparameter optimization.
problem Stability issues in bi-level hyperparameter optimization.
method Uses Moreau-Yosida regularization to stabilize convergence.
result Significant improvement in loss values with fixed computation budget.
New framework tackles bi-level optimization without LLS condition.
problem Bi-level optimization problems without LLS condition.
method Bi-level Descent Aggregation (BDA) framework, proving convergence without LLS condition.
result Proves convergence of BDA without LLS condition.
Optimizes weights for better model performance in shifting data.
problem Improper importance weighting leads to poor model performance in data shifts.
method Interprets weights as a bias-variance trade-off and optimizes them simultaneously with model parameters.
result Optimizing weights significantly improves model generalization performance.
A game-theoretic approach simplifies MBRL design and improves sample efficiency.
problem Designing stable and efficient MBRL algorithms using rich function approximators.
method Develops a game-theoretic framework where MBRL is modeled as a Stackelberg game between policy and model players.
result Proposed algorithms are highly sample efficient and match asymptotic performance of model-free policy gradient.
Paper explores generalization of AID-based bi-level optimization methods.
problem Uncertainty in generalization properties of AID-based bi-level optimization methods.
method Uniform stability analysis and convergence study of AID-based methods.
result AID-based methods can achieve similar generalization as single-level nonconvex problems.
A new stochastic method tackles bi-level optimization problems in deep learning.
problem Bi-level optimization problems in deep learning, including hyperparameter optimization and meta learning.
method Turning a BLO problem into a stochastic optimization, using SGLD MCMC and a recurrent algorithm to compute MC-estimated hypergradient.
result Our method is more robust to suboptimal inner optimization and non-unique inner minima, leading to more reliable solutions.
A new algorithm improves Wasserstein discriminant analysis for better data classification.
problem Improving data classification in machine learning.
method Bi-level nonlinear eigenvector algorithm (WDA-nepv) for optimal transport and trace ratio optimizations.
result WDA-nepv enhances classification accuracy and scalability.
Biases in observational data of treatments pose a major challenge to estimating expected treatment outcomes in different populations. An important technique that accounts for these biases is reweighting samples to minimize the discrepancy between treatment groups. We present a novel reweighting approach that uses bi-le…
A method for fair representation learning through bi-level optimization and implicit differentiation.
problem Ensuring fair predictors invariant across sub-groups.
method Bi-level optimization with inner-loop for invariant predictors, implicit path alignment for efficiency.
result Consistently better trade-off in prediction performance and fairness measurement.
Bi-GNN models drug interactions using a bi-level graph approach.
problem Predicting drug-drug interactions using machine learning.
method Bi-level graph neural networks that consider both interaction graph and representation graphs of drugs.
result Bi-GNN model improves DDI prediction accuracy compared to existing methods.
This paper proves IRM minimizes o.o.d. risk under certain conditions.
problem Deep networks can fail to generalize to new domains with different distributions.
method Proves IRM minimizes o.o.d. risk through a bi-level optimization problem.
result IRM minimizes o.o.d. risk under specific conditions.
New algorithm tackles nested bi-level optimization problems for robust feature learning.
problem Nested compositional bi-level optimization problems in machine learning.
method Stochastic approximation algorithms for solving nested compositional bi-level optimization problems without matrix inversions.
result Achieves an ε-stationary solution with an oracle complexity of approximately O_T(1/ε^2).
SHINE uses forward pass quasi-Newton matrices to approximate Jacobian inverses for faster bi-level optimization.
problem Efficiently solving bi-level optimization problems with large Jacobian matrices.
method Proposes using quasi-Newton matrices from the forward pass to approximate the inverse Jacobian matrix.
result Empirically shows SHINE reduces computational cost of the backward pass for various problems.
Paper solves inverse optimal transport problem with convex optimization and neural network.
problem Learning the cost function for optimal transport from observed data.
method Unconstrained convex optimization, Sinkhorn-Knopp algorithm, and deep neural network parameterization.
result Novel framework avoids repeated OT solving, demonstrating efficiency and accuracy.
Predicting traffic incident duration is a major challenge for many traffic centres around the world. Most research studies focus on predicting the incident duration on motorways rather than arterial roads, due to a high network complexity and lack of data. In this paper we propose a bi-level framework for predicting th…
HRT uses bi-level reinforcement learning to optimize stock selection and execution in multi-asset equity markets.
problem Optimizing automated equity trading decisions under risk, turnover, and transaction costs.
method Hierarchical Reinforced Trader (HRT) framework that separates selection and execution decisions.
result HRT outperforms other methods in learning-based return-risk-cost trade-offs, improving Sharpe ratio and reducing turnover.
Event ticket price prediction is important to marketing strategy for any sports team or musical ensemble. An accurate prediction model can help the marketing team to make promotion plan more effectively and efficiently. However, given all the historical transaction records, it is challenging to predict the sale price o…
DGKIP extends KIP for dataset distillation without bi-level optimization.
problem Efficiently distill datasets for various loss functions.
method Leverages duality theory to avoid bi-level optimization.
result DGKIP supports a wider range of loss functions.
Paper tackles NAS problem by modeling it as a sparse supernet.
problem Neural Architecture Search (NAS) problem, particularly Mixed-Path Search.
method Model NAS as a sparse supernet with sparsity constraints. Use hierarchical accelerated proximal gradient algorithm for optimization.
result Proposed method finds compact, general, and powerful neural architectures.
This paper improves HNNs by learning optimal curvature for better generalization.
problem Inappropriate curvatures in HNNs lead to suboptimal performance.
method Sharpness-aware curvature learning method to smooth loss landscape.
result Proposed method improves HNNs' generalization across various settings.
Develops a generic two-layer framework for adaptive ABMs.
problem Bi-level adaptation problem in ABMs: agents adapt to environment, and environment adapts to agents.
method Formalizes bi-level problem as a Stackelberg game with conditional policies, solving coupled non-linear equations.
result Unified framework for adaptive ABMs, addressing traditional ABM limitations.
Proposes efficient, modular method for implicit differentiation.
problem Implicit differentiation of optimization problems.
method Automatic implicit differentiation using autodiff and implicit function theorem.
result Automatic differentiation of optimization problems is made easier and more modular.
Two algorithms improve GP bandits by selecting priors and minimizing regret.
problem Selecting appropriate GP priors for unknown functions.
method Developed two algorithms: Prior-Elimination GP-TS and HyperPrior GP-TS.
result Established sublinear regret bound for HyperPrior GP-TS.
pFedMe uses Moreau envelopes to improve personalized FL performance.
problem Statistical diversity among clients limits FL performance.
method pFedMe employs Moreau envelopes as clients' regularized loss functions in a bi-level optimization problem.
result pFedMe achieves state-of-the-art convergence rates and superior empirical performance.
New algorithm improves dataset distillation with 108% improvement on ImageNet.
problem Improving dataset distillation performance.
method Reparameterization and convexification of implicit gradients (RCIG).
result Establishes new state-of-the-art on various dataset distillation tasks.
Sample selection approaches are popular in robust learning from noisy labels. However, how to properly control the selection process so that deep networks can benefit from the memorization effect is a hard problem. In this paper, motivated by the success of automated machine learning (AutoML), we model this issue as a …
Study resolves conjecture on overparameterized linear models' generalization.
problem Asymptotic generalization of multiclass classification with overparameterized models.
method Gaussian covariates bi-level model, Hanson-Wright inequality variant.
result Min-norm interpolating classifier can be suboptimal compared to noninterpolating classifiers.
The paper examines how to protect LASSO-based feature selection from adversarial attacks.
problem Adversarial attacks on LASSO-based feature selection.
method Formulated as a bi-level optimization problem, reformulated LASSO with linear inequality constraints, solved using interior-point method, and modified using projected gradient descent.
result Demonstrated the effectiveness of the proposed method in protecting LASSO-based feature selection from adversarial attacks.
Paper learns dictionaries for sparse signal recovery using automatic differentiation.
problem Learning dictionaries for sparse signal recovery from noisy data.
method Approximates reconstructions using FB algorithm and learns dictionaries with projected gradient descent.
result Successfully learns 1D TV dictionary from piecewise constant signals.
RODE learns roles to simplify multi-agent tasks.
problem Efficiently discovering roles for complex multi-agent tasks.
method Clustering actions based on effects, bi-level learning hierarchy, integrating action effects into role policies.
result RODE outperforms state-of-the-art MARL algorithms on StarCraft II benchmarks.
New method uses diffusion models to optimize experimental design efficiently.
problem Optimizing experimental design for high-dimensional and complex settings.
method Introduces a pooled posterior distribution and uses diffusion-based samplers for efficient sampling and optimization.
result Extends Bayesian Optimal Experimental Design to practical scenarios.
Paper proposes a new trading strategy using corporate event detection from news articles.
problem Predicting stock movements based on corporate events from news articles.
method Bi-level event detection model: low-level for token-level event identification, high-level for article-level event identification.
result The proposed strategy outperforms existing models in stock prediction metrics.
MixMin finds optimal data mixtures for better model performance.
problem Finding the best data mixtures for improved model performance is challenging.
method Developed a gradient-based approach for optimizing a convex bi-level objective.
result MixMin mixtures uniformly improved model performance across various tasks.
We provide a novel approach and an exploratory study for modelling life event choices and occurrence from a probabilistic perspective through causal discovery and survival analysis. Our approach is formulated as a bi-level problem. In the upper level, we build the life events graph, using causal discovery tools. In the…
Leveraging weak or noisy supervision for building effective machine learning models has long been an important research problem. Its importance has further increased recently due to the growing need for large-scale datasets to train deep learning models. Weak or noisy supervision could originate from multiple sources i…
Machine learning algorithms are vulnerable to poisoning attacks: An adversary can inject malicious points in the training dataset to influence the learning process and degrade the algorithm's performance. Optimal poisoning attacks have already been proposed to evaluate worst-case scenarios, modelling attacks as a bi-le…
Optimal quantization improves dataset distillation for faster training.
problem Efficiently train models with synthetic data.
method Reformulate disentangled methods as optimal quantization problems.
result Better performance and generalization on ImageNet-1K and subsets.
The paper proposes an efficient nested simulation design using likelihood ratio method.
problem Designing nested simulations with fixed outer scenarios and minimizing simulation effort.
method Proposes a bi-level optimization problem to decide inner replications and pooling strategies.
result Optimized design achieves $\cO(Γ^{-1})$ mean squared error of estimators.
Importance sampling is often used in machine learning when training and testing data come from different distributions. In this paper we propose a new variant of importance sampling that can reduce the variance of importance sampling-based estimates by orders of magnitude when the supports of the training and testing d…
Paper proposes a method to estimate variance reduction in DNN training using importance sampling.
problem Challenges in assessing variance reduction during DNN training using importance sampling.
method Proposes a method for estimating variance reduction using minibatches sampled under importance sampling.
result Demonstrates consistent reduction in variance, improved training efficiency, and enhanced model accuracy.
Importance sampling is widely used in machine learning and statistics, but its power is limited by the restriction of using simple proposals for which the importance weights can be tractably calculated. We address this problem by studying black-box importance sampling methods that calculate importance weights for sampl…
The paper analyzes and improves privacy in machine learning through importance sampling.
problem Ensuring privacy in machine learning while maintaining utility and efficiency.
method Individualized privacy analysis of importance sampling, proposing two approaches for constructing sampling distributions.
result Proposed approaches optimize privacy-efficiency trade-off and outperform uniform sampling.
Our model predicts stock market intervals using chaotic fusion and graph convolutional networks.
problem Uncertainty in financial market predictions without quantified uncertainty.
method Bi-level chaotic fusion, graph convolutional networks, volatility-aware gating, temporal dependencies.
result Significant improvements in prediction intervals and coverage compared to existing methods.
The paper explains how importance sampling can be used for optimization of rare events.
problem Minimizing tail risks in stochastic optimization formulations.
method Importance sampling for reducing sample requirements in estimating rare events.
result Effective importance sampling techniques for optimization of rare events.
Paper introduces a new sampling method combining Consistency Models with importance sampling.
problem Inherent errors in samples and high NFEs for high-quality samples in Boltzmann distributions.
method Combines Consistency Models with importance sampling to produce unbiased samples with minimal NFEs.
result Produces unbiased samples using only 6-25 NFEs, comparable to 100 NFEs for DDPMs.