New framework tackles bi-level optimization without LLS condition.
problem Bi-level optimization problems without LLS condition.
method Bi-level Descent Aggregation (BDA) framework, proving convergence without LLS condition.
result Proves convergence of BDA without LLS condition.
New method uses bi-level optimization to learn useful representations for imitation learning.
problem Learning useful representations for multiple tasks in imitation learning settings.
method Formulates representation learning as a bi-level optimization problem.
result Bi-level optimization framework provides sample complexity benefits for imitation learning.
A method for fair representation learning through bi-level optimization and implicit differentiation.
problem Ensuring fair predictors invariant across sub-groups.
method Bi-level optimization with inner-loop for invariant predictors, implicit path alignment for efficiency.
result Consistently better trade-off in prediction performance and fairness measurement.
Paper explores generalization of AID-based bi-level optimization methods.
problem Uncertainty in generalization properties of AID-based bi-level optimization methods.
method Uniform stability analysis and convergence study of AID-based methods.
result AID-based methods can achieve similar generalization as single-level nonconvex problems.
New algorithm stabilizes bi-level hyperparameter optimization.
problem Stability issues in bi-level hyperparameter optimization.
method Uses Moreau-Yosida regularization to stabilize convergence.
result Significant improvement in loss values with fixed computation budget.
BiDVL improves EBLVMs for visual tasks by optimizing two variational distributions.
problem Training EBLVMs is challenging due to intractable distributions.
method Bi-level doubly variational learning with two tractable distributions.
result BiDVL achieves impressive image generation and reconstruction performance.
Bi-GNN models drug interactions using a bi-level graph approach.
problem Predicting drug-drug interactions using machine learning.
method Bi-level graph neural networks that consider both interaction graph and representation graphs of drugs.
result Bi-GNN model improves DDI prediction accuracy compared to existing methods.
New algorithm tackles nested bi-level optimization problems for robust feature learning.
problem Nested compositional bi-level optimization problems in machine learning.
method Stochastic approximation algorithms for solving nested compositional bi-level optimization problems without matrix inversions.
result Achieves an ε-stationary solution with an oracle complexity of approximately O_T(1/ε^2).
A new stochastic method tackles bi-level optimization problems in deep learning.
problem Bi-level optimization problems in deep learning, including hyperparameter optimization and meta learning.
method Turning a BLO problem into a stochastic optimization, using SGLD MCMC and a recurrent algorithm to compute MC-estimated hypergradient.
result Our method is more robust to suboptimal inner optimization and non-unique inner minima, leading to more reliable solutions.
SHINE uses forward pass quasi-Newton matrices to approximate Jacobian inverses for faster bi-level optimization.
problem Efficiently solving bi-level optimization problems with large Jacobian matrices.
method Proposes using quasi-Newton matrices from the forward pass to approximate the inverse Jacobian matrix.
result Empirically shows SHINE reduces computational cost of the backward pass for various problems.
A new algorithm improves Wasserstein discriminant analysis for better data classification.
problem Improving data classification in machine learning.
method Bi-level nonlinear eigenvector algorithm (WDA-nepv) for optimal transport and trace ratio optimizations.
result WDA-nepv enhances classification accuracy and scalability.
This paper proves IRM minimizes o.o.d. risk under certain conditions.
problem Deep networks can fail to generalize to new domains with different distributions.
method Proves IRM minimizes o.o.d. risk through a bi-level optimization problem.
result IRM minimizes o.o.d. risk under specific conditions.
This paper improves HNNs by learning optimal curvature for better generalization.
problem Inappropriate curvatures in HNNs lead to suboptimal performance.
method Sharpness-aware curvature learning method to smooth loss landscape.
result Proposed method improves HNNs' generalization across various settings.
Develops a generic two-layer framework for adaptive ABMs.
problem Bi-level adaptation problem in ABMs: agents adapt to environment, and environment adapts to agents.
method Formalizes bi-level problem as a Stackelberg game with conditional policies, solving coupled non-linear equations.
result Unified framework for adaptive ABMs, addressing traditional ABM limitations.
pFedMe uses Moreau envelopes to improve personalized FL performance.
problem Statistical diversity among clients limits FL performance.
method pFedMe employs Moreau envelopes as clients' regularized loss functions in a bi-level optimization problem.
result pFedMe achieves state-of-the-art convergence rates and superior empirical performance.
Proposes efficient, modular method for implicit differentiation.
problem Implicit differentiation of optimization problems.
method Automatic implicit differentiation using autodiff and implicit function theorem.
result Automatic differentiation of optimization problems is made easier and more modular.
HRT uses bi-level reinforcement learning to optimize stock selection and execution in multi-asset equity markets.
problem Optimizing automated equity trading decisions under risk, turnover, and transaction costs.
method Hierarchical Reinforced Trader (HRT) framework that separates selection and execution decisions.
result HRT outperforms other methods in learning-based return-risk-cost trade-offs, improving Sharpe ratio and reducing turnover.
DGKIP extends KIP for dataset distillation without bi-level optimization.
problem Efficiently distill datasets for various loss functions.
method Leverages duality theory to avoid bi-level optimization.
result DGKIP supports a wider range of loss functions.
Paper solves inverse optimal transport problem with convex optimization and neural network.
problem Learning the cost function for optimal transport from observed data.
method Unconstrained convex optimization, Sinkhorn-Knopp algorithm, and deep neural network parameterization.
result Novel framework avoids repeated OT solving, demonstrating efficiency and accuracy.
RODE learns roles to simplify multi-agent tasks.
problem Efficiently discovering roles for complex multi-agent tasks.
method Clustering actions based on effects, bi-level learning hierarchy, integrating action effects into role policies.
result RODE outperforms state-of-the-art MARL algorithms on StarCraft II benchmarks.
Paper learns dictionaries for sparse signal recovery using automatic differentiation.
problem Learning dictionaries for sparse signal recovery from noisy data.
method Approximates reconstructions using FB algorithm and learns dictionaries with projected gradient descent.
result Successfully learns 1D TV dictionary from piecewise constant signals.
New algorithm improves dataset distillation with 108% improvement on ImageNet.
problem Improving dataset distillation performance.
method Reparameterization and convexification of implicit gradients (RCIG).
result Establishes new state-of-the-art on various dataset distillation tasks.
Paper tackles NAS problem by modeling it as a sparse supernet.
problem Neural Architecture Search (NAS) problem, particularly Mixed-Path Search.
method Model NAS as a sparse supernet with sparsity constraints. Use hierarchical accelerated proximal gradient algorithm for optimization.
result Proposed method finds compact, general, and powerful neural architectures.
MixMin finds optimal data mixtures for better model performance.
problem Finding the best data mixtures for improved model performance is challenging.
method Developed a gradient-based approach for optimizing a convex bi-level objective.
result MixMin mixtures uniformly improved model performance across various tasks.
A game-theoretic approach simplifies MBRL design and improves sample efficiency.
problem Designing stable and efficient MBRL algorithms using rich function approximators.
method Develops a game-theoretic framework where MBRL is modeled as a Stackelberg game between policy and model players.
result Proposed algorithms are highly sample efficient and match asymptotic performance of model-free policy gradient.
A new method estimates corporate bond defaults in financial networks efficiently.
problem Challenges in valuing corporate bonds in interconnected financial systems.
method Bi-Level Importance Sampling with Splitting
result The method efficiently estimates rare default events in financial networks.
Study resolves conjecture on overparameterized linear models' generalization.
problem Asymptotic generalization of multiclass classification with overparameterized models.
method Gaussian covariates bi-level model, Hanson-Wright inequality variant.
result Min-norm interpolating classifier can be suboptimal compared to noninterpolating classifiers.
The paper examines how to protect LASSO-based feature selection from adversarial attacks.
problem Adversarial attacks on LASSO-based feature selection.
method Formulated as a bi-level optimization problem, reformulated LASSO with linear inequality constraints, solved using interior-point method, and modified using projected gradient descent.
result Demonstrated the effectiveness of the proposed method in protecting LASSO-based feature selection from adversarial attacks.
Our model predicts stock market intervals using chaotic fusion and graph convolutional networks.
problem Uncertainty in financial market predictions without quantified uncertainty.
method Bi-level chaotic fusion, graph convolutional networks, volatility-aware gating, temporal dependencies.
result Significant improvements in prediction intervals and coverage compared to existing methods.
Predicting traffic incident duration is a major challenge for many traffic centres around the world. Most research studies focus on predicting the incident duration on motorways rather than arterial roads, due to a high network complexity and lack of data. In this paper we propose a bi-level framework for predicting th…
Sample selection approaches are popular in robust learning from noisy labels. However, how to properly control the selection process so that deep networks can benefit from the memorization effect is a hard problem. In this paper, motivated by the success of automated machine learning (AutoML), we model this issue as a …
Paper proposes a new trading strategy using corporate event detection from news articles.
problem Predicting stock movements based on corporate events from news articles.
method Bi-level event detection model: low-level for token-level event identification, high-level for article-level event identification.
result The proposed strategy outperforms existing models in stock prediction metrics.
Event ticket price prediction is important to marketing strategy for any sports team or musical ensemble. An accurate prediction model can help the marketing team to make promotion plan more effectively and efficiently. However, given all the historical transaction records, it is challenging to predict the sale price o…
Novel approach trains ASR models with less supervision using bilevel optimization.
problem Training acoustic models for ASR with minimal supervision.
method Bilevel optimization with unsupervised and supervised losses.
result Achieves superior performance compared to existing methods.
This work shows how disentangled and sparse representations improve multi-task learning.
problem Improving generalization in multi-task learning with disentangled and sparse representations.
method Proved a new identifiability result and proposed a practical approach using sparsity-promoting bi-level optimization.
result Maximally sparse base-predictors yield disentangled representations under certain conditions.
Machine learning (ML) methods have been developing rapidly, but configuring and selecting proper methods to achieve a desired performance is increasingly difficult and tedious. To address this challenge, automated machine learning (AutoML) has emerged, which aims to generate satisfactory ML configurations for given tas…
Framework learns linear programs from optimal decisions.
problem Learning linear programs from optimal decisions is challenging.
method Gradient-based framework for learning linear programs from optimal decisions.
result Successfully learns linear programs and multi-commodity flow instances.
Optimal quantization improves dataset distillation for faster training.
problem Efficiently train models with synthetic data.
method Reformulate disentangled methods as optimal quantization problems.
result Better performance and generalization on ImageNet-1K and subsets.
New method tests conditional independence using spectral representations.
problem Untestable conditional independence in many settings.
method Spectral representations of partial covariance operators, bi-level contrastive learning.
result Asymptotic validity and power guarantees for CI testing.
FATE framework attacks graph learning models to amplify bias deceptively.
problem Achieving poisoning attacks on graph learning models to exacerbate bias deceptively.
method Bi-level optimization problem and meta learning-based framework named FATE.
result FATE amplifies bias of graph neural networks while maintaining downstream task utility.
Meta-learning shows negative transfer between tasks, which MetaCRL addresses.
problem Negative transfer between tasks in meta-learning.
method Structural Causal Models (SCMs) and MetaCRL to eliminate task confounders.
result MetaCRL achieves state-of-the-art performance in various benchmark datasets.
Novel approach models life events using causal discovery and survival analysis.
problem Modeling life event choices and occurrence from a probabilistic perspective.
method Bi-level problem formulation: causal discovery for life events graph, survival analysis for time-to-event modeling.
result Identification of causal relationships and factors influencing transition rates between life events.
Weakly supervised data are widespread and have attracted much attention. However, since label quality is often difficult to guarantee, sometimes the use of weakly supervised data will lead to unsatisfactory performance, i.e., performance degradation or poor performance gains. Moreover, it is usually not feasible to man…
Machine learning algorithms are vulnerable to poisoning attacks: An adversary can inject malicious points in the training dataset to influence the learning process and degrade the algorithm's performance. Optimal poisoning attacks have already been proposed to evaluate worst-case scenarios, modelling attacks as a bi-le…
Unified proof for scalable personalized federated learning.
problem Personalized federated learning under asynchronous updates.
method Unified proof for asynchronous federated learning with bounded staleness applied to MAML and ME personalization frameworks.
result Unified proof for convergence to first-order stationary point for smooth and non-convex functions.
A new algorithm improves federated learning by combining knowledge distillation and weighted combination loss.
problem Non-IID client data in federated learning leads to model drift and poor generalization.
method pFedKD-WCL integrates knowledge distillation with bi-level optimization to address non-IID challenges.
result pFedKD-WCL outperforms state-of-the-art algorithms in accuracy and convergence speed.
New deep learning method simplifies parameter estimation design.
problem Optimal experimental design for parameter estimation with non-linear systems.
method Training a deep network as a Likelihood Free Estimator to simplify design process.
result Deep design improves parameter recovery quality and simplifies design process.
GBML with deep nets converges globally and generalizes well.
problem Theoretical guarantees for few-shot learning with deep nets.
method Proving global convergence and generalization bounds for GBML with over-parameterized DNNs.
result GBML with over-parameterized DNNs converges globally to the optimum at a linear rate and achieves good generalization.