The study clarifies fairness guarantees from unconstrained machine learning, showing group calibration is often satisfied but at the cost of other criteria.
problem Fairness guarantees from unconstrained machine learning.
method Characterization and proof of upper and lower bounds on the deviation from group calibration.
result Group calibration is the fairness criterion implicitly favored by unconstrained learning, but it comes at the cost of violating other criteria.
Unconstrained MLIPs outperform constrained ones in accuracy and speed.
problem Improving the efficiency and accuracy of machine-learned interatomic potentials.
method Investigated unconstrained models trained on large datasets compared to physically constrained models.
result Unconstrained MLIPs can be superior in accuracy and speed compared to physically constrained models.
Unconstrained models learn physical symmetries effectively with simple data augmentation.
problem Ensuring physical symmetries in machine learning models.
method Rigorous metrics to measure symmetry content, data augmentation strategy, architectural analysis.
result Unconstrained models can learn approximate equivariant behavior with simple data augmentation.
New algorithms reduce online learning regret by tracking gradient variation.
problem Online learning with unconstrained losses and gradient variation.
method Parameter-free algorithms with adaptive updates for L L L -smooth convex losses. result Regret bounds of order O ~ ( ∥ u ∥ V T ( u ) + L ∥ u ∥ 2 + G 4 ) \widetilde{O}(\|u\|\sqrt{V_T(u)} + L\|u\|^2+G^4) O ( ∥ u ∥ V T ( u ) + L ∥ u ∥ 2 + G 4 ) achieved without prior knowledge of comparator norm or Lipschitz constant. Proposes ConstraintMatch for semi-supervised clustering with unconstrained data.
problem Leveraging unconstrained data alongside constraints for clustering models.
method Semi-supervised context with pseudo-constraining and pseudo-labeling mechanisms.
result Demonstrates effectiveness of ConstraintMatch over baselines.
VAV method optimizes learning rate for faster, stable SGD convergence.
problem Optimizing learning rate for efficient and stable machine learning models.
method Energy-based self-adaptive learning rate with auxiliary variable r r r . result VAV method achieves faster convergence and superior stability with larger learning rates.
New algorithms reduce regret in unconstrained online learning with unknown parameters.
problem Online convex optimization with unknown Lipschitz constant and comparison point.
method Developed algorithms with polynomial bounds that adapt to unknown parameters.
result Achieved improved regret bounds with polynomial dependence on all parameters.
This paper extends neural collapse to class-imbalanced datasets using an unconstrained ReLU feature model.
problem Understanding neural collapse in class-imbalanced datasets with cross-entropy loss.
method Generalized neural collapse to class-imbalanced settings using an unconstrained ReLU feature model.
result Class-means converge to orthogonal vectors with different lengths, and classifier weights align to these vectors.
Our paper explains deep neural collapse in multiple layers.
problem Understanding deep neural collapse in multi-layered neural networks.
method Generalized unconstrained features model for deep networks.
result Deep unconstrained features model exhibits deep neural collapse.
Cookbook transforms constrained statistical inference into unconstrained problems.
problem Transforming constrained statistical inference into unconstrained problems.
method Bijective and diffeomorphisms parametrizations.
result Maintains statistical inference properties like identifiability.
New methods solve complex optimization problems in machine learning.
problem Challenges in stochastic bilevel optimization with constraints and high variables.
method Inexact bilevel stochastic gradient methods for constrained and unconstrained lower-level problems.
result Comprehensive convergence theory for both unconstrained and constrained cases.
New approach reduces unconstrained linear bandits to simpler optimization problems.
problem Unconstrained linear bandits problem.
method Perturbation-based approach combined with comparator-adaptive OLO algorithms.
result First high-probability guarantees for both static and dynamic regret in unconstrained linear bandits.
Develops parameter-free online mirror descent for optimal dynamic regret.
problem Optimal online linear optimization in unbounded domains.
method Modified online mirror descent framework for parameter-free algorithms.
result First unconstrained online linear optimization achieving optimal dynamic regret.
Paper proves fair classification can be done via simple thresholding.
problem Achieving fair binary classification subject to group fairness constraints.
method Proves Bayes optimal fair learning rule is a group-wise thresholding rule over the Bayes regressor with randomization.
result Proposes an efficient unconstrained optimization algorithm for post-processing fair classification.
Deep learning gait recognition from smartphone data in unconstrained settings.
problem Gait recognition in unconstrained environments using smartphones.
method Hybrid deep neural network combining convolutional and recurrent neural networks for gait feature extraction.
result Achieves over 93% accuracy in person identification and authentication.
Develops methods for estimating constrained function-valued parameters in infinite-dimensional models.
problem Estimating function-valued parameters with structural constraints in complex models.
method Characterizes constrained solutions as minimizers of penalized population risk, using a Lagrange-type formulation and path through unconstrained space.
result Proposes estimators that achieve optimal risk and constraint satisfaction, applicable across various statistical learning approaches.
We consider a variant of online convex optimization in which both the instances (input vectors) and the comparator (weight vector) are unconstrained. We exploit a natural scale invariance symmetry in our unconstrained setting: the predictions of the optimal comparator are invariant under any linear transformation of th…
A new optimization algorithm improves convergence in unconstrained problems.
problem Unconstrained optimization problems.
method Element-wise relaxed scalar auxiliary variable (E-RSAV) algorithm.
result Improved convergence and alignment of modified and original energy.
Quantum computing improves feature selection in machine learning.
problem Optimizing feature selection in machine learning problems.
method Formulated feature selection as a QUBO problem and compared quantum and classical methods.
result Quantum computing can outperform classical methods in feature selection, depending on data set.
The natural gradient of ELBO vanishes in unconstrained optimization, simplifying learning.
problem The gap between evidence and ELBO has a vanishing natural gradient.
method Analyzes the Fisher-Rao gradient of ELBO and its implications for learning.
result Maximizing ELBO is equivalent to minimizing KL divergence, simplifying learning.
The paper explores solving inverse problems for ODEs with and without constraints.
problem Understanding when second order ODEs can represent Lagrangian models with or without constraints.
method Geometric techniques to address the inverse problem for both constrained and unconstrained systems of second order ODEs.
result The constrained case presents more ambiguities and complexities than the unconstrained one.
Study uses RL to optimize investment with financial constraints, showing exploration benefits.
problem Optimal investment with financial constraints in continuous time.
method Reinforcement learning framework, focusing on Gaussian and truncated Gaussian distributions.
result Exploration leads to more dispersed wealth distribution with heavier tails, especially with smaller exploration parameters.
Adam converges with high probability under unconstrained non-convex smooth stochastic optimizations.
problem Theoretical limitations of Adam's convergence under unconstrained non-convex smooth stochastic optimizations.
method Deep analysis of Adam's convergence rate under affine variance noise, without bounded gradient assumptions.
result Adam converges to the stationary point with a high probability rate of $\mathcal{O}\left({
m poly}(\log T)/\sqrt{T}
ight)$ .
SurvLIME explains survival models by approximating them with Cox models.
problem Explaining complex survival models in machine learning.
method Applies Cox proportional hazards model to approximate survival model locally.
result Demonstrates efficiency through numerical experiments.
Paper analyzes regret bounds for unconstrained online optimization.
problem Minimizing regret in dynamic online learning for strongly convex and smooth functions.
method Preconditioned OGD, Online Optimistic Newton (OON), multiple gradient queries.
result Achieves O ( C 2 , T ∗ ) O(C^*_{2,T}) O ( C 2 , T ∗ ) regret bound with one gradient query per round. Training shapes the geometry of neural network feature maps, revealing local area magnification.
problem Understanding how training affects the geometric structure of neural network feature maps.
method Analyzing the Riemannian geometry induced by neural network feature maps at infinite width and after training.
result Training breaks the symmetry of the geometry induced by random neural network feature maps, magnifying local areas along decision boundaries.
A new L-BFGS method tackles large-scale optimization with fewer evaluations.
problem Efficiently solving large-scale unconstrained optimization problems.
method Proposes a regularized L-BFGS method with line search techniques.
result Shows global convergence and robust performance in numerical tests.
Despite significant progress made over the past twenty five years, unconstrained face verification remains a challenging problem. This paper proposes an approach that couples a deep CNN-based approach with a low-dimensional discriminative embedding learned using triplet probability constraints to solve the unconstraine…
New algorithm minimizes cumulative loss in dynamic linear bandits without prior knowledge of comparator switches.
problem Minimizing cumulative loss in dynamic linear bandits with unknown number of switches.
method Combining several bandit algorithms to adapt to unknown number of switches without prior knowledge.
result First algorithm achieving optimal regret guarantee of O ( d ( 1 + S T ) T ) \mathcal{O}\big(\sqrt{d(1+S_T) T}\big) O ( d ( 1 + S T ) T ) up to poly-logarithmic terms. This paper extends neural collapse to imbalanced data under cross-entropy loss.
problem Analyzing neural collapse in deep networks with imbalanced data.
method Using the unconstrained feature model and cross-entropy loss, the paper studies neural collapse in imbalanced datasets.
result Feature vectors within the same class collapse to a single mean vector, but angles between them depend on sample size.
New method uses neural networks to solve PDEs without grid discretization.
problem Solving PDEs with boundary conditions efficiently and accurately.
method Combining neural networks with TFC to transform PDEs into unconstrained optimization problems.
result Deep TFC method provides closed-form, differentiable approximations of PDE solutions.
We propose an online convex optimization algorithm (RescaledExp) that achieves optimal regret in the unconstrained setting without prior knowledge of any bounds on the loss functions. We prove a lower bound showing an exponential separation between the regret of existing algorithms that require a known bound on the los…
VoxCeleb 2019 challenge assesses speaker recognition in uncontrolled settings.
problem Evaluate speaker recognition technology in unconstrained data.
method Public dataset, challenge, and workshop at Interspeech 2019.
result Baseline results and discussions provided.
Pairwise learning usually refers to a learning task which involves a loss function depending on pairs of examples, among which most notable ones include ranking, metric learning and AUC maximization. In this paper, we study an online algorithm for pairwise learning with a least-square loss function in an unconstrained …
Neural networks have been used prominently in several machine learning and statistics applications. In general, the underlying optimization of neural networks is non-convex which makes their performance analysis challenging. In this paper, we take a novel approach to this problem by asking whether one can constrain neu…
Paper tackles constrained learning with non-convex losses, overcoming challenges with new approach.
problem Challenges in learning with non-convex losses and statistical constraints.
method Learning in the empirical dual domain, bounding empirical duality gap.
result Established a constrained counterpart to classical learning theory.
PROPEL learns interpretable programmatic policies using imitation and projection.
problem Learning interpretable programmatic policies in reinforcement learning.
method PROPEL is a meta-algorithm based on three insights: optimization in policy space, neural-program mixing, and imitation synthesis.
result PROPEL significantly outperforms state-of-the-art approaches in learning programmatic policies.
Fourier Neural Operators accurately predict dynamics of high-dimensional ionic models.
problem Approximating stiff, multiscale ionic models using neural networks.
method Fourier Neural Operators for learning dynamics of high-dimensional ionic models.
result Fourier Neural Operators can accurately predict dynamics of high-dimensional ionic models.
New theory tackles AGI by breaking data constraints and minimizing global risk.
problem Current AI's limitations in handling complex real-world data and making reasonable judgments.
method Developed subjectivity learning theory to break data constraints and minimize global risk.
result Subjectivity learning holds a lower risk bound than traditional machine learning.
OMWU shows last iterate convergence in convex-concave games.
problem Optimizing in constrained min-max optimization landscapes.
method OMWU (Optimistic Multiplicative-Weights Update) in the no-regret online learning framework.
result OMWU exhibits last iterate convergence for convex-concave games, generalizing previous results.
Solves VaR-constrained portfolio optimization in markets with stochastic volatility.
problem Optimizing portfolio in markets with stochastic volatility under VaR constraints.
method Dynamic programming approach to Heston's stochastic volatility model.
result Optimal investment strategy linked to unconstrained problem via a vega-neutral derivative.
DP-GD achieves dimension-independent convergence for unconstrained private GLMs.
problem Differentially private empirical risk minimization for unconstrained GLMs.
method Differentially private gradient descent (DP-GD).
result DP-GD achieves an excess empirical risk of $ ilde O\left(\sqrt{ exttt{rank}}/εn
ight)$ for unconstrained GLMs.
Yau's Affine Normal Descent optimizes smooth unconstrained problems with geometrically adapted directions.
problem Optimizing smooth unconstrained problems with geometrically adapted directions.
method Yau's Affine Normal Descent (YAND) uses the equi-affine normal of level-set hypersurfaces as search directions.
result YAND converges globally under standard smoothness assumptions and locally quadratically near nondegenerate minimizers.
Study optimal portfolio management with periodic evaluations in stochastic models, considering convex constraints.
problem Optimal portfolio management under ratio-type periodic evaluations in stochastic factor models with convex trading constraints.
method Transformed infinite horizon optimal control problem into an auxiliary terminal wealth optimization problem. Introduced an auxiliary unconstrained optimization problem in a modified market model. Used martingale duality approach to establish dual minimizer and optimal unconstrained wealth process.
result Derived and verified the optimal constrained portfolio process for the original problem over an infinite horizon.
CPCMs integrate causal drivers for robust portfolio optimization.
problem Degradation of classical portfolio models under structural breaks and lack of arbitrage consistency in machine learning.
method Causal PDE-Control Models integrating structural causal drivers, nonlinear filtering, and forward-backward PDE control.
result CPCM solvers achieve higher Sharpe ratios and lower turnover than benchmarks.
This paper benchmarks monotone-constrained models for credit PD across datasets and finds constraints are mostly costless.
problem Aligning machine learning model behavior with domain knowledge in credit risk.
method Benchmarked monotone-constrained versus unconstrained gradient boosting models across five datasets and three libraries, defining the Price of Monotonicity (PoM) as the relative change in AUC.
result Monotonicity constraints are almost costless on large datasets and most costly on smaller datasets, with PoM ranging from essentially zero to about 2.9 percent.
New optimization method for sampling from unknown density measures.
problem Sampling from measures with unknown normalization constants.
method Mollified Interaction Energy Descent (MIED) method.
result Gradient flow of MIE converges to chi-square divergence.
New method parameterizes solutions to linearized vacuum constraints on Einstein manifolds.
problem Parameterizing solutions to linearized vacuum constraints on Einstein manifolds.
method Parameterize solutions using unconstrained potentials and shield linearized gravitational fields.
result Showed how to shield linearized gravitational fields without TT gauge for any value of cosmological constant.