Intersectional constraints improve selection outcomes by reducing inequality.
problem Persistent inequality and reduced utility in selection processes due to implicit bias.
method Introducing intersectional constraints to mitigate the adverse effects of implicit bias in selection processes.
result Intersectional constraints can recover almost all the utility achievable in the absence of implicit bias, offering a significant advantage over non-intersectional constraints.
This work investigates implicit bias in multiclass separable data using a novel geometry-aware optimizer.
problem Understanding implicit bias in overparameterized models on multiclass separable data.
method Introduces NucGD, a geometry-aware optimizer enforcing low-rank structures through nuclear norm constraints.
result NucGD enables scalable training and characterizes the impact of stochastic optimization dynamics.
Paper introduces CageBO for optimizing complex public policy problems.
problem Complex decision-making and implicit constraints in public policy.
method CageBO framework using conditional variational autoencoder.
result CageBO outperforms baselines in optimizing large-scale police redistricting.
New implicit Krasulina's k-PCA update avoids QR-decomposition and improves convergence.
problem Online k-PCA problem with orthonormality constraint.
method Derived an implicit form of Krasulina's update that bypasses orthonormality constraint.
result The new update avoids costly QR-decomposition and yields superior convergence.
Factored gradient descent finds unique rank-r solution in PSD matrix sensing.
problem Finding a unique rank-r solution in over-parameterized matrix sensing.
method Factored gradient descent with PSD constraints.
result PSD constraint alone leads to a unique rank-r matrix recovery.
Generalizes Routh's reduction method for Lagrangian systems with symmetry.
problem Generalizing Routh's reduction method for Lagrangian systems with symmetry.
method Using implicit Lagrange-Routh equations and Dirac structures, the paper generalizes Routh's reduction method for Lagrangian systems with symmetry.
result The reduced implicit Lagrange-Routh equations can be described in the context of dynamical systems associated to Dirac structures.
SGM generalizes well with varying number of passes and step sizes.
problem Generalization of stochastic gradient methods with multiple passes.
method Analysis of stability and approximation properties controlled by step-size and number of passes.
result The number of passes and step-size control implicit regularization.
Proves solutions to Einstein and scalar field constraints form a Hilbert manifold.
problem Proving the structure of solutions to coupled Einstein and scalar field equations.
method Used weighted Sobolev spaces and Implicit Function Theorem.
result The set of solutions has a Hilbert manifold structure.
Bayesian method uses human similarity constraints for better image representation learning.
problem Losing semantic structure in high-dimensional parametric models.
method Generative unsupervised feature learning with probabilistic treatment of oracle information.
result Oracle triplet information improves representation learning and outperforms previous methods.
Proposes a new model for image restoration combining deep learning and total variation.
problem Restoring images from limited data with low-rank constraints insufficient.
method Regularized Deep Matrix Factorized (RDMF) model using deep neural network's low-rank bias and total variation.
result Outperforms state-of-the-art models in image restoration from few observations.
Study discretizes Dirac and port-Hamiltonian systems using manifolds.
problem Discretization of Dirac and port-Hamiltonian systems.
method Retraction and discretization maps on manifolds for Dirac structures, applied to port-Hamiltonian systems.
result Numerical integrators for port-Hamiltonian systems derived from discretization techniques.
Novel semi-supervised classifier minimizes squared loss without explicit assumptions.
problem Improving classification accuracy with limited labeled data.
method Implicitly constrained least squares (ICLS) classifier that minimizes squared loss on labeled data.
result In limited settings, ICLS never performs worse than supervised classifier.
Method estimates posterior model for boundary value problems with uncertain constraints.
problem Estimating posterior probability model for stochastic boundary value problems with uncertain constraints.
method Probabilistic learning inference using Kullback-Leibler divergence and MCMC.
result Method successfully estimates posterior probability measure with constraints.
Paper tackles efficient SGD methods for constrained bilevel optimization.
problem Stochastic bilevel optimization with equality constraints.
method Alternating implicit projected SGD and its variants.
result Achieves sample complexity matching state-of-the-art for unconstrained problems.
Two effective methods for writing the dynamical equations for non-holonomic systems are illustrated. They are based on the two types of representation of the constraints: by parametric equations or by implicit equations. They can be applied to linear as well as to non-linear constraints. Only the basic notions of vecto…
Graph-dependent implicit regularisation improves Distributed SGD for convex problems.
problem Improving convergence rates in distributed stochastic subgradient descent.
method Graph-dependent implicit regularisation strategies for Distributed SGD.
result Established statistical learning rates retaining centralised guarantees.
We present a unified approach to constrained implicit Lagrangian and Hamiltonian systems based on the introduced concept of Dirac algebroid. The latter is a certain almost Dirac structure associated with the Courant algebroid on the dual E∗ to a vector bundle E. If this almost Dirac structure is integrable (Dir…
New framework improves robustness of implicit neural networks.
problem Ill-posedness and convergence instability in implicit neural networks.
method NEMON framework based on contraction theory for ℓ∞ norm, including well-posedness condition, average iteration, and input-output Lipschitz constant regularization. result Improved accuracy and robustness of implicit models with smaller input-output Lipschitz bounds.
A new method for optimizing non-decomposable metrics with constraints.
problem Optimizing complex machine learning objectives with thresholded constraints.
method Formulate rate-constrained optimization using the Implicit Function theorem and solve with gradient-based methods.
result Demonstrated effectiveness over existing methods on benchmark datasets.
Develops implicit MAML for efficient few-shot learning.
problem Efficient few-shot learning with limited data.
method Implicit differentiation for inner loop optimization.
result Agrees with inner loop optimizer choice and handles many gradient steps.
Although conservative Hamiltonian systems with constraints can be formulated in terms of Dirac structures, a more general framework is necessary to cover also dissipative systems such as gradient and metriplectic systems with constraints. We define Leibniz-Dirac structures which lead to a natural generalization of Dira…
The paper connects geometric structures to mechanical systems and introduces new numerical methods.
problem Analyzing and simulating systems with constraints.
method Introduces Dirac integrators based on generalized geometry.
result Dirac integrators can conserve constraints and provide new insights.
New dual formulation reduces generalization error for ERM-fDR.
problem Generalization error in constrained optimization problems.
method Introduces a dual formulation of ERM-fDR using Legendre-Fenchel transform and implicit function theorem.
result Explicit characterizations of generalization error for algorithms under mild conditions.
Lasso method applied to polynomial models with hierarchy constraints.
problem Estimating parameters in polynomial models with hierarchy constraints.
method Using lasso and standard quadratic programming techniques to estimate parameters.
result The proposed methodology outperforms existing techniques in terms of validation error and model size.
Algorithm learns fair ranking from biased data.
problem Unfair ranking policies from biased implicit feedback.
method Policy-gradient approach with amortized fairness constraints.
result Efficient algorithm FULTR learns fair policies.
Optimizes storage valuation using variational analysis.
problem Optimizing intrinsic storage valuation.
method Variational analysis to solve the problem.
result The optimal exercise strategy is revealed, and the solution depends on different constraint types.
Solves complex equation with singularities using transformations and numerical methods.
problem Solving a semilinear parabolic HJB equation with a singular initial condition.
method Transformed the equation to remove singularity, then constructed numerical schemes.
result Proved convergence of numerical schemes for the transformed equation.
The purpose of this paper is to define the concept of multi-Dirac structures and to describe their role in the description of classical field theories. We begin by outlining a variational principle for field theories, referred to as the Hamilton-Pontryagin principle, and we show that the resulting field equations are t…
Proposes a probabilistic CCA with implicit distributions for multi-view data.
problem Overcoming the deficiency of linear correlation in practical multi-view learning tasks.
method Probabilistic interpretation of CCA based on implicit distributions, using Conditional Mutual Information (CMI) and Adversarial CCA (ACCA).
result Achieves superior alignment of multi-view data with implicit distributions.
The paper solves stochastic control problems with implicit objectives, finding equilibrium strategies.
problem Stochastic control problems with implicitly defined objectives leading to time-inconsistency.
method Closed-loop equilibrium solutions in a controlled diffusion framework, providing sufficient and necessary conditions.
result Explicit characterization of equilibrium portfolio strategies in terms of ordinary differential equations.
Most existing distance metric learning methods assume perfect side information that is usually given in pairwise or triplet constraints. Instead, in many real-world applications, the constraints are derived from side information, such as users' implicit feedbacks and citations among articles. As a result, these constra…
AdamW optimizes a constrained loss with ℓ∞ norm constraint.
problem Understanding the optimization behavior of AdamW with ℓ∞ norm constraint. method Analyzing AdamW as a smoothed version of SignGD and connecting it to Frank-Wolfe optimization.
result AdamW implicitly performs constrained optimization with ℓ∞ norm constraint. Semi-supervised learning is an important and active topic of research in pattern recognition. For classification using linear discriminant analysis specifically, several semi-supervised variants have been proposed. Using any one of these methods is not guaranteed to outperform the supervised classifier which does not t…
New framework improves reliability of learned representations by modeling uncertainty and structural constraints.
problem Uncertainty in learned representations treated as deterministic, leading to unreliable models.
method Proposes a principled framework for reliable representation learning with uncertainty-aware regularization and structural constraints.
result Improves stability, calibration, and robustness of learned representations.
New method avoids failures in physics-constrained systems using active learning.
problem Handling fatal failures in systems governed by physics constraints.
method Develops a novel active learning method that considers implicit physics constraints.
result Achieves zero-failure in composite fuselage assembly process without explicit failure regions.
We propose a fast algorithm for spectral embedding using stochastic gradient descent.
problem Scalability issue in spectral embedding due to eigendecomposition bottleneck.
method Reformulate spectral embedding as a stochastic optimization problem, replacing orthogonality constraint with an orthogonalization matrix.
result Efficient algorithm based on mini-batch gradient descent that outperforms existing techniques in execution speed.
Images seen during test time are often not from the same distribution as images used for learning. This problem, known as domain shift, occurs when training classifiers from object-centric internet image databases and trying to apply them directly to scene understanding tasks. The consequence is often severe performanc…
Novel optimizer mitigates instability in asynchronous data parallel optimization.
problem Instability in asynchronous data parallel optimization.
method Pushes a normalized sequence of first-order gradients to a parameter server.
result Achieves better convergence rate compared to other optimizers.
This paper integrates auto-encoders and GANs using variational inference.
problem Preventing mode collapse in generative models.
method Develops a principle to combine variational auto-encoders and GANs, using synthetic likelihoods and implicit posterior distributions.
result Unified objective for optimizing the fusion of variational auto-encoders and GANs.
Extends initial data to asymptotically flat solutions in general relativity.
problem Solving constraint equations for asymptotically flat solutions in general relativity.
method New method for solving prescribed divergence equation and geometric variant of conformal method.
result Global solution extending initial data to asymptotically flat solutions.
A novel semi-supervised least squares classifier that minimizes squared loss.
problem Improving classification accuracy with limited labeled data.
method Implicitly constrained least squares (ICLS) classifier that minimizes squared loss on labeled data among all possible unlabeled data labelings.
result The ICLS classifier can be formulated as a quadratic programming problem and its solution found using gradient descent.
Study reveals biases in gradient descent for GLNs, improving neural network performance.
problem Understanding and improving the inductive biases of deep neural networks.
method Derive infinite-time training limit of gated linear networks and generalize to other networks.
result Theoretical framework captures key inductive biases of ReLU networks.
The paper solves Lovelock gravity's constraint equations, generalizing the σk-Yamabe problem.
problem Solving Lovelock gravity's constraint equations in a specific manifold setting.
method Using the σk-Yamabe problem as a basis, the paper extends solutions to the Lovelock constraint equations and studies concavity properties. result Several cases where conformal solutions exist, including when Lovelock theories are close to General Relativity.
Study develops a new method for creating fair models.
problem Ensuring equal outcomes for different protected groups.
method Introduces a new group-fair constraint based on transport maps.
result Develops a novel algorithm FTM for training group-fair models.
Muon optimizer improves deep learning with spectral norm constraints.
problem Improving optimization algorithms in deep learning.
method Theoretical analysis of Muon optimizer within the Lion-K family. result Muon implicitly solves an optimization problem enforcing spectral norm constraints.
Automatic human matting without user interaction.
problem High quality extraction of humans from natural images.
method SHM learns semantic constraints from data and jointly fits both semantic information and high quality details with deep networks.
result SHM achieves comparable results with state-of-the-art interactive matting methods.
A novel approach learns constraints and maximizes rewards for autonomous agents.
problem Ensuring autonomous agents align with societal norms and values.
method Inverse reinforcement learning for constraints, contextual bandit orchestrator for policy mixing.
result Agent learns to act optimally within constraints and maximize rewards.
A four-dimensional Walker geometry is a four-dimensional manifold M with a neutral metric g and a parallel distribution of totally null two-planes. This distribution has a natural characterization as a projective spinor field subject to a certain constraint. Spinors therefore provide a natural tool for studying Walker …