The paper develops a method for Gaussian Process regression with linear operator constraints.
problem Modeling functions with multiple linear constraints in high-consequence engineering systems.
method Develops a method for constrained Gaussian Process regression using linear operator constraints.
result Derives the exact posterior for a conjugate likelihood under linear operator constraints.
Neural networks learn vector fields constrained by linear operators.
problem Learning vector fields from physical systems with linear operator constraints.
method Model the target function as a linear transformation of a potential field, which is a neural network.
result Predictions of the target function satisfy the linear operator constraints.
New classifiers ensure fairness by adjusting a base classifier's operating characteristics.
problem Ensuring fairness in binary classification with multiple group constraints.
method Intervening directly on a base classifier's operating characteristics using group-wise ROC convex hulls and post-processing.
result Methods satisfy multiple fairness constraints (DP, EO, PP) with minimal interventions and near-oracle accuracy.
We show that, for generative classifiers, conditional independence corresponds to linear constraints for the induced discrimination functions. Discrimination functions of undirected Markov network classifiers can thus be characterized by sets of linear constraints. These constraints are represented by a second order fi…
POLICE enforces linear constraints on deep neural networks efficiently.
problem Enforcing constraints on deep neural networks without affecting optimization.
method Provably optimal affine constraint enforcement method that minimally modifies DNNs.
result POLICE ensures DNNs fulfill affine constraints during training and testing.
Proposes a method to impose linear inequality constraints on neural networks.
problem Imposing prior knowledge on neural network activations.
method Directly incorporates constraints into the network architecture using stochastic gradient descent.
result Significantly speeds up inference at test time with up to two orders of magnitude improvement.
We present a learning theory for the training of a linear system operator having an input compositional variable and propose a Bayesian inversion method for inferring the unknown variable from an output of a noisy linear system. We assume that we have partial or even no knowledge of the operator but have training data …
We consider a modification of the covariance function in Gaussian processes to correctly account for known linear constraints. By modelling the target function as a transformation of an underlying function, the constraints are explicitly incorporated in the model such that they are guaranteed to be fulfilled by any sam…
Positive definite operator-valued kernels generalize the well-known notion of reproducing kernels, and are naturally adapted to multi-output learning situations. This paper addresses the problem of learning a finite linear combination of infinite-dimensional operator-valued kernels which are suitable for extending func…
The paper generalizes equivariant neural networks on homogeneous spaces to the non-linear setting.
problem Equivariant neural networks on homogeneous spaces.
method Deriving generalized steerability constraints for non-linear equivariant layers.
result The universality of the derived construction for non-linear equivariant layers.
Physics-informed neural networks improve by measuring effective dimensionality of constraints.
problem Task interference in physics-informed neural networks due to shared parameter space.
method Introduce effective dimensionality (deff) as an operator invariant to quantify constraints. result Effective dimensionality measures unconstrained parameter directions, independent of network architecture.
The cardinality constraint is an intrinsic way to restrict the solution structure in many domains, for example, sparse learning, feature selection, and compressed sensing. To solve a cardinality constrained problem, the key challenge is to solve the projection onto the cardinality constraint set, which is NP-hard in ge…
Algorithm solves covariant exterior derivative equations in small regions.
problem Solving covariant exterior derivative equations in geometric and algorithmic ways.
method Linear homotopy operator of the Poincare lemma, constraints for parallel transport equations.
result Solves covariant constant and related equations in a geometric and algorithmic way.
Study shows how varying levels of supervision and orthonormality constraints affect generalization errors in subspace fitting.
problem Effects of varying levels of supervision and orthonormality constraints on generalization errors in subspace fitting.
method Flexible family of problems connecting unsupervised and supervised subspace fitting tasks, explored over a supervision-orthonormality plane.
result Generalization errors of subspace fitting problems follow double descent trends as they become more supervised and less orthonormally constrained.
A new method solves variational inequality problems with multiple constraints without needing optimal Lagrange multipliers.
problem Solving variational inequality problems with multiple functional constraints efficiently.
method Constrained Gradient Method (CGM) for Minty variational inequality problems.
result The Constrained Gradient Method achieves complexity similar to projection-based methods but with cheaper oracles.
Iterative thresholding algorithms seek to optimize a differentiable objective function over a sparsity or rank constraint by alternating between gradient steps that reduce the objective, and thresholding steps that enforce the constraint. This work examines the choice of the thresholding operator, and asks whether it i…
Paper provides linear convergence guarantees for KZIHT and KZPT methods.
problem Solving linear equation systems with sparse constraints.
method Combines Kaczmarz and iterative thresholding methods, using reshuffling data sampling.
result KZIHT and KZPT converge linearly to sparse solutions.
New algorithm solves composite optimization problems with unknown expectations.
problem Solving composite optimization problems with unknown statistical expectations.
method Proposes a new stochastic primal-dual algorithm for composite optimization problems with unknown statistical expectations.
result Converges to a saddle point of the Lagrangian function.
New methods solve saddle point problems without line search.
problem Solving saddle point problems efficiently and adaptively.
method Auto-conditioned primal-dual hybrid gradient (AC-PDHG) and auto-conditioned ADMM (AC-ADMM) methods.
result Methods achieve optimal complexity and convergence guarantees.
Scoring systems are linear classification models that only require users to add, subtract and multiply a few small numbers in order to make a prediction. These models are in widespread use by the medical community, but are difficult to learn from data because they need to be accurate and sparse, have coprime integer co…
First order Hamiltonian operators of differential-geometric type were introduced by Dubrovin and Novikov in 1983, and thoroughly investigated by Mokhov. In 2D, they are generated by a pair of compatible flat metrics g and g~ which satisfy a set of additional constraints coming from the skew-symmetry condition…
In this paper, we show that the generating function for linear Hodge integrals over moduli spaces of stable maps to a nonsingular projective variety X can be connected to the generating function for Gromov-Witten invariants of X by a series of differential operators {Lm∣m≥1} after a suitable change…
Formula for Laplace-Beltrami on orthogonal group in Euclidean coords.
problem Computing Laplace-Beltrami on constrained submanifolds.
method Embedded gradient vector field method, explicit formula derivation.
result Explicit formula for Laplace-Beltrami on orthogonal group.
Adapts Bartnik method to Hilbert manifold structure for vacuum constraint equations.
problem Vacuum constraint equations on compact manifolds of any dimension ≥ 3.
method Adapts Bartnik method to provide Hilbert manifold structure.
result Fibers of scalar curvature and constraint operator are Hilbert submanifolds.
This paper considers online convex optimization over a complicated constraint set, which typically consists of multiple functional constraints and a set constraint. The conventional online projection algorithm (Zinkevich, 2003) can be difficult to implement due to the potentially high computation complexity of the proj…
Holistic GLMs add constraints for better model quality.
problem Improving classical linear regression models.
method Sparsity-inducing, sign-coherence, and linear constraints.
result Holistic GLMs reliably solve GLMs for various responses.
Formula derived for Laplace-Beltrami on Stiefel manifold.
problem Finding Laplace-Beltrami operator on Stiefel manifold.
method Using the general framework of Laplace operators on constraint manifolds, derived the explicit formula in terms of ambient Euclidean coordinates.
result Extended previously known formulas for sphere and special orthogonal group.
The paper improves SVR with linear constraints for better model properties.
problem Improving Support Vector Regression with linear constraints.
method Generalized SMO algorithm for solving optimization with linear constraints.
result The proposed method shows better practical performance on various datasets.
Quantum algorithms improve regret bounds for bandits with knapsacks.
problem Combining stochastic integer programming and online learning.
method Quantum algorithms for BwK with improved regret and time complexities.
result Quantum algorithms achieve better regret bounds than classical methods.
Regularized least-squares approaches have been successfully applied to linear system identification. Recent approaches use quadratic penalty terms on the unknown impulse response defined by stable spline kernels, which control model space complexity by leveraging regularity and bounded-input bounded-output stability. T…
Graphical notation simplifies complex polynomial constraints in linear models.
problem Complex polynomial constraints in linear structural equation models are impractical.
method Developed a graphical notation to represent these constraints.
result The graphical notation simplifies the representation of many polynomial constraints.
PIKS combines physics principles with kernel methods for universal consistency.
problem Lack of learning theory for physics-informed machine learning.
method Physics-Informed Kernel methodS (PIKS) for linear differential constraints.
result PIKS achieves universal consistency for linear differential constraints with Gaussian or Matérn kernels.
New algorithm tackles complex optimization problems with inexact and stochastic methods.
problem Solving complex optimization problems with inexact and stochastic methods.
method Developed ICGALP algorithm for composite minimization problems with inexact computations.
result Convergence of Lagrangian to an optimum and asymptotic feasibility of the affine constraint.
New method for linear connections in ODEs with constraints.
problem Constructing linear connections for ODEs with and without constraints.
method Novel method using submodule covariant derivatives.
result Closed form expressions for Massa-Pagani connection and its extension.
Proposes D-LADMM for solving constrained optimization problems.
problem Constrained optimization problems with ill-posed inverse problems.
method Introduces Differentiable Linearized ADMM (D-LADMM) with learnable weights and activation functions.
result Rigorously proves globally converged solutions for D-LADMM.
We consider machine learning techniques to develop low-latency approximate solutions to a class of inverse problems. More precisely, we use a probabilistic approach for the problem of recovering sparse stochastic signals that are members of the ℓp-balls. In this context, we analyze the Bayesian mean-square-error …
We study the supersymmetric Wilson loop as introduced by Caron-Huot, which attaches to lightlike polygons certain edge and vertex operators, whose shape is determined by supersymmetry constraints. We state explicit formulas for the vertex operators to all orders in the Graßmann expansion, thus filling a gap in the lite…
Regulating causal effects through averaged constraints fails to enforce conditional independence.
problem Enforcing conditional independence in regulatory and analytic settings.
method Formulated causal masking as a linear program and analyzed the resulting enforcement problem from both regulator and optimizer perspectives.
result Averaged-constraint optimization often violates stratum-wise requirements while satisfying the averaged one exactly, and detection requires conditional-independence tests.
Study proves stability of slowly rotating Kerr black holes.
problem Stability of slowly rotating Kerr black holes.
method Linear stability analysis using wave map/DeTurck gauge, microlocal analysis, and non-elliptic Fredholm theory.
result Linearized perturbations decay at an inverse polynomial rate.
New DAG constraints improve differentiable DAG learning.
problem Recovering DAG structures from observational data is hard due to combinatorial optimization.
method Developed analytic functions to formulate DAG constraints, closed under differentiation, summation, and multiplication.
result Analytic DAG constraints outperform previous methods in various settings.
ACOL learns constraints from human preferences in driving simulations.
problem Learning constraints from human preferences in driving simulations.
method Adaptive Constraint Learning (ACOL) algorithm for constrained linear best-arm identification.
result ACOL's sample complexity matches worst-case lower bound and is significantly tighter in the average case.
The one-bit quantization is implemented by one single comparator that operates at low power and a high rate. Hence one-bit compressive sensing (1bit-CS) becomes attractive in signal processing. When measurements are corrupted by noise during signal acquisition and transmission, 1bit-CS is usually modeled as minimizing …
We prove that the extended Toda hierarchy of \cite{CDZ} admits nonabelian Lie algebra of infinitesimal symmetries isomorphic to the half of the Virasoro algebra. The generators Lm, m≥−1 of the Lie algebra act by linear differential operators onto the tau function of the hierarchy. We also prove that the tau fu…
Con-TS optimizes wireless link throughput with latency constraints.
problem Optimizing rate selection for wireless links with latency constraints.
method Proposes Con-TS, a constrained Thompson sampling algorithm for stochastic MAB problems.
result Con-TS achieves upper bounds on expected constraint violations and throughput loss.
Unified framework for integrating linear constraints in time series forecasting.
problem Challenges in traditional time series forecasting algorithms.
method Unified framework combining linear constraints in time series forecasting.
result Exact minimizer of the constrained empirical risk can be computed efficiently using linear algebra.
Bitcoin's monetary velocity is constrained by network friction, leading to significant utility contraction during shocks.
problem Bitcoin's monetary velocity is limited by network congestion, causing significant utility loss during economic shocks.
method Empirical analysis using Transaction Cost Index and threshold regression to identify structural breaks and velocity contraction.
result Network friction significantly reduces Bitcoin's monetary velocity, leading to a net utility contraction of -9.39% during shocks.
This paper extends forecast reconciliation to non-linearly constrained time series.
problem Forecasting time series with non-linear constraints.
method Non-linearly Constrained Reconciliation (NLCR) algorithm that adjusts forecasts to meet non-linear constraints.
result NLCR significantly improves forecast accuracy compared to benchmarks.
We introduce a near-linear complexity (geometric and meshless/algebraic) multigrid/multiresolution method for PDEs with rough (L∞) coefficients with rigorous a-priori accuracy and performance estimates. The method is discovered through a decision/game theory formulation of the problems of (1) identifying restri…