A framework estimates categorical distributions under constraints, ensuring generality and uniqueness.
problem Estimating categorical distributions summarizing sample data under marginal constraints.
method Theoretical framework + Iterative Proportional Fitting (IPF) to estimate the distribution.
result A unique categorical distribution of Maximum Entropy under marginal constraints exists and is estimated.
Paper finds robust Λ-quantiles equal to extremal distributions.
problem Investigating robust models for Λ-quantiles with partial loss information. method Extending classical quantiles using Λ-quantiles and applying results from robust quantiles. result Robust Λ-quantiles equal to Λ-quantiles of extremal distributions. Proposes a method to generate text that adheres to logical constraints.
problem Generating text that respects logical constraints is hard for autoregressive models.
method Bayesian conditioning to draw samples subject to a constraint, considering the entire sequence and inducing a local, factorized distribution.
result Our approach generates samples that closely approximate the target distribution and are guaranteed to satisfy the constraints.
Study binary hypothesis testing with privacy and communication constraints.
problem Binary hypothesis testing under local differential privacy and communication constraints.
method Qualifies results as minimax or instance optimal, develops instance-optimal algorithms.
result Achieves minimum possible sample complexity under both privacy and communication constraints.
OLLA framework efficiently samples from constrained distributions with nonconvex constraints.
problem Sampling from constrained distributions with nonconvex constraints is challenging.
method Overdamped Langevin with Landing (OLLA) framework that handles both equality and inequality constraints.
result OLLA converges exponentially fast to the constrained target density in W2 distance. New algorithm tackles optimization with distributed constraints.
problem Optimization problems with generalized orthogonality constraints in a decentralized setting.
method Introduced a novel algorithm that tracks gradients and Jacobians simultaneously.
result Global convergence with an iteration complexity established.
This paper studies nonholonomic constraints in Hamiltonian systems, deriving equations and theorems.
problem Analyzing nonholonomic constraints in Hamiltonian systems.
method Deriving distributional RCH systems, geometric constraint conditions, and Hamilton-Jacobi theorems.
result Derives precise geometric constraint conditions and Hamilton-Jacobi theorems for nonholonomic systems.
In this paper, the extension of the framework of Learning from Constraints (LfC) to a distributed setting where multiple parties, connected over the network, contribute to the learning process is studied. LfC relies on the generic notion of "constraint" to inject knowledge into the learning problem and, due to its gene…
We study distributed estimation methods under communication constraints in a distributed version of the nonparametric random design regression model. We derive minimax lower bounds and exhibit methods that attain those bounds. Moreover, we show that adaptive estimation is possible in this setting.
New framework for distributed nonparametric estimation under slow communication.
problem Efficiently estimate nonparametric models across multiple nodes with limited communication.
method Developed a general framework for nonparametric estimation under communication constraints.
result Derived minimax lower and upper bounds for various models.
Enhances flexibility in data reweighting with optimal transport and maximum entropy principles.
problem Adapting empirical distributions to predefined constraints on moments, tail behavior, etc.
method Nonparametric distributional constraints, maximum entropy principle, optimal transport.
result Maximum entropy weight adjusted empirical distribution close to a specified distribution in optimal transport metric.
Develops integrators for nonholonomic systems on Lie groups.
problem Nonholonomic constraints on Lie groups.
method Using retraction maps and Hamel formulation.
result Structure-preserving numerical integrators for nonholonomic systems.
A Bayesian approach termed BAyesian Least Squares Optimization with Nonnegative L1-norm constraint (BALSON) is proposed. The error distribution of data fitting is described by Gaussian likelihood. The parameter distribution is assumed to be a Dirichlet distribution. With the Bayes rule, searching for the optimal parame…
Optimal transport with path constraints for distributions of different masses.
problem Comparing distributions with different total masses under path constraints.
method Introduces a model for unbalanced optimal transport with path constraints, proving existence of solutions.
result Existence of solutions to path constrained unbalanced optimal transport for various constraints.
The distributional category bounds manifold invariants and imposes constraints.
problem Bounding manifold invariants and understanding constraints.
method Using geometric conditions like non-negative Ricci curvature, the distributional category bounds invariants such as the first Betti number and macroscopic dimension.
result Equality of bounds imposes specific constraints on the manifold.
Paper studies optimal federated learning for nonparametric regression with privacy constraints.
problem Federated learning for nonparametric regression with heterogeneous differential privacy constraints.
method Proposes distributed privacy-preserving estimators and investigates their risk properties.
result Establishes matching minimax lower bounds for global and pointwise estimation.
We address the problem of solving convex optimization problems with many convex constraints in a distributed setting. Our approach is based on an extension of the alternating direction method of multipliers (ADMM) that recently gained a lot of attention in the Big Data context. Although it has been invented decades ago…
This paper considers distributed online optimization with time-varying coupled inequality constraints. The global objective function is composed of local convex cost and regularization functions and the coupled constraint function is the sum of local convex functions. A distributed online primal-dual dynamic mirror des…
This paper studies Hamilton-Jacobi equations for magnetic systems with constraints.
problem Understanding dynamics of magnetic systems with geometric constraints.
method Developed Hamilton-Jacobi equations for magnetic systems with nonholonomic constraints.
result Revealed relationships between magnetic structures, constraints, and dynamics.
The paper develops methods for sampling from log-concave distributions with constraints.
problem Sampling from log-concave distributions with constraints.
method Randomized midpoint discretization of Langevin diffusions with various projections.
result New convergence guarantees for constrained Langevin algorithms.
Voltage control plays an important role in the operation of electricity distribution networks, especially with high penetration of distributed energy resources. These resources introduce significant and fast varying uncertainties. In this paper, we focus on reactive power compensation to control voltage in the presence…
Improved sampling from Gaussian distributions with privacy constraints.
problem Sampling from unbounded Gaussian distributions with differential privacy.
method First $\widetilde{\mathcal{O}}\left(d
ight)$-sample algorithm for unbounded Gaussians under $\left(\varepsilon, δ
ight)$-differential privacy.
result A quadratic improvement over previous results, settling an open question.
Many machine learning approaches are characterized by information constraints on how they interact with the training data. These include memory and sequential access constraints (e.g. fast first-order methods to solve stochastic optimization problems); communication constraints (e.g. distributed learning); partial acce…
Paper tackles SMPC for linear systems with unknown noise distribution.
problem Stochastic MPC for linear systems with chance state constraints and unknown noise distribution.
method Reformulate chance constraints, design robust benchmark SMPC, and develop adaptive SMPC with online noise statistics learning.
result Adaptive SMPC guarantees time-uniform satisfaction of unknown reformulated state constraints with high probability.
Study magnetic Hamiltonian systems with constraints, deriving Hamilton-Jacobi equations.
problem Understanding dynamics of controlled magnetic Hamiltonian systems with constraints.
method Defined CMH system, derived Hamilton-Jacobi equations for different constraints.
result Invariant solutions of Hamilton-Jacobi equations under CMH-equivalence.
Optimal distributed testing under communication constraints with shared randomness.
problem Signal detection in a distributed system with limited communication.
method Derivation of minimax testing errors, distributed testing algorithms, and theoretical lower bounds.
result Consistent nonparametric distributed testing is possible even with minimal communication.
Characterizes causal structure dominance for latent variables.
problem Determining dominance relations between causal structures with latent variables.
method Complete characterization for three visible variables, partial for four; uses nontrivial inequality constraints.
result Equivalence classes with nontrivial inequality constraints become ubiquitous as the number of visible variables increases.
The constraints arising from DAG models with latent variables can be naturally represented by means of acyclic directed mixed graphs (ADMGs). Such graphs contain directed and bidirected arrows, and contain no directed cycles. DAGs with latent variables imply independence constraints in the distribution resulting from a…
This work presents entropic constraints from DAGs with hidden variables.
problem Characterizing causal relations in systems with hidden variables.
method Entropic inequality constraints derived from e-separation relations. result These constraints can learn about true causal models from observed data.
Generative models often misrepresent class frequencies; this paper calibrates them.
problem Miscalibration of class frequencies in generative models.
method Formulated as constrained optimization, using surrogate objectives to approximate constraints.
result Significant reduction in calibration error across various models and applications.
We consider an optimal stopping problem where a constraint is placed on the distribution of the stopping time. Reformulating the problem in terms of so-called measure-valued martingales allows us to transform the marginal constraint into an initial condition and view the problem as a stochastic control problem; we esta…
SHIFT method optimally estimates heterogeneous discrete distributions with limited communication.
problem Collaborative learning of discrete distributions under heterogeneity and communication constraints.
method Two-stage method: First, users learn a central distribution; then, fine-tune this to estimate individual distributions.
result SHIFT is minimax optimal in the model of heterogeneity and under communication constraints.
Some high-dimensional data.sets can be modelled by assuming that there are many different linear constraints, each of which is Frequently Approximately Satisfied (FAS) by the data. The probability of a data vector under the model is then proportional to the product of the probabilities of its constraint violations. We …
The paper improves Gaussian processes by adding sum constraints, enhancing prediction accuracy.
problem Improving Gaussian process predictions with background knowledge constraints.
method Conditioning the prior distribution on sum constraints to ensure fulfillment of linear and nonlinear constraints.
result The approach fulfills constraints with high precision and improves prediction accuracy.
Paper extends LME models to allow sign constraints on coefficients with SDTN random effects.
problem Inference with sign constraints on random effects in LME models.
method Proposes SDTN distribution for random effects and develops likelihood-based approaches for estimation.
result Proposed constrained model improves real-world interpretations and achieves satisfactory performance.
Distributed sensors compress and send features to a fusion center for linear regression.
problem Efficiently compress and transmit features from distributed sensors to a fusion center under varying communication constraints.
method Designs a distributed and adaptive feature compression scheme using optimal quantizers and simple adaptive strategies.
result Demonstrates improved inference performance through simulated experiments.
The study establishes risk bounds for distributional regression estimators.
problem Estimating distributional regression models with nonparametric methods.
method Theoretical bounds for CRPS and MSE are derived for convex and non-convex constraints.
result Theoretical risk bounds are validated through experiments on simulated and real data.
Framework learns stochastic dynamics from endpoint and intermediate distributions using soft energy constraints.
problem Learning stochastic dynamics from endpoint and intermediate distributional observations.
method Formulates generation as a McKean-Vlasov control problem with soft energy constraints, solving it through FBSDE.
result Model learns coherent stochastic trajectories matching prescribed marginal laws.
New neural network models extreme value distributions with preserved shape constraints.
problem Modeling multivariate extreme value distributions with preserved shape constraints.
method d-max-decreasing neural network architecture for non-parametric calibration and generation of MEVs.
result The proposed architecture approximates the dependence structure of MEVs at parametric rate and preserves essential shape constraints.
Theory of learning with weight-distribution constraints.
problem Understanding how structure influences function in neural networks.
method Statistical mechanical theory and optimal transport.
result Reduction in capacity due to constrained weight-distribution is related to Wasserstein distance.
In the world of modern financial theory, portfolio construction has traditionally operated under at least one of two central assumptions: the constraints are derived from a utility function and/or the multivariate probability distribution of the underlying asset returns is fully known. In practice, both the performance…
New algorithm samples constrained distributions efficiently.
problem Sampling from distributions with statistical constraints.
method Primal-dual Langevin Monte Carlo (PD-LMC) using gradient descent-ascent dynamics.
result PD-LMC algorithm successfully samples constrained distributions.
We study distribution testing with communication and memory constraints in the following computational models: (1) The {\em one-pass streaming model} where the goal is to minimize the sample complexity of the protocol subject to a memory constraint, and (2) A {\em distributed model} where the data samples reside at mul…
Study hypothesis testing under quantized samples with communication constraints, achieving near-optimal sample complexity.
problem Optimizing hypothesis testing with quantized samples and communication constraints.
method Developed a polynomial-time algorithm achieving near-optimal sample complexity under communication constraints.
result Achieved near-optimal sample complexity under communication constraints, with a logarithmic factor increase over unconstrained setting.
Paper tackles nonparametric classification with privacy constraints, achieving optimal accuracy.
problem Nonparametric classification under distributed differential privacy constraints.
method Minimax and adaptive transfer learning, considering privacy, sample sizes, and heterogeneity.
result Developed an adaptive classifier achieving optimal misclassification rate with privacy constraints.
New method learns DAGs from data without acyclicity constraint.
problem Learning DAGs from data without imposing acyclicity.
method Sparse matrix factorization and ℓ1-penalized optimization. result Empirical success in recovering true graphs and almost-DAG graphs.
Statistical models with constrained probability distributions are abundant in machine learning. Some examples include regression models with norm constraints (e.g., Lasso), probit, many copula models, and latent Dirichlet allocation (LDA). Bayesian inference involving probability distributions confined to constrained d…
Optimization results are one method for understanding neural computation from Nature's perspective and for defining the physical limits on neuron-like engineering. Earlier work looks at individual properties or performance criteria and occasionally a combination of two, such as energy and information. Here we make use …