arXiv research

Differentiable cutting-plane layers solve parametric mixed-integer linear optimization problems.

Unified analysis for nonlinear parametric models in Bayesian optimization.

Optimal learning for parametric prophet inequalities with exponential-type distributions

Over-parametrization speeds up learning a single neuron model.

Unified parametric assumption improves convergence guarantees for nonconvex optimization.

Non-parametric bootstrap improves robust portfolio and trading strategy optimization.

Parametric UMAP learns a mapping from data to embeddings.

We introduce a novel approach to perform first-order optimization with orthogonal and unitary constraints. This approach is based on a parametrization stemming from Lie group theory through the exponential map. The parametrization transforms the constrained optimization problem into an unconstrained one over a Euclidea…

Adversarially robust machine learning has received much recent attention. However, prior attacks and defenses for non-parametric classifiers have been developed in an ad-hoc or classifier-specific basis. In this work, we take a holistic look at adversarial examples for non-parametric classifiers, including nearest neig…

NPOD algorithm improves efficiency in estimating pharmacokinetic parameters.

Paper proves CFlows can approximate any diffeomorphism and applies it in Bayesian optimization.

Parametric t-SNE improves generalization for streaming data.

We establish an optimal regularity result for parametrized two-dimensional stationary varifolds. Namely, we show that the parametrization map is a smooth minimal branched immersion and that the multiplicity function is constant. We provide some applications of this regularity result, especially in the calculus of varia…

We consider off-policy evaluation and optimization with continuous action spaces. We focus on observational data where the data collection policy is unknown and needs to be estimated. We take a semi-parametric approach where the value function takes a known parametric form in the treatment, but we are agnostic on how i…

Neural network training is usually accomplished by solving a non-convex optimization problem using stochastic gradient descent. Although one optimizes over the networks parameters, the main loss function generally only depends on the realization of the neural network, i.e. the function it computes. Studying the optimiz…

Parametric adversarial divergences, which are a generalization of the losses used to train generative adversarial networks (GANs), have often been described as being approximations of their nonparametric counterparts, such as the Jensen-Shannon divergence, which can be derived under the so-called optimal discriminator …

Proof of learning rate transfer in MLPs with $μ$P parameterization.

Near-optimal tests and confidence sequences for non-parametric data.

We consider an investor, whose portfolio consists of a single risky asset and a risk free asset, who wants to maximize his expected utility of the portfolio subject to the Value at Risk assuming a heavy tail distribution of the stock prices return. We use Markov Decision Process and dynamic programming principle to get…

New function class characterizes loss landscape of deep neural networks without over-parametrization.

Improved MLE for Hawkes Processes stabilizes unstable optimization.

Estimates conditional Brenier maps using entropic optimal transport.

This paper presents a semi-parametric algorithm for online learning of a robot inverse dynamics model. It combines the strength of the parametric and non-parametric modeling. The former exploits the rigid body dynamics equa- tion, while the latter exploits a suitable kernel function. We provide an extensive comparison …

DPPS uses DP priors for Bayesian non-parametric multi-arm bandits.

Given a set of observations generated by an optimization process, the goal of inverse optimization is to determine likely parameters of that process. We cast inverse optimization as a form of deep learning. Our method, called deep inverse optimization, is to unroll an iterative optimization process and then use backpro…

Optimizes predictions for specific tasks using parametrized decision analysis.

The method of covariate adjustment is often used for estimation of population average treatment effects in observational studies. Graphical rules for determining all valid covariate adjustment sets from an assumed causal graphical model are well known. Restricting attention to causal linear models, a recent article der…

Recent policy optimization approaches have achieved substantial empirical success by constructing surrogate optimization objectives. The Approximate Policy Iteration objective (Schulman et al., 2015a; Kakade and Langford, 2002) has become a standard optimization target for reinforcement learning problems. Using this ob…

New unbiased gradient estimators for complex optimization problems.

We give a parametrization of test configurations in the sense of Donaldson via spherical buildings, and show the existence of "optimal" destabilizing test configurations for unstable varieties, in the wake of Mumford and Kempf. We also give an account of the recent slight amendment to definition of K-stability after Li…

This work introduces benchmarks for evaluating nanophotonic structures in design simulations.

Kernel ridge regression (KRR) is a standard method for performing non-parametric regression over reproducing kernel Hilbert spaces. Given $n$ samples, the time and space complexity of computing the KRR estimate scale as $\mathcal{O}(n^3)$ and $\mathcal{O}(n^2)$ respectively, and so is prohibitive in many cases. We prop…

New algorithm reduces dynamic regret for noisy gradient feedback with piecewise polynomial comparators.

Cookbook transforms constrained statistical inference into unconstrained problems.

We introduce a new framework for comparing parametric network families.

This paper studies the effect of discretizing the parametrization of a dictionary used for Matching Pursuit decompositions of signals. Our approach relies on viewing the continuously parametrized dictionary as an embedded manifold in the signal space on which the tools of differential (Riemannian) geometry can be appli…

The paper proposes a semi-parametric Bayesian network model using Gaussian Processes and Horseshoe priors.

This research uses DPPs to improve semi-parametric regression models.

Parametric insurance offers better risk-sharing in high-risk settings than traditional indemnity insurance.

New MD algorithms using Tempesta logarithms for machine learning.

Develops a new method for learning non-parametric DAGs using RKHS.

Improved complexity for machine learning optimization methods.

New analysis shows optimal embedding learning rate depends on vocabulary size, not just model width.

Method identifies shifts leading to large model performance differences.

The paper sets bounds on how much regret is unavoidable in adaptive LQR with unknown B-matrix.

Efficiently optimizes orthogonal and Stiefel matrices on parallel units.

Study optimizes estimating linear functionals from observational data without strict overlap.

Although various distributed machine learning schemes have been proposed recently for pure linear models and fully nonparametric models, little attention has been paid on distributed optimization for semi-paramemetric models with multiple-level structures (e.g. sparsity, linearity and nonlinearity). To address these is…