arXiv research

Deep nets' optima are surprisingly connected by simple paths.

In this survey article, we review the relation between heat kernels and path integrals. In particular, we review recent results on the approximation of the Wiener measure on compact manifold by measures on (finite-dimensional) spaces of piece-wise geodesics.

Given two points on a soup can or conical cup with lid, we find and classify all paths of minimal length connecting them. When the number of minimal paths is finite, there are at most four on a can and three on a cup. At worst, minimal paths are piece-wise smooth with three components, each of which is a classical geod…

XOFM explains attribute effects in ordinal regression using piece-wise linear functions.

Paper reformulates UOT as non-negative penalized linear regression for efficient algorithms.

L*ReLU improves deep learning for fine-grained image classification.

AdaPID optimizes diffusion-based samplers by dynamically adjusting schedules.

DAMI uses interpretable regions to select informative samples for deep learning models.

CTR prediction in real-world business is a difficult machine learning problem with large scale nonlinear sparse data. In this paper, we introduce an industrial strength solution with model named Large Scale Piece-wise Linear Model (LS-PLM). We formulate the learning problem with $L_1$ and $L_{2,1}$ regularizers, leadin…

We introduce a new activation function using Chebyshev-Lagrange polynomials for improved neural network performance.

RUMBoost combines RUMs and deep learning for better choice modelling.

A new algorithm finds optimal solutions for constrained decision processes.

PyChEst detects changes in non-stationary time series without distributional assumptions.

This paper proposes a method to approximate non-Gaussian likelihoods in Gaussian Processes.

Estimates change-points and graph structures in a time-varying Ising model.

Method to create rational Seifert surfaces for knots in Lens space.

Let S be a triangulated 2-sphere with fixed triangulation T. We apply the methods of thin position from knot theory to obtain a simple version of the three geodesics theorem for the 2-sphere [5]. In general these three geodesics may be unstable, corresponding, for example, to the three equators of an ellipsoid. Using a…

Neural models can realize decision trees with parameter sharing and improved performance.

This paper studies a class of continuous-time scalar-state stochastic Linear-Quadratic (LQ) optimal control problem with the linear control constraints. Applying the state separation theorem induced from its special structure, we develop the explicit solution for this class of problem. The revealed optimal control poli…

Bayesian method detects change points and clusters in piece-wise constant signals.

New methods for calculating curvature in graph theory.

The linear transports along paths in vector bundles introduced in Ref. [1] are applied to the special case of tensor bundles over a given differentiable manifold. Links with the transports along paths generated by derivations of tensor algebras are investigated. A possible generalization of the theory of geodesics is p…

Diagonal linear networks converge to lasso regularization path during training.

Efficiently infers switching nonlinear systems with collapsed amortized variational inference.

In many applications we seek to maximize an expectation with respect to a distribution over discrete variables. Estimating gradients of such objectives with respect to the distribution parameters is a challenging problem. We analyze existing solutions including finite-difference (FD) estimators and continuous relaxatio…

Global approximation for piecewise linear paths via signatures.

We propose a strategy for approximating Pareto optimal sets based on the global analysis framework proposed by Smale (Dynamical systems, New York, 1973, pp. 531-544). The method highlights and exploits the underlying manifold structure of the Pareto sets, approximating Pareto optima by means of simplicial complexes. Th…

The importance of interpretability of machine learning models has been increasing due to emerging enterprise predictive analytics, threat of data privacy, accountability of artificial intelligence in society, and so on. Piecewise linear models have been actively studied to achieve both accuracy and interpretability. Th…

The (parallel) linear transports along paths in vector bundles are axiomatically described. Their general form and certain properties are found. It is shown that these transports are locally (i.e. along every fixed path) always Euclidean ones in a senses that there exist frames in which their matrices are unit. The inv…

Paper investigates a new type of elastica formed during beam rupture.

Optimal order execution strategies for brokers under reference benchmarks.

The regularization path of the Lasso can be shown to be piecewise linear, making it possible to "follow" and explicitly compute the entire path. We analyze in this paper this popular strategy, and prove that its worst case complexity is exponential in the number of variables. We then oppose this pessimistic result to a…

Curvature and torsion of linear transports along paths in, respectively, vector bundles and the tangent bundle to a differentiable manifold are defined and certain their properties are derived.

We propose an algorithm for exploring the entire regularization path of asymmetric-cost linear support vector machines. Empirical evidence suggests the predictive power of support vector machines depends on the regularization parameters of the training algorithms. The algorithms exploring the entire regularization path…

We study in this paper a class of constrained linear-quadratic (LQ) optimal control problem formulations for the scalar-state stochastic system with multiplicative noise, which has various applications, especially in the financial risk management. The linear constraint on both the control and state variables considered…

New algorithm speeds up path computation for optimal models.

A geometric interpretation of curvature and torsion of linear transports along paths is presented. A number of (Bianchi type) identities satisfied by these quantities are derived. The obtained results contain as special cases the corresponding classical ones concerning curvature and torsion of linear connections.

New method generates realistic financial price paths with drawdowns.

The paper proves signatures of non-geometric rough paths can approximate functionals uniformly.

Although various linear log-distance path loss models have been developed, advanced models are requiring to more accurately and flexibly represent the path loss for complex environments such as the urban area. This letter proposes an artificial neural network (ANN) based multi-dimensional regression framework for path …

A new method to measure neural network expressiveness using tighter upper bounds.

Proposes an efficient shrinkage path for ridge regression.

We consider the generic regularized optimization problem $\hat{\mathsfβ}(λ)=\arg \min_βL({\sf{y}},X{\sfβ})+λJ({\sfβ})$. Efron, Hastie, Johnstone and Tibshirani [Ann. Statist. 32 (2004) 407--499] have shown that for the LASSO--that is, if $L$ is squared error loss and $J(β)=\|β\|_1$ is the $\ell_1$ norm of $β$--the opti…

A positive path in the linear symplectic group $\Sp(2n)$ is a smooth path which is everywhere tangent to the positive cone. These paths are generated by negative definite (time-dependent) quadratic Hamiltonian functions on Euclidean space. A special case are autonomous positive paths, which are generated by time-indepe…

Algorithm reduces regret in SSP problems with LFA.

We introduce linear holonomy on Poisson manifolds. The linear holonomy of a Poisson structure generalizes the linearized holonomy on a regular symplectic foliation. However, for singular Poisson structures the linear holonomy is defined for the lifts of tangential path to the cotangent bundle (cotangent paths). The lin…

This paper extends depth separation results to piece-wise oscillatory functions.

Introduces a neural network-based method for efficient state and parameter estimation in complex systems.