arXiv research

A new algorithm finds optimal solutions for constrained decision processes.

CTR prediction in real-world business is a difficult machine learning problem with large scale nonlinear sparse data. In this paper, we introduce an industrial strength solution with model named Large Scale Piece-wise Linear Model (LS-PLM). We formulate the learning problem with $L_1$ and $L_{2,1}$ regularizers, leadin…

New methods for calculating curvature in graph theory.

DAMI uses interpretable regions to select informative samples for deep learning models.

Deep neural networks paved the way for significant improvements in image visual categorization during the last years. However, even though the tasks are highly varying, differing in complexity and difficulty, existing solutions mostly build on the same architectural decisions. This also applies to the selection of acti…

RUMBoost combines RUMs and deep learning for better choice modelling.

While neural networks have achieved high performance in different learning tasks, their accuracy drops significantly in the presence of small adversarial perturbations to inputs. Defenses based on regularization and adversarial training are often followed by new attacks to defeat them. In this paper, we propose attack-…

PyChEst detects changes in non-stationary time series without distributional assumptions.

We show how neural models can be used to realize piece-wise constant functions such as decision trees. The proposed architecture, which we call locally constant networks, builds on ReLU networks that are piece-wise linear and hence their associated gradients with respect to the inputs are locally constant. We formally …

We provide larger step-size restrictions for which gradient descent based algorithms (almost surely) avoid strict saddle points. In particular, consider a twice differentiable (non-convex) objective function whose gradient has Lipschitz constant L and whose Hessian is well-behaved. We prove that the probability of init…

This paper proposes a method to approximate non-Gaussian likelihoods in Gaussian Processes.

Ordinal regression predicts the objects' labels that exhibit a natural ordering, which is important to many managerial problems such as credit scoring and clinical diagnosis. In these problems, the ability to explain how the attributes affect the prediction is critical to users. However, most, if not all, existing ordi…

Method to create rational Seifert surfaces for knots in Lens space.

Let S be a triangulated 2-sphere with fixed triangulation T. We apply the methods of thin position from knot theory to obtain a simple version of the three geodesics theorem for the 2-sphere [5]. In general these three geodesics may be unstable, corresponding, for example, to the three equators of an ellipsoid. Using a…

This work focuses on the estimation of multiple change-points in a time-varying Ising model that evolves piece-wise constantly. The aim is to identify both the moments at which significant changes occur in the Ising model, as well as the underlying graph structures. For this purpose, we propose to estimate the neighbor…

This paper studies a class of continuous-time scalar-state stochastic Linear-Quadratic (LQ) optimal control problem with the linear control constraints. Applying the state separation theorem induced from its special structure, we develop the explicit solution for this class of problem. The revealed optimal control poli…

In this paper, we develop a novel {\bf ho}moto{\bf p}y {\bf s}moothing (HOPS) algorithm for solving a family of non-smooth problems that is composed of a non-smooth term with an explicit max-structure and a smooth term or a simple non-smooth term whose proximal mapping is easy to compute. The best known iteration compl…

In this paper, we investigate a transition from an elastica to a piece-wised elastica whose connected point defines the hinge angle $φ_0$; we refer the piece-wised elastica $Λ_{φ_0}$-elastica or $Λ$-elastica. The transition appears in the bending beam experiment; we compress elastic beams gradually and then suddenly du…

In many applications we seek to maximize an expectation with respect to a distribution over discrete variables. Estimating gradients of such objectives with respect to the distribution parameters is a challenging problem. We analyze existing solutions including finite-difference (FD) estimators and continuous relaxatio…

We propose a strategy for approximating Pareto optimal sets based on the global analysis framework proposed by Smale (Dynamical systems, New York, 1973, pp. 531-544). The method highlights and exploits the underlying manifold structure of the Pareto sets, approximating Pareto optima by means of simplicial complexes. Th…

We consider billiard ball motion in a convex domain of the Euclidean plane bounded by a piece-wise smooth curve influenced by the constant magnetic field. We show that if there exists a polynomial in velocities integral of the magnetic billiard flow then every smooth piece $γ$ of the boundary must be algebraic and eith…

The importance of interpretability of machine learning models has been increasing due to emerging enterprise predictive analytics, threat of data privacy, accountability of artificial intelligence in society, and so on. Piecewise linear models have been actively studied to achieve both accuracy and interpretability. Th…

Mode connectivity is a surprising phenomenon in the loss landscape of deep nets. Optima -- at least those discovered by gradient-based optimization -- turn out to be connected by simple paths on which the loss function is almost constant. Often, these paths can be chosen to be piece-wise linear, with as few as two segm…

In this survey article, we review the relation between heat kernels and path integrals. In particular, we review recent results on the approximation of the Wiener measure on compact manifold by measures on (finite-dimensional) spaces of piece-wise geodesics.

Optimal order execution strategies for brokers under reference benchmarks.

We study in this paper a class of constrained linear-quadratic (LQ) optimal control problem formulations for the scalar-state stochastic system with multiplicative noise, which has various applications, especially in the financial risk management. The linear constraint on both the control and state variables considered…

We seek to improve the data efficiency of neural networks and present novel implementations of parameterized piece-wise polynomial activation functions. The parameters are the y-coordinates of n+1 Chebyshev nodes per hidden unit and Lagrangian interpolation between the nodes produces the polynomial on [-1, 1]. We show …

Study reveals convergence properties of SGD with random learning rate.

A new method to measure neural network expressiveness using tighter upper bounds.

We study the problem of estimating a temporally varying coefficient and varying structure (VCVS) graphical model underlying nonstationary time series data, such as social states of interacting individuals or microarray expression profiles of gene networks, as opposed to i.i.d. data from an invariant model widely consid…

Paper reformulates UOT as non-negative penalized linear regression for efficient algorithms.

This paper extends depth separation results to piece-wise oscillatory functions.

Probabilistic solvers improve stability for stiff systems.

This work simplifies adversarial attacks using neural networks, reducing computation and improving training convergence.

Novel method converts time series data into functional data for high dimensional classification.

The paper proposes new cross-correlators using Price's Theorem and piecewise-linear decomposition.

Most of machine learning approaches have stemmed from the application of minimizing the mean squared distance principle, based on the computationally efficient quadratic optimization methods. However, when faced with high-dimensional and noisy data, the quadratic error functionals demonstrated many weaknesses including…

Considering Wirtinger's inequality for piece-wise equipartite functions we find a discrete version of this classical inequality. The main tool we use is the theorem of classification of isometries. Our approach provides a new elementary proof of Wirtinger's inequality that also allows to study the case of equality. Mor…

The paper proves Sard's theorem for polynomial maps in infinite dimensions.

Proposes a method for inference in high-dimensional classification with non-differentiable surrogate losses.

A new method for creating simpler models from complex ones.

Paper proposes a novel SVM method for creating survival trees.

It is shown that most of the well-known basic results for Sobolev-Slobodeckii and Bessel potential spaces, known to hold on bounded smooth domains in $\mathbb{R}^n$, continue to be valid on a wide class of Riemannian manifolds with singularities and boundary, provided suitable weights, which reflect the nature of the s…

Given two points on a soup can or conical cup with lid, we find and classify all paths of minimal length connecting them. When the number of minimal paths is finite, there are at most four on a can and three on a cup. At worst, minimal paths are piece-wise smooth with three components, each of which is a classical geod…

Fisheye cameras are commonly used in applications like autonomous driving and surveillance to provide a large field of view ($>180^{\circ}$). However, they come at the cost of strong non-linear distortions which require more complex algorithms. In this paper, we explore Euclidean distance estimation on fisheye cameras …

Boundary of fiber convex domains is a cohomological sphere.

In this survey, we present and compare different approaches to estimate Mutual Information (MI) from data to analyse general dependencies between variables of interest in a system. We demonstrate the performance difference of MI versus correlation analysis, which is only optimal in case of linear dependencies. First, w…

Optimal control in changing systems without strong convexity assumptions.