arXiv research

TVS-FNNs can approximate any continuous function on expanded input spaces.

We present a novel framework for specifying and verifying correctness globally for neural networks on perception tasks. Most previous works on neural network verification for perception tasks focus on robustness verification. Unlike robustness verification, which aims to verify that the prediction of a network is stabl…

Functional input neural networks approximate continuous functions on weighted spaces.

Study of deep neural networks using finite-time Lyapunov exponents.

Following great success in the image processing field, the idea of adversarial training has been applied to tasks in the natural language processing (NLP) field. One promising approach directly applies adversarial training developed in the image processing field to the input word embedding space instead of the discrete…

Improves sample efficiency in reinforcement learning with input representation.

Aims to optimize complex multivariate systems with constraints.

The sparse pseudo-input Gaussian process (SPGP) is a new approximation method for speeding up GP regression in the case of a large number of data points N. The approximation is controlled by the gradient optimization of a small set of M `pseudo-inputs', thereby reducing complexity from N^3 to NM^2. One limitation of th…

FoRDE uses input gradients to improve neural network ensembles.

State-space systems generate probabilistic dependencies between inputs and outputs.

Study efficient neural operator learning using variation spaces.

LOL-BO improves latent space Bayesian optimization over structured inputs.

This paper studies the generalization error of invariant classifiers. In particular, we consider the common scenario where the classification task is invariant to certain transformations of the input, and that the classifier is constructed (or learned) to be invariant to these transformations. Our approach relies on fa…

MINs learn inverse mappings for high-dimensional optimization problems.

Many engineering problems require identifying feasible domains under implicit constraints. One example is finding acceptable car body styling designs based on constraints like aesthetics and functionality. Current active-learning based methods learn feasible domains for bounded input spaces. However, we usually lack pr…

A new method reduces both input and output dimensions for better goal-oriented analysis.

New recursive algorithm estimates conditional kernel mean embeddings in Hilbert space.

Develops a new approach to establish universality for any-dimensional machine learning models.

One popular hypothesis of neural network generalization is that the flat local minima of loss surface in parameter space leads to good generalization. However, we demonstrate that loss surface in parameter space has no obvious relationship with generalization, especially under adversarial settings. Through visualizing …

A method constructs a stochastic surrogate from dimensionality reduction results for high-dimensional uncertainty quantification.

Improves active learning efficiency by warping input space based on observed outputs.

Paper addresses hypothesis space misspecification in learning from human demonstrations and corrections.

Active learning selects inputs for GPSSM to learn latent states.

New neural network models learn symmetric functions of varying input sizes.

Although deep reinforcement learning has advanced significantly over the past several years, sample efficiency remains a major challenge. Careful choice of input representations can help improve efficiency depending on the structure present in the problem. In this work, we present an attention-based method to project i…

We propose regularizing the empirical loss for semi-supervised learning by acting on both the input (data) space, and the weight (parameter) space. We show that the two are not equivalent, and in fact are complementary, one affecting the minimality of the resulting representation, the other insensitivity to nuisance va…

Model captures system input variations in latent space for actionable dynamics.

In this paper we introduce the deep kernelized autoencoder, a neural network model that allows an explicit approximation of (i) the mapping from an input space to an arbitrary, user-specified kernel space and (ii) the back-projection from such a kernel space to input space. The proposed method is based on traditional a…

Proposes a framework to fuse heterogeneous data sources for better modeling.

Instance embeddings are an efficient and versatile image representation that facilitates applications like recognition, verification, retrieval, and clustering. Many metric learning methods represent the input as a single point in the embedding space. Often the distance between points is used as a proxy for match confi…

This work uses sampling theory to analyze smoothness and error bounds of finite neural networks.

δ-CLUE generates diverse explanations for model uncertainty.

Random feature model approximates PDE solutions efficiently.

Random projections improve GP regression performance, reducing high-dimensional inputs to 1D.

Within machine learning, the supervised learning field aims at modeling the input-output relationship of a system, from past observations of its behavior. Decision trees characterize the input-output relationship through a series of nested $if-then-else$ questions, the testing nodes, leading to a set of predictions, th…

Geometry-aware noise improves model generalization on complex manifolds.

A new faster neural network training method using backprojection.

Manifold learning has been successfully applied to a variety of medical imaging problems. Its use in real-time applications requires fast projection onto the low-dimensional space. To this end, out-of-sample extensions are applied by constructing an interpolation function that maps from the input space to the low-dimen…

JES optimizes expensive functions by considering joint entropy over input and output spaces.

BOSS optimizes string inputs using string kernels and genetic algorithms.

Despite recent advances, large scale visual artifacts are still a common occurrence in images generated by GANs. Previous work has focused on improving the generator's capability to accurately imitate the data distribution $p_{data}$. In this paper, we instead explore methods that enable GANs to actively avoid errors b…

Deep neural networks are vulnerable to adversarial attacks and hard to interpret because of their black-box nature. The recently proposed invertible network is able to accurately reconstruct the inputs to a layer from its outputs, thus has the potential to unravel the black-box model. An invertible network classifier c…

We study the geometry of deep (neural) networks (DNs) with piecewise affine and convex nonlinearities. The layers of such DNs have been shown to be {\em max-affine spline operators} (MASOs) that partition their input space and apply a region-dependent affine mapping to their input to produce their output. We demonstrat…

LESS combines local predictors for subsets to learn from heterogeneous input-output pairs.

Develops neural network approximations for infinite-dimensional input-output maps.

Adversarial Reprogramming has demonstrated success in utilizing pre-trained neural network classifiers for alternative classification tasks without modification to the original network. An adversary in such an attack scenario trains an additive contribution to the inputs to repurpose the neural network for the new clas…

We consider a neural network architecture with randomized features, a sign-splitter, followed by rectified linear units (ReLU). We prove that our architecture exhibits robustness to the input perturbation: the output feature of the neural network exhibits a Lipschitz continuity in terms of the input perturbation. We fu…

Deep single-index Fréchet regression for metric space-valued outputs