arXiv research

Paper tackles fault tolerance in distributed linear regression.

A data filtering method for cluster analysis is proposed, based on minimizing a least squares function with a weighted $\ell_0$-norm penalty. To overcome the discontinuity of the objective function, smooth non-convex functions are employed to approximate the $\ell_0$-norm. The convergence of the global minimum points o…

Method introduces topological regularization using information filtering networks.

Pruning CNNs by removing less important filters based on empirical loss changes.

We discuss the problem of adaptive discrete-time signal denoising in the situation where the signal to be recovered admits a "linear oracle" -- an unknown linear estimate that takes the form of convolution of observations with a time-invariant filter. It was shown by Juditsky and Nemirovski (2009) that when the $\ell_2…

Kernel adaptive filters, a class of adaptive nonlinear time-series models, are known by their ability to learn expressive autoregressive patterns from sequential data. However, for trivial monotonic signals, they struggle to perform accurate predictions and at the same time keep computational complexity within desired …

Paper addresses fault-tolerance in distributed machine learning with stochastic gradient descent.

We propose two sparsity-aware normalized subband adaptive filter (NSAF) algorithms by using the gradient descent method to minimize a combination of the original NSAF cost function and the l1-norm penalty function on the filter coefficients. This l1-norm penalty exploits the sparsity of a system in the coefficients upd…

Improved robustness for high-dimensional Kalman filtering.

We introduce a family of adaptive estimators on graphs, based on penalizing the $\ell_1$ norm of discrete graph differences. This generalizes the idea of trend filtering [Kim et al. (2009), Tibshirani (2014)], used for univariate nonparametric regression, to graphs. Analogous to the univariate case, graph trend filteri…

Proposes a new graph trend filtering model for inhomogeneous graph signals.

Kähler information manifolds for signal filters in weighted Hardy spaces are explored.

New bounds improve deep learning performance efficiently.

The paper learns an autoregressive filter for unknown dynamical systems with robust guarantees.

We prove the correspondence between the information geometry of a signal filter and a Kähler manifold. The information geometry of a minimum-phase linear system with a finite complex cepstrum norm is a Kähler manifold. The square of the complex cepstrum norm of the signal filter corresponds to the Kähler potential. The…

Bayesian neural network improves feature selection and prediction.

A new filter design improves system identification accuracy.

New method quantifies market shocks and their effects.

Low-rank matrix recovery has found many applications in science and engineering such as machine learning, signal processing, collaborative filtering, system identification, and Euclidean embedding. But the low-rank matrix recovery problem is an NP hard problem and thus challenging. A commonly used heuristic approach is…

In this work, we study stability of distributed filtering of Markov chains with finite state space, partially observed in conditionally Gaussian noise. We consider a nonlinear filtering scheme over a Distributed Network of Agents (DNA), which relies on the distributed evaluation of the likelihood part of the centralize…

Paper tackles label insufficiency and inaccuracy in semi-supervised learning.

Study on discrepancy principle for learning algorithms in nonparametric regression.

Dropout neural networks can approximate any function with high probability.

We compute different versions of link Floer homology $HFL^{-}$ and $\widehat{HFL}$ for any $L$-space link with two components. The main approach is to compute the $h$-function of the filtered chain complex which is determined by the Alexander polynomials of every sublink of the $L$-space link. As an application, Thurst…

Most traditional online learning algorithms are based on variants of mirror descent or follow-the-leader. In this paper, we present an online algorithm based on a completely different approach, tailored for transductive settings, which combines "random playout" and randomized rounding of loss subgradients. As an applic…

Improved generalization bounds for multi-class CNNs without explicit class dependence.

Two-layer CNN learns key patterns in specific data distributions.

A new method for joint noise removal and trend estimation from sparse signals.

Algorithm estimates covariance from noisy data efficiently.

We describe a model for capturing the statistical structure of local amplitude and local spatial phase in natural images. The model is based on a recently developed, factorized third-order Boltzmann machine that was shown to be effective at capturing higher-order structure in images by modeling dependencies among squar…

New nonconvex regularizer speeds up low-rank matrix completion.

We propose algorithms for approximate filtering and smoothing in high-dimensional Factorial hidden Markov models. The approximation involves discarding, in a principled way, likelihood factors according to a notion of locality in a factor graph associated with the emission distribution. This allows the exponential-in-d…

FPCA optimizes fairness in target vectors' span.

Algorithm efficiently learns deep ReLU networks with polynomial runtime in depth and parameters.

Adaptive filtering algorithms operating in reproducing kernel Hilbert spaces have demonstrated superiority over their linear counterpart for nonlinear system identification. Unfortunately, an undesirable characteristic of these methods is that the order of the filters grows linearly with the number of input data. This …

Singular values of a data in a matrix form provide insights on the structure of the data, the effective dimensionality, and the choice of hyper-parameters on higher-level data analysis tools. However, in many practical applications such as collaborative filtering and network analysis, we only get a partial observation.…

This paper introduces structure learning for autoencoder recommenders to improve performance and generalization.

This paper focuses on spectral filters on graphs, namely filters defined as elementwise multiplication in the frequency domain of a graph. In many graph signal processing settings, it is important to transfer a filter from one graph to another. One example is in graph convolutional neural networks (ConvNets), where the…

SPCA improves PCA by learning from simple to complex samples.

Simplifies and optimizes learning from untrusted batches with structure.

We study a set $\mathcal{M}_{K,N}$ parameterizing filtered $SL(K)$-Higgs bundles over $\mathbb{CP}^1$ with an irregular singularity at $z = \infty$, such that the eigenvalues of the Higgs field grow like $\lvert λ\rvert \sim \lvert z ^{N/K} \mathrm{d} z \rvert$, where $K$ and $N$ are coprime. $\mathcal{M}_{K,N}$ carrie…

A new SOHP filter improves trend estimation in economic time series.

Low-rank modeling plays a pivotal role in signal processing and machine learning, with applications ranging from collaborative filtering, video surveillance, medical imaging, to dimensionality reduction and adaptive filtering. Many modern high-dimensional data and interactions thereof can be modeled as lying approximat…

Deep density methods improve filtering in high-dimensional systems.

The study examines how weight sharing, equivariance, and locality affect the sample complexity of neural networks.

The sophisticated structure of Convolutional Neural Network (CNN) allows for outstanding performance, but at the cost of intensive computation. As significant redundancies inevitably present in such a structure, many works have been proposed to prune the convolutional filters for computation cost reduction. Although ex…

Efficiently estimates covariance matrix for elliptical distributions under strong contamination.

Gradient filters track moving parameters under noisy data and misspecification.