arXiv research

Study introduces a new method for multiple parameter regularization in polynomial functional regression.

A new method debiases multiple target parameters without IFs.

The paper studies how regularization parameters affect sparsity in deep neural networks.

This paper presents a bias-variance tradeoff of graph Laplacian regularizer, which is widely used in graph signal processing and semi-supervised learning tasks. The scaling law of the optimal regularization parameter is specified in terms of the spectral graph properties and a novel signal-to-noise ratio parameter, whi…

We solve ElasticNet regularization tuning across multiple instances with provable guarantees.

Method estimates multiple related Gaussian distributions using Laplacian regularization.

Within a statistical learning setting, we propose and study an iterative regularization algorithm for least squares defined by an incremental gradient method. In particular, we show that, if all other parameters are fixed a priori, the number of passes over the data (epochs) acts as a regularization parameter, and prov…

Improved covariance matrix estimation for multiple classes with limited data.

Novel optimization method detects change points in Gaussian data.

Proposes a method to infer complex network topologies from multiple graphs.

The (stochastic) gradient descent and the multiplicative update method are probably the most popular algorithms in machine learning. We introduce and study a new regularization which provides a unification of the additive and multiplicative updates. This regularization is derived from an hyperbolic analogue of the entr…

Proposes a new model for predicting chronic conditions over time.

Multiple generalized additive models (GAMs) are a type of distributional regression wherein parameters of probability distributions depend on predictors through smooth functions, with selection of the degree of smoothness via $L_2$ regularization. Multiple GAMs allow finer statistical inference by incorporating explana…

SeReNe prunes neurons with low sensitivity to reduce network size.

This work improves deep learning from noisy crowdsourced labels.

We study the generalization properties of stochastic gradient methods for learning with convex loss functions and linearly parameterized functions. We show that, in the absence of penalizations or constraints, the stability and approximation properties of the algorithm can be controlled by tuning either the step-size o…

Conditional meta-learning improves meta-learning performance in diverse task environments.

Manifold regularization, such as laplacian regularized least squares (LapRLS) and laplacian support vector machine (LapSVM), has been widely used in semi-supervised learning, and its performance greatly depends on the choice of some hyper-parameters. Cross-validation (CV) is the most popular approach for selecting the …

Proves $C^{1,1}$ regularity for multiple membrane solutions.

In this paper we discuss Bayesian nonconvex penalization for sparse learning problems. We explore a nonparametric formulation for latent shrinkage parameters using subordinators which are one-dimensional Lévy processes. We particularly study a family of continuous compound Poisson subordinators and a family of discrete…

Due to the growing ubiquity of unlabeled data, learning with unlabeled data is attracting increasing attention in machine learning. In this paper, we propose a novel semi-supervised kernel learning method which can seamlessly combine manifold structure of unlabeled data and Regularized Least-Squares (RLS) to learn a ne…

This paper solves the convergence problem for estimating MGGD parameters with a convex formulation.

Regularization plays an important role in generalization of deep neural networks, which are often prone to overfitting with their numerous parameters. L1 and L2 regularizers are common regularization tools in machine learning with their simplicity and effectiveness. However, we observe that imposing strong L1 or L2 reg…

New neural network training method uses bi-fidelity data to reduce errors.

We empirically investigate the best trade-off between sparse and uniformly-weighted multiple kernel learning (MKL) using the elastic-net regularization on real and simulated datasets. We find that the best trade-off parameter depends not only on the sparsity of the true kernel-weight spectrum but also on the linear dep…

Sparse linear regression -- finding an unknown vector from linear measurements -- is now known to be possible with fewer samples than variables, via methods like the LASSO. We consider the multiple sparse linear regression problem, where several related vectors -- with partially shared support sets -- have to be recove…

We present a novel methodology able to distinguish meaningful level shifts from typical signal fluctuations. A two-stage regularization filtering can accurately identify the location of the significant level-shifts with an efficient parameter-free algorithm. The developed methodology demands low computational effort an…

A new method for multi-task learning improves performance without weakening inductive bias.

A tuning-free method recovers jointly sparse signals in MMV using implicit regularization.

A new method estimates protein evolutionary fields and couplings from alignments.

Kernel fusion is a popular and effective approach for combining multiple features that characterize different aspects of data. Traditional approaches for Multiple Kernel Learning (MKL) attempt to learn the parameters for combining the kernels through sophisticated optimization procedures. In this paper, we propose an a…

We propose and evaluate new techniques for compressing and speeding up dense matrix multiplications as found in the fully connected and recurrent layers of neural networks for embedded large vocabulary continuous speech recognition (LVCSR). For compression, we introduce and study a trace norm regularization technique f…

The inverse Potts problem to infer a Boltzmann distribution for homologous protein sequences from their single-site and pairwise amino acid frequencies recently attracts a great deal of attention in the studies of protein structure and evolution. We study regularization and learning methods and how to tune regularizati…

Reduces constructing multiplicative connections to simpler tasks.

We identify a phenomenon, which we refer to as multi-model forgetting, that occurs when sequentially training multiple deep networks with partially-shared parameters; the performance of previously-trained models degrades as one optimizes a subsequent one, due to the overwriting of shared parameters. To overcome this, w…

The paper analyzes the risk of CV-tuned regularized estimators and connects it to SURE.

A new method prunes neural networks efficiently without losing effectiveness.

The paper develops RL methods for optimal switching between multiple states.

Multi-task learning (MTL) is a common paradigm that seeks to improve the generalization performance of task learning by training related tasks simultaneously. However, it is still a challenging problem to search the flexible and accurate architecture that can be shared among multiple tasks. In this paper, we propose a …

Entropy regularization improves sparse model discovery in federated learning.

The paper proposes a new system ID method from noisy data.

Study improves boundary smoothness for area-minimizing currents with complex boundaries.

Let $G$ be a connected, linear, real reductive Lie group with compact centre. Let $K<G$ be compact. Under a condition on $K$, which holds in particular if $K$ is maximal compact, we give a geometric expression for the multiplicities of the $K$-types of any tempered representation (in fact, any standard representation) …

Proposes a method for evaluating multiple dimensions of organizational effectiveness using DEA.

Solutions to a quadratic matrix equation are linked to strongly regular graphs and multiplicative characters.

We present an efficient approach for leveraging the knowledge from multiple modalities in training unimodal 3D convolutional neural networks (3D-CNNs) for the task of dynamic hand gesture recognition. Instead of explicitly combining multimodal information, which is commonplace in many state-of-the-art methods, we propo…

Proposes a method for multi-view clustering that integrates consistent and complementary graph regularizers.

Study optimal execution in a transient price impact model with multiple traders.