arXiv research

Curvature penalties improve interpretability of KANs without sacrificing accuracy.

Improved online penalty selection for time series models.

Paper proposes efficient algorithms for designing SLOPE penalty sequences.

Proposes using MLP for predicting optimal penalty in changepoint detection.

New method approximates sampling from smooth potential distributions using a vanishing penalty.

Two new methods improve block-sparse signal recovery from noisy data.

In the classic sparsity-driven problems, the fundamental L-1 penalty method has been shown to have good performance in reconstructing signals for a wide range of problems. However this performance relies on a good choice of penalty weight which is often found from empirical experiments. We propose an algorithm called t…

Deep Penalty Method solves high-dimensional optimal stopping problems using deep learning.

This paper proposes a new interpretation of sparse penalties such as the elastic-net and the group-lasso. Beyond providing a new viewpoint on these penalization schemes, our approach results in a unified optimization strategy. Our experiments demonstrate that this strategy, implemented on the elastic-net, is computatio…

PASE method protects machine learning models from membership inference attacks without significant accuracy loss.

A new method reduces bias in adaptive Lasso estimates.

Study improves understanding of non-differentiable penalties in high-dimensional settings.

Unified framework for fair regression in aware and unaware settings.

We study epidemic forecasting on real-world health data by a graph-structured recurrent neural network (GSRNN). We achieve state-of-the-art forecasting accuracy on the benchmark CDC dataset. To improve model efficiency, we sparsify the network weights via transformed-$\ell_1$ penalty and maintain prediction accuracy at…

HALO learns to prune neural networks by adaptively shrinking weights.

Stochastic approach improves neural network training for kinetic simulations.

We study channel number reduction in combination with weight binarization (1-bit weight precision) to trim a convolutional neural network for a keyword spotting (classification) task. We adopt a group-wise splitting method based on the group Lasso penalty to achieve over 50% channel sparsity while maintaining the netwo…

ShuffleNet is a state-of-the-art light weight convolutional neural network architecture. Its basic operations include group, channel-wise convolution and channel shuffling. However, channel shuffling is manually designed empirically. Mathematically, shuffling is a multiplication by a permutation matrix. In this paper, …

Study the distribution for low-rank matrix learning, improving inference methods.

A new method for combining multiple data views in supervised learning.

We provide novel theoretical results regarding local optima of regularized $M$-estimators, allowing for nonconvexity in both loss and penalty functions. Under restricted strong convexity on the loss and suitable regularity conditions on the penalty, we prove that \emph{any stationary point} of the composite objective f…

Joint peak detection is a central problem when comparing samples in genomic data analysis, but current algorithms for this task are unsupervised and limited to at most 2 sample types. We propose PeakSegJoint, a new constrained maximum likelihood segmentation model for any number of sample types. To select the number of…

New method predicts customer churn using mixed-penalty logistic regression.

New fair regression method improves fairness in chronic kidney disease classification.

Proposes a fair classification model using robust optimization.

Regularization methods are often employed in deep learning neural networks (DNNs) to prevent overfitting. For penalty based DNN regularization methods, convex penalties are typically considered because of their optimization guarantees. Recent theoretical work have shown that nonconvex penalties that satisfy certain reg…

WrapNet optimizes inference for low-resolution neural networks by using 8-bit additions.

ResRep prunes CNNs without losing accuracy by separating remembering and forgetting.

Improved equation learning accuracy via comprehensive R²-elimination and Bayesian model selection.

We build a deep reinforcement learning (RL) agent that can predict the likelihood of an individual testing positive for malaria by asking questions about their household. The RL agent learns to determine which survey question to ask next and when to stop to make a prediction about their likelihood of malaria based on t…

Deep learning models reconstruct volatility surfaces from noisy data under no-arbitrage constraints.

Proposes a non-crossing deep neural network quantile regression method.

A new method solves bilevel optimization problems without Hessian inversion.

Sparse regression models are increasingly prevalent due to their ease of interpretability and superior out-of-sample performance. However, the exact model of sparse regression with an $\ell_0$ constraint restricting the support of the estimators is a challenging (\NP-hard) non-convex optimization problem. In this paper…

UCPO improves diversity in reinforcement learning models, maintaining high accuracy.

Single deep model detects out-of-distribution data with single forward pass.

Gradient penalty improves GAN performance by inducing a large-margin classifier.

One-bit measurements widely exist in the real world, and they can be used to recover sparse signals. This task is known as the problem of learning halfspaces in learning theory and one-bit compressive sensing (1bit-CS) in signal processing. In this paper, we propose novel algorithms based on both convex and nonconvex s…

This paper deals with the numerical approximation of American-style option values governed by partial differential complementarity problems. For a variety of one- and two-asset American options we investigate by ample numerical experiments the temporal convergence behaviour of three modern splitting methods: the explic…

The paper studies robust risk measures with linear penalties under uncertain distributions.

Efficiently performs robust and sparse kernel regression.

The use of machine-learning in neuroimaging offers new perspectives in early diagnosis and prognosis of brain diseases. Although such multivariate methods can capture complex relationships in the data, traditional approaches provide irregular (l2 penalty) or scattered (l1 penalty) predictive pattern with a very limited…

A new training method speeds up ResNet training by 3x with minimal accuracy loss.

Many problems in machine learning and other fields can be (re)for-mulated as linearly constrained separable convex programs. In most of the cases, there are multiple blocks of variables. However, the traditional alternating direction method (ADM) and its linearized version (LADM, obtained by linearizing the quadratic p…

We introduce a general method of performing Residual Network inference and learning in the JPEG transform domain that allows the network to consume compressed images as input. Our formulation leverages the linearity of the JPEG transform to redefine convolution and batch normalization with a tune-able numerical approxi…

Improved Kriging model reduces prediction errors.

New method for sparse kernel selection improves prediction accuracy.

New sparse penalty improves biclustering for gene expression data.