A new algorithm optimizes L1-norm error fitting problems efficiently.
problem Optimizing L1-norm error fitting models with data aggregation.
method Data aggregation-based algorithm with monotonic convergence to global optimum.
result The proposed algorithm optimally solves any L1-norm error fitting model.
BALSON optimizes parameters with Bayesian approach and Dirichlet distribution.
problem Data fitting with nonnegative L1-norm constraints.
method Bayesian approach, Gaussian likelihood, Dirichlet distribution, sampling methods.
result BALSON outperforms conventional methods in polynomial fitting.
Principal component analysis (PCA) is often used to reduce the dimension of data by selecting a few orthonormal vectors that explain most of the variance structure of the data. L1 PCA uses the L1 norm to measure error, whereas the conventional PCA uses the L2 norm. For the L1 PCA problem minimizing the fitting error of…
Proposes an optimization framework for sparse robust subspace estimation.
problem Sparse robust one-dimensional subspace estimation.
method l1-norm regularization, linear relaxation, simple ratios, sorting techniques.
result Achieves global optimality for sparse robust subspace with polynomial time efficiency.
A typical approach in estimating the learning rate of a regularized learning scheme is to bound the approximation error by the sum of the sampling error, the hypothesis error and the regularization error. Using a reproducing kernel space that satisfies the linear representer theorem brings the advantage of discarding t…
Study investigates estimation error in EMHMM simulations.
problem Estimation error in Hidden Markov Models (HMMs) with EMHMM.
method Simulation study using variational Bayesian inference.
result KL divergence and L1-norm relate to estimation error and ground-truth HMM parameters.
Study tightens bounds for interpolating noisy data using minimum l1-norm.
problem Predicting noisy data with minimum l1-norm interpolation.
method Provided matching upper and lower bounds for prediction error.
result Tight consistency up to negligible terms for d≫n. Novel L1-norm and L2-norm LDA methods improve discriminant analysis.
problem Improving linear discriminant analysis for robustness and adaptability.
method Proposes L1BLDA and L2BLDA using Bhattacharyya error bound, maximizing between-class scatters and minimizing within-class scatters.
result Proposed methods avoid SSS and have no rank limit, demonstrating robust performance and effectiveness.
A l1-norm penalized orthogonal forward regression (l1-POFR) algorithm is proposed based on the concept of leaveone- out mean square error (LOOMSE). Firstly, a new l1-norm penalized cost function is defined in the constructed orthogonal space, and each orthogonal basis is associated with an individually tunable regulari…
QAPCA uses quantum annealing for robust PCA.
problem Outliers in data skew L2-norm principal components.
method Quantum annealing for L1-norm optimization.
result QAPCA's reconstruction error is comparable to L1-BF.
Proposes a new method for hyperspectral image dimensionality reduction.
problem Highly correlated noisy hyperspectral images.
method Trace Lasso-L1 Graph Cut method using L1-norm for robustness and sparsity.
result Optimal projection matrix maximizing between-class dispersion to within-class dispersion.
Study models arrival rates and cancellation rates of limit orders in Borsa Istanbul.
problem Understanding order dynamics in Borsa Istanbul's stock market.
method Used limit order book data from Garanti Bank. Tested three discrete probability distributions and two theoretical models for arrival rates. Examined cancellation rates using L1 norms.
result Modelled daily, weekly, and monthly arrival rates of limit orders in the first fifteen bid and ask price levels.
Uniform convergence of interpolators proven for Gaussian data.
problem Interpolation learning in high-dimensional linear regression with Gaussian data.
method Generic uniform convergence guarantee in terms of Gaussian width.
result Consistency of interpolators for minimum-norm and near-minimal-norm cases.
A fast algorithm for L1-norm kernel PCA with convergence analysis.
problem Finding an optimal solution for L1-norm kernel PCA due to its non-convexity and non-smoothness.
method A fixed-point type algorithm that iteratively computes binary weights for each observation, based on a geometrically interpretable reformulation of the problem.
result The algorithm converges to a local optimal solution in a finite number of steps and the sequence of objective values converges at a linear rate.
Smoothed analysis shows that many classes become learnable from positive-only samples.
problem Learning from positive-only samples is challenging due to negative results in worst-case settings.
method Smoothed analysis of positive-only learning, assuming samples from a reference distribution smooth with respect to the true distribution.
result All VC classes become learnable in the smoothed model with O(VC/ε2) positive samples for ε classification error. We study the deformation of the three-dimensional conformal structures by the Ricci flow. We drive the evolution equation of Cotton-York tensor and the L1-norm of it under the Ricci flow. In particular, we investigate the behavior of the L1-norm of the Cotton-York tensor under the Ricci flow on three-dimensional simply…
New method solves rank-1 L1-norm TUCKER2 decomposition efficiently.
problem Exact solution to rank-1 L1-norm TUCKER2 decomposition of tensors.
method Proved equivalent to combinatorial optimization, derived two algorithms.
result L1-TUCKER2 outperforms other methods in tensor approximation for outlier-corrupted data.
Minimal SVM reduces support vectors for better classification.
problem Finding optimal hyperplane for classification with fewer support vectors.
method Proposes a Minimal SVM using L0.5 norm on slack variables.
result Increases classification performance by reducing support vectors.
A new algorithm calculates L1-norm principal components efficiently for large datasets.
problem Efficiently calculating L1-norm principal components for large datasets.
method Suboptimal algorithm with cost comparable to standard PCA, using bit flipping.
result Achieves higher L1-PC optimization metric than alternatives.
Approximate dynamic programming is a popular method for solving large Markov decision processes. This paper describes a new class of approximate dynamic programming (ADP) methods- distributionally robust ADP-that address the curse of dimensionality by minimizing a pessimistic bound on the policy loss. This approach tur…
Targeting at sparse learning, we construct Banach spaces B of functions on an input space X with the properties that (1) B possesses an l1 norm in the sense that it is isometrically isomorphic to the Banach space of integrable functions on X with respect to the counting measure; (2) point evaluations are continuous lin…
To recover a sparse signal from an underdetermined system, we often solve a constrained L1-norm minimization problem. In many cases, the signal sparsity and the recovery performance can be further improved by replacing the L1 norm with a "weighted" L1 norm. Without any prior information about nonzero elements of the si…
New theoretical framework improves error rates for sparse learning with convex regularization.
problem Improving error rates for sparse learning with convex regularization.
method Proposed a new theoretical framework using common assumptions to derive high-dimensional estimation bounds.
result Improved error rates for L1, Slope, and Group L1-L2 regularizations, matching or exceeding existing results.
Two sparsity-aware NSAF algorithms improve sparse system identification with lower complexity.
problem Sparse system identification with improved performance and lower complexity.
method Gradient descent method to minimize combined cost function and l1-norm penalty on filter coefficients.
result Proposed algorithms achieve comparable performance with lower computational complexity.
Vertex distortion measures how far lattice knots deviate from straight lines.
problem Measuring how much lattice knots deviate from straight paths.
method Analogous to smooth knots, study vertex distortion in lattice knots.
result Vertex distortion is 1 only for the unknot and can be arbitrarily high.
Proposes a method to emulate sparse priors using L1 regularization without complex transformations.
problem Sparse priors in under-determined estimation problems.
method Parameter transform to emulate sparse priors under L2 regularization.
result L1 regularization can be achieved with a remapping of parameters under normal priors.
Network anomaly detection is still a vibrant research area. As the fast growth of network bandwidth and the tremendous traffic on the network, there arises an extremely challengeable question: How to efficiently and accurately detect the anomaly on multiple traffic? In multi-task learning, the traffic consisting of flo…
Proposes a new SVM model for binary classification with theoretical and practical advantages.
problem Binary classification in supervised learning.
method Quadratic surface support vector machine with L1 norm regularization.
result The model can detect true sparsity patterns and is efficient for both synthetic and real data.
Develops efficient method for nonconvex problems using Regula Falsi.
problem Nonconvex inverse problems with likelihood constraints.
method Regula Falsi root-finding techniques applied to level-set formulations.
result Proves extension of level-set methods to nonconvex problems.
Develops high-dimensional measurement error models for non-linear loss functions.
problem Measurement errors in ultrahigh-dimensional biomedical data.
method Lipschitz loss functions, L1 norm minimization, Lasso analog.
result Improved accuracy in classification and quantile regression problems.
Study shows how varying levels of supervision and orthonormality constraints affect generalization errors in subspace fitting.
problem Effects of varying levels of supervision and orthonormality constraints on generalization errors in subspace fitting.
method Flexible family of problems connecting unsupervised and supervised subspace fitting tasks, explored over a supervision-orthonormality plane.
result Generalization errors of subspace fitting problems follow double descent trends as they become more supervised and less orthonormally constrained.
Noise injection before gradient steps helps in regularization for neural networks.
problem Improving generalization in overparametrized neural networks.
method Injecting small noise perturbations before computing gradient steps, especially in layer-wise fashion.
result Small noise perturbations can explicitly regularize neural networks without variance explosion.
Study evaluates different mathematical models for three case studies using statistical fitting.
problem Estimating outcomes in population dynamics, temperature variations, and market equilibrium.
method Applied various statistical equations (e.g., fractional exponential, sinusoidal) to three case studies.
result Optimal models differ by case study (fractional exponential for population dynamics, sinusoidal for temperature and market equilibrium).
New algorithm targets nonsmooth constraints for robust data interpolation and denoising.
problem Robust data interpolation and denoising with large outliers and varying amplitudes.
method Flexible algorithmic framework targeting nonsmooth level-set constraints (L1, Linf, L0 norms).
result Improved robustness to large outliers and significant amplitude variations in seismic data.
This paper addresses the problem of sparsity penalized least squares for applications in sparse signal processing, e.g. sparse deconvolution. This paper aims to induce sparsity more strongly than L1 norm regularization, while avoiding non-convex optimization. For this purpose, this paper describes the design and use of…
Estimates hybrid dynamical systems with polynomial expansions and Markovian switching.
problem Identifying hybrid dynamical systems with nonlinear autoregressive exogenous (NARX) components and Markovian switching.
method Probabilistic framework using Expectation Maximization for parameter estimation, including submodel coefficients, hidden state values, and transition probabilities. Disentangles mode classification and NARX regression tasks. Uses soft-labels and coordinate descent approach for parameter fitting.
result Demonstrated on a SMNARX problem with three nonlinear sub-models, achieving parsimonious models through l1-norm bridge estimation and hard-thresholding.
A novel approach to regression fitting over various quantiles of target variable.
problem Error between predicted and actual values has varying behavior across quantiles of dependent variable.
method Segmented behavior understanding and retrospective fitting based on each quantile behavior.
result Significantly improved eccentric behavior of error distance between predicted and actual values.
For supervised and unsupervised learning, positive definite kernels allow to use large and potentially infinite dimensional feature spaces with a computational cost that only depends on the number of observations. This is usually done through the penalization of predictor functions by Euclidean or Hilbertian norms. In …
New method for sparse data using L1-NMF with improved sparsity control.
problem Sparse data with false zeros and heavy-tailed noise.
method Component-wise L1-NMF with weighted penalization and coordinate descent.
result Effective in handling sparse data with false zeros.
AHS framework selects hyperparameters for FQE with error guarantees.
problem Hyperparameter selection for FQE is challenging and affects its utility.
method AHS framework defines optimality criteria without hyperparameters.
result Error bounds match empirical observations.
Most of the existing methods for sparse signal recovery assume a static system: the unknown signal is a finite-length vector for which a fixed set of linear measurements and a sparse representation basis are available and an L1-norm minimization program is solved for the reconstruction. However, the same representation…
New method stabilizes FQE by reweighting Bellman targets.
problem Stability guarantees for FQE often rely on Bellman completeness, which can fail with function approximation.
method Proposes stationary-weighted FQE, reweighting Bellman targets by stationary target-to-behavior density ratio.
result Proves finite-sample linear convergence to stationary projected Bellman fixed point without Bellman completeness.
Near-convex archetypal analysis improves interpretability and fitting error in NMF.
problem High data fitting error in traditional archetypal analysis.
method Introduces near-convex archetypal analysis (NCAA) that combines AA and NMF.
result NCAA achieves lower data fitting error than state-of-the-art methods.
When the in-sample Sharpe ratio is obtained by optimizing over a k-dimensional parameter space, it is a biased estimator for what can be expected on unseen data (out-of-sample). We derive (1) an unbiased estimator adjusting for both sources of bias: noise fit and estimation error. We then show (2) how to use the adjust…
Suppose that two large, multi-dimensional data sets are each noisy measurements of the same underlying random process, and principle components analysis is performed separately on the data sets to reduce their dimensionality. In some circumstances it may happen that the two lower-dimensional data sets have an inordinat…
Paper proposes a new method for recovering missing samples in images.
problem Missing sample recovery in image signals.
method Iterative sparse recovery algorithm using constrained l1-norm minimization with a new CSIM fidelity metric. result Simulation results demonstrate the efficiency of the proposed method.
Iterative algorithms are ubiquitous in the field of data mining. Widely known examples of such algorithms are the least mean square algorithm, backpropagation algorithm of neural networks. Our contribution in this paper is an improvement upon this iterative algorithms in terms of their respective performance metrics an…
This study evaluates Lx-norm penalties for resolving complex LC-MS data.
problem Resolving complex LC-MS data with rotational ambiguity.
method Simulated LC-MS data and grid search strategy to compare L0-, L1-, and L2-norm penalties.
result L1-norm penalty (Lasso) provides more sparse solutions and reduces rotational ambiguity.