Paper accelerates NMF algorithms using extrapolation.
problem Efficiently solving nonnegative matrix factorization problems.
method Inspired by convex optimization, uses extrapolation for two-block exact coordinate descent algorithms.
result Significantly accelerates NMF algorithms on various data sets.
Proposes a partitioned least squares model for feature grouping.
problem Modeling with feature groups to assess variable importance.
method Two methods: alternating least squares and exact reformulation.
result Exact method provides better results in less time.
This paper compares LSM and ANN/GBM for pricing American put options under a complex model.
problem Pricing American put options using advanced techniques.
method Least-Squares Monte Carlo (LSM) and Artificial Neural Network (ANN) and Gradient Boosted Machine (GBM) Trees.
result LSM outperforms ANN and GBM in pricing American put options.
New algorithm for online collaborative filtering using linear bandits and alternating least squares.
problem Online collaborative filtering with item recommendations over time.
method Combines linear bandits and alternating least squares for matrix factorization.
result Superior performance in cumulative regret and average cumulative NDCG over state-of-the-art algorithms.
Private ALS method improves matrix completion with tighter rates and better privacy.
problem Differential privacy in matrix completion for user-level privacy.
method Joint differentially private ALS method with tighter sample complexity and privacy trade-offs.
result Achieves nearly optimal sample complexity and best privacy/utility trade-off.
A new method speeds up ALS for recommender systems by subsampling key elements.
problem High computational cost of ALS for large-scale datasets.
method Core-elements subsampling method for efficient ALS approximation.
result Achieves similar accuracy with significantly reduced computational time.
Introduces BMF for efficient matrix factorization of large data.
problem Efficiently factorizing large scale matrices with limited memory.
method Uses block matrix approach and factorization at a block level.
result Demonstrates faster convergence on large matrices.
Proposes expectile matrix factorization for skewed data analysis.
problem Skewed and extreme data cannot be explained by least squares-based matrix factorization.
method Introduces asymmetric least squares into matrix factorization framework and proposes an efficient algorithm.
result The proposed scheme achieves lower recovery errors than least squares-based methods in synthetic and real-world data.
Estimates network structure and interaction rules from multiple agent trajectories.
problem Modeling multi-agent systems on networks from data.
method Jointly infers network topology and interaction kernels using non-convex optimization.
result ORALS estimator is consistent and asymptotically normal under coercivity conditions.
New method improves accuracy of compressed least-squares solutions.
problem Error and noise introduced by compression in least-squares problems.
method Proposes robust partially-compressed least-squares models and algorithms.
result Robust solutions more accurate than classical compressed variants.
New IRLS algorithms for SVM fitting via MM approach.
problem Fitting support vector machines (SVMs) via quadratic programming.
method Majorization--Minimization (MM) paradigm for iteratively-reweighted least-squares (IRLS) algorithms.
result IRLS algorithms for SVM risk minimization problems with various losses and penalties.
New tensor method for faster, more accurate convolutional dictionary learning.
problem Learning shift-invariant models with additional invariances.
method Developed novel tensor decomposition algorithms based on alternating least squares with efficient projections onto circulant matrices.
result Our method converges faster and more accurately to the dictionary compared to existing methods.
The ratio of two probability densities can be used for solving various machine learning tasks such as covariate shift adaptation (importance sampling), outlier detection (likelihood-ratio test), and feature selection (mutual information). Recently, several methods of directly estimating the density ratio have been deve…
OLS predictions are shown to be similar to attention mechanisms in models.
problem OLS in traditional statistics and econometrics.
method Rewriting OLS as an attention mechanism in a transformed space.
result OLS can be understood as minimizing squared prediction errors via optimal embedding and decoding.
Optimizes mixture models without parametrizing distributions using tensor decomposition.
problem Estimating conditionally-independent mixture models in high dimensions.
method Alternating least squares optimization scheme for tensor decomposition.
result Competitive performance and applicability to various models and applications.
Proposes a new tensor grid method for image completion.
problem Image completion from missing data.
method Low-rank tensor grid with two-stage density matrix renormalization group initialization and alternating least squares factorization.
result The proposed tensor grid method outperforms existing methods in image recovery accuracy.
Enhances NMF for better time series recovery and prediction using side information.
problem Reconstruct and predict electricity consumption time series.
method Extends NMF with side information, proposes HALSX algorithm.
result Improved recovery and prediction performance validated on various datasets.
Improved neural network training with ADMM for hardware compatibility.
problem Training scalable and parallelizable neural networks.
method Alternating Direction Method of Multipliers (ADMM) combined with iterative least-square methods.
result Achieved significant accuracy improvements over SGD and Adam.
We propose a stochastic approximation (SA) based method with randomization of samples for policy evaluation using the least squares temporal difference (LSTD) algorithm. Our proposed scheme is equivalent to running regular temporal difference learning with linear function approximation, albeit with samples picked unifo…
Functional PLS improves prediction and inference for scalar responses from functional predictors.
problem Estimating scalar responses from functional predictors in an ill-posed inverse problem.
method Functional partial least squares (PLS) estimator with adaptive early stopping and new tests.
result PLS attains nearly minimax-optimal convergence rates and detects local alternatives.
Unified analysis of reweighted least-squares algorithms for linear models.
problem Recovering unknown signals from linear measurements using reweighted least squares.
method Unified asymptotic analysis of IRLS, lin-RFM, and alternating minimization algorithms.
result The algorithms can achieve favorable performance in a few iterations with appropriate reweighting.
New AMP algorithms reveal phase transitions in tensor recovery.
problem Understanding algorithmic behavior of low-rank tensor decompositions.
method Derive Bayesian AMP algorithms and use dynamic mean field theory.
result Reveals phase transitions between easy, hard, and impossible inference regimes.
Paper presents novel online MTL methods using WRLS and OSLSSVR.
problem Online Multi-Task Learning (MTL) Regression Problems
method Develops recursive versions of WRLS and OSLSSVR for MTL.
result Achieves exact and approximate recursions with quadratic cost.
Develops a Gaussian-based message-passing algorithm for noisy matrix completion.
problem Noisy matrix completion problems.
method Gaussian parameterized belief propagation algorithm.
result Advantages over traditional methods in non-Gaussian noise scenarios.
New algorithm extracts shared latent space for cortico-muscular interactions.
problem Challenges of high dimensionality and limited sample sizes in multivariate cortico-muscular analysis.
method Structured and sparse partial least squares coherence (ssPLSC) algorithm.
result ssPLSC achieves competitive or better performance in scenarios with limited sample sizes and high noise levels.
Paper proposes a new method to improve classification performance over PCA.
problem Improving classification performance over PCA for classification problems.
method Minimizes the difference in margin distribution between original and projected data.
result Proposed method typically outperforms PCA in classification tasks.
Combines BTEM and T-PLS for accurate spectral recovery and calibration.
problem Calibrating pure spectra of minority components in mixtures without prior knowledge.
method Band target entropy minimization (BTEM) and target partial least squares (T-PLS).
result Estimated amounts from BTEM-T-PLS similar to MCR-ALS on simple mixtures, superior on complex ones.
Bayesian l0-regularized least squares for high-dimensional predictors.
problem Optimizing a non-convex objective function over model space.
method Spike-and-slab priors with single Best Replacement (SBR) for scalability.
result SBR can find the spike-and-slab estimator, bridging Bayesian regularization and proximal updating.
MPI-FAUN tackles NMF for big data, offering scalable parallel algorithms.
problem Efficient parallel algorithms for NMF on big data.
method MPI-based framework for NMF, solving alternating NLS subproblems.
result Significant performance improvements over baseline implementations.
This study examines the relationship between PLS and OLS regression using eigenvalue distributions.
problem Analyzing the difference between PLS and OLS regression in terms of eigenvalue distributions.
method Examined the distance between PLS and OLS regression coefficients using the Mahalanobis distance and eigenvalue distributions of the regressor covariance matrix.
result Provided a bound on the distance between PLS and OLS regression coefficients that depends only on the eigenvalue distribution of the regressor covariance matrix.
Estimates manifold dimension using local graph structure.
problem Estimating the intrinsic dimension of manifolds from data.
method Regression on local PCA coordinates, focusing on local graph structure.
result Proposed QE and TLS estimators outperform existing methods.
Federated learning is protected against adversarial attacks with residual-based reweighting.
problem Adversarial attacks on federated learning's aggregation process.
method Residual-based reweighting combined with iteratively reweighted least squares.
result Our aggregation algorithm outperforms other methods in label-flipping and backdoor attacks.
We propose directed time series regression, a new approach to estimating parameters of time-series models for use in certainty equivalent model predictive control. The approach combines merits of least squares regression and empirical optimization. Through a computational study involving a stochastic version of a well …
KANOP uses KANs to efficiently price American options.
problem Efficiently pricing American options with limited data.
method Combines KANs with LSMC to estimate continuation value.
result KANOP provides more accurate option value estimates.
A novel AIRLS algorithm for multiaffine variable relations in high-dimensional problems.
problem Challenges in Maximum Likelihood Estimation in high-dimensional settings with complex variable relations.
method Proposes an Alternating and Iteratively-Reweighted Least Squares (AIRLS) algorithm for multiaffine variable relations.
result Proves convergence for problems with Generalized Normal Distributions and shows empirically super-linear convergence rate.
Paper proposes monotonous NMF to handle scale and ordering ambiguities in nonnegative matrix factorization.
problem Scale and ordering ambiguities in nonnegative matrix factorization.
method Alternating least-squares algorithm for monotonous NMF, relaxed mixing matrix assumption.
result Monotonous NMF estimates monotonous signals better than standard NMF when sources are monotonous.
The telegraph process models a random motion with finite velocity and it is usually proposed as an alternative to diffusion models. The process describes the position of a particle moving on the real line, alternatively with constant velocity +v or −v. The changes of direction are governed by an homogeneous Poisso…
A new tensor completion method using tensor networks with Tucker wrapper.
problem Low-rank tensor completion in various applications.
method Solving LRTC as a system of nonlinear equations using a two-level alternative least squares method.
result The method converges to the exact solution at a linear rate with high probability.
New algorithms improve NMF for extracting patterns from time series data.
problem Extracting short-lived temporal motifs from high-dimensional time series data.
method Extended HALS and ANLS algorithms for CNMF model.
result Improved performance on large-scale data compared to multiplicative updates.
New method guarantees simultaneous decomposition of tensor components.
problem Existing methods fail to recover all tensor components simultaneously.
method S-ASI method using slicing initialization and subspace iterations.
result Guaranteed recovery of top r components simultaneously for symmetric tensors.
Study shows how optimization affects deep neural network performance as model size increases.
problem Understanding the performance of deep neural networks as model size increases.
method Careful study of learning dynamics for least squares scenario, providing an excess risk bound.
result Excess risk bound depends on the smallest non-zero eigenvalue of the covariance matrix of input features, showing double descent behavior.
We consider the problem of sparse coding, where each sample consists of a sparse linear combination of a set of dictionary atoms, and the task is to learn both the dictionary elements and the mixing coefficients. Alternating minimization is a popular heuristic for sparse coding, where the dictionary and the coefficient…
ADMM-Softmax improves classification accuracy for multiclass problems.
problem Multinomial logistic regression for classification tasks with many examples and features.
method Alternating direction method of multipliers (ADMM) for decoupling and solving the problem into efficient steps.
result ADMM-Softmax leads to improved generalization compared to other methods on two image classification problems.
Paper tackles multivariate shape-constrained convex regression problems.
problem Fitting a convex function to data with component-wise monotonicity and uniform Lipschitz continuity.
method Least squares estimator via solving a constrained convex quadratic programming problem. Efficient algorithms designed: sGS-ADMM and pALM.
result Both proposed algorithms outperform state-of-the-art methods in numerical experiments.
Develops a cumulant-based algorithm for optimizing investment portfolios.
problem Optimizing investment portfolios with low variability in non-Gaussian data.
method Alternating Least Square method applied to 2nd-6th cumulants of multidimensional random variables.
result The algorithm outperforms benchmarks and other methods during recent crashes.
New algorithm trains ReLU networks via alternating minimization.
problem Training deep neural networks with ReLU activations.
method Alternating minimization of activation patterns and weight updates.
result Proves linear convergence for recovering true parameters.
Paper proposes robust LAD estimators for 2D sinusoidal model, proving consistency and normality.
problem Estimation of parameters in 2D sinusoidal models with outliers or heavy-tailed noise.
method Least absolute deviation (LAD) estimators for robust parameter estimation.
result Strong consistency and asymptotic normality of LAD estimators for 2D sinusoidal model parameters.
New algorithm for tensor factorization with outlying slabs.
problem Factoring low-rank tensors with outlying slabs in real-world data.
method Group-sparsity promoting formulation and alternating optimization framework.
result Proposed algorithm converges and performs well on various real-world data.