Paper proves supermodularity of AG-SSL objective and proposes a greedy sampling algorithm.
problem Improving semi-supervised learning with limited labeled data.
method Proves supermodularity of AG-SSL objective under Stieltjes regularization and proposes a greedy sampling algorithm.
result Proposed method achieves superior classification accuracy compared to state-of-the-art methods.
Efficient methods for Lévy models using SINH-regular processes.
problem Efficient numerical methods for evaluating Lévy models.
method Defining SL-processes and sSL-processes, deriving properties of characteristic exponent, and showing all popular Lévy processes can be subordinated to Brownian motion.
result All crucial properties of characteristic exponent are consequences of a specific representation, and all popular Lévy processes are SL- or sSL-subordinated Brownian motion.
The paper connects Riemannian Gaussian distributions to random matrix theory and diffusion kernels.
problem Analyzing Riemannian Gaussian distributions on symmetric spaces.
method Analytical computation of marginals using orthogonal and skew orthogonal polynomials, and diffusion kernels.
result Riemannian Gaussian distributions are random matrix types, and their probability density functions can be computed analytically.
Study shows deterministic equivalent for neural network kernel convergence.
problem Understanding convergence of neural network kernels.
method Analyzes empirical spectral distribution of Conjugate Kernel, proving convergence to a deterministic limit.
result Obtains a deterministic equivalent for the Stieltjes transform and resolvent of the Conjugate Kernel.
Researchers create minimal surfaces with Scherk ends and find catenoid limits.
problem Constructing minimal surfaces with specific end types and understanding their limits.
method Constructing families of embedded, singly periodic minimal surfaces with Scherk-type ends and analyzing their limits.
result The limit of the constructed surfaces are catenoid necks connecting planes, determined by Stieltjes polynomials.
Paper defines quasi-Strebel structures for meromorphic k-differentials and proves their existence.
problem Existence of quasi-Strebel structures for meromorphic k-differentials.
method Introduced quasi-Strebel structures and proved their existence for meromorphic k-differentials.
result Every differential of even order k > 2 satisfying certain conditions admits a quasi-Strebel structure.
New method extrapolates spectral densities from smaller models to larger ones.
problem Limited practical computations for large machine learning models.
method Algebraic spectral curve theory for free decompression.
result Framework enables extrapolation of spectral densities with multiple or multi-modal bulks.
Study spectral density of neural networks using resolvent method.
problem Investigate spectral density of neural networks with random feature matrices.
method Use resolvent method from random matrix theory, cumulant expansion.
result Impossible to preserve singular value distribution with additive bias.
We show that a trader, who starts with no initial wealth and is not allowed to borrow money or short sell assets, is theoretically able to attain positive wealth by continuous trading, provided that she has perfect foresight of future asset prices, given by a continuous semimartingale. Such an arbitrage strategy can be…
New nonconvex regularizers improve low-rank matrix recovery efficiency and accuracy.
problem Efficiently recover low-rank matrices from incomplete data.
method Factor group-sparse regularization, related to Schatten-p norms.
result Improved generalization error bounds for Schatten-p norms as p decreases.
Gradient descent in deep matrix factorization favors low-rank solutions, improving recovery accuracy.
problem Understanding the generalization in deep learning models.
method Study of gradient descent over deep linear neural networks for matrix completion and sensing.
result Adding depth enhances an implicit tendency towards low-rank solutions, leading to more accurate recovery.
Efficiently preconditions machine learning problems with adaptive regularization.
problem Prohibitively expensive full-matrix adaptive regularization for large parameter problems.
method Modified full-matrix adaptive regularization with efficient inverse square root computation.
result Improved convergence rates and better solutions through careful preconditioning.
The paper studies the loss landscape of regularized deep matrix factorization, revealing unique and sharp minimizers.
problem Understanding the loss landscape and minimizers of regularized deep matrix factorization problems.
method Theoretical analysis of ℓ 2 \ell^2 ℓ 2 -regularized deep matrix factorization/deep linear network training problems with squared-error loss. result The unique end-to-end minimizer exists for all target matrices except for a set of Lebesgue measure zero.
New insights into how deep models generalize, focusing on matrix factorization.
problem Understanding how deep models generalize and why they work well.
method Using Morse functions and dynamical systems to study implicit regularization.
result Solved a conjecture on implicit regularization in matrix factorization.
Proposes a new method for selecting regularization parameters in sparse precision matrix estimation.
problem Selecting an appropriate regularization parameter for sparse precision matrix estimation.
method Developed a closed-form matrix-valued regularization parameter based on the sampling distribution of optimality conditions.
result The proposed method achieves comparable estimation accuracy and superior support recovery to cross-validation, with significant runtime improvements.
In this paper, we develop a relative error bound for nuclear norm regularized matrix completion, with the focus on the completion of full-rank matrices. Under the assumption that the top eigenspaces of the target matrix are incoherent, we derive a relative upper bound for recovering the best low-rank approximation of t…
Gradient descent on matrix factorization leads to minimum nuclear norm solution.
problem Optimizing underdetermined quadratic objectives over matrices.
method Gradient descent on a full dimensional factorization of the matrix.
result Gradient descent converges to the minimum nuclear norm solution.
New nonconvex regularizer speeds up low-rank matrix completion.
problem Low-rank matrix completion with good theoretical and empirical performance.
method Proposes a new nonconvex regularizer with adaptive shrinkage, scalable, and fast optimization.
result Proposed method achieves state-of-the-art recovery performance and is the fastest.
Researchers developed a rigorous mathematical formulation of functional integration for fields on paracompact manifolds.
problem Overcoming the problematic aspect of measures in the space of fields.
method Using Schwartz-Sobolev spaces, open coverings with subordinate partition-of-unity test functions, and convolution operations.
result Validated the basic assumption of differential geometry that fields live on differentiable manifolds.
New proof shows norms can't explain deep learning's implicit regularization.
problem Understanding the implicit regularization in deep learning.
method Mathematical proof on matrix factorization problems.
result Implicit regularization drives norms towards infinity, suggesting rank minimization is key.
Solutions to a quadratic matrix equation are linked to strongly regular graphs and multiplicative characters.
problem Solving a specific quadratic matrix equation in Riemannian geometry.
method Constructing nonzero solutions using group rings and multiplicative characters of finite fields.
result Solutions relate to strongly regular graphs and multiplicative characters of finite fields.
Improved covariance matrix estimation for multiple classes with limited data.
problem Estimating covariance matrices for multiple classes with scarce data.
method Coupled regularized sample covariance matrix estimator (RSCM) that combines pooled SCM and scaled identity matrix for regularization.
result The coupled RSCM estimators outperform cross-validation in classification tasks with comparable accuracy but faster computation.
SNN architecture shows gradient descent converges to regularized solution in matrix sensing problems.
problem Understanding implicit regularization in neural networks for matrix sensing.
method Developed Spectral Neural Networks (SNN) for matrix learning problems, rigorously demonstrating implicit regularization.
result Gradient descent converges to the solution of a regularized learning problem in matrix sensing problems.
New algorithm speeds up matrix learning with nonconvex regularizers.
problem Efficiently learn low-rank matrices using nonconvex regularizers.
method Developed an efficient proximal gradient algorithm with inexact proximal splitting.
result Achieved a convergence rate of O(1/T) and nearly linear speedup.
A novel BMC model with nonconvex regularizers and accelerated proximal algorithm for binary matrix completion.
problem Recovering a binary matrix from partial observed positive elements.
method Proposes a novel BMC model with nonconvex regularizers and accelerates proximal algorithm for solving the nonconvex optimization problem.
result The proposed model and algorithm outperform other methods in both synthetic and real-world data sets.
The paper analyzes error bounds and KL properties for noisy matrix recovery problems.
problem Noisy low-rank matrix recovery problems.
method Squared F-norm regularization, accelerated alternating minimization method.
result Established error bounds and KL properties for critical points and global minimizers.
This paper proposes robust matrix variate regression models with rank constraints and vector regularization.
problem High dimensional and noisy matrix-valued predictors in regression models.
method Rank constraint, vector regularization, alternating projected gradient descent algorithm.
result The proposed method achieves the minimax rate of estimation errors.
Paper tackles low-rank matrix recovery with column ℓ 2 , 0 \ell_{2,0} ℓ 2 , 0 -norm regularization.
problem Low-rank matrix recovery problems with column sparsity constraints.
method Developed alternating majorization-minimization (AMM) methods with extrapolation and hybrid AMM.
result Global convergence analysis and superior performance in matrix completion problems.
Dropout improves matrix factorization by acting as a low-rank regularizer.
problem Improving matrix factorization performance through regularization.
method Using Bernoulli random variables to drop columns of factors, demonstrating equivalence to a deterministic model with sum of squared Euclidean norms.
result Dropout achieves the global minimum of a convex approximation problem with squared nuclear norm regularization.
Paper proposes a method to improve autoencoder-based matrix completion models.
problem Overfitting in autoencoder-based models with limited training data.
method Data-dependent regularization using manifold learning as an auxiliary task.
result The proposed model achieves high reconstruction accuracy in well-known datasets.
Improved gradient descent for rectangular matrix completion without ℓ 2 , ∞ \ell_{2,\infty} ℓ 2 , ∞ regularization.
problem Nonconvex rectangular matrix completion without ℓ 2 , ∞ \ell_{2,\infty} ℓ 2 , ∞ regularization. method Gradient Descent without ℓ 2 , ∞ \ell_{2,\infty} ℓ 2 , ∞ regularization. result Improved sampling rate from O ( poly ( κ ) μ 3 r 3 log 3 n / n ) O(\operatorname{poly}(κ)μ^3 r^3 \log^3 n/n ) O ( poly ( κ ) μ 3 r 3 log 3 n / n ) to O ( μ 2 r 2 κ 14 log n / n ) O(μ^2 r^2 κ^{14} \log n/n ) O ( μ 2 r 2 κ 14 log n / n ) . Factored gradient descent finds unique rank-r solution in PSD matrix sensing.
problem Finding a unique rank-r solution in over-parameterized matrix sensing.
method Factored gradient descent with PSD constraints.
result PSD constraint alone leads to a unique rank-r matrix recovery.
Dropout improves matrix factorization by controlling factor size.
problem Understanding regularization properties of dropout for matrix factorization.
method Theoretical analysis of dropout's equivalence to a deterministic model with adaptive dropout rates.
result Dropout's regularization effect is limited by the fixed dropout rate, suggesting adaptive rates.
Method improves clarity in forecasting spatio-temporal data.
problem Forecasting spatio-temporal data with clarity and interpretability.
method Supervised semi-nonnegative matrix factorization with frequency regularization.
result Method offers clearer interpretability in forecasting spatio-temporal data.
Spectral methods are popular in detecting global structures in the given data that can be represented as a matrix. However when the data matrix is sparse or noisy, classic spectral methods usually fail to work, due to localization of eigenvectors (or singular vectors) induced by the sparsity or noise. In this work, we …
The paper examines how gradient descent stabilizes low-rank matrix factorization in noisy conditions.
problem Stability of low-rank implicit regularization in perturbed deep matrix factorization.
method Derives spectral conditions for gradient descent to exhibit a low-rank phase in noiseless settings and analyzes perturbed dynamics.
result Gradient descent converges to a low-rank solution under perturbation, with explicit dependence on perturbation size.
DeepVir uses deep matrix factorization to predict antivirals for COVID-19.
problem Predicting effective antivirals for COVID-19 using known drug-virus associations.
method Graphical deep matrix factorization with HyPALM optimization.
result DeepVir outperforms state-of-the-art techniques in predicting antivirals for COVID-19.
AIR-Net adapts low-rank regularization dynamically for better image completion.
problem Fixed low-rank regularization limits adaptability to different images.
method AIR-Net uses adaptive and implicit regularization parameterized by a dynamic Laplacian matrix.
result AIR-Net enhances implicit regularization and outperforms fixed methods in non-uniform missing data scenarios.
Matrix Chernoff bound for Markov chains applied to co-occurrence matrices.
problem Analyzing the behavior of co-occurrence statistics in sequential data.
method Proved a matrix Chernoff-type bound for sums of matrix-valued random variables sampled via a regular Markov chain.
result Achieved exponentially fast convergence rate and sample complexity analysis for co-occurrence matrices.
Matrix factorization generates investment recommendations for investors.
problem Generating accurate investment recommendations for investors.
method Used matrix factorization and an iterative conjugate gradient method to optimize investment recommendations.
result Achieved highest average prediction accuracy of 13.3% for investors.
Over the past few years, trace regression models have received considerable attention in the context of matrix completion, quantum state tomography, and compressed sensing. Estimation of the underlying matrix from regularization-based approaches promoting low-rankedness, notably nuclear norm regularization, have enjoye…
Method estimates noise transition matrix from noisy labels without relying on unreliable class-posterior estimation.
problem Estimating noise transition matrix from noisy data.
method Total variation regularization to encourage distinguishable predicted probabilities.
result Consistent estimator of the noise transition matrix under mild assumptions.
Gradient descent implicitly regularizes over-parameterized matrix factorization and neural networks with quadratic activations.
problem Implicit regularization in over-parameterized models with quadratic activations.
method Gradient descent applied to parameterizing U U o p UU^ op U U o p with U ∈ R d i m e s d U\in \mathbb R^{d imes d} U ∈ R d im es d to recover a rank r r r positive semidefinite matrix X ⋆ X^{\star} X ⋆ . result Gradient descent recovers X ⋆ X^{\star} X ⋆ in i l d e O ( r ) ilde{O}(\sqrt{r}) i l d e O ( r ) iterations starting from a small initialization. Sparse group matrix completion reduces complexity and improves performance.
problem Matrix completion with non-informative side features.
method Group-Lasso regularization for feature selection in matrix factorization.
result Theoretical sample complexity is significantly lower than competitors.
Derives adjoint formulas for matrix operations and applies them to specific cases.
problem Computing adjoints for matrix operations and specific matrix types.
method Derives adjoint formulas for matrix operations and applies them to specific cases.
result Closed-form expressions for adjoints in specific matrix types.
Solves weakly supervised regression using low-rank approximations and manifold regularization.
problem Weakly supervised regression with known, unknown, and uncertain labels.
method Combines manifold regularization and low-rank matrix decomposition for optimization.
result Improves solution quality and stability for large datasets.
Paper tackles low-rank matrix recovery with KL property and DC reformulation.
problem Low-rank matrix recovery with coarse rank estimation.
method Adds ℓ 2 , 0 \ell_{2,0} ℓ 2 , 0 -norm and balanced terms to factorized loss function; establishes KL property and DC reformulations. result Establishes KL property of exponent 1 / 2 1/2 1/2 for the composite function and its global minimizers. New framework explains why nonconvex methods work well in low-rank matrix estimation.
problem Nonconvex low-rank matrix estimation problems in machine learning.
method Developed a theoretical framework revealing a benign regularizer.
result Nonconvex procedures can behave well due to a disguised convexity.