Study efficient power iteration for tensor models, proving convergence under specific conditions.
problem Simultaneous alternating power iteration for fixed-order asymmetric rank-one spiked tensor models.
method Finite-iteration local theory, geometrically decaying transient, fixed-order multilinear noise event, warm-start mechanism.
result Convergence to the unique informative local fixed point under specific conditions.
A new algorithm reduces communication in distributed SVD by p p p factors.
problem Efficiently compute SVD in distributed systems.
method LocalPower algorithm with weighted aggregation and periodic decay of iterations.
result Reduces communication cost by a factor of p p p . A non-convex algorithm recovers low-rank matrices from one-bit labels efficiently.
problem Learning with one-bit labels in multi-label scenarios.
method Formulated as one-bit rank-one matrix sensing, developed an alternating power iteration algorithm.
result Achieves linear convergence and nearly optimal sampling complexity.
A stable approach to eigendecomposition for deep learning networks.
problem Numerical instability in backpropagation of eigendecomposition results.
method A numerically stable and differentiable approach to eigendecomposition.
result Better robustness of the new approach over standard methods for ZCA whitening and PCA denoising.
Alternating Minimization is a widely used and empirically successful heuristic for matrix completion and related low-rank optimization problems. Theoretical guarantees for Alternating Minimization have been hard to come by and are still poorly understood. This is in part because the heuristic is iterative and non-conve…
Sharp analysis of power iteration for tensor PCA, improving convergence and stopping criteria.
problem Analyzing the power iteration algorithm for tensor PCA to improve convergence and stopping criteria.
method Sharp bounds on the number of iterations, revealing a smaller algorithmic threshold, proposing a stopping criterion.
result Sharp bounds on the number of iterations required for power method to converge, revealing a smaller algorithmic threshold than previously conjectured.
This paper proposes an alternating back-propagation algorithm for learning the generator network model. The model is a non-linear generalization of factor analysis. In this model, the mapping from the continuous latent factors to the observed signal is parametrized by a convolutional neural network. The alternating bac…
FedPower improves eigenspace estimation privacy in federated learning.
problem Privacy breaches and communication challenges in federated eigenspace estimation.
method FedPower uses a power method with local power iterations and global aggregation, weighted by OPT, and adds Gaussian noise for privacy.
result FedPower provides convergence bounds and demonstrates effectiveness in experiments.
Paper refutes conjecture on tensor power iteration convergence in overcomplete models.
problem Understanding convergence of tensor power iteration in overcomplete random tensors.
method Analysis of tensor power iteration dynamics from random initialization.
result Polynomially many steps are necessary for convergence, refutes logarithmic conjecture.
New method guarantees simultaneous decomposition of tensor components.
problem Existing methods fail to recover all tensor components simultaneously.
method S-ASI method using slicing initialization and subspace iterations.
result Guaranteed recovery of top r components simultaneously for symmetric tensors.
Efficiently compress pretrained models using RSI for improved predictive accuracy.
problem Efficiently compressing large pretrained models for practical deployment.
method Randomized subspace iteration (RSI) for low-rank approximation of pretrained models.
result RSI achieves near-optimal approximation quality and outperforms RSVD in predictive accuracy.
Tensor CANDECOMP/PARAFAC (CP) decomposition has wide applications in statistical learning of latent variable models and in data mining. In this paper, we propose fast and randomized tensor CP decomposition algorithms based on sketching. We build on the idea of count sketches, but introduce many novel ideas which are un…
We propose a new method for robust PCA -- the task of recovering a low-rank matrix from sparse corruptions that are of unknown value and support. Our method involves alternating between projecting appropriate residuals onto the set of low-rank matrices, and the set of sparse matrices; each projection is {\em non-convex…
Deep neural networks improve real-time power system state estimation and forecasting.
problem Real-time monitoring of power grids with large-scale renewable generation and electric vehicles.
method Developed a novel model-specific DNN for real-time PSSE and used deep RNNs for forecasting.
result Improved performance compared to existing alternatives, including Gauss-Newton PSSE solver.
New method explains GNNs using power iteration clustering.
problem Mysterious mechanism of message passing in GNNs.
method Subspace power iteration clustering (SPIC) models.
result Message passing in GNNs can be understood through power iteration.
Improved clustering algorithm for large datasets.
problem Finding alternative partitions in large datasets.
method Iterative Spectral Method (ISM) for alternative clustering.
result Significantly improved scalability and computation time.
In this paper, we provide local and global convergence guarantees for recovering CP (Candecomp/Parafac) tensor decomposition. The main step of the proposed algorithm is a simple alternating rank- 1 1 1 update which is the alternating version of the tensor power iteration adapted for asymmetric tensors. Local convergence g…
New method learns collective variables using autoencoders for molecular simulations.
problem Learning low-dimensional slow degrees of freedom (collective variables) for molecular simulations.
method Iterative method involving CV learning with autoencoders and reweighting scheme.
result Achieves convergence of learned collective variables.
Paper uses SC to estimate hidden interference for WSRM.
problem Maximizing sum-rate with hidden interfering sources.
method Synthetic control (SC) for estimating counterfactual interference in WMMSE.
result SC-WMMSE outperforms original WMMSE in convergence and objective.
The paper analyzes the variance of different shuffling methods in stochastic gradient descent.
problem Understanding the variance of different shuffling methods in stochastic gradient descent.
method Power spectral density analysis to study the noise sequences of stochastic gradients.
result The stationary variances of iterates decrease in the order of SGD, SGD-RR, and SGD-SO.
NumPyro accelerates probabilistic programming with JAX transformations.
problem Efficiently handling probabilistic models with hardware acceleration.
method Composable effects and program transformations in JAX.
result Iterative NUTS formulation JIT compiled for speed.
Robust PCA reduces to power iterations for outlier-resilient feature extraction.
problem Sensitivity of PCA to non-Gaussian samples and outliers.
method Robust formulation of PCA based on maximum correntropy criterion.
result MCPI reduces to power iterations, making PCA more robust to outliers.
SCI-PI solves scale invariant problems efficiently.
problem Solving scale invariant problems in optimization.
method Introduces SCI-PI and proves its convergence.
result SCI-PI achieves local linear convergence.
A simple power iteration with momentum achieves optimal PCA in stochastic settings.
problem Accelerating PCA in the stochastic setting with limited data.
method A simple variant of the power iteration with momentum.
result Achieves optimal sample and iteration complexity of O ( 1 / Δ ) \mathcal{O}(1/\sqrt{Δ}) O ( 1/ Δ ) . A method to give users control over automated decisions by enumerating decision subspaces.
problem Users lack control over automated decision-making processes.
method Formalizes the problem as an evasion attack and uses subspace enumeration.
result Implemented for decision forests, showing how to map the problem to k k k -clique enumeration. New algorithm solves minimax games with linear constraints.
problem Nonconvex minimax games with coupled linear constraints.
method Primal-dual alternating proximal gradient (PDAPG) algorithm.
result Achieves ε-stationary solution within O(ε^(-2)) iterations for strongly concave settings.
Testing independence is of significant interest in many important areas of large-scale inference. Using extreme-value form statistics to test against sparse alternatives and using quadratic form statistics to test against dense alternatives are two important testing procedures for high-dimensional independence. However…
Characterizes a specific type of neural network for alternating group equivariance.
problem Understanding and characterizing neural networks with alternating group equivariance.
method Characterization of all possible A n A_n A n -equivariant neural networks using tensor powers of R n \mathbb{R}^{n} R n . result Found a basis of matrices for learnable, linear A n A_n A n -equivariant layer functions. Nonparametric two sample testing deals with the question of consistently deciding if two distributions are different, given samples from both, without making any parametric assumptions about the form of the distributions. The current literature is split into two kinds of tests - those which are consistent without any a…
Fast algorithm recovers principal eigenvector from noisy matrices.
problem Recovering the first principal eigenvector from noisy positive semidefinite matrices.
method Cone projected power iteration algorithm.
result Achieves polynomial time complexity and small error for certain convex cones.
We present a novel analysis of the dynamics of tensor power iterations in the overcomplete regime where the tensor CP rank is larger than the input dimension. Finding the CP decomposition of an overcomplete tensor is NP-hard in general. We consider the case where the tensor components are randomly drawn, and show that …
Paper proposes P-ADMM for ADMM in distributed medical machine learning with differential privacy.
problem Privacy leakage in ADMM for distributed machine learning with sensitive data.
method Integrates Gaussian noise with linearly decaying variance to provide dynamic zCDP.
result P-ADMM achieves the same convergence rate as non-private ADMM while ensuring differential privacy.
ParPIC clusters directed graphs using random walks and diffusion operators.
problem Challenges in vertex-level clustering for directed graphs due to edge directionality.
method Parametrized Power-Iteration Clustering (ParPIC) based on reversible random walks and diffusion operators.
result ParPIC achieves competitive clustering accuracy with improved scalability compared to spectral and teleportation-based methods.
AGD converges in polynomial iterations to optimal matrix factorization.
problem Matrix factorization optimization with alternating gradient descent.
method Alternating gradient descent with fixed step size, proving convergence in polynomial iterations.
result AGD reaches ε-optimal factorization in T iterations with high probability.
New algorithms solve complex minimax problems without needing derivatives.
problem Solving nonconvex-concave minimax problems efficiently.
method Zeroth-order alternating and proximal gradient algorithms.
result Iteration complexity and function value estimation bounds established.
Critical volatility triggers log-normal to power-law transitions in interconnected systems.
problem Understanding the transition from log-normal to power-law distributions in interconnected systems.
method Analyzing an infinite option-on-option chain model, deriving a critical volatility threshold.
result A critical volatility threshold of approximately 250.66% for unconditional cases, dropping to 125.3% with selective survival.
Study spectral learning for odeco tensors, addressing initialization bottlenecks.
problem Recovering orthogonally decomposable tensors under noise.
method Investigates perturbation bounds, non-convex optimization, and initialization strategies.
result Initialization is the main bottleneck for efficient algorithms.
This chapter deals with decentralized learning algorithms for in-network processing of graph-valued data. A generic learning problem is formulated and recast into a separable form, which is iteratively minimized using the alternating-direction method of multipliers (ADMM) so as to gain the desired degree of paralleliza…
Efficiently estimates Cox model coefficients without sharing data.
problem Privacy and ownership concerns in multi-center biomedical studies.
method Communication-efficient iterative distributed algorithms for estimation and inference.
result Achieves convergence rate of full-sample estimator with minimal iterations.
A new deep learning model speeds up MRI by reconstructing from undersampled data.
problem Slow MRI due to undersampling in k-space.
method Unrolling primal-dual hybrid gradient algorithm into a deep network, gradually relaxing constraints.
result Superior MR reconstructions from highly undersampled data.
Implicit models can match or exceed explicit models with more test-time compute.
problem Understanding the expressive power and scaling of implicit models.
method Nonparametric analysis of expressive power, mathematical characterization of implicit operators, and test-time scaling experiments.
result Implicit models can progressively express more complex mappings through iteration, matching a richer function class with test-time compute.
New knot concordance invariants from cyclic covers of prime power.
problem Constructing new knot concordance invariants.
method Considering m-fold cyclic branched covers with m a prime power.
result Computations of new invariants for some families of knots.
QAOA matches classical tensor power iteration in spiked tensor model recovery.
problem Statistical estimation in spiked tensor model with computational gap.
method Analysis of QAOA performance on spiked tensor model.
result QAOA weak recovery threshold matches tensor power iteration.
This dissertation advances scalable Gaussian processes using iterative methods and pathwise conditioning.
problem The classical Gaussian process formulation is not scalable for large datasets and modern hardware.
method Combining iterative methods and pathwise conditioning to improve scalability.
result Significantly reduced memory requirements and facilitated application to larger datasets.
SMPI recovers tensor spikes from noisy data with improved performance.
problem Recovering tensor spikes corrupted by Gaussian noise.
method Selective Multiple Power Iterations (SMPI) with polynomial random initializations and symmetrized tensor power iterations.
result SMPI outperforms existing algorithms and approaches theoretical optimal recovery.
Enhances single-step adversarial training to defend against iterative adversarial examples.
problem Defending against iterative adversarial examples in neural networks.
method Identified and leveraged empirical properties of Iter-Adv to improve Single-Adv.
result Enhanced Single-Adv to defend against iterative adversarial examples with improved accuracy and reduced training cost.
Novel algorithm for Markov decision processes using rank-one approximation.
problem Solving planning and learning problems of Markov decision processes.
method Policy iteration with rank-one approximation of transition probability matrix.
result The proposed algorithm consistently outperforms first-order algorithms and their accelerated versions.
Enhances power of covariance matrix tests for high-dimensional data.
problem Testing large covariance matrices in high-dimensional data.
method Proposes a new Fisher's combined probability test for quadratic form and maximum form statistics.
result Boosts power against more general alternatives.