Proves well-posedness for hard phase model in general relativity.
problem Modeling fluid dynamics in curved spacetime.
method A priori estimates and well-posedness in Sobolev spaces, coupled interior-boundary system of wave equations.
result Proves existence and uniqueness of solutions in Sobolev spaces.
An algorithmically hard phase was described in a range of inference problems: even if the signal can be reconstructed with a small error from an information theoretic point of view, known algorithms fail unless the noise-to-signal ratio is sufficiently small. This hard phase is typically understood as a metastable bran…
We find optimal learning rate schedules for a random feature model.
problem Choosing optimal learning rates for deep learning models.
method We analyze a powerlaw random feature model trained with SGD, considering optimal schedules as numerical and analytical problems.
result We discover two regimes: easy and hard phases, with different optimal learning rate schedules.
Meta-learning strategy improves few-shot classification performance.
problem Few-shot classification with deep neural networks struggles when labeled samples are limited.
method Proposes an easy-to-hard expert meta-training strategy to arrange training tasks based on task hardness.
result Meta-learners achieve better results with the proposed expert training strategy.
Study shows it's impossible to count communities without finding them.
problem Determining the number and sizes of communities in random graph models.
method Hypothesis testing between models with different community structures, using low-degree polynomial framework.
result Testing between two different planted distributions is as hard as finding the communities.
Tyler's M-estimator's phase transition at DS-SNR = 1 is resolved.
problem Robust Subspace Recovery
method Tyler's M-estimator
result TME converges exactly to the true subspace for DS-SNR >= 1 under a new stability condition.
Paper eliminates warm-up phase for PO in linear MDPs, achieving optimal regret.
problem Costly warm-up phase in PO algorithms for linear MDPs.
method Simple contraction mechanism replaces warm-up phase.
result Achieves rate-optimal regret with improved dependence on problem parameters.
Predicting labels of nodes in a network, such as community memberships or demographic variables, is an important problem with applications in social and biological networks. A recently-discovered phase transition puts fundamental limits on the accuracy of these predictions if we have access only to the network topology…
Combinatorial auctions are formulated as frustrated lattice gases on sparse random graphs, allowing the determination of the optimal revenue by methods of statistical physics. Transitions between computationally easy and hard regimes are found and interpreted in terms of the geometric structure of the space of solution…
Boolean logic used for neural network training and inference, with convergence analysis.
problem Discrete optimization in neural networks with Boolean logic.
method Boolean logic backpropagation with convergence analysis.
result First convergence analysis for Boolean logic in neural networks.
Sharp results link DLN gradient flow to basis pursuit optimization and GHA phase transitions.
problem Understanding implicit regularization in Diagonal Linear Networks.
method Sharp convergence bounds and characterization of ℓ1 minimizers. result Gradient flow of DLNs with tiny initialization approximates minimizers of basis pursuit optimization problem.
Study reveals phase transition in neural networks near interpolation.
problem Understanding generalization and learning transitions in neural networks.
method Effective theory for approximating Bayes-optimal generalisation error.
result Unveils a discontinuous phase transition between universal and specialisation phases.
Optimal learning rates decay to zero in easy tasks and maintain a warmup phase in hard tasks.
problem Optimizing learning rates under functional scaling laws for model training.
method Deriving optimal learning-rate schedules based on exponents s and β. result Sharp phase transition between easy and hard tasks, with different decay behaviors.
New PSDMF algorithms derived from PR and ARM methods.
problem Positive semidefinite matrix factorization (PSDMF) challenges.
method Design PSDMF algorithms based on phase retrieval (PR) and affine rank minimization (ARM) methods.
result New PSDMF algorithms inherit numerical properties from PR and ARM methods.
This paper develops the Jungle model in a credit portfolio framework. The Jungle model is able to model credit contagion, produce doubly-peaked probability distributions for the total default loss and endogenously generate quasi phase transitions, potentially leading to systemic credit events which happen unexpectedly …
The excluded area between a pair of two-dimensional hard particles with given relative orientation is the region in which one particle cannot be located due to the presence of the other particle. The magnitude of the excluded area as a function of the relative particle orientation plays a major role in the determinatio…
Stochastic Gradient Descent phases explained for deep networks.
problem Understanding the different regimes of SGD in deep learning.
method Teacher-student perceptron model, phase diagram analysis.
result SGD phases separated by batch size B∗, scaling with training set size P. We consider the problem of estimating the phases of K mixed complex signals from a multichannel observation, when the mixing matrix and signal magnitudes are known. This problem can be cast as a non-convex quadratically constrained quadratic program which is known to be NP-hard in general. We propose three approaches t…
This paper proposes an end-to-end approach for single-channel speaker-independent multi-speaker speech separation, where time-frequency (T-F) masking, the short-time Fourier transform (STFT), and its inverse are represented as layers within a deep network. Previous approaches, rather than computing a loss on the recons…
The Bethe free energy approximation is reliable when convex on a submanifold, the 'Bethe box'.
problem Accuracy of the Bethe free energy approximation in probabilistic inference.
method Analysis of convexity and verification conditions based on the Bethe Hessian matrix.
result The Bethe approximation is mostly accurate if it is convex on a submanifold, the 'Bethe box'.
Many optimization problems can be cast into the maximum satisfiability (MAX-SAT) form, and many solvers have been developed for tackling such problems. To evaluate a MAX-SAT solver, it is convenient to generate hard MAX-SAT instances with known solutions. Here, we propose a method of generating weighted MAX-2-SAT insta…
Classification and regression tasks in overparameterized models show different generalization properties.
problem Comparing classification and regression in overparameterized models.
method Comparison of least-squares minimum-norm interpolation and hard-margin SVM using different loss functions.
result Interpolating solutions generalize well with 0-1 loss but not with square loss.
AIHT improves online high-dimensional quantile regression by separating support discovery and refinement.
problem Online high-dimensional quantile regression with structural sparsity.
method Adaptive Iterative Hard Thresholding (AIHT) alternates stochastic updates with adaptive hard-thresholding steps.
result AIHT achieves logarithmic regret for the sliding-window objective in high-dimensional settings.
Reduces learning periodic neural networks to lattice problems, proving hardness under cryptographic assumptions.
problem Learning single periodic neurons in noisy environments.
method Reduction to worst-case lattice problems, using LLL algorithm.
result Polynomial-time algorithms for learning these functions are hard under cryptographic assumptions.
Optimal feature transfer identified through bias-variance analysis.
problem Optimizing feature transfer in transfer learning.
method Simple linear model with fine-grained bias-variance decomposition.
result Optimal pretrained feature transform is naturally sparse.
Study on InstaHide's security, linking to phase retrieval problem.
problem Security of InstaHide scheme for private dataset sharing.
method Design of a provable algorithm for private vector recovery.
result Private vectors can be recovered using synthetic vectors and public vectors.
Adaptive algorithm identifies best arm with abstention, showing phase transition from polynomial to exponential error probability.
problem Bayesian best-arm identification with abstention to reduce undetected error.
method Adaptive algorithm PGWS that optimally uses abstention budget.
result Introducing any positive abstention budget induces an exponential decay in undetected error probability.
Recently, it was shown that there is a phase transition in the community detection problem. This transition was first computed using the cavity method, and has been proved rigorously in the case of q=2 groups. However, analytic calculations using the cavity method are challenging since they require us to understand p…
We consider the problem of sparse phase retrieval from Fourier transform magnitudes to recover the k-sparse signal vector and its support T. We exploit extended support estimate E with size larger than k satisfying E⊇T and obtained by a trained deep neural net…
Maximizes Rényi entropy for efficient exploration in reward-free RL.
problem Challenges of exploration in reward-free reinforcement learning.
method Maximizes Rényi entropy over state-action space in exploration phase; uses batch RL for planning phase.
result Effective and sample-efficient exploration leading to superior policies.
Sparse phase retrieval plays an important role in many fields of applied science and thus attracts lots of attention. In this paper, we propose a \underline{sto}chastic alte\underline{r}nating \underline{m}inimizing method for \underline{sp}arse ph\underline{a}se \underline{r}etrieval (\textit{StormSpar}) algorithm whi…
Study local convergence of GDA for training GANs with kernel-based discriminators.
problem Analyzing the local dynamics of GDA for GANs with kernel-based discriminators.
method Linearization of a non-linear dynamical system, under an isolated points model assumption.
result Showed phase transitions indicating convergence, oscillation, or divergence of GDA.
The tree reconstruction problem is to collect and analyze massive data at the nth level of the tree, to identify whether there is non-vanishing information of the root, as n goes to infinity. Its connection to the clustering problem in the setting of the stochastic block model, which has wide applications in machin…
Paper shows how to integrate quantization into neural compression models.
problem Integrating quantization into neural compression models.
method Integrates uniform noise channel at test time using universal quantization.
result Eliminates mismatch between training and test phases while maintaining differentiability.
This paper studies the problem of detecting the presence of a small dense community planted in a large Erdős-Rényi random graph G(N,q), where the edge probability within the community exceeds q by a constant factor. Assuming the hardness of the planted clique detection problem, we show that the computatio…
In this paper, we propose a general framework for tensor singular value decomposition (tensor SVD), which focuses on the methodology and theory for extracting the hidden low-rank structure from high-dimensional tensor data. Comprehensive results are developed on both the statistical and computational limits for tensor …
Tensor PCA problem analyzed with statistical query lower bounds.
problem Estimating the expected value of a rank-1 tensor from Gaussian samples.
method Sharp analysis of optimal sample complexity in the Statistical Query model.
result SQ algorithms with polynomial query complexity fail in the conjectured hard phase and have sub-optimal sample complexity.
Proposes a new model for community detection using neural priors.
problem Community detection in graphs with node attributes.
method Neural-prior stochastic block model with belief propagation and approximate message passing algorithm.
result Identifies phase transitions and algorithmically hard regions for community detection.
Neural networks learn simpler features first, then more complex ones; Fourier analysis reveals this pattern.
problem Understanding the learning dynamics of neural networks, especially with natural image data.
method Fourier analysis of translation-invariant and power-law spectra to study feature learning.
result Simple neural networks first rely on amplitude information, then phase information, and power-law spectra can accelerate learning phase information.
Paper explores limits of high-order clustering with planted structures.
problem Statistical and computational limits of high-order clustering with planted structures.
method Developed methods for detection and recovery of clusters, identified signal-to-noise ratio boundaries.
result Sharp boundaries of signal-to-noise ratio for statistical and computational feasibility.
Autodock is a widely used molecular modeling tool which predicts how small molecules bind to a receptor of known 3D structure. The current version of AutoDock uses meta-heuristic algorithms in combination with local search methods for doing the conformation search. Appropriate settings of hyperparameters in these algor…
DiTSNe-Ia model accurately reconstructs supernovae spectra from light curves.
problem Difficult identification and interpretation of diverse sub-populations of supernovae.
method Variational diffusion-based generative model conditioned on light curves.
result DiTSNe-Ia achieves significantly more accurate reconstructions than SALT3 across all phases.
New insights into neural networks reveal unexpected phase transitions.
problem Understanding how overparameterized neural networks learn and generalize.
method Statistical physics of disordered systems applied to non-convex binary neural networks.
result Atypical phase transitions lead to better generalization in neural networks.
TREK uses distillation to help students solve hard problems.
problem Stalled progress on hard prompts when current policy lacks useful reasoning trajectories.
method TREK combines distillation and reinforcement learning to expand student support.
result TREK significantly improves student performance on mathematical reasoning and agentic tasks.
Study shows how transformers learn to combine simple tasks into complex ones.
problem Understanding how transformers learn to perform complex tasks not seen during training.
method Controlled setting involving variable assignment and modular addition; partitioned training data analysis.
result Small transformers can generalize to unseen combinations of variables and numbers.
We propose an efficient protocol for decentralized training of deep neural networks from distributed data sources. The proposed protocol allows to handle different phases of model training equally well and to quickly adapt to concept drifts. This leads to a reduction of communication by an order of magnitude compared t…
Efficiently transforms Gaussian data to simulate various target distributions.
problem Generating observations from different target distributions given a single Gaussian observation.
method Designs computationally efficient procedures to approximate target distributions.
result Establishes reduction-based computational lower bounds for high-dimensional statistical models.
Polynomial-time algorithm solves random parity games with high probability.
problem Solving random parity games efficiently.
method SWCP algorithm based on cycles in subgraphs.
result Polynomial-time solution for large-degree games with high probability.