Validates replica trick for simple models using replica analytic continuation.
problem Validating replica trick for complex systems.
method Applying replica analysis to simple models, focusing on replica analytic continuation.
result Replica analytic continuation is a robust procedure in replica analysis.
Replica analysis assesses portfolio optimization with correlated assets.
problem Investment risk with correlated asset returns.
method Replica analysis applied to single-factor model portfolio optimization.
result Increased investment risk with correlated returns compared to independent returns.
Dense Associative Memories outperform classical networks in robustness and signal processing.
problem Improving neural network performance in adversarial attacks and weak signal processing.
method Relaxing replica symmetry in statistical mechanics of spin glasses to analyze unsupervised and supervised learning.
result Explicit analytical investigation of phase diagrams and storage capacities for Dense Associative Memories.
We use variational Gaussian approximations to analyze parametric models with unknown data-generating distributions.
problem Analyzing inference and learning in parametric models with unknown or intractable data-generating distributions.
method Replica method with variational Gaussian approximation in grand canonical formalism.
result Stationarity conditions adaptively determine parameters of the trial Hamiltonian for each dataset.
AMP algorithm analyzes SCAD nonconvex regularization for sparse regression.
problem Sparse regression with nonconvex SCAD regularization under Gaussian data.
method Approximate message passing (AMP) algorithm for SCAD-AMP, stability and asymptotic analysis.
result SCAD-AMP achieves optimal performance and identifies phase transitions.
Proves formula for reconstruction performance in generalized linear models.
problem Analyzing reconstruction performance in generalized linear models with arbitrary bounded spectrum.
method Message passing algorithms and dynamical system stability analysis.
result Analytical formula confirms replica method conjecture for convex models.
Optimizes investment risk with cost using replica analysis.
problem Minimizing investment risk with cost.
method Replica analysis of Hamiltonians in mean-variance model.
result Derives minimal investment risk with cost and optimal portfolio investment concentration.
Paper optimizes portfolios with non-identical asset return variances using statistical mechanics.
problem Optimizing portfolios with assets having different return variance.
method Replica analysis of statistical mechanical informatics.
result Asymptotic behaviors of minimal investment risk and concentrated investment level determined analytically.
A fast, approximate method for variable selection in GLMs tackles correlated data.
problem Variable selection in generalized linear models with correlated data.
method Replica method of statistical mechanics and vector approximate message passing.
result The proposed algorithm provides fast convergence and high approximation accuracy.
The paper analyzes maximizing and minimizing investment concentration under budget and risk constraints.
problem Maximizing and minimizing investment concentration with budget and risk constraints.
method Replica analysis and the method of steepest descent based on Lagrange's method of undetermined multipliers.
result Optimal solutions are verified to be dual to the portfolio optimization problem.
Study on signal recovery from low-rank matrix with sparse noise.
problem Inference of a rank-one signal in the presence of sparse noise.
method Replica method from statistical physics, recursive distributional equations, population dynamics algorithm.
result Critical signal strength for recovery via top eigenvector identified.
SOCRATES uses LLMs to automate simulation optimization of complex systems.
problem Optimizing complex, expensive-to-sample stochastic systems.
method Two-stage procedure: replica construction and meta-optimization.
result Adaptive hybrid optimization schedule for real systems.
Improved reSGLD accelerates convergence in non-convex learning problems.
problem Inefficient swaps due to noisy energy estimators in reSGLD.
method Variance reduction for noisy energy estimators, theoretical analysis, and numerical experiments.
result Exponential acceleration in convergence for non-convex learning problems.
Statistical physics helps solve complex machine learning problems.
problem Large dimensional inference problems in machine learning.
method Replica symmetric level analysis and cavity methods.
result General framework for solving various problems with weak long-range interactions.
Study analyzes Bayesian inference algorithms using dynamical functional approach.
problem Analysis of approximate inference algorithms for large Gaussian latent variable models.
method Dynamical functional approach to model nontrivial dependencies and obtain exact effective stochastic process.
result Closed-form expressions for the rate of convergence are derived and validated.
Using generating functional and replica techniques, respectively, we study the dynamics and statics of a spherical Minority Game (MG), which in contrast with a spherical MG previously presented in J.Phys A: Math. Gen. 36 11159 (2003) displays a phase with broken ergodicity and dependence of the macroscopic stationary s…
Study analyzes sparse linear regression with SCAD penalty under noise, providing theoretical insights and practical tools.
problem Signal reconstruction in sparse linear regression with piecewise continuous nonconvex penalties.
method Theoretical analysis using replica method, development of cross-validation error formula, and annealing procedure.
result The SCAD estimator outperforms ℓ1 in a wide parameter range, with the global minimum of mean square error in the replica symmetric phase. Study on how kernel regression models generalize to out-of-distribution data.
problem Understanding generalization in machine learning models under distributional shifts.
method Replica method from statistical physics to derive analytical formula for generalization error.
result Identified overlap matrix as key determinant of generalization performance under distribution shift.
Replica exchange Langevin diffusion accelerates nonconvex optimization.
problem Nonconvex optimization challenges in machine learning.
method Replica exchange Langevin diffusion, discretization analysis.
result Replica exchange accelerates convergence to global minima.
Analyzes impermanent loss in decentralized exchanges and provides a replication formula.
problem Impermanent loss in decentralized exchanges like Uniswap and Balancer.
method Analytical static replication formula using European calls and puts.
result Guaranteed coverage for pool value within a predefined range.
We rigorously prove statistical physics predictions for non-convex GLMs in high dimensions.
problem Analyzing high-dimensional optimization problems in non-convex Generalized Linear Models.
method Developed a systematic framework using the Gaussian Min-Max Theorem and AMP to rigorously prove replica-symmetric formulas.
result Validated statistical physics predictions for non-convex GLMs, aligning with physicist's conjectures.
Analytic method optimizes portfolio variance with asymmetric ℓ1 constraint.
problem Optimizing portfolio variance under budget and asymmetric ℓ1 constraints. method Replica method from disordered systems theory.
result Regularization extends optimization interval and suppresses large sample fluctuations.
We study high-dimensional Gaussian mixture classification using statistical physics methods.
problem Classifying high-dimensional Gaussian mixture with general covariance matrices.
method Replica method from statistical physics for asymptotic analysis of convex classifiers.
result Construction and validation of a de-biased estimator for variable selection.
Paper uses replica analysis to optimize net present value in investment portfolios.
problem Maximizing net present value in portfolios of multiple development projects.
method Replica analysis applied to optimization problem with budget and investment constraints.
result Replica analysis yields higher net present value than conventional methods.
This paper optimizes deep learning training by efficiently sharding weight updates across replicas.
problem Redundant weight update computation on all replicas in data-parallel training.
method Automatic sharding of weight updates using static analysis and transformations on the training graph.
result Substantial speedups achieved on large-scale models using Cloud TPUs.
New method for Bayesian learning on large datasets using replica-exchange Nosé-Hoover dynamics.
problem Bayesian learning on complex posterior distributions with multiple isolated modes and mini-batch noise.
method Simulating replicas in parallel with different temperatures, applying Nosé-Hoover dynamics, and developing a noise-aware exchange protocol.
result Significant improvements over strong baselines in deep Bayesian neural networks on large-scale datasets.
Paper tackles investment risk with cost and return constraints using replica analysis.
problem Investment risk minimization under cost and return constraints.
method Replica analysis for portfolio optimization problems.
result Derivation of macroscopic theory for optimal solution.
Review of mean-field methods for neural network inference.
problem Understanding neural network learning from a theoretical perspective.
method Mean-field methods, high-temperature expansions, replica method, message passing algorithms.
result Equivalences and complementarities of mean-field methods.
Statistical physics approaches can be used to derive accurate predictions for the performance of inference methods learning from potentially noisy data, as quantified by the learning curve defined as the average error versus number of training examples. We analyse a challenging problem in the area of non-parametric inf…
Study on generalisation in random feature learning and hidden manifold models.
problem Generalisation in high-dimensional learning problems.
method Replica method from statistical physics for asymptotic generalisation performance.
result Closed-form expression for generalisation performance in various high-dimensional settings.
Statistical learning theory connects to spin glass models via Rademacher complexity and replica theory.
problem Bounding generalization gap in statistical learning theory.
method Linking Rademacher complexity in statistical learning to synthetic models in statistical physics.
result Rademacher complexity is closely related to ground state energy in spin glass models.
Binary perceptron's instability linked to replica symmetry breaking.
problem Understanding the relationship between algorithmic instability and replica symmetry breaking in binary perceptron learning.
method Established the connection between algorithmic instability and replica symmetry breaking by comparing the instability condition around the fixed point to the instability for breaking the replica symmetric solution of the free energy function.
result The instability condition around the algorithmic fixed point is identical to the instability for breaking the replica symmetric saddle point solution of the free energy function.
The study calculates the injectivity capacity of ReLU networks using a novel mathematical approach.
problem Determining the injectivity capacity of ReLU networks layers.
method Employing fully lifted random duality theory (fl RDT) to handle the ℓ0 spherical perceptron and implicitly the ReLU layers injectivity. result The lifting mechanism converges remarkably fast with relative corrections not exceeding 0.1%.
The replica method solves mean-variance portfolio optimization without symmetry assumptions.
problem Mean-variance portfolio optimization for a generic covariance matrix.
method Replica method from statistical physics applied to optimization problem.
result Replica symmetry emerges as the unique solution of the optimization problem.
Deep networks preferentially learn shared features, avoiding memorization in early layers.
problem Understanding how deep neural networks generalize vs. memorize training data.
method Replica-based mean field geometric analysis of deep neural networks.
result Deep layers predominantly memorize, while early layers are minimally affected.
Review of tools from RMT for estimating large covariance matrices.
problem Estimating large covariance matrices from noisy data.
method Random Matrix Theory (RMT) methods and analytical techniques.
result Rotationally Invariant Estimators (RIE) are superior to existing methods.
Study analyzes eigenvalue distributions of non-i.i.d. Wishart matrices using replica analysis and belief propagation.
problem Eigenvalue distribution of non-i.i.d. Wishart matrices.
method Replica analysis and belief propagation.
result Determines asymptotic eigenvalue distribution and proposes an algorithm based on belief propagation.
Replica analysis reveals dual structure in portfolio optimization.
problem Optimizing investment risk and return under constraints.
method Replica analysis in statistical mechanics.
result Optimal portfolios exhibit primal-dual structure.
The paper estimates key metrics for linear models with Markov or hidden Markov sources.
problem Estimating free energy, mutual information, and MMSE for linear models with specific signal priors.
method Replica analysis in statistical physics, focusing on Markov and hidden Markov sources.
result The linear model with Markov or hidden Markov sources can be simplified into decoupled AWGN channels.
Loopy belief propagation (LBP), which is equivalent to the Bethe approximation in statistical mechanics, is a message-passing-type inference method that is widely used to analyze systems based on Markov random fields (MRFs). In this paper, we propose a message-passing-type method to analytically evaluate the quenched a…
New algorithm speeds up MCMC for deep learning models.
problem Large biases in SGMCMC for big data.
method Adaptive replica exchange SGMCMC (reSGMCMC).
result Achieves state-of-the-art results on various datasets.
Injectivity of ReLU networks studied using statistical physics.
problem When can the input of a ReLU neural network be inferred from its output?
method Connection to spherical integral geometry and statistical physics.
result Replica symmetry-breaking theory and Gordon's min--max theorem provide insights into the injectivity threshold.
The typical behavior of optimal solutions to portfolio optimization problems with absolute deviation and expected shortfall models using replica analysis was pioneeringly estimated by S. Ciliberti and M. Mézard [Eur. Phys. B. 57, 175 (2007)]; however, they have not yet developed an approximate derivation method for fin…
Study investigates learning performance in inverse Ising problems with sparse teacher couplings.
problem Learning performance in inverse Ising problems with sparse teacher couplings.
method Pseudolikelihood maximization method, replica and cavity methods from statistical mechanics.
result Perfect inference of teacher's couplings is possible in the thermodynamic limit for certain conditions.
New neural networks model complex phenomena with fewer parameters.
problem Challenges in studying higher-order interactions in neural networks.
method Introducing curved neural networks using the maximum entropy principle.
result Curved neural networks accelerate memory retrieval and exhibit explosive phase transitions.
2D-PT improves sampling in constrained optimization problems.
problem Sampling Boltzmann distributions with soft constraints.
method Two-dimensional extension of parallel tempering.
result 2D-PT achieves near-ideal mixing in constrained problems.
BLADE uses Bayesian methods to discover complex systems from scarce data.
problem Efficiently discovering governing equations of complex dynamical systems from limited data.
method Combines replica-exchange stochastic gradient Langevin Monte Carlo with active learning.
result Reduces measurement requirements by 60% for Lotka-Volterra and 40% for Burgers' equation.
Study high-dimensional logistic regression with missing data, providing exact error characterizations.
problem High-dimensional logistic regression with missing or corrupted covariates.
method Exact characterizations of prediction and estimation errors under independence and moment conditions.
result Characterizations are universal and hold for various imputation strategies.