We use variational Gaussian approximations to analyze parametric models with unknown data-generating distributions.
problem Analyzing inference and learning in parametric models with unknown or intractable data-generating distributions.
method Replica method with variational Gaussian approximation in grand canonical formalism.
result Stationarity conditions adaptively determine parameters of the trial Hamiltonian for each dataset.
The typical behavior of optimal solutions to portfolio optimization problems with absolute deviation and expected shortfall models using replica analysis was pioneeringly estimated by S. Ciliberti and M. Mézard [Eur. Phys. B. 57, 175 (2007)]; however, they have not yet developed an approximate derivation method for fin…
Validates replica trick for simple models using replica analytic continuation.
problem Validating replica trick for complex systems.
method Applying replica analysis to simple models, focusing on replica analytic continuation.
result Replica analytic continuation is a robust procedure in replica analysis.
Review of tools from RMT for estimating large covariance matrices.
problem Estimating large covariance matrices from noisy data.
method Random Matrix Theory (RMT) methods and analytical techniques.
result Rotationally Invariant Estimators (RIE) are superior to existing methods.
Dense Associative Memories outperform classical networks in robustness and signal processing.
problem Improving neural network performance in adversarial attacks and weak signal processing.
method Relaxing replica symmetry in statistical mechanics of spin glasses to analyze unsupervised and supervised learning.
result Explicit analytical investigation of phase diagrams and storage capacities for Dense Associative Memories.
Replica exchange Langevin diffusion accelerates nonconvex optimization.
problem Nonconvex optimization challenges in machine learning.
method Replica exchange Langevin diffusion, discretization analysis.
result Replica exchange accelerates convergence to global minima.
Study on generalisation in random feature learning and hidden manifold models.
problem Generalisation in high-dimensional learning problems.
method Replica method from statistical physics for asymptotic generalisation performance.
result Closed-form expression for generalisation performance in various high-dimensional settings.
Paper uses replica analysis to optimize net present value in investment portfolios.
problem Maximizing net present value in portfolios of multiple development projects.
method Replica analysis applied to optimization problem with budget and investment constraints.
result Replica analysis yields higher net present value than conventional methods.
This paper optimizes deep learning training by efficiently sharding weight updates across replicas.
problem Redundant weight update computation on all replicas in data-parallel training.
method Automatic sharding of weight updates using static analysis and transformations on the training graph.
result Substantial speedups achieved on large-scale models using Cloud TPUs.
New method for Bayesian learning on large datasets using replica-exchange Nosé-Hoover dynamics.
problem Bayesian learning on complex posterior distributions with multiple isolated modes and mini-batch noise.
method Simulating replicas in parallel with different temperatures, applying Nosé-Hoover dynamics, and developing a noise-aware exchange protocol.
result Significant improvements over strong baselines in deep Bayesian neural networks on large-scale datasets.
Paper tackles investment risk with cost and return constraints using replica analysis.
problem Investment risk minimization under cost and return constraints.
method Replica analysis for portfolio optimization problems.
result Derivation of macroscopic theory for optimal solution.
Optimizes investment risk with cost using replica analysis.
problem Minimizing investment risk with cost.
method Replica analysis of Hamiltonians in mean-variance model.
result Derives minimal investment risk with cost and optimal portfolio investment concentration.
Paper optimizes portfolios with non-identical asset return variances using statistical mechanics.
problem Optimizing portfolios with assets having different return variance.
method Replica analysis of statistical mechanical informatics.
result Asymptotic behaviors of minimal investment risk and concentrated investment level determined analytically.
Replica analysis assesses portfolio optimization with correlated assets.
problem Investment risk with correlated asset returns.
method Replica analysis applied to single-factor model portfolio optimization.
result Increased investment risk with correlated returns compared to independent returns.
AMP algorithm analyzes SCAD nonconvex regularization for sparse regression.
problem Sparse regression with nonconvex SCAD regularization under Gaussian data.
method Approximate message passing (AMP) algorithm for SCAD-AMP, stability and asymptotic analysis.
result SCAD-AMP achieves optimal performance and identifies phase transitions.
Statistical learning theory connects to spin glass models via Rademacher complexity and replica theory.
problem Bounding generalization gap in statistical learning theory.
method Linking Rademacher complexity in statistical learning to synthetic models in statistical physics.
result Rademacher complexity is closely related to ground state energy in spin glass models.
Binary perceptron's instability linked to replica symmetry breaking.
problem Understanding the relationship between algorithmic instability and replica symmetry breaking in binary perceptron learning.
method Established the connection between algorithmic instability and replica symmetry breaking by comparing the instability condition around the fixed point to the instability for breaking the replica symmetric solution of the free energy function.
result The instability condition around the algorithmic fixed point is identical to the instability for breaking the replica symmetric saddle point solution of the free energy function.
The replica method solves mean-variance portfolio optimization without symmetry assumptions.
problem Mean-variance portfolio optimization for a generic covariance matrix.
method Replica method from statistical physics applied to optimization problem.
result Replica symmetry emerges as the unique solution of the optimization problem.
SOCRATES uses LLMs to automate simulation optimization of complex systems.
problem Optimizing complex, expensive-to-sample stochastic systems.
method Two-stage procedure: replica construction and meta-optimization.
result Adaptive hybrid optimization schedule for real systems.
Study analyzes eigenvalue distributions of non-i.i.d. Wishart matrices using replica analysis and belief propagation.
problem Eigenvalue distribution of non-i.i.d. Wishart matrices.
method Replica analysis and belief propagation.
result Determines asymptotic eigenvalue distribution and proposes an algorithm based on belief propagation.
A fast, approximate method for variable selection in GLMs tackles correlated data.
problem Variable selection in generalized linear models with correlated data.
method Replica method of statistical mechanics and vector approximate message passing.
result The proposed algorithm provides fast convergence and high approximation accuracy.
Replica analysis reveals dual structure in portfolio optimization.
problem Optimizing investment risk and return under constraints.
method Replica analysis in statistical mechanics.
result Optimal portfolios exhibit primal-dual structure.
The paper estimates key metrics for linear models with Markov or hidden Markov sources.
problem Estimating free energy, mutual information, and MMSE for linear models with specific signal priors.
method Replica analysis in statistical physics, focusing on Markov and hidden Markov sources.
result The linear model with Markov or hidden Markov sources can be simplified into decoupled AWGN channels.
New algorithm speeds up MCMC for deep learning models.
problem Large biases in SGMCMC for big data.
method Adaptive replica exchange SGMCMC (reSGMCMC).
result Achieves state-of-the-art results on various datasets.
2D-PT improves sampling in constrained optimization problems.
problem Sampling Boltzmann distributions with soft constraints.
method Two-dimensional extension of parallel tempering.
result 2D-PT achieves near-ideal mixing in constrained problems.
BLADE uses Bayesian methods to discover complex systems from scarce data.
problem Efficiently discovering governing equations of complex dynamical systems from limited data.
method Combines replica-exchange stochastic gradient Langevin Monte Carlo with active learning.
result Reduces measurement requirements by 60% for Lotka-Volterra and 40% for Burgers' equation.
Proposes r2SGLD for efficient constrained exploration in non-convex learning.
problem Stagnation in high-temperature chains of reSGLD in distribution tails.
method r2SGLD: replica exchange with reflection steps in a bounded domain.
result Reflection steps enhance mixing rates with quadratic improvement in domain diameter.
Derives a recursion formula for irregular spectral curves.
problem Calculating the mean of irregular spectral curves.
method Variant of replica method by Brezín and Hikami, generalized to generalized Laguerre polynomial case.
result Derives a recursion formula for special times where terms are polynomials.
The paper analyzes maximizing and minimizing investment concentration under budget and risk constraints.
problem Maximizing and minimizing investment concentration with budget and risk constraints.
method Replica analysis and the method of steepest descent based on Lagrange's method of undetermined multipliers.
result Optimal solutions are verified to be dual to the portfolio optimization problem.
Recent experimental advances in neuroscience have opened new vistas into the immense complexity of neuronal networks. This proliferation of data challenges us on two parallel fronts. First, how can we form adequate theoretical frameworks for understanding how dynamical network processes cooperate across widely disparat…
Develops a new theory for neural systems stability and width effects.
problem Stability and finite-width effects in deep neural systems.
method Gauge-covariant stochastic effective field theory using classical commuting fields.
result Predicts the edge of chaos and low-frequency spectral deformation.
New sampler tackles complex discrete energy landscapes efficiently.
problem Stagnation in gradient-based discrete samplers for non-convex settings.
method DREXEL sampler with Replica Exchange and Adjusted Metropolis.
result Proves samplers satisfy detailed balance and converge to target distribution.
Paper uses replica method to study overfitting in Cox model.
problem Overfitting in Cox model when p ~ N.
method Replica method from statistical physics.
result Established relationship between optimal regularization and p/N.
Improved reSGLD accelerates convergence in non-convex learning problems.
problem Inefficient swaps due to noisy energy estimators in reSGLD.
method Variance reduction for noisy energy estimators, theoretical analysis, and numerical experiments.
result Exponential acceleration in convergence for non-convex learning problems.
Study binary perceptrons' capacity using random duality theory.
problem Characterize the capacity of binary perceptrons with general thresholds.
method Utilized fully lifted random duality theory (fl RDT) to characterize the capacity.
result Characterizations match replica symmetry breaking predictions and uncover the capacity for zero-threshold scenario.
New method improves convergence and reduces variance in noisy optimization problems.
problem Computing exact minimizers with noisy gradient information.
method Stochastic mirror descent with interacting particles.
result Interaction helps improve convergence and reduce variance.
We use a replica approach to deal with portfolio optimization problems. A given risk measure is minimized using empirical estimates of asset values correlations. We study the phase transition which happens when the time series is too short with respect to the size of the portfolio. We also study the noise sensitivity o…
Paper analyzes minimal investment risk with budget and concentration constraints.
problem Minimal investment risk in portfolio optimization with budget and concentration constraints.
method Replica analysis to consider the minimal investment risk.
result Minimal investment risk with concentration constraint is larger than without.
This research highlights the secrecy potential of nonlinear generative models and their all-or-nothing phase transition.
problem Secrecy potential of nonlinear generative models in statistical learning.
method Replica method to derive asymptotic normalized cross entropy and statistical decoupling of Bayesian estimator.
result Strictly nonlinear models exhibit an all-or-nothing phase transition, leading to perfect secrecy.
ADA augments data using AR replicas for robust regression.
problem Improving robustness in nonlinear over-parametrized regression.
method Extends Anchor regression (AR) for data augmentation, using replicas of modified samples.
result ADA provides more robust regression predictions compared to state-of-the-art solutions.
Paper introduces supervised and unsupervised TAM models for binary neurons.
problem Learning and retrieval of structured triplets of patterns in neural networks.
method Extends Hebbian paradigm to supervised and unsupervised protocols, using glassy statistical mechanical techniques.
result Obtained self-consistency equations for critical dataset sizes and retrieval performance.
We rigorously prove statistical physics predictions for non-convex GLMs in high dimensions.
problem Analyzing high-dimensional optimization problems in non-convex Generalized Linear Models.
method Developed a systematic framework using the Gaussian Min-Max Theorem and AMP to rigorously prove replica-symmetric formulas.
result Validated statistical physics predictions for non-convex GLMs, aligning with physicist's conjectures.
This study analyzes quantization in deep learning models using statistical physics methods.
problem The computational resource requirements for large-scale data analysis models.
method Typical case analysis from statistical physics, specifically the replica method.
result Optimal quantization width minimizes error and delays overfitting.
Investigates optimal portfolios with risk-free assets, minimizing investment risk.
problem Investment risk minimization with budget and return constraints.
method Replica analysis and exploration of implications of a risk-free asset.
result Implications of a risk-free asset on optimal portfolio and investment risk.
Paper proposes a new algorithm combining gradient descent and Langevin dynamics.
problem Gradient descent can get stuck in local minima, while Langevin dynamics can explore but is slow.
method Replica exchange mechanism swaps positions if Langevin yields a lower objective function.
result New algorithm converges to global minimum linearly with high probability.
RBM generates complex, graded data features.
problem Extracting complex features from high-dimensional data.
method Characterized structural conditions for RBM to generate compositional representations.
result RBM can operate in a compositional phase under specific conditions.
Study analyzes sparse linear regression with SCAD penalty under noise, providing theoretical insights and practical tools.
problem Signal reconstruction in sparse linear regression with piecewise continuous nonconvex penalties.
method Theoretical analysis using replica method, development of cross-validation error formula, and annealing procedure.
result The SCAD estimator outperforms ℓ1 in a wide parameter range, with the global minimum of mean square error in the replica symmetric phase. Proves formula for reconstruction performance in generalized linear models.
problem Analyzing reconstruction performance in generalized linear models with arbitrary bounded spectrum.
method Message passing algorithms and dynamical system stability analysis.
result Analytical formula confirms replica method conjecture for convex models.