We discuss replica analytic continuation using several simple models in order to prove mathematically the validity of replica analysis, which is used in a wide range of fields related to large scale complex systems. While replica analysis consists of two analytical techniques, the replica trick (or replica analytic con…
Paper uses replica analysis to optimize net present value in investment portfolios.
problem Maximizing net present value in portfolios of multiple development projects.
method Replica analysis applied to optimization problem with budget and investment constraints.
result Replica analysis yields higher net present value than conventional methods.
Paper tackles investment risk with cost and return constraints using replica analysis.
problem Investment risk minimization under cost and return constraints.
method Replica analysis for portfolio optimization problems.
result Derivation of macroscopic theory for optimal solution.
Replica analysis assesses portfolio optimization with correlated assets.
problem Investment risk with correlated asset returns.
method Replica analysis applied to single-factor model portfolio optimization.
result Increased investment risk with correlated returns compared to independent returns.
Optimizes investment risk with cost using replica analysis.
problem Minimizing investment risk with cost.
method Replica analysis of Hamiltonians in mean-variance model.
result Derives minimal investment risk with cost and optimal portfolio investment concentration.
The portfolio optimization problem in which the variances of the return rates of assets are not identical is analyzed in this paper using the methodology of statistical mechanical informatics, specifically, replica analysis. We define two characteristic quantities of an optimal portfolio, namely, minimal investment ris…
AMP algorithm analyzes SCAD nonconvex regularization for sparse regression.
problem Sparse regression with nonconvex SCAD regularization under Gaussian data.
method Approximate message passing (AMP) algorithm for SCAD-AMP, stability and asymptotic analysis.
result SCAD-AMP achieves optimal performance and identifies phase transitions.
Replica analysis reveals dual structure in portfolio optimization.
problem Optimizing investment risk and return under constraints.
method Replica analysis in statistical mechanics.
result Optimal portfolios exhibit primal-dual structure.
Binary perceptron's instability linked to replica symmetry breaking.
problem Understanding the relationship between algorithmic instability and replica symmetry breaking in binary perceptron learning.
method Established the connection between algorithmic instability and replica symmetry breaking by comparing the instability condition around the fixed point to the instability for breaking the replica symmetric solution of the free energy function.
result The instability condition around the algorithmic fixed point is identical to the instability for breaking the replica symmetric saddle point solution of the free energy function.
The typical behavior of optimal solutions to portfolio optimization problems with absolute deviation and expected shortfall models using replica analysis was pioneeringly estimated by S. Ciliberti and M. Mézard [Eur. Phys. B. 57, 175 (2007)]; however, they have not yet developed an approximate derivation method for fin…
The paper estimates key metrics for linear models with Markov or hidden Markov sources.
problem Estimating free energy, mutual information, and MMSE for linear models with specific signal priors.
method Replica analysis in statistical physics, focusing on Markov and hidden Markov sources.
result The linear model with Markov or hidden Markov sources can be simplified into decoupled AWGN channels.
This paper optimizes deep learning training by efficiently sharding weight updates across replicas.
problem Redundant weight update computation on all replicas in data-parallel training.
method Automatic sharding of weight updates using static analysis and transformations on the training graph.
result Substantial speedups achieved on large-scale models using Cloud TPUs.
Paper uses replica method to study overfitting in Cox model.
problem Overfitting in Cox model when p ~ N.
method Replica method from statistical physics.
result Established relationship between optimal regularization and p/N.
We use variational Gaussian approximations to analyze parametric models with unknown data-generating distributions.
problem Analyzing inference and learning in parametric models with unknown or intractable data-generating distributions.
method Replica method with variational Gaussian approximation in grand canonical formalism.
result Stationarity conditions adaptively determine parameters of the trial Hamiltonian for each dataset.
This study analyzes quantization in deep learning models using statistical physics methods.
problem The computational resource requirements for large-scale data analysis models.
method Typical case analysis from statistical physics, specifically the replica method.
result Optimal quantization width minimizes error and delays overfitting.
New algorithm speeds up MCMC for deep learning models.
problem Large biases in SGMCMC for big data.
method Adaptive replica exchange SGMCMC (reSGMCMC).
result Achieves state-of-the-art results on various datasets.
Dense Associative Memories outperform classical networks in robustness and signal processing.
problem Improving neural network performance in adversarial attacks and weak signal processing.
method Relaxing replica symmetry in statistical mechanics of spin glasses to analyze unsupervised and supervised learning.
result Explicit analytical investigation of phase diagrams and storage capacities for Dense Associative Memories.
Investigates optimal portfolios with risk-free assets, minimizing investment risk.
problem Investment risk minimization with budget and return constraints.
method Replica analysis and exploration of implications of a risk-free asset.
result Implications of a risk-free asset on optimal portfolio and investment risk.
Replica exchange Langevin diffusion accelerates nonconvex optimization.
problem Nonconvex optimization challenges in machine learning.
method Replica exchange Langevin diffusion, discretization analysis.
result Replica exchange accelerates convergence to global minima.
In this paper, as a first step in examining the properties of a feasible portfolio subset that is characterized by budget and risk constraints, we assess the maximum and minimum of the investment concentration using replica analysis. To do this, we apply an analytical approach of statistical mechanics. We note that the…
RBM generates complex, graded data features.
problem Extracting complex features from high-dimensional data.
method Characterized structural conditions for RBM to generate compositional representations.
result RBM can operate in a compositional phase under specific conditions.
Improved reSGLD accelerates convergence in non-convex learning problems.
problem Inefficient swaps due to noisy energy estimators in reSGLD.
method Variance reduction for noisy energy estimators, theoretical analysis, and numerical experiments.
result Exponential acceleration in convergence for non-convex learning problems.
Paper derives a Pythagorean theorem for Sharpe ratio and validates a new method for portfolio optimization.
problem Optimizing investment risk with budget and return constraints.
method Replica analysis for portfolio optimization under non-specific probability distribution constraints.
result Derives a Pythagorean theorem for Sharpe ratio and validates new method effectiveness.
New method for Bayesian learning on large datasets using replica-exchange Nosé-Hoover dynamics.
problem Bayesian learning on complex posterior distributions with multiple isolated modes and mini-batch noise.
method Simulating replicas in parallel with different temperatures, applying Nosé-Hoover dynamics, and developing a noise-aware exchange protocol.
result Significant improvements over strong baselines in deep Bayesian neural networks on large-scale datasets.
Paper examines the relationship between maximizing and minimizing expected return in portfolio optimization.
problem Investment risk and return optimization in portfolio problems.
method Lagrange undetermined multiplier method and replica analysis.
result Derived mean square error and correlation coefficient of optimal portfolios as functions of risk tolerance.
Statistical learning theory connects to spin glass models via Rademacher complexity and replica theory.
problem Bounding generalization gap in statistical learning theory.
method Linking Rademacher complexity in statistical learning to synthetic models in statistical physics.
result Rademacher complexity is closely related to ground state energy in spin glass models.
The Lasso performs well in ultra-sparse linear models with finite support size.
problem Performance analysis of Lasso in ultra-sparse linear models.
method Novel application of replica method from statistical physics, rigorous analysis of average case performance.
result Average performance of Lasso assessed without scaling assumptions, offering sample complexity bounds.
This research highlights the secrecy potential of nonlinear generative models and their all-or-nothing phase transition.
problem Secrecy potential of nonlinear generative models in statistical learning.
method Replica method to derive asymptotic normalized cross entropy and statistical decoupling of Bayesian estimator.
result Strictly nonlinear models exhibit an all-or-nothing phase transition, leading to perfect secrecy.
In the present work, eigenvalue distributions defined by a random rectangular matrix whose components are neither independently nor identically distributed are analyzed using replica analysis and belief propagation. In particular, we consider the case in which the components are independently but not identically distri…
SOCRATES uses LLMs to automate simulation optimization of complex systems.
problem Optimizing complex, expensive-to-sample stochastic systems.
method Two-stage procedure: replica construction and meta-optimization.
result Adaptive hybrid optimization schedule for real systems.
Study dynamics of alternating minimization for bilinear regression under large system limits.
problem Understanding the time evolution of alternating minimization for bilinear regression.
method Replica method applied to a multi-temperature glassy system.
result Dynamics of alternating minimization can be described by a two-dimensional discrete stochastic process.
Dreaming neural networks learn and consolidate patterns during sleep.
problem Maximizing information storage and critical capacity in neural networks.
method Daily routine of learning during awake state and consolidation during sleep, using Guerra's interpolation techniques.
result The network achieves perfect retrieval regime after sleep, storing the same number of patterns as neurons.
Study analyzes Bayesian inference algorithms using dynamical functional approach.
problem Analysis of approximate inference algorithms for large Gaussian latent variable models.
method Dynamical functional approach to model nontrivial dependencies and obtain exact effective stochastic process.
result Closed-form expressions for the rate of convergence are derived and validated.
A fast, approximate method for variable selection in GLMs tackles correlated data.
problem Variable selection in generalized linear models with correlated data.
method Replica method of statistical mechanics and vector approximate message passing.
result The proposed algorithm provides fast convergence and high approximation accuracy.
We rigorously prove statistical physics predictions for non-convex GLMs in high dimensions.
problem Analyzing high-dimensional optimization problems in non-convex Generalized Linear Models.
method Developed a systematic framework using the Gaussian Min-Max Theorem and AMP to rigorously prove replica-symmetric formulas.
result Validated statistical physics predictions for non-convex GLMs, aligning with physicist's conjectures.
Transfer learning improves prediction quality in high-dimensional sparse regression.
problem Selecting hyperparameters for Lasso-based transfer learning algorithms.
method Asymptotic analysis using the replica method.
result Ignoring one type of transferred information has minimal impact on performance.
Proves formula for reconstruction performance in generalized linear models.
problem Analyzing reconstruction performance in generalized linear models with arbitrary bounded spectrum.
method Message passing algorithms and dynamical system stability analysis.
result Analytical formula confirms replica method conjecture for convex models.
Study analyzes sparse linear regression with SCAD penalty under noise, providing theoretical insights and practical tools.
problem Signal reconstruction in sparse linear regression with piecewise continuous nonconvex penalties.
method Theoretical analysis using replica method, development of cross-validation error formula, and annealing procedure.
result The SCAD estimator outperforms ℓ1 in a wide parameter range, with the global minimum of mean square error in the replica symmetric phase. In the present paper, the minimal investment risk for a portfolio optimization problem with imposed budget and investment concentration constraints is considered using replica analysis. Since the minimal investment risk is influenced by the investment concentration constraint (as well as the budget constraint), it is i…
PLS-SVD struggles with missing data in multimodal datasets, showing a phase transition in performance.
problem Missing data in PLS-SVD for multimodal datasets.
method Replica-symmetric analysis of spiked rectangular random matrices with missing entries.
result PLS-SVD performance transitions from uninformative to informative singular vectors at a critical signal-to-noise threshold.
2D-PT improves sampling in constrained optimization problems.
problem Sampling Boltzmann distributions with soft constraints.
method Two-dimensional extension of parallel tempering.
result 2D-PT achieves near-ideal mixing in constrained problems.
BLADE uses Bayesian methods to discover complex systems from scarce data.
problem Efficiently discovering governing equations of complex dynamical systems from limited data.
method Combines replica-exchange stochastic gradient Langevin Monte Carlo with active learning.
result Reduces measurement requirements by 60% for Lotka-Volterra and 40% for Burgers' equation.
Deep networks preferentially learn shared features, avoiding memorization in early layers.
problem Understanding how deep neural networks generalize vs. memorize training data.
method Replica-based mean field geometric analysis of deep neural networks.
result Deep layers predominantly memorize, while early layers are minimally affected.
Derives asymptotic generalization error for large-margin classifiers.
problem Understanding the generalization error of large-margin classifiers.
method Statistical physics replica method for deriving asymptotic expression.
result Establishes phase transition boundary for class separability.
Proposes r2SGLD for efficient constrained exploration in non-convex learning.
problem Stagnation in high-temperature chains of reSGLD in distribution tails.
method r2SGLD: replica exchange with reflection steps in a bounded domain.
result Reflection steps enhance mixing rates with quadratic improvement in domain diameter.
New sampler tackles complex discrete energy landscapes efficiently.
problem Stagnation in gradient-based discrete samplers for non-convex settings.
method DREXEL sampler with Replica Exchange and Adjusted Metropolis.
result Proves samplers satisfy detailed balance and converge to target distribution.
New method improves sparse signal reconstruction using 1RSB-AMP.
problem Sparse signal reconstruction with improved accuracy.
method Developed 1RSB-AMP and 1RSB-SE for SCAD penalty minimization.
result 1RSB-AMP achieves improved reconstruction compared to RS-AMP.
Toy model study shows resampling/reweighting can improve feature learning in imbalanced classification.
problem Improving feature learning in imbalanced classification problems.
method High-dimensional toy model with replica method, class-wise resampling/reweighting, and simplified model.
result No resampling/reweighting can sometimes give best feature learning performance.