Survey on learning Boolean functions in computational theory.
problem Learning Boolean function classes in computational theory.
method Overview of known results in PAC and related models.
result Discussion of various learning results for Boolean functions.
New theory challenges traditional machine learning assumptions.
problem Traditional machine learning theories are critiqued.
method A new theory is proposed and discussed.
result Learning true probabilities is not equivalent to other learning goals.
A new framework for information theory considers computational constraints.
problem Understanding information in complex systems with computational limitations.
method Variational extension of Shannon's information theory with computational constraints.
result Predictive V-information can be created through computation and reliably estimated from data. Study on test risk dynamics in learning theory with stochastic gradient flow.
problem Understanding test risk in stochastic gradient flow dynamics.
method Path integral formulation for small learning rates, explicit computation for weak features.
result Explicit corrections due to stochastic term in dynamics, good agreement with simulations.
The paper uses machine learning to compute rare event probabilities in stochastic systems.
problem Characterizing rare events in stochastic dynamical systems with weak noise.
method Developed a neural network framework for computing quasipotential, most probable paths, and prefactors.
result Demonstrated higher effectiveness and accuracy of the algorithm in calculating mean exit times.
Simplified neural network EFTs reveal a single critical condition.
problem Understanding neuron statistics in neural networks at initialization.
method Diagrammatic approach to effective field theories (EFTs).
result A single condition governs criticality of all neuron preactivations.
The paper uses topological concepts to analyze neural networks, revealing complex structure and dynamics.
problem Understanding the structure and dynamics of deep learning models.
method Topological dynamical systems, index theory, and computational homology.
result Neurons correspond to simplexes in a simplicial complex, and topological invariants can be computed.
Quantum Kerr learning shows enhancements in convergence and generalization for kernel-based methods.
problem Improving convergence and generalization in kernel-based methods for quantum computing.
method Combining quantum mechanics with neural tangent kernel theory and first-order perturbation theory.
result Quantum enhancements in terms of convergence time and generalization error.
Improved method for computing Fréchet means on SPD matrices.
problem Computing Fréchet means on the manifold of SPD matrices.
method Random matrix theory-based approach for estimating Fréchet means.
result Significantly outperforms state-of-the-art methods in experiments.
Machine learning methods for solving the equations of dynamical mean-field theory are developed. The method is demonstrated on the three dimensional Hubbard model. The key technical issues are defining a mapping of an input function to an output function, and distinguishing metallic from insulating solutions. Both meta…
This paper explores the interactions between knot theory and quantum computing. On one side, knot theory has been used to create models of quantum computing, and on the other, it is a source of computational problems. Knot theory is often used to introduce topological idea to people without a formal mathematical backgr…
The paper explores the difficulty of robust machine learning models.
problem Understanding the vulnerability of machine learning models to adversarial attacks.
method The study uses computational learning theory to analyze the feasibility of robust learning from both sample and computational complexity perspectives.
result No non-trivial concept class can be robustly learned in the distribution-free setting against a single-bit adversary, and the class of monotone conjunctions cannot be robustly learned under the uniform distribution against an adversary that can perturb ω(logn) bits. Variational quantum computing faces a flat optimization landscape problem.
problem Barren Plateaus (BP) in optimization landscapes.
method Theoretical and heuristic methods to understand and mitigate BPs.
result All algorithm components can lead to BPs if not well-suited.
We study the problem of identifying a probability distribution for some given randomly sampled data in the limit, in the context of algorithmic learning theory as proposed recently by Vinanyi and Chater. We show that there exists a computable partial learner for the computable probability measures, while by Bienvenu, M…
We propose using category theory to unify deep learning architectures.
problem Lack of a coherent bridge between model constraints and implementations.
method Apply category theory to unify neural network design.
result Theory recovers constraints from geometric deep learning and encodes standard constructs.
We introduce a theory-driven mechanism for learning a neural network model that performs generative topology design in one shot given a problem setting, circumventing the conventional iterative process that computational design tasks usually entail. The proposed mechanism can lead to machines that quickly response to n…
Serial problems can't be efficiently parallelized, affecting machine learning models.
problem Inefficiency of parallelization in inherently serial problems.
method Formalized distinction in complexity theory, demonstrated with diffusion models.
result Diffusion models cannot solve inherently serial problems.
This paper improves understanding of GAIL's generalization and computational efficiency.
problem Understanding the theoretical properties of GAIL, especially its generalization and computational aspects.
method Investigates GAIL's theoretical properties, showing guarantees for generalization and computational efficiency.
result GAIL can be efficiently solved by stochastic first order optimization algorithms with sublinear convergence.
TKFT models computation via smooth vector fields, simulating functions in a single dynamical step.
problem Modeling computation in a single step.
method Established Topological Kleene Field Theory (TKFT) as a new model of computation.
result Any computable function can be simulated in a single go of a dynamical system.
We provide a proof of backpropagation algorithm in matrix notation.
problem The lack of a full induction proof of backpropagation algorithm in matrix notation.
method We provide a full induction proof of the BP algorithm in matrix notation, situating it in the framework of matrix differential calculus.
result We prove the validity of the backpropagation algorithm in inductive form.
This research simplifies computation of feature attribution methods under certain conditions.
problem Computational complexity of feature attribution methods, especially power indices.
method Identifying conditions for polynomial computation and introducing new indices.
result Conditions for efficient computation of feature attribution methods are identified.
This paper extends financial theory to measure learnable market structure under computational constraints.
problem Understanding learnable market structure under bounded computational capacity.
method Introduces financial epiplexity as a measure of learnable market structure, extending classical information theory.
result Proves that equal entropy does not imply equal epiplexity and derives thresholds for useful regimes.
Despite recent advances in regularisation theory, the issue of parameter selection still remains a challenge for most applications. In a recent work the framework of statistical learning was used to approximate the optimal Tikhonov regularisation parameter from noisy data. In this work, we improve their results and ext…
Statistical learning theory connects to spin glass models via Rademacher complexity and replica theory.
problem Bounding generalization gap in statistical learning theory.
method Linking Rademacher complexity in statistical learning to synthetic models in statistical physics.
result Rademacher complexity is closely related to ground state energy in spin glass models.
A new method reduces communication costs in distributed learning.
problem Reduces communication bottlenecks in distributed learning.
method Local SGD with communication-computation overlap and delay-corrected sparse model averaging.
result Theoretical convergence guarantees for smooth non-convex objectives.
Transformers predict scattering amplitudes in theoretical physics.
problem Computing exact coefficients of scattering amplitudes in N = 4 SYM theory.
method Applied Transformers to predict integer coefficients of scattering amplitudes.
result Transformers achieve high (> 98%) accuracy on predicting scattering amplitudes.
Computes immersions of C2-projective spaces using K-theory.
problem Computing immersions of equivariant projective spaces.
method Geometric filtration and localized slice spectral sequence.
result Obtained equivariant analogue of James periodicity.
The pathwise coordinate optimization is one of the most important computational frameworks for high dimensional convex and nonconvex sparse learning problems. It differs from the classical coordinate optimization algorithms in three salient features: {\it warm start initialization}, {\it active set updating}, and {\it …
A computational theory reduces agent evaluation errors and speeds up processes.
problem Efficient evaluation of mini agents at reduced cost.
method Developed a computational theory and a meta-learner to handle heterogeneous agents.
result Reduced evaluation errors by 24.1% to 99.0% across various scenarios.
Physics-informed ML models improve turbulence understanding in fusion plasmas.
problem Improving turbulence modeling in fusion plasma devices.
method Physics-informed deep learning framework constrained by PDEs.
result Direct quantitative comparisons of turbulent fields between theory and gyrokinetic models.
The holy grail of deep learning is to come up with an automatic method to design optimal architectures for different applications. In other words, how can we effectively dimension and organize neurons along the network layers based on the computational resources, input size, and amount of training data? We outline prom…
Multiview representation learning is very popular for latent factor analysis. It naturally arises in many data analysis, machine learning, and information retrieval applications to model dependent structures among multiple data sources. For computational convenience, existing approaches usually formulate the multiview …
SLT explains neural network success by closing theory-practice gap.
problem Failure of classical inference and learning theory in modern neural networks.
method Physics-inspired Singular Learning Theory (SLT) applied to neural networks.
result SLT recovers known and novel scaling laws for neural network phase transitions.
The common view that our creativity is what makes us uniquely human suggests that incorporating research on human creativity into generative deep learning techniques might be a fruitful avenue for making their outputs more compelling and human-like. Using an original synthesis of Deep Dream-based convolutional neural n…
This paper presents a distance-based discriminative framework for learning with probability distributions. Instead of using kernel mean embeddings or generalized radial basis kernels, we introduce embeddings based on dissimilarity of distributions to some reference distributions denoted as templates. Our framework exte…
Machine Learning benefits from prior information and computational power for better performance and understanding.
problem Improper use of Machine Learning methods leads to lack of understanding and performance issues.
method Employing prior information and computational power to solve learning problems, emphasizing interpretability and performance.
result Combining prior information and computational power can lead to better understanding and performance in Machine Learning.
Learning in restricted Boltzmann machine is typically hard due to the computation of gradients of log-likelihood function. To describe the network state statistics of the restricted Boltzmann machine, we develop an advanced mean field theory based on the Bethe approximation. Our theory provides an efficient message pas…
Researchers compute differential K-theory for moduli stacks.
problem Computing differential K-theory for moduli stacks of principal G-bundles.
method Using homotopy theory of presheaves of spaces and spectra, they formulate results in terms of invariant polynomials and representation rings.
result They successfully compute the connective differential K-theory and differential cohomology of moduli stacks.
Canonical Correlation Analysis (CCA) is a widely used statistical tool with both well established theory and favorable performance for a wide range of machine learning problems. However, computing CCA for huge datasets can be very slow since it involves implementing QR decomposition or singular value decomposition of h…
GEORCE computes geodesics quickly and accurately.
problem Computing geodesics on Riemannian and Finsler manifolds is difficult and inefficient.
method GEORCE transforms geodesic computation into a discrete control problem.
result GEORCE achieves global convergence and quadratic local convergence.
Simplified proof for approximations of set systems.
problem Approximations of set systems in various fields.
method Modular, self-contained proof using Chernoff's bound.
result Accessible proof for a wider audience.
Deep learning enhances Hamiltonian Monte Carlo for sampling gauge field configurations.
problem Sampling from complex gauge field topologies efficiently.
method Stacked neural networks to generalize Hamiltonian Monte Carlo.
result Significantly reduces computational cost for generating gauge field configurations.
Paper connects RL and non-equilibrium statistical mechanics for entropy-regularized RL.
problem Obtaining analytical solutions for entropy-regularized RL.
method Mapping RL to non-equilibrium statistical mechanics, applying large deviation theory.
result Derives exact analytical results for optimal policy and dynamics in MDPs.
GQML uses symmetries from representation theory to improve quantum machine learning.
problem Creating quantum models with symmetries to improve performance.
method Introduction to representation theory for quantum learning, focusing on group actions and symmetries.
result Effective implementation of GQML requires knowledge of group representation theory.
DisCoPyro combines category theory with machine learning for program learning.
problem Applying category theory to machine learning tasks.
method Introducing DisCoPyro, a framework combining categorical structures with amortized variational inference.
result DisCoPyro can be applied in program learning for variational autoencoders and potentially contributes to AGI.
We present an exploration of the rich theoretical connections between several classes of regularized models, network flows, and recent results in submodular function theory. This work unifies key aspects of these problems under a common theory, leading to novel methods for working with several important models of inter…
We prove that there is no parity anomaly in M-theory in the low-energy field theory approximation. Our approach is computational. We determine generators for the 12-dimensional bordism group of pin manifolds with a w_1-twisted integer lift of w_4; these are the manifolds on which Wick-rotated M-theory exists. The anoma…
Quantum map counts BPS states in special theories.
problem Computing protected spin characters in class S theories.
method Geometric approach from 5D supersymmetric Yang-Mills theory.
result Explicit computation of protected spin characters in various examples.