Kolmogorov neural networks can represent various types of functions.
problem Representing different types of functions with neural networks.
method Continuous, discontinuous bounded or unbounded activation functions in a two hidden layer model.
result Kolmogorov neural networks can represent continuous, discontinuous bounded and all unbounded multivariate functions.
Revisits the connection between neural networks and the Kolmogorov-Arnold theorem.
problem Explains the limitations of using the Kolmogorov-Arnold theorem to explain neural networks with multiple hidden layers.
method Derives modifications of the Kolmogorov-Arnold representation that transfer smoothness properties to the outer function and can be well approximated by ReLU networks.
result Shows that a deep neural network with most layers approximating the interior function is a more natural interpretation of the Kolmogorov-Arnold representation.
Poor approximators found in neural networks and random feature models.
problem Understanding why certain neural networks and models perform poorly in approximating functions.
method Established a scale separation of Kolmogorov width type and applied it to neural networks and random feature models.
result Reproducing kernel Hilbert spaces and two-layer neural networks are poor L2-approximators for certain functions. NN-Turb generates turbulent velocity statistics using neural networks.
problem Creating a 1D field with turbulent velocity statistics.
method Fully-convolutional neural network (NN-Turb) to generate the field.
result NN-Turb generates a 1D field that satisfies Kolmogorov's 2/3 and 4/5 laws, exhibiting intermittency.
Presented are two neural network architectures for convex functions, demonstrating competitive performance.
problem Approximating convex functions efficiently and accurately.
method Developed two neural network architectures: one based on linear-by-part representation and the other on cubic splines.
result Cubic ICKAN networks produce results similar to classical ICNNs in solving convex approximation problems.
Kolmogorov-Arnold Networks promise scalable performance in high dimensions.
problem Curse of dimensionality in multilayer perceptrons.
method Kolmogorov-Arnold representation theorem and interpolation methods.
result Kolmogorov-Arnold Networks achieve true freedom from the curse of dimensionality.
K-DAREK improves KKANs for efficient function approximation with robust error bounds.
problem Efficient function approximation with uncertainty quantification for large-scale problems.
method Developed a novel learning algorithm, K-DAREK, for KKANs.
result Established robust error bounds that are distance-aware, improving efficiency and scalability.
Kolmogorov-Arnold network improves GW catalog posterior construction.
problem Efficiently constructing posterior distributions for GW catalogs.
method Using the Kolmogorov-Arnold network to create lightweight neural density estimators.
result Kolmogorov-Arnold network achieves superior interpretability and accuracy in posterior construction.
causalKANs provides interpretable treatment effect estimates using neural networks.
problem The opacity of deep neural networks limits their adoption in sensitive domains.
method Proposes causalKANs, a framework that transforms neural estimators into interpretable closed-form formulas.
result causalKANs performs on par with neural baselines in CATE error metrics and offers a favorable accuracy--interpretability trade-off.
New test uses neural networks to compare distributions, outperforming traditional methods.
problem Comparing distributions in high dimensions and higher orders of smoothness.
method Integral probability metrics with Radon bounded variation functions and neural networks.
result The Radon-Kolmogorov-Smirnov (RKS) test outperforms traditional methods in distinguishing distributions.
Paper develops a neural model to assess cascading extreme events.
problem Risk assessment of domino effects like earthquakes and tsunamis.
method Develops a Kolmogorov-Arnold neural network (KANE) framework.
result Estimates the probability of one extreme event triggering another.
The paper proves neural networks' consistency and optimal convergence rates for various function classes.
problem Proving neural networks' consistency and optimal convergence rates for diverse function classes.
method Analyzes wide and deep ReLU neural networks trained on logistic loss and Kolmogorov-Donoho optimal function classes.
result Proves universal consistency and minimax optimal convergence rates for neural networks.
A new KAN variant uses sinusoidal activations to approximate functions.
problem Approximating multivariable functions using neural networks.
method Replacing inner and outer functions in Kolmogorov-Arnold representation with weighted sinusoidal functions.
result The new KAN variant outperforms fixed-frequency Fourier transform and achieves comparable performance to MLPs.
Wav-KAN improves neural network interpretability and performance.
problem Challenges in interpretability, training speed, robustness, and performance of traditional neural networks.
method Integrates wavelet functions into the Kolmogorov-Arnold network structure for efficient data representation.
result Enhanced accuracy, faster training speeds, and increased robustness compared to existing methods.
Neural operators solve families of 2BSDEs efficiently.
problem Solving infinite families of 2BSDEs on bounded domains.
method Introduces a mild generative neural operator model to approximate solutions.
result Solution operators can be approximated by neural operators with polynomial parameters.
S2KAN integrates symbolic primitives into neural network activations for improved interpretability.
problem Training activations in KANs often lack symbolic fidelity, leading to unintelligible models.
method Softly Symbolified Kolmogorov-Arnold Networks (S2KAN) integrates symbolic primitives into training with learnable gates and a Minimum Description Length objective.
result S2KAN discovers interpretable forms when symbolic terms suffice, gracefully degrading to dense splines when necessary.
Proposes a new neural network architecture combining MLP and basis functions.
problem Function approximation and operator learning in scientific machine learning.
method Combines robust MLP inner functions with flexible basis functions outer functions.
result KKAN outperforms MLPs and KANs in function approximation and operator learning tasks.
Kolmogorov-Arnold Networks offer interpretable models for energy applications.
problem Lack of interpretability in modern machine learning methods for sensitive industries.
method Symbolic regression with Kolmogorov-Arnold Networks compared to traditional feedforward neural networks.
result Kolmogorov-Arnold Networks yield perfectly interpretable models and learn real, physical relations.
The study reveals simplicity bias in neural networks leading to better compositional mappings.
problem Understanding when and how to encourage neural networks to learn compositional mappings.
method Examined compositional mappings through coding length and gradient descent dynamics.
result Neural networks tend to learn the simplest bijections, explaining their good generalization.
Measures neural network information transfer for generalization.
problem Estimating the generalizable information in neural networks.
method Proposes Information Transfer (LIT) based on prequential coding. result Consistently correlates with generalizable information in neural networks.
The development of new classification and regression algorithms based on empirical risk minimization (ERM) over deep neural network hypothesis classes, coined deep learning, revolutionized the area of artificial intelligence, machine learning, and data analysis. In particular, these methods have been applied to the num…
Random neural nets learn Black-Scholes PDEs without dimensionality issues.
problem Learning Black-Scholes type PDEs efficiently in high dimensions.
method Random feature neural networks applied to Kolmogorov PDEs.
result Random neural nets avoid the curse of dimensionality for Black-Scholes PDEs.
New measure LMN explains neural network grokking.
problem Delayed generalization after memorization in neural networks.
method Defined LMN to measure network complexity, showing LMN correlates with test losses linearly.
result LMN reveals intriguing XOR network behavior and is a promising complexity measure.
Researchers use estimated Kolmogorov complexity for better link prediction in graphs.
problem Improving link prediction accuracy in complex networks.
method Regularization based on an approximation of Kolmogorov complexity, which is differentiable and compatible with recent link prediction algorithms.
result The regularization method shows good performance on diverse real-world networks, but the success is likely due to an aggregation method rather than actual estimation of Kolmogorov complexity.
A new Kolmogorov-Arnold network improves function approximation and optimization.
problem Approximating potentially irregular functions in high dimensions.
method Proposes a new Kolmogorov-Arnold network (KAN) and provides error bounds and universal approximation theorems.
result Outperforms multilayer perceptrons in accuracy and convergence speed for irregular functions.
KANs replace fixed MLP weights with learnable edge functions, improving accuracy and interpretability.
problem Lack of interpretability and scalability in MLPs.
method KANs use learnable activation functions on edges instead of fixed weights, replacing weights with spline functions.
result KANs outperform MLPs in accuracy and interpretability with smaller models.
GKAN extends KAN to graphs, improving graph-based learning.
problem Graph-based learning tasks, especially semi-supervised.
method Learnable spline-based functions applied to graph data.
result GKAN achieves higher accuracy in graph semi-supervised learning.
SVGP KAN integrates uncertainty quantification into Kolmogorov-Arnold networks.
problem Uncertainty quantification in scientific machine learning models.
method Sparse variational Gaussian process inference with Kolmogorov-Arnold topology.
result Demonstrated ability to distinguish aleatoric and epistemic uncertainty in various scientific applications.
Sharp lower bounds on shallow neural networks' approximation rates are derived.
problem The efficiency of shallow neural networks in approximating functions.
method Lower bounding the L2-metric entropy and Kolmogorov n-widths of the convex hull of neural network basis functions. result Sharp lower bounds on the approximation rates for shallow neural networks are provided.
The paper introduces a spline-based method for calibrating neural networks.
problem Ensuring neural network outputs are reliable for safety-critical applications.
method Approximating the empirical cumulative distribution function using splines to map network outputs to calibrated probabilities.
result The spline-based recalibration consistently outperforms existing methods on calibration measures.
MAGDiff detects data shifts in neural networks without retraining.
problem Neural networks' sensitivity to data distribution shifts.
method Extracts MAGDiff representations from neural networks to detect shifts.
result MAGDiff representations improve data set shift detection.
This paper develops fundamental limits of deep neural network learning by characterizing what is possible if no constraints are imposed on the learning algorithm and on the amount of training data. Concretely, we consider Kolmogorov-optimal approximation through deep neural networks with the guiding theme being a relat…
No free lunch theorems suggest inductive biases are needed, but we show neural networks prefer low-complexity data.
problem The need for inductive biases in machine learning.
method Analysis of Kolmogorov complexity and neural network behavior on various datasets.
result Neural networks prefer low-complexity data, suggesting inductive biases are not always necessary.
Spectral analysis shows neural networks separate from linear methods in approximating functions.
problem Separating two-layer neural networks from linear methods in function approximation.
method Spectral-based approach using Kolmogorov width and kernel spectrum.
result Upper and lower bounds on separation, explicit hard functions identified.
Deep learning solves high-dimensional PDEs efficiently.
problem Solving high-dimensional Kolmogorov PDEs numerically.
method Reformulating as a statistical learning problem using Feynman-Kac formula.
result Single neural network learns entire family of PDEs.
Bayesian KANs achieve near-minimax posterior contraction rates in anisotropic Besov spaces.
problem Statistical foundation for Bayesian Kolmogorov-Arnold networks in anisotropic Besov spaces.
method Sparse Bayesian KANs with spike-and-slab priors, hyperprior on model size, and approximation complexity bounds.
result Posterior contraction rates depend on intrinsic anisotropic smoothness and effective dimension of the compositional structure.
Wavelet Kolmogorov-Arnold Networks improve federated learning performance.
problem Improving performance in federated learning with heterogeneous data.
method Implemented Wav-KAN with CWT and DWT for multiresolution capability, integrating wavelet-based activation functions.
result Significant improvements in computational efficiency, robustness, and accuracy in federated learning.
KACDP model improves credit default prediction with enhanced interpretability.
problem Insufficient interpretability and limited performance in credit default prediction.
method Kolmogorov-Arnold Networks (KANs) for handling complex multi-dimensional data.
result KACDP model outperforms mainstream models in performance metrics.
This study compares MLPs and KANs in low-data regimes, finding MLPs with personalized activation functions outperform KANs.
problem Comparing MLPs and KANs in low-data regimes.
method Introduced an effective technique for designing MLPs with unique, parameterized activation functions for each neuron.
result MLPs with personalized activation functions achieve significantly higher predictive accuracy with only a modest increase in parameters, especially in low-data regimes.
Smooth KANs improve model reliability in computational biomedicine.
problem Limited convergence of KANs in representing generic smooth functions.
method Introducing smooth, structurally informed KANs that can approximate MLPs in specific function classes.
result Smooth KANs can achieve equivalence to MLPs in specific function classes, enhancing model reliability and performance.
Neural Markov models improve time series analysis by balancing deep learning and classical models.
problem Modeling non-stationary time series with high data sparsity.
method Hybrid approach using neural networks to parameterize stochastic matrices, estimating time-inhomogeneous Markov chains.
result Reduction of Chapman-Kolmogorov discrepancy and superior likelihood in financial markets.
The paper develops a computational method for efficient online filtering of diffusion processes.
problem Online filtering of discretely observed nonlinear diffusion processes.
method The approach involves Doob's h-transforms approximated by solving backward Kolmogorov equations using nonlinear Feynman-Kac formulas and neural networks. result The proposed method can be orders of magnitude more efficient than state-of-the-art particle filters.
Kolmogorov-Arnold Networks achieve optimal convergence rates in nonparametric regression.
problem Nonparametric function approximation in multivariate settings.
method Structured additive and multiplicative KANs using B-splines.
result Achieve minimax-optimal convergence rate O(n−2r/(2r+1)) for Sobolev space functions. The paper introduces a new method for multivariate density estimation using deep neural mixture models.
problem Multivariate density estimation is a fundamental but underexplored task in machine learning.
method The paper extends Neural Mixture Densities (NMMs) to multivariate Deep Neural Mixture Models (DNMMs) using maximum-likelihood algorithm.
result The DNMMs can model any probability density function to any degree of precision and outperform traditional statistical estimation techniques.
Optimized neural network approximates high-dimensional functions with minimal parameters.
problem Achieving optimal approximation of high-dimensional continuous functions with minimal parameters.
method Developed a neural network with a specific activation function and architecture to achieve super approximation property.
result A composed network with at most 10889d + 10887 nonzero parameters achieves super approximation property, suggesting optimality in parameter growth.
Kolmogorov-Arnold Networks improve deep learning adaptivity and can approximate Besov functions optimally.
problem Improving deep learning adaptivity and understanding approximation rates.
method Analyzing Besov norms and using Res-KANs for approximation.
result KANs can optimally approximate Besov functions at the optimal rate.
GC-KAN uses KANs to detect Granger causality in time series data.
problem Detecting causal relationships in nonlinear time series data.
method Developed GC-KAN framework using Kolmogorov-Arnold networks for Granger causality detection.
result KANs outperform MLPs in identifying sparse Granger causal relationships.
This paper reports empirical evidence that a neural networks model is applicable to the statistically reliable prediction of foreign exchange rates. Time series data and technical indicators such as moving average, are fed to neural nets to capture the underlying "rules" of the movement in currency exchange rates. The …