Researchers use estimated Kolmogorov complexity for better link prediction in graphs.
problem Improving link prediction accuracy in complex networks.
method Regularization based on an approximation of Kolmogorov complexity, which is differentiable and compatible with recent link prediction algorithms.
result The regularization method shows good performance on diverse real-world networks, but the success is likely due to an aggregation method rather than actual estimation of Kolmogorov complexity.
We consider several ways to measure the `geometric complexity' of an embedding from a simplicial complex into Euclidean space. One of these is a version of `thickness', based on a paper of Kolmogorov and Barzdin. We prove inequalities relating the thickness and the number of simplices in the simplicial complex, general…
Efficient algorithm approximates discrete random variables with minimal Kolmogorov distance.
problem Estimating the probability of missing deadlines in series-parallel schedules.
method An efficient algorithm that computes a random variable with minimal Kolmogorov distance to a given discrete random variable.
result The algorithm efficiently approximates the probability of missing deadlines with minimal Kolmogorov distance.
SVGP KAN integrates uncertainty quantification into Kolmogorov-Arnold networks.
problem Uncertainty quantification in scientific machine learning models.
method Sparse variational Gaussian process inference with Kolmogorov-Arnold topology.
result Demonstrated ability to distinguish aleatoric and epistemic uncertainty in various scientific applications.
Kolmogorov-Arnold Networks improve deep learning adaptivity and can approximate Besov functions optimally.
problem Improving deep learning adaptivity and understanding approximation rates.
method Analyzing Besov norms and using Res-KANs for approximation.
result KANs can optimally approximate Besov functions at the optimal rate.
Recent developments have linked causal inference with Algorithmic Information Theory, and methods have been developed that utilize Conditional Kolmogorov Complexity to determine causation between two random variables. We present a method for inferring causal direction between continuous variables by using an MDL Binnin…
Proposes a new neural network architecture combining MLP and basis functions.
problem Function approximation and operator learning in scientific machine learning.
method Combines robust MLP inner functions with flexible basis functions outer functions.
result KKAN outperforms MLPs and KANs in function approximation and operator learning tasks.
K-DAREK improves KKANs for efficient function approximation with robust error bounds.
problem Efficient function approximation with uncertainty quantification for large-scale problems.
method Developed a novel learning algorithm, K-DAREK, for KKANs.
result Established robust error bounds that are distance-aware, improving efficiency and scalability.
S2KAN integrates symbolic primitives into neural network activations for improved interpretability.
problem Training activations in KANs often lack symbolic fidelity, leading to unintelligible models.
method Softly Symbolified Kolmogorov-Arnold Networks (S2KAN) integrates symbolic primitives into training with learnable gates and a Minimum Description Length objective.
result S2KAN discovers interpretable forms when symbolic terms suffice, gracefully degrading to dense splines when necessary.
No free lunch theorems suggest inductive biases are needed, but we show neural networks prefer low-complexity data.
problem The need for inductive biases in machine learning.
method Analysis of Kolmogorov complexity and neural network behavior on various datasets.
result Neural networks prefer low-complexity data, suggesting inductive biases are not always necessary.
Kolmogorov-Arnold Networks promise scalable performance in high dimensions.
problem Curse of dimensionality in multilayer perceptrons.
method Kolmogorov-Arnold representation theorem and interpolation methods.
result Kolmogorov-Arnold Networks achieve true freedom from the curse of dimensionality.
We consider options that pay the complexity deficiency of a sequence of up and down ticks of a stock upon exercise. We study the price of European and American versions of this option numerically for automatic complexity, and theoretically for Kolmogorov complexity. We also consider run complexity, which is a restricte…
Kolmogorov neural networks can represent various types of functions.
problem Representing different types of functions with neural networks.
method Continuous, discontinuous bounded or unbounded activation functions in a two hidden layer model.
result Kolmogorov neural networks can represent continuous, discontinuous bounded and all unbounded multivariate functions.
Revisits the connection between neural networks and the Kolmogorov-Arnold theorem.
problem Explains the limitations of using the Kolmogorov-Arnold theorem to explain neural networks with multiple hidden layers.
method Derives modifications of the Kolmogorov-Arnold representation that transfer smoothness properties to the outer function and can be well approximated by ReLU networks.
result Shows that a deep neural network with most layers approximating the interior function is a more natural interpretation of the Kolmogorov-Arnold representation.
KANOP uses KANs to efficiently price American options.
problem Efficiently pricing American options with limited data.
method Combines KANs with LSMC to estimate continuation value.
result KANOP provides more accurate option value estimates.
New architectures improve KANs, making them more interpretable and accurate.
problem Improving Kolmogorov-Arnold networks while maintaining interpretability.
method Overprovisioned architectures combined with sparsification, deep supervision, and depth selection, optimized with a minimum description length objective.
result Combining sparsification with depth selection achieves competitive or superior accuracy while discovering smaller models.
Kolmogorov-Arnold Networks offer improved interpretability and parsimony in science tasks.
problem Improving interpretability and parsimony in science-oriented tasks.
method Theoretical analysis of Kolmogorov-Arnold Networks (KAN) with generalization bounds and model complexity.
result Generalization bounds for KAN with various activation functions, scaling with the l1 norm of coefficient matrices and Lipschitz constants. Enhances KANs for accuracy and interpretability with multi-exit architecture.
problem Unclear optimal depth for KANs and difficulty in optimization and interpretation.
method Introduces multi-exit KANs with each layer having its own prediction branch.
result Multi-exit KANs outperform single-exit versions on various datasets.
The study reveals simplicity bias in neural networks leading to better compositional mappings.
problem Understanding when and how to encourage neural networks to learn compositional mappings.
method Examined compositional mappings through coding length and gradient descent dynamics.
result Neural networks tend to learn the simplest bijections, explaining their good generalization.
HaKAN uses Hahn-KAN blocks to forecast multivariate time series.
problem Long-term time series forecasting challenges with high complexity and spectral bias.
method HaKAN integrates channel independence, patching, and a stack of Hahn-KAN blocks with residual connections. It uses Hahn polynomial-based learnable activation functions.
result HaKAN consistently outperforms state-of-the-art methods on various forecasting benchmarks.
Stochastic differential equations (SDEs) and the Kolmogorov partial differential equations (PDEs) associated to them have been widely used in models from engineering, finance, and the natural sciences. In particular, SDEs and Kolmogorov PDEs, respectively, are highly employed in models for the approximative pricing of …
A new Kolmogorov-Arnold network improves function approximation and optimization.
problem Approximating potentially irregular functions in high dimensions.
method Proposes a new Kolmogorov-Arnold network (KAN) and provides error bounds and universal approximation theorems.
result Outperforms multilayer perceptrons in accuracy and convergence speed for irregular functions.
Study shows priors are crucial for accurate causal learning from unlabeled data.
problem Improving causal learning from unlabeled data.
method Investigated causal learning using Bayesian methods and analyzed the impact of priors.
result Factorized priors lead to factorized posteriors, aligning with independent causal mechanisms.
In Maslov (2003), a two level model of the occurrence of financial pyramid (bubbles) has been considered. We also considered the mathematical analogy of this model to Bose condensation. In the present paper, we explain why Ponzi schemes and bubbles result in a crisis in real economics. In Maslov (2005), the law of incr…
A new feature selection method using random forest and Kolmogorov filter.
problem Ultra-high dimensional data feature selection.
method Fused Kolmogorov filter with random forest based recursive feature elimination.
result Selection and L2 consistency under weak conditions. We prove that ``almost generically'' for a one-relator group Delzant's T-invariant (which measures the smallest size of a finite presentation for a group) is comparable in magnitude with the length of the defining relator. The proof relies on our previous results regarding isomorphism rigidity of generic one-relator …
The usual derivation of the Fokker-Planck partial differential eqn. assumes the Chapman-Kolmogorov equation for a Markov process. Starting instead with an Ito stochastic differential equation we argue that finitely many states of memory are allowed in Kolmogorov's two pdes, K1 (the backward time pde) and K2 (the Fokker…
Neuroimaging datasets keep growing in size to address increasingly complex medical questions. However, even the largest datasets today alone are too small for training complex machine learning models. A potential solution is to increase sample size by pooling scans from several datasets. In this work, we combine 12,207…
We study the solution to Kolmogorov-Feller equation and by using it provide pricing formulas of well known some options under jump-diffusion model.
Bayesian KANs achieve near-minimax posterior contraction rates in anisotropic Besov spaces.
problem Statistical foundation for Bayesian Kolmogorov-Arnold networks in anisotropic Besov spaces.
method Sparse Bayesian KANs with spike-and-slab priors, hyperprior on model size, and approximation complexity bounds.
result Posterior contraction rates depend on intrinsic anisotropic smoothness and effective dimension of the compositional structure.
Study finds conjugate points in geodesics of Kolmogorov flows on torus.
problem Characterizing pairs of integers (m,n) for which geodesics have conjugate points.
method Analysis of geodesics in the group of volume-preserving diffeomorphisms of a torus using stream functions.
result Existence of conjugate points for all pairs of strictly positive integers (m,n).
A new machine learning method solves high-dimensional Kolmogorov PDEs efficiently.
problem Solving high-dimensional Kolmogorov PDEs and SDEs.
method Stochastic weighted minimization and stochastic gradient descent with Malliavin weights.
result Accurate approximation of high-dimensional Kolmogorov PDEs and SDEs without curse of dimensionality.
New measure LMN explains neural network grokking.
problem Delayed generalization after memorization in neural networks.
method Defined LMN to measure network complexity, showing LMN correlates with test losses linearly.
result LMN reveals intriguing XOR network behavior and is a promising complexity measure.
SVGP KAN integrates sparse variational GP with KANs for scalable probabilistic inference.
problem Lack of probabilistic outputs in standard KANs and cubic scaling of Gaussian Process methods.
method Sparse Variational GP-KAN combines KAN topology with sparse variational inference and permutation-based importance analysis.
result Enables probabilistic KANs to handle larger datasets with linear computational complexity.
A new autoencoder combines deep learning with SVD to reduce model complexity.
problem Overcoming the Kolmogorov barrier in high-dimensional systems.
method Learnable weighted hybrid autoencoder combining SVD and deep learning.
result Empirically, the model exhibits a sharpness thousands of times smaller than other models.
Smooth KANs improve model reliability in computational biomedicine.
problem Limited convergence of KANs in representing generic smooth functions.
method Introducing smooth, structurally informed KANs that can approximate MLPs in specific function classes.
result Smooth KANs can achieve equivalence to MLPs in specific function classes, enhancing model reliability and performance.
We study gradient bounds and other functional inequalities for the diffusion semigroup generated by Kolmogorov type operators. The focus is on two different methods: coupling techniques and generalized Γ-calculus techniques. The advantages and drawbacks of each of these methods are discussed.
KOLMOGOROV-OPTIMAL RESOLUTION ESTIMATION (KORE) solves spline regression without exhaustive search
problem Hyperparameter tuning in spline regression
method Solving for optimal resolution analytically
result KORE matches exhaustive cross-validation and outperforms tuned models
Kolmogorov-Arnold Networks achieve optimal convergence rates in nonparametric regression.
problem Nonparametric function approximation in multivariate settings.
method Structured additive and multiplicative KANs using B-splines.
result Achieve minimax-optimal convergence rate O(n−2r/(2r+1)) for Sobolev space functions. Randomness and regularities in Finance are usually treated in probabilistic terms. In this paper, we develop a completely different approach in using a non-probabilistic framework based on the algorithmic information theory initially developed by Kolmogorov (1965). We present some elements of this theory and show why i…
High order splitting schemes with complex timesteps are applied to Kolmogorov backward equations stemming from stochastic differential equations in Stratonovich form. In the setting of weighted spaces, the necessary analyticity of the split semigroups can be easily proved. A numerical example from interest rate theory,…
Paper proposes a hybrid MTL framework for improved stock market prediction accuracy.
problem Inaccurate stock market predictions due to financial data's complexities.
method Multi-layer hybrid MTL structure with Transformer, BiGRU, and KAN.
result Achieved low MAE (1.078), MAPE (0.012), and high R^2 (0.98) compared to other models.
KSGAN uses KS distance for deep generative modeling.
problem Deep generative modeling challenges, especially for multivariate distributions.
method Formulates adversarial training as minimization of KS distance, using quantile function as critic.
result KSGAN trained distributions closely match target distributions.
This study compares MLPs and KANs in low-data regimes, finding MLPs with personalized activation functions outperform KANs.
problem Comparing MLPs and KANs in low-data regimes.
method Introduced an effective technique for designing MLPs with unique, parameterized activation functions for each neuron.
result MLPs with personalized activation functions achieve significantly higher predictive accuracy with only a modest increase in parameters, especially in low-data regimes.
NN-Turb generates turbulent velocity statistics using neural networks.
problem Creating a 1D field with turbulent velocity statistics.
method Fully-convolutional neural network (NN-Turb) to generate the field.
result NN-Turb generates a 1D field that satisfies Kolmogorov's 2/3 and 4/5 laws, exhibiting intermittency.
We link SVEs to SPDEs and derive Kolmogorov equations for singular kernels.
problem Solving stochastic Volterra equations with singular kernels.
method Establishing connections between SVEs and SPDEs, using stochastic calculus in Hilbert spaces.
result Solutions of SVEs can be expressed in terms of backward Kolmogorov equations.
Deep learning solves high-dimensional PDEs efficiently.
problem Solving high-dimensional Kolmogorov PDEs numerically.
method Reformulating as a statistical learning problem using Feynman-Kac formula.
result Single neural network learns entire family of PDEs.
Measures neural network information transfer for generalization.
problem Estimating the generalizable information in neural networks.
method Proposes Information Transfer (LIT) based on prequential coding. result Consistently correlates with generalizable information in neural networks.