The paper shows neural networks can approximate functions over non-compact domains with non-polynomial activation.
problem Approximating functions over non-compact domains using neural networks.
method Using single-hidden-layer feedforward neural networks with non-polynomial activation functions over non-compact subsets of Euclidean spaces.
result Neural networks can approximate functions in weighted Ck-spaces and weighted Sobolev spaces over unbounded domains. New bounds show complex neural networks need many queries to learn.
problem Learning non-polynomial activation functions with Gaussian marginals.
method Gradient boosting procedure to amplify lower bounds on SQ dimension of neural networks.
result Statistical-query lower bounds for ReLU regression with 2ncε queries. Neural networks can approximate functions uniformly across various measures.
problem Universal approximation of functions across different probability measures.
method Proving neural networks are dense in Orlicz spaces, extending classical theorems.
result Neural networks uniformly approximate functions for weakly compact families of measures.
New method learns low-dimensional models for systems with non-polynomial terms.
problem Modeling systems with non-polynomial nonlinear terms that are spatially local and given in analytic form.
method Non-intrusive model reduction method that learns operators for linear and polynomially nonlinear dynamics via a least-squares problem incorporating given non-polynomial terms.
result Comparable accuracy to intrusive methods that require full knowledge of governing equations.
Wide neural networks can be closely approximated by Gaussian processes, with rates depending on the activation function's properties.
problem Approximating the behavior of wide neural networks using Gaussian processes.
method Established convergence rates for the central limit theorem in an infinite-dimensional functional space, using a transportation distance metric.
result Explicit convergence rates for neural networks approximated by Gaussian processes, varying based on the activation function's properties.
The paper proves deep neural networks with analytic activation can approximate any function.
problem Approximating functions with neural networks using analytic activation functions.
method Elementary proofs for real and complex networks, Stone-Weierstrass theorem, Mergelyan's theorem.
result Closure of neural network classes equals space of polynomials for analytic activation.
New framework establishes positivity of DNTK for PINNs.
problem Establishing positivity of NTK for PINNs with multiple differential operators.
method Proposed Differential Neural Tangent Kernel (DNTK) for PINNs.
result Positivity of infinite width DNTK for various activation functions and differential operators.
Non-polynomial growth harmonic maps from the complex plane to the hyperbolic space are studied. Some non-surjectivity results are obtained. Moreover, images of such harmonic maps are investigated with reference to their Hopf differentials.
CVNNs improve performance in tasks with complex-valued inputs.
problem Improving performance in tasks with complex-valued inputs.
method Analyze the approximation properties of complex-valued neural networks (CVNNs).
result Quantitative approximation bounds for CVNNs, showing error scales as m−k/(2n). UDENet and ResNet can approximate any function, with ODENet showing UAP for continuous functions.
problem Approximating any function using ODENet and ResNet.
method Proved UAP for ODENet and ResNet, derived gradient, and applied to various problems.
result UDENet and ResNet can approximate any function, with ODENet showing UAP for continuous functions.
Quantum K-theory of quintic 3-fold conjectured with non-polynomial coefficients.
problem Reconstructing quantum K-theory for quintic 3-fold.
method Formulated explicit conjecture for small J-function and its q-difference equation.
result Coefficients of q-difference equations are non-polynomial functions of Gopakumar-Vafa invariants.
The action of the mapping class group of the thrice-punctured projective plane on its GL(2,C) character variety produces an algorithm for generating the simple length spectra of quasi-Fuchsian thrice-punctured projective planes. We apply this algorithm to quasi-Fuchsian representations of the corres…
We study the uniqueness of a vortex equation involving an entire function on the complex plane. As geometric applications, we show that there is a unique harmonic map u:C→H2 satisfying ∂u=0 with prescribed polynomial Hopf differential; there is a unique affine spherical imm…
Study examines null vector fields on Lorentzian manifolds.
problem Understanding the structure of null vector fields on Lorentzian manifolds.
method Investigates the bundle structure and ternary product of nowhere vanishing null vector fields.
result Null tangent bundle is a non-polynomial graded bundle with a para-associative ternary product.
Data symmetries in neural networks can generate conserved quantities.
problem Conservation laws in neural networks
method Using tensorizable networks
result Data augmentation can induce conserved quantities
Much combinatorial optimisation problems constitute a non-polynomial (NP) hard optimisation problem, i.e., they can not be solved in polynomial time. One such problem is finding the shortest route between two nodes on a graph. Meta-heuristic algorithms such as A∗ along with mixed-integer programming (MIP) methods …
Neural networks can approximate any continuous function with a single hidden layer.
problem Approximating any continuous function using neural networks.
method Direct algebraic proof and explicit quantification of hidden units required.
result Explicit bounds on the number of hidden units needed for approximation.
Ehlers-Kundt conjecture is a physical assertion about the fundamental role of plane waves for the description of gravitational waves. Mathematically, it becomes equivalent to a problem on the Euclidean plane R2 with a very simple formulation in Classical Mechanics: given a non-necessarily autonomous potent…
Detecting correlated trees helps align sparse graphs.
problem Detecting correlation between trees for sparse random graphs.
method MPAlign message-passing algorithm for graph alignment.
result MPAlign succeeds in polynomial time for partial alignment.
Cryptotree enables accurate predictions on encrypted data using Random Forests.
problem Applying machine learning to private data while preserving confidentiality.
method Adapting Neural RF to CKKS scheme for HE operations on encrypted data.
result Cryptotree achieves better prediction results on encrypted data than regular RF.
FNOs learn solution operators of dissipative equations efficiently via spectral methods.
problem Learning and approximation of solution operators for dissipative equations.
method Introducing spectral methods and deriving FNO approximation bounds and sample complexity guarantees.
result Polynomial sample complexity guarantees for FNOs learning solution operators of dissipative equations.
In order to identify important variables that are involved in making optimal treatment decision, Lu et al. (2013) proposed a penalized least squared regression framework for a fixed number of predictors, which is robust against the misspecification of the conditional mean model. Two problems arise: (i) in a world of ex…
New method for MAP inference using Benders' decomposition.
problem Finite-time convergence guarantee for MAP inference.
method Sequentially adding constraints using Benders' decomposition.
result Higher optimal posterior value compared to other methods.
Quantum computing optimizes ESG portfolios efficiently.
problem Optimizing investment portfolios with risk, return, and ESG considerations.
method Formulated discrete Markowitz portfolio theory (DMPT) for quantum annealers, incorporating ESG ratings.
result Discrete portfolios converge to continuous solutions as budgets increase, outperforming traditional methods.
Improved MoE performance through perturbing cosine router.
problem Representation collapse and parameter redundancy in MoE models.
method Least square estimation of cosine router in MoE, followed by noise addition to improve convergence rates.
result Perturbed cosine router leads to polynomial convergence rates for MoE models.
At initialization, artificial neural networks (ANNs) are equivalent to Gaussian processes in the infinite-width limit, thus connecting them to kernel methods. We prove that the evolution of an ANN during training can also be described by a kernel: during gradient descent on the parameters of an ANN, the network functio…
We present and analyze a central cutting surface algorithm for general semi-infinite convex optimization problems, and use it to develop a novel algorithm for distributionally robust optimization problems in which the uncertainty set consists of probability distributions with given bounds on their moments. Moments of a…
Many activation functions have been proposed in the past, but selecting an adequate one requires trial and error. We propose a new methodology of designing activation functions within a neural network at each layer. We call this technique an "activation ensemble" because it allows the use of multiple activation functio…
This paper studies activation sparsity in large language models, finding key trends and implications.
problem Activation sparsity in large language models (LLMs) can be improved for efficiency and interpretability.
method Proposes PPL-p% sparsity, analyzes trends with training data, width-depth ratio, and parameter scale. result ReLU is more efficient for sparsity than SiLU, and deeper architectures can improve sparsity.
Activity recognition from sensor data deals with various challenges, such as overlapping activities, activity labeling, and activity detection. Although each challenge in the field of recognition has great importance, the most important one refers to online activity recognition. The present study tries to use online hi…
Many neural network architectures rely on the choice of the activation function for each hidden layer. Given the activation function, the neural network is trained over the bias and the weight parameters. The bias catches the center of the activation, and the weights capture the scale. Here we propose to train the netw…
BinaryDuo improves BNNs by coupling binary activations, outperforming state-of-the-art models.
problem Gradient mismatch in BNNs due to binarizing activations.
method Using gradient of smoothed loss function to estimate gradient mismatch, proposing BinaryDuo scheme with coupled ternary activations.
result BinaryDuo outperforms state-of-the-art BNNs on various benchmarks.
Evolutionary algorithms improve neural network performance by discovering better activation functions.
problem The choice of activation function affects neural network performance, but ReLU remains dominant.
method Defined a tree-based search space of candidate activation functions and used evolutionary algorithms (mutation, crossover, exhaustive search) to explore and discover better functions.
result Replacing ReLU with evolved activation functions statistically significantly increases network accuracy.
Active learning method balances bias and variance under class imbalance.
problem Active learning under label shift when class proportions differ.
method Mediated Active Learning under Label Shift (MALLS) using a 'medial distribution'.
result MALLS reduces asymptotic sample complexity under arbitrary label shift.
Study active learning of PTFs with derivative access.
problem Active learning of polynomial threshold functions (PTFs).
method Algorithm for active learning degree-d univariate PTFs with derivative access. result Computational efficient algorithm for active learning degree-d univariate PTFs. Collaborative filtering is a useful technique for exploiting the preference patterns of a group of users to predict the utility of items for the active user. In general, the performance of collaborative filtering depends on the number of rated examples given by the active user. The more the number of rated examples giv…
Bayesian adaptive designs can be biased by active learning, especially with misspecified models.
problem Active learning bias in Bayesian adaptive experimental designs.
method Analysis of linear and preference learning models, empirical testing.
result Model misspecification and noise influence active learning bias in Bayesian designs.
Derives time-averaged active inference from control principles.
problem Finite-horizon or discounted-surprise problems in active inference.
method Derives infinite-horizon, average-surprise active inference from optimal control principles.
result Unified objective functional for sensorimotor control.
RAN model recognizes multiple activities from unlabeled sensor data.
problem Handling weakly labeled multi-activity data from wearable sensors.
method Recurrent Attention Networks (RAN) for sequential multi-activity recognition and localization.
result RAN model can infer multiple activities and determine activity locations from unlabeled data.
A new indicator measures project risk from activity durations.
problem Managing project risks throughout the lifecycle.
method Activity Risk Index (ARI) based on Schedule Risk Baseline.
result Identifies activities contributing most to project uncertainty.
Hidden Markov Models detect hand gestures from wearable sEMG signals.
problem Detecting activity regions in continuous sEMG signals.
method Hidden Markov Models applied to sEMG signals for gesture recognition.
result Average accuracy of 96.25% for activity onsets and 87.5% for activity terminations.
Active testing reduces label costs for efficient model evaluation.
problem Real-world applications require expensive test labels, disconnecting from existing model evaluation methods.
method Derives acquisition strategies to select test points efficiently, addressing label bias and variance.
result Active testing improves model evaluation efficiency without sacrificing accuracy.
Survey of trainable activation functions in neural networks.
problem Improving neural network performance through trainable activation functions.
method Taxonomy and comparison of recent and past models of trainable activation functions.
result Many trainable activation functions are equivalent to adding neuron layers with fixed activation functions and simple constraints.
This paper analyzes and improves active learning techniques for real-world projects.
problem Reducing labelling effort in machine learning models with real-world constraints.
method Systematic study of active learning issues, proposing techniques to address model convergence, annotation error, and dataset imbalance.
result Presentation of two techniques to speed up active learning: partial uncertainty sampling and larger query size.
Study active nematic forces on curved surfaces, revealing new coupling mechanisms.
problem Understanding active nematic forces on curved surfaces.
method Developed a thermodynamically consistent surface model with nematic activity, analyzed topological defects.
result Active defects contribute both tangential and normal forces on curved surfaces.
This paper provides an overview of activation functions in neural networks.
problem Confusion in activation function selection and properties in deep learning.
method Analytic review of popular activation functions.
result Clarification of activation function properties and selection.
Bayesian active learning method improved for censored regression data.
problem Challenges in estimating BALD for censored regression data.
method Derived entropy and mutual information for censored distributions, developed C-BALD objective, proposed novel modelling approach. result Demonstrated C-BALD outperforms other methods in censored regression. DUNs improve active learning by dynamically adjusting model complexity.
problem Active learning datasets grow, leading to model bias or overfitting.
method Depth Uncertainty Networks (DUNs) infer model depth for flexibility and complexity.
result DUNs outperform other BNN variants on active learning tasks, reducing overfitting.