A theorem for debiasing machine learning with finite sample guarantees.
problem Calculating confidence intervals for machine learning functionals.
method Debiased machine learning based on bias correction and sample splitting.
result Nonasymptotic debiased machine learning theorem with finite sample guarantees.
Study compares FDA and ML methods for time series classification.
problem Comparing functional data analysis and machine learning for time series classification.
method Functional generalized additive models, feature extraction, basis representations, support vector machines, classification trees.
result Benchmarking and ranking of methods for non-expert practitioners.
New approach to bilevel optimization for machine learning using functional methods.
problem Solving bilevel optimization problems in machine learning, especially with over-parameterized neural networks.
method Functional point of view, scalable and efficient algorithms for functional bilevel optimization.
result Demonstrates benefits of functional approach on instrumental regression and reinforcement learning tasks.
Examines challenges and proposes new approaches in machine learning theory.
problem Challenges in machine learning as a function approximation and optimization.
method Mathematical analysis of gradient descent, fixed network limitations, and RNNs.
result New insights and mathematical approaches to improve machine learning.
Proposes a new loss function for causal machine-learning.
problem Lack of a well-defined loss function for causal machine-learning.
method Introduces a novel loss function equal to MSE in a standard regression problem.
result Demonstrates that gradient descent can be directly applied to this loss function.
Develops a direct debiased machine learning framework using Bregman divergence.
problem Reduces bias in machine learning estimates of causal effects or structural models.
method Neyman targeted estimation and generalized Riesz regression using Bregman divergence.
result Improves estimation of parameters of interest in causal models.
Machine learning impacts computational math, offering new functions approximations.
problem Machine learning's black box nature hinders further progress in computational math.
method Analyzes machine learning's impact on computational math and vice versa.
result Integrating computational math with machine learning can enhance both fields.
In this paper, we aim at introducing a new machine learning model, namely reconciled polynomial machine, which can provide a unified representation of existing shallow and deep machine learning models. Reconciled polynomial machine predicts the output by computing the inner product of the feature kernel function and va…
Mathematical approach defines stability conditions for ML models.
problem Ensuring stability of machine learning models.
method Adopted topological and metric spaces theory to define stability.
result Stability of ML models depends on topological properties of classification sets.
RBM and DBM are represented as 2D tensor networks, revealing their expressive power and efficiency.
problem Understanding and optimizing RBM and DBM models.
method Representing RBM and DBM as 2D tensor networks and developing an efficient tensor network contraction algorithm.
result The proposed algorithm for computing partition functions is more accurate than state-of-the-art methods.
This paper connects functional data analysis with machine learning techniques.
problem Lack of theoretical analysis for functional depths.
method Viewing functional depths as kernel mean embeddings in machine learning.
result Facilitates answers to open questions about functional depths.
Paper examines power consumption in neural networks using various activation functions.
problem Power consumption in machine learning models.
method Examines power consumption for different activation functions.
result Substantial differences in power consumption exist between activation functions.
DML addresses biases in machine learning by estimating nuisance functions.
problem Bias in machine learning models due to nuisance functions.
method Double/Debiased Machine Learning (DML) approach to reduce biases.
result DML allows flexible estimation of nuisance functions without auxiliary assumptions.
Improved machine learning model performance through data augmentation, custom loss functions, and transfer learning.
problem Poor performance of a traditional engineering model due to limited training data.
method Data augmentation, custom loss functions, transfer learning.
result Improvement of at least 38% in performance across five models.
Quantum machine learning models can approximate any continuous function.
problem Theoretical understanding of quantum feature maps in machine learning.
method Proving universal approximation property of quantum machine learning models in quantum-enhanced feature spaces.
result Quantum machine learning models are universal approximators of continuous functions.
Machine learning uses invariant theory to restrict function classes.
problem Creating function classes that respect physical law constraints.
method Using equivariant machine learning and Malgrance's method to parameterize functions.
result Explicitly parameterizes equivariant functions between linear spaces.
Estimates impulse response functions using machine learning in time series data.
problem Estimating causal effects of discrete treatments over time with flexible models.
method Double/debiased machine learning for nonparametric time series data.
result Consistent and asymptotically normal estimator for impulse response functions.
Though the deep learning is pushing the machine learning to a new stage, basic theories of machine learning are still limited. The principle of learning, the role of the a prior knowledge, the role of neuron bias, and the basis for choosing neural transfer function and cost function, etc., are still far from clear. In …
A new machine learning framework reduces IoT data transfer by two orders of magnitude.
problem Reducing data transfer in IoT devices over wireless channels.
method Developed a machine learning framework for distributed functional compression over GMAC and AWGN channels.
result The framework reduces communication by two orders of magnitude compared to cloud-based methods.
Machine learning predicts liquid water properties from cluster data.
problem Accuracy of bulk properties from machine-learned potentials is limited by training data.
method Local, atom-centred descriptors enable prediction of bulk properties from cluster data.
result Excellent agreement with experimental and theoretical counterparts of liquid water properties.
Teaching is critical to human society: it is with teaching that prospective students are educated and human civilization can be inherited and advanced. A good teacher not only provides his/her students with qualified teaching materials (e.g., textbooks), but also sets up appropriate learning objectives (e.g., course pr…
Machine learning methods for solving the equations of dynamical mean-field theory are developed. The method is demonstrated on the three dimensional Hubbard model. The key technical issues are defining a mapping of an input function to an output function, and distinguishing metallic from insulating solutions. Both meta…
A new algorithm for learning from functional data across multiple machines.
problem Handling large-scale functional data analysis.
method Distributed Gradient Descent Functional Learning (DGDFL) algorithm.
result First theoretical understanding and optimal learning rates for DGDFL.
Method uses DNNs to approximate functions with specific asymptotic behavior.
problem Approximating functions with given asymptotic behavior.
method Specifically constructed terms combined with unconstrained DNN.
result Enforcing asymptotic behavior leads to better approximation and faster convergence.
Machine learning should incorporate maximum likelihood for better estimation.
problem Lack of rigorous foundational theory in machine learning.
method Integrate maximum likelihood estimation into machine learning models.
result Foundationally rigorous machine learning models have greater practical impact.
This paper explores ratio-based loss functions for machine learning.
problem Margin-based and distance-based loss functions for classification and regression.
method Investigation of ratio-based loss functions' properties.
result Proposed new ratio-based loss functions for regression.
A machine learning method selects optimal orthonormal bases for functional data analysis.
problem Lack of formal criteria for choosing initial orthonormal bases in functional data methods.
method Proposes a machine learning algorithm to learn and place knots for efficient orthogonal spline bases (splinets).
result Demonstrates efficiency, especially for sparse functional data and complex physical systems.
SGD with machine learning noise converges to global minimum exponentially fast.
problem Optimizing machine learning models with stochastic gradient descent.
method Analysis of SGD with machine learning noise, focusing on energy landscapes and gradient noise.
result SGD converges to the global minimum exponentially fast under certain conditions.
Teaches matrix calculus for machine learning and optimization.
problem Computing derivatives of functions involving matrices.
method Extends differential calculus to vector spaces, focusing on practical applications in machine learning.
result Introduction of adjoint and reverse-mode differentiation for efficient computation.
The article proposes optimal learning strategies for machine learning-based reliability analysis.
problem Improving computational efficiency and accuracy in machine learning-based reliability analysis.
method Theorems and mathematical proofs for optimal learning strategies considering and neglecting correlations among design samples.
result The optimal learning strategy considering Kriging correlation outperforms other methods in terms of reduced evaluations of performance functions.
FiberNet integrates geometry into machine learning for clearer classification.
problem Lack of interpretability in traditional deep learning.
method Reformulates classification as geometric optimization on fiber bundles, introducing learnable Riemannian metrics and variational prototype optimization.
result Clear geometric interpretability and efficiency in classification.
Function trees simplify complex ML models for better understanding.
problem Understanding and interpreting machine learning model predictions.
method Representing a multivariate function as a tree of simpler functions.
result Function trees reveal the global internal structure of functions.
Localized debiased machine learning simplifies estimating quantile treatment effects.
problem Estimating quantile treatment effects in causal inference with many covariates and flexible relationships.
method Localized debiased machine learning (LDML) avoids learning the full nuisance function by estimating only at a single initial guess.
result LDML enables practically-feasible and theoretically-grounded efficient estimation of quantile treatment effects.
New tests compare regression functions using machine learning, overcoming dimensionality issues.
problem Comparing regression functions in high-dimensional settings.
method Generalized kernel-based conditional mean dependence, machine learning methods for flexible estimation.
result Established asymptotic properties of tests under fixed and high-dimensional regimes.
Prediction markets show considerable promise for developing flexible mechanisms for machine learning. Here, machine learning markets for multivariate systems are defined, and a utility-based framework is established for their analysis. This differs from the usual approach of defining static betting functions. It is sho…
New method improves submodular maximization for machine learning applications.
problem Inexact monotonicity in submodular functions limits traditional algorithms' performance.
method Introduces monotonicity ratio as a continuous version of monotonicity, leading to improved approximation guarantees.
result Improved approximation ratios for movie recommendation, quadratic programming, and image summarization.
DIGEN benchmark provides synthetic datasets for ML algorithm evaluation.
problem Understanding and comparing machine learning algorithms' performance.
method Synthetic datasets generated using 40 mathematical functions to evaluate machine learning algorithms.
result DIGEN resource facilitates understanding why algorithms perform poorly and provides ideas for improvement.
m-arcsinh improves SVM and MLP reliability and speed in scikit-learn.
problem Improving SVM and MLP reliability and speed in scikit-learn.
method Modified arcsinh function for kernel and activation in SVM and MLP.
result Competitive classification performance and reliability of SVM and MLP with m-arcsinh.
Gradient span algorithms show consistent progress in high dimensions.
problem Understanding consistent training progress in large machine learning models.
method Proving deterministic behavior of gradient span algorithms on Gaussian random functions.
result Gradient span algorithms have asymptotically deterministic behavior in high dimensions.
This note shows how to transform high-probability to in-expectation guarantees in machine learning.
problem The challenge of constructing reliable machine learning models due to sampling randomness.
method Transforming high-probability to in-expectation guarantees using a witness condition for unbounded loss functions.
result A technical transformation method for generalization guarantees in machine learning.
Paper uses machine learning to optimize VNF placement for reduced delay.
problem Optimizing VNF placement for reduced delay and cost.
method Developed a machine learning decision tree model to learn from VNF placement data.
result Model reduces delay between VNF instances and across SFC.
Bayesian nonparametric machine learning improves instrumental variable inference.
problem Estimating causal effects with nonlinear relationships.
method Bayesian Additive Regression Trees (BART) for estimating functions and Dirichlet Process mixtures for error terms.
result Dramatic improvements in inference with nonlinear data, no manual tuning required.
RS-HDMR-GPR simplifies complex functions with machine-learned lower-dimensional terms.
problem Representing and understanding complex multidimensional functions with sparse data.
method Random Sampling High Dimensional Model Representation Gaussian Process Regression (RS-HDMR-GPR).
result Facilitates recovery of functional dependence and adds insight into input variable importance.
New theory challenges traditional machine learning assumptions.
problem Traditional machine learning theories are critiqued.
method A new theory is proposed and discussed.
result Learning true probabilities is not equivalent to other learning goals.
AI can learn true probabilities if data and assumptions align.
problem Understanding when AI models can accurately represent true objective probabilities.
method Proved conditions under which AI can learn true probabilities.
result Conditions for learning true probabilities are identified.
Develops a mathematical model for automatic differentiation in machine learning.
problem Current automatic differentiation lacks a simple mathematical model for machine learning.
method Articulates relationships between program differentiation and nonsmooth functions, provides a class of functions and nonsmooth calculus.
result Shows how nonsmooth calculus applies to stochastic approximation methods and evidence of artificial critical points.
Kernel for STL formulae enables machine learning in temporal logic.
problem Lack of a kernel for STL formulae.
method Define a kernel for STL formulae and embed them into a Hilbert space.
result Kernel-based machine learning algorithms can now be applied to STL formulae.
SGLB boosts machine learning with Langevin diffusion for multimodal loss functions.
problem Dealing with multimodal loss functions in machine learning.
method Stochastic Gradient Langevin Boosting (SGLB) based on Langevin diffusion equation.
result SGLB guarantees global convergence for multimodal loss functions.