Unified model for shallow and deep learning models.
problem Unified representation of shallow and deep learning models.
method Reconciled polynomial machine predicting output via feature kernel and variable reconciling function.
result Existing models can be reduced to reconciled polynomial machine representations.
Study on functions computed by deep-layered machines finds same distribution in neural networks and Boolean circuits.
problem Understanding the space of functions computed by deep-layered machines.
method Investigation of Boolean functions on random-layered machines, including neural networks and Boolean circuits.
result The space of functions computed at large depth limit is characterized and the macroscopic entropy of Boolean functions is either monotonically increasing or decreasing with depth.
RBM and DBM are represented as 2D tensor networks, revealing their expressive power and efficiency.
problem Understanding and optimizing RBM and DBM models.
method Representing RBM and DBM as 2D tensor networks and developing an efficient tensor network contraction algorithm.
result The proposed algorithm for computing partition functions is more accurate than state-of-the-art methods.
Deep networks are mathematically equivalent to kernel machines learned by gradient descent.
problem Understanding the learned representations of deep learning models.
method Using gradient descent to learn deep networks, showing they are equivalent to kernel machines.
result Deep network weights are a superposition of training examples, revealing the learned function.
New CFNN architecture approximates functions with machine accuracy.
problem Function approximation with high precision.
method Chebyshev Feature Neural Network (CFNN) with learnable frequencies.
result Achieves machine accuracy in function approximation.
DFMs generalize neural networks for topological layer design.
problem Designing neural networks that can handle high-resolution data without parameter dependence.
method Introducing deep function machines (DFMs) that are invariant to input dimensionality.
result DFMs can approximate bounded non-linear operators between function spaces.
FiberNet integrates geometry into machine learning for clearer classification.
problem Lack of interpretability in traditional deep learning.
method Reformulates classification as geometric optimization on fiber bundles, introducing learnable Riemannian metrics and variational prototype optimization.
result Clear geometric interpretability and efficiency in classification.
Deep Sets models for set-based tasks, invariant to permutations.
problem Designing models for tasks defined on sets, invariant to permutations.
method Characterized permutation invariant functions, designed deep network architecture.
result Deep network architecture for set-based tasks, invariant to permutations.
Method uses DNNs to approximate functions with specific asymptotic behavior.
problem Approximating functions with given asymptotic behavior.
method Specifically constructed terms combined with unconstrained DNN.
result Enforcing asymptotic behavior leads to better approximation and faster convergence.
Proposes a new loss function for causal machine-learning.
problem Lack of a well-defined loss function for causal machine-learning.
method Introduces a novel loss function equal to MSE in a standard regression problem.
result Demonstrates that gradient descent can be directly applied to this loss function.
Proposes MFI for non-linear learning to explain complex feature interactions.
problem Lack of straightforward interpretation for non-linear learning models.
method Introduces Measure of Feature Importance (MFI) for any learning machine.
result MFI detects and explains complex feature interactions.
This work uses variational quantum circuits for deep reinforcement learning.
problem Intractability of deep quantum circuits on existing quantum computing platforms.
method Reshaping classical deep reinforcement learning algorithms into variational quantum circuits and using quantum information encoding.
result First proof-of-principle demonstration of variational quantum circuits for deep reinforcement learning.
Introduces epistemic deep learning for better uncertainty estimation in neural networks.
problem Uncertainty quantification in deep neural networks.
method Random-set convolutional neural networks with belief function-based loss functions.
result Epistemic approach produces better performance in uncertainty estimation.
Deep Boltzmann machines are in principle powerful models for extracting the hierarchical structure of data. Unfortunately, attempts to train layers jointly (without greedy layer-wise pretraining) have been largely unsuccessful. We propose a modification of the learning algorithm that initially recenters the output of t…
Deep Causal Graphs model complex causal relationships using neural networks.
problem Limited applicability of parametric causal models to real-life datasets with non-linear relationships.
method Deep Causal Graphs, an abstract specification for neural networks to model causal distributions.
result Demonstrates expressive power in modelling complex interactions and provides true causal counterfactuals.
Deep neural networks with piecewise-polynomial activations can approximate smooth functions and their derivatives.
problem Approximating smooth functions and their derivatives with neural networks.
method Derives the depth, width, and sparsity required for approximation in Hölder norms.
result Deep neural networks with bounded weights can approximate Hölder smooth functions and their derivatives.
Deep learning reduces complex data to simpler predictors.
problem High-dimensional data reduction in input-output models.
method Hierarchical layers of latent features for constructing predictors.
result Deep learning is a black-box method for high-dimensional function estimation.
Deep neural networks are optimizable due to their multilayered structure.
problem Understanding why deep neural networks are easily optimizable despite their non-convex loss functions.
method Analysis of a spin glass model of deep neural networks using random matrix theory and algebraic geometry.
result The multilayered structure of deep neural networks leads to fewer stationary points, more clustered minima, and less severe tradeoffs between depth and width of minima.
A theorem for Hilbert space-valued functions simplifies machine learning models.
problem Learning functions in Hilbert space-valued spaces.
method Generalized Representer Theorem for Hilbert space-valued functions.
result Unified view of supervised and semi-supervised learning methods.
Optimizes kernel machines using deep learning for efficiency and end-to-end learning.
problem Limited training data and lack of explicit feature sources.
method DKMO (Deep Kernel Machine Optimization) framework using Nystrom kernel approximations and deep learning.
result Improved training convergence and effectiveness over conventional model inferencing techniques.
This paper identifies duplicate questions on Quora using machine and deep learning models.
problem Detecting semantically identical questions on Quora to improve user experience.
method Applied machine learning and deep learning techniques on Quora's dataset.
result Xgboost model with character level term frequency and inverse term frequency achieved 85.82% accuracy.
Survey of deep learning methods for fMRI natural image reconstruction.
problem Reconstructing natural images from fMRI brain activity.
method Survey of deep learning approaches, including architectural design, datasets, and evaluation metrics.
result Performance evaluation across standardized metrics.
Estimates and convergence of neural network approximations without structural assumptions.
problem Estimating and understanding the generalization error of neural networks.
method Introducing a new approach to estimate and analyze the convergence of neural network approximations.
result Estimates of the error without structural assumptions and convergence under mild regularity assumptions.
Deep neural network predicts molecular wave functions in minimal basis.
problem Improving accuracy and efficiency in quantum chemistry calculations.
method Adapted SchNet for Orbitals (SchNOrb) model in quasi-atomic minimal basis.
result Model accurately predicts molecular orbital energies and wavefunctions for large molecules.
Study explores bias-variance in adversarial machine learning.
problem Understanding adversarial machine learning's impact on bias and variance.
method Investigates bias-variance trade-offs in deep neural networks using MSE and cross-entropy.
result Derives bias-variance trade-offs for classification and regression.
Paper reveals hidden convexities in deep learning models using sparse signal processing.
problem Non-convex loss functions in deep learning models complicate optimization and theoretical understanding.
method Developed convex equivalences of ReLU NNs and their connections to sparse signal processing models.
result Recent research has uncovered hidden convexities in certain NN architectures, notably two-layer ReLU networks and other architectures.
DeepMed uses DNNs to estimate causal mediation effects without sparsity constraints.
problem Estimating Natural Direct and Indirect Effects in mediation analysis.
method DeepMed employs deep neural networks to cross-fit infinite-dimensional nuisance functions.
result DeepMed achieves semiparametric efficiency bound and adapts to low-dimensional nuisance structures.
SGD with machine learning noise converges to global minimum exponentially fast.
problem Optimizing machine learning models with stochastic gradient descent.
method Analysis of SGD with machine learning noise, focusing on energy landscapes and gradient noise.
result SGD converges to the global minimum exponentially fast under certain conditions.
Deep learning networks learn complex data structures.
problem Understanding the internal functioning of deep learning networks.
method Using complex network analysis techniques to study deep belief networks.
result Gained insights into the structural and functional properties of deep learning networks.
Approach uses machine learning to identify structural modal parameters from output-only data.
problem Identifying modal parameters from output-only data for structural health monitoring.
method Unsupervised learning using a self-coding deep neural network to separate modal responses from vibration data.
result The approach effectively identifies structural modal parameters from system responses.
Enhances demand models with deep learning for personalized pricing.
problem Capturing rich heterogeneity in demand models.
method Integrates deep neural networks into structural models with economic structure.
result Captures rich heterogeneity and creates personalized pricing.
Sherpa automates hyperparameter tuning for machine learning models.
problem Hyperparameter optimization for computationally expensive models.
method Powerful and interchangeable algorithms, interactive dashboard.
result Automates tedious aspects of model tuning.
Deep QMC methods use neural networks to solve quantum chemistry problems.
problem Solving the electronic Schrödinger equation from first principles.
method Quantum Monte Carlo with neural network wavefunctions.
result Highly accurate solutions at reduced computational cost.
Framework enhances DNNs against malware attacks.
problem Adversarial malware detection in DNNs.
method Hashing Transformation Deep Neural Networks (HashTran-DNN) with DAE regularizer.
result HashTran-DNN effectively defends against all four known attacks.
Deep learning improves optimal trading strategies in complex markets.
problem Finding optimal trading strategies in markets with multiple time scales.
method Deep differentiable reinforcement learning applied to optimal trading.
result Deep learning leads to more accurate and stable optimal trading strategies.
Deep Sets approximates functions on sets with high-dimensional latent space.
problem Modeling functions of sets (permutation-invariant functions).
method Deep Sets, a method known to be a universal approximator for continuous set functions.
result Deep Sets' universal approximation property is only guaranteed with a sufficiently high-dimensional latent space.
New method trains deep neural networks for non-interacting kinetic-energy functionals in DFT.
problem Lack of exact relationship between electron density and non-interacting kinetic energy.
method Variational principle to regularize machine-learned density functionals.
result Excellent results on kinetic-energy functionals for various systems.
Deep learning predicts personalized drug responses from medical data.
problem Predicting personalized drug responses from medical data.
method Deep learning algorithms applied to large datasets.
result Deep learning improves prediction of personalized drug responses.
Influence functions are inaccurate in deep learning models, especially for deeper networks.
problem Inaccuracies in influence functions in deep learning models.
method Empirical study of influence functions in neural network models trained on various datasets.
result Influence estimates are often erroneous for deeper networks and require regularization.
MBPEP algorithm improves uncertainty prediction for unknown input distributions.
problem Difficulty in providing high-quality machine learning solutions for unknown input distributions.
method Margin-based Pareto deep ensemble pruning (MBPEP) model using deep ensemble networks and unique loss functions.
result Achieves small prediction interval width and high confidence of prediction interval coverage probability.
MASS Learning trains models to use minimal sufficient statistics, improving performance and uncertainty quantification.
problem Training deep networks to use minimal sufficient statistics for better performance and uncertainty quantification.
method MASS Learning trains models to produce minimal sufficient statistics with respect to a class of functions, using Conserved Differential Information (CDI).
result Deep networks trained with MASS Learning achieve competitive performance on supervised learning and uncertainty quantification benchmarks.
Paper proposes quantum methods for optimizing machine learning functions.
problem Machine learning optimization problem
method Average approach and Partial Swap Test Cut-off method (PSTC)
result Potential to improve PSTC to O ( ∣ Θ ∣ ⋅ s u b l i n e a r N ) O(\sqrt{|Θ|} \cdot sublinear \ N) O ( ∣Θ∣ ⋅ s u b l in e a r N ) Bayesian optimization has become a successful tool for hyperparameter optimization of machine learning algorithms, such as support vector machines or deep neural networks. Despite its success, for large datasets, training and validating a single configuration often takes hours, days, or even weeks, which limits the ach…
FiBiNET combines feature importance and bilinear interactions for CTR prediction.
problem Improving click-through rate prediction in advertising and feed ranking systems.
method FiBiNET dynamically learns feature importance via SENET and bilinear feature interactions.
result FiBiNET outperforms shallow and deep models on real-world datasets.
Paper presents deep learning and ML for automated student performance estimation.
problem Evaluation of students' performance during the pandemic.
method In-depth analysis of deep learning and machine learning approaches.
result Better performance across different prediction tasks with fully data-driven approach.
This paper surveys deep learning applications in machine health monitoring.
problem Data-driven machine health monitoring in modern manufacturing systems.
method Review of deep learning techniques and their applications in machine health monitoring.
result Deep learning provides useful tools for processing and analyzing machinery data.
A framework uses deep learning for spatio-temporal data prediction.
problem Interpolation of continuous spatio-temporal fields on irregular points.
method Decomposes spatio-temporal processes into products of basis functions and spatial coefficients.
result Effectiveness in reconstructing coherent spatio-temporal fields.
This work analyzes how different layers in deep neural networks contribute to generalization error.
problem Understanding the role of each layer in deep neural networks for generalization.
method Spectral analysis, Neural Tangent Kernel, Hermite polynomials, Spherical Harmonics.
result Initial layers in deep neural networks have a larger bias towards high-frequency functions.