Echo optimizes LSTM RNN training on GPUs by reducing memory footprint.
problem Memory bottleneck in LSTM RNN training on GPUs.
method Compiler-based feature map recomputation to estimate and balance memory and execution time.
result Average 1.89X reduction in GPU memory footprint.
A new method reduces memory usage in neural networks by 36-81%.
problem Minimizing memory usage in neural networks during backpropagation.
method Formalized as a graph theory problem, uses dynamic programming to minimize computational overhead.
result Reduces peak memory consumption by 36-81% on various networks.
Paper proposes a non-iterative method to speed up DCNN training.
problem Training deep convolutional neural networks (DCNN) is time-consuming.
method Eliminates backpropagation at dense layers using Moore-Penrose Inverse.
result Significant performance improvements over 30 state-of-the-art methods.
A new algorithm reduces memory usage for deep learning models.
problem Training deep learning models requires significant memory.
method Dynamic Tensor Rematerialization (DTR) is a greedy online algorithm that dynamically plans recomputations.
result DTR achieves comparable performance to optimal static checkpointing with only a small memory budget.
Bieri, Geoghegan and Kochloukova computed the BNSR-invariants Σ m ( F ) Σ^m(F) Σ m ( F ) of Thompson's group F F F for all m m m . We recompute these using entirely geometric techniques, making use of the Stein--Farley CAT(0) cube complex on which F F F acts.
ERNet reduces CNN bandwidth needs for imaging tasks.
problem High DRAM bandwidth requirements for CNNs in imaging.
method ERNet uses block-based inference and temporary layer expansion.
result ERNet outperforms state-of-the-art models in image denoising and super-resolution.
UAVs learn to collect data from IoT sensors efficiently.
problem Efficient UAV path planning for wireless data collection.
method Deep reinforcement learning (DDQN) with experience replay and convolutional layers.
result UAV control policy generalizes over changing scenario parameters.
ACP-UCB1 ranks arms based on upper-tail performance, improving stochastic bandit algorithms.
problem Stochastic bandit algorithms often favor arms with strong upper-tail performance, which is not well-addressed by classical mean-reward criteria.
method ACP-UCB1 combines an adaptive conformal estimate of the upper endpoint with a UCB-type optimism bonus.
result ACP-UCB1 achieves logarithmic upper-quantile regret with per-arm contribution \(O(
icefrac{\log n}{Δ_j^{\mathrm{ACP}}})\).
New ODE solvers improve training efficiency and accuracy.
problem Training Neural ODEs requires efficient and accurate gradient calculation.
method Presented algebraically reversible ODE solvers that are time and memory efficient, calculate exact gradients, and are numerically stable.
result Reversible solvers strictly improve upon previous architectures in efficiency and accuracy.
iCVI-ARTMAP accelerates clustering with adaptive resonance theory and validity indices.
problem Improving clustering efficiency and accuracy using adaptive resonance theory.
method Integrates adaptive resonance theory (ARTMAP) with incremental cluster validity indices (iCVIs) for clustering.
result Significantly reduces clustering time and outperforms other methods on synthetic and real-world data.
Combines experience replay techniques to improve reinforcement learning.
problem Improving reinforcement learning algorithms.
method Combines CER, PER, and HER with DDPG and DQN.
result Effective combinations of these techniques in various environments.
Random feature maps improve forecasting with cheaper computation.
problem Improving forecasting accuracy with random feature maps.
method Developed a hit-and-run algorithm to select optimal internal weights.
result Optimal internal weights lead to superior forecasting skill.
Quantum machine learning models can approximate any continuous function.
problem Theoretical understanding of quantum feature maps in machine learning.
method Proving universal approximation property of quantum machine learning models in quantum-enhanced feature spaces.
result Quantum machine learning models are universal approximators of continuous functions.
Unified feature maps for graph kernels improve efficiency without sacrificing accuracy.
problem Efficiently applying non-linear kernel methods to large-scale graph data.
method Constructing feature maps for graph kernels, analyzing feasibility, and proposing algorithms.
result Explicit feature maps can achieve similar accuracy to kernel trick methods but with significantly reduced computation time.
Neural networks can separate non-separable data using feature maps.
problem Non-separable data in neural networks.
method Characterization of feedforward neural networks and use of feature maps.
result ReLU neural networks can separate concentric data.
A new method, k-SVRG, speeds up large-scale optimization.
problem Efficiently solving large-scale optimization problems with variance reduction.
method k-SVRG, which uses available memory and minimizes stalling phases.
result Proves linear convergence on strongly convex problems and convergence to stationary points on non-convex problems.
We analyze random feature maps for high-dimensional data using spectral methods.
problem Understanding the spectrum of random feature maps for high-dimensional data.
method We use concentration phenomena from random matrix theory to analyze the Gram matrix of random feature maps for Gaussian mixture models.
result Our results provide insights into the interplay between nonlinearity and data statistics.
Random feature maps improve forecasting of chaotic dynamical systems.
problem Forecasting chaotic dynamical systems with high accuracy.
method Data-driven random feature maps with tanh activation, skip connections, and localization.
result Effective forecasting skill for dynamical systems with dimensions up to 512.
This paper proposes an online knowledge distillation method that transfers feature map information in addition to class probabilities.
problem Previous online knowledge distillation methods only utilized class probabilities, missing feature map information.
method Adversarial training framework to transfer feature map information; multiple networks trained simultaneously with discriminators.
result Our method performs better than direct alignment methods and is more suitable for online distillation.
ERM performs well in feature learning with minimal feature maps.
problem Empirical risk minimization in feature learning with square loss.
method Asymptotic and non-asymptotic analysis of ERM performance.
result Excess risk quantiles of ERM match those of oracle procedure under certain conditions.
A new Gaussian process framework uses neural feature maps for scalable, accurate inference.
problem Efficient and accurate Gaussian process inference for diverse data types.
method Neural feature maps to construct expressive kernels, with theoretical guarantees and practical scalability.
result The approach outperforms existing methods in accuracy and efficiency across various data modalities.
RFSVM with random features achieves faster learning rates.
problem Improving the learning rate of SVM with random features.
method Support Vector Machine with N ≪ m N\ll m N ≪ m random features, optimized feature map, and reweighted feature selection. result RFSVM achieves faster learning rates than O ( 1 / m ) O(1/\sqrt{m}) O ( 1/ m ) under low noise assumptions. Improved disentanglement in VAEs using aggregated feature maps.
problem Improving disentanglement in Variational Autoencoders (VAEs).
method Regionally aggregated feature maps extracted from pre-trained CNNs on ImageNet.
result 2nd place in NeurIPS 2019 disentanglement challenge.
Improves machine learning models by incorporating physical laws into feature maps.
problem Lack of model interpretability in classical machine learning approaches.
method Physics-informed feature maps constructed from physical laws and dimensional analysis.
result Enhanced model interpretability and potential discovery of new physical equations.
Improved disentanglement through learned feature aggregation.
problem Disentangling latent factors in images.
method Variational autoencoder trained on regionally aggregated feature maps from ImageNet.
result 2nd place in NeurIPS 2019 disentanglement challenge.
New feature map for topological data analysis improves classification performance.
problem Lack of effective feature maps for topological data analysis.
method Realize barcodes as paths in a vector space, compute path signature, resulting in a feature map.
result Achieves state-of-the-art results on classification benchmarks.
Aligns databases with Gaussian features using MAP estimation and thresholding.
problem Aligning databases with jointly Gaussian features.
method Two algorithms: MAP estimation and thresholding of log likelihood ratios.
result Conditions on mutual information for reliable performance of algorithms.
Improves neural network mapping functionality using latent feature generation.
problem Improving neural network performance in visual recognition tasks.
method Reversible learning for generating and learning latent features.
result The proposed method outperforms existing state-of-the-art methods in visual recognition.
New feature mapping approach improves recommendation accuracy and explainability.
problem Balancing recommendation accuracy and explainability using metadata.
method Maps uninterpretable features to interpretable aspect features, minimizing both prediction and interpretation losses.
result Strong performance in recommendation and explainability, eliminating metadata need.
Lower bound proves ridgeless regression performs poorly near interpolation threshold.
problem Proving performance of ridgeless regression near interpolation threshold.
method Distribution-independent lower bound for mean squared error in noisy ridgeless linear regression.
result Lower bound implies ridgeless regression performs poorly near interpolation threshold.
Proposes a method for kernel learning using feature maps.
problem Improving SVM margin through iterative refinement.
method Fourier-analytic characterization and iterative feature maps.
result Optimal and generalization guarantees for SVM margin improvement.
Deep learning maps road safety features from streetview imagery.
problem Accurate mapping of road safety features for safer roads.
method Proposes a deep learning approach with LSTM to capture spatial autocorrelation.
result Model outperforms baseline methods in mapping road safety features.
We propose different schemes for option hedging when asset returns are modeled using a general class of GARCH models. More specifically, we implement local risk minimization and a minimum variance hedge approximation based on an extended Girsanov principle that generalizes Duan's (1995) delta hedge. Since the minimal m…
Paper proposes an efficient RL algorithm for discounted MDPs using feature mapping.
problem Efficient reinforcement learning for large state and action spaces.
method Uses feature mapping to represent states and actions in a low-dimensional space, proposing a novel algorithm with polynomial regret bound.
result Achieves a O ( d T / ( 1 − γ ) 2 ) O(d\sqrt{T}/(1-γ)^2) O ( d T / ( 1 − γ ) 2 ) regret bound, near-optimal up to a ( 1 − γ ) − 0.5 (1-γ)^{-0.5} ( 1 − γ ) − 0.5 factor. Proposes a new method to prevent overfitting in deep neural networks.
problem Overfitting in deep neural networks with many trainable parameters.
method Randomly replaces elements in feature maps with specific values during training.
result Improves the testing performance of deep neural networks on benchmark datasets.
A planning mechanism improves sequence-to-sequence model performance.
problem Improving sequence-to-sequence model performance for tasks like translation and graph analysis.
method Developed a model that plans ahead using attention and differentiable operations.
result Outperforms strong baselines on character-level translation, graph analysis, and question generation.
Characterizes test error in learning with deep, structured feature maps.
problem Characterizing test error in learning with deep, structured feature maps.
method Asymptotic analysis of feature covariance and population covariance.
result Closed-form formula for feature covariance in Gaussian rainbow neural networks.
This paper examines how energy in feature maps decays in deep neural networks.
problem Understanding energy decay in deep convolutional neural networks.
method Analyzes energy conservation and decay rates in various deep neural network architectures.
result Energy in feature maps decays polynomially or exponentially across layers.
Framework improves ML flood mapping generalization.
problem Improving machine learning models' ability to generalize to new conditions.
method Dimensionless, multi-scale features constrained by the Buckingham Π theorem.
result Model outperformed dimensional features, improving AUC in unmapped areas.
Paper explores Polya's characterization of positive-definite kernels and random feature maps.
problem Characterizing positive-definite kernels and their random feature maps.
method Study Polya's criterion and derive novel kernels; compare random Fourier and binning feature maps.
result Random binning feature map yields a closer Euclidean inner product to the kernel.
NEU learns feature maps for any model class preserving UAP.
problem Effective feature representation for predictive performance.
method Meta-procedure NEU for UAP-invariant feature maps.
result NEU learns feature maps with UAP for most model classes.
Sensitivity Maps improve HSIC for better interpretability and scalability.
problem High computational cost and lack of interpretability in HSIC.
method Introduce Sensitivity Maps (SMs) for HSIC, approximate kernels using random features, and provide convergence bounds.
result RHSIC and SMs efficiently approximate HSIC and provide scalable solutions.
Map matching of GPS trajectories from a sequence of noisy observations serves the purpose of recovering the original routes in a road network. In this work in progress, we attempt to share our experience of feature construction in a spatial database by reporting our ongoing experiment of feature extrac-tion in Conditio…
Method learns feature map between source and target domains for high-dimensional regression with missing features.
problem High-dimensional regression with differing feature sets in target and source domains.
method First learns a feature map between missing and observed features using source data, then imputes missing features in target domain, and performs two-step transfer learning for penalized regression.
result Developed upper bounds on estimation and prediction errors for HTL, showing dependence on model complexity, sample size, feature map quality, and domain differences.
Kernel approximation using randomized feature maps has recently gained a lot of interest. In this work, we identify that previous approaches for polynomial kernel approximation create maps that are rank deficient, and therefore do not utilize the capacity of the projected feature space effectively. To address this chal…
Novel approach uses neural networks to enhance CI testing for feature selection.
problem Challenges in implementing Markov blanket feature selection due to CI testing limitations.
method Two-step approach: feature mapping followed by CI testing using k k k -NN. result The method boosts CI testing performance, leading to improved feature selection.
The paper develops a method to select features from multiple kernels for efficient risk minimization.
problem Identifying promising features leading to satisfactory out-of-sample performance in nonlinear kernel approximation.
method A greedy selection process using a correlation metric to choose features from multiple kernels.
result An out-of-sample error bound capturing trade-offs between approximation and spectral errors, showing poly-logarithmic scaling with data.
Paper tackles speaker verification by removing reverberation using deep LSTM networks.
problem Improving speaker verification accuracy in reverberant environments.
method Dual-label deep LSTM networks trained to map reverberant to clean speech features.
result Evaluates performance using EERs, showing improved accuracy.