Random feature model approximates PDE solutions efficiently.
problem Approximating solutions to PDEs with high-dimensional inputs and outputs.
method Random feature model applied to infinite-dimensional operators.
result Efficient and accurate approximation of PDE solutions.
DANCE improves saliency maps by adding subtle input variations.
problem Poor performance of saliency methods in saturated gradients, adversarial perturbations, and inter-feature dependence.
method Two-step procedure: 1) Perturbation mechanism, 2) Aggregation of saliency maps.
result DANCE saliency method outperforms existing methods qualitatively and quantitatively.
NEU learns feature maps for any model class preserving UAP.
problem Effective feature representation for predictive performance.
method Meta-procedure NEU for UAP-invariant feature maps.
result NEU learns feature maps with UAP for most model classes.
New method for visualizing high-level concepts in generative models.
problem Challenges in evaluating and visualizing concepts in generative models.
method Introduces a method to compute concept saliency maps for latent representations of known or novel high-level concepts.
result Concept saliency maps highlight input features important for high-level concepts.
Lower bound proves ridgeless regression performs poorly near interpolation threshold.
problem Proving performance of ridgeless regression near interpolation threshold.
method Distribution-independent lower bound for mean squared error in noisy ridgeless linear regression.
result Lower bound implies ridgeless regression performs poorly near interpolation threshold.
Many practical machine learning tasks employ very deep convolutional neural networks. Such large depths pose formidable computational challenges in training and operating the network. It is therefore important to understand how fast the energy contained in the propagated signals (a.k.a. feature maps) decays across laye…
Method sanitizes IFM in CNN layers to control privacy loss.
problem Controlling privacy loss in CNNs using input feature maps.
method Sample-and-hold approximation scheme to sanitize IFM, unfolding tensors for independence from CNN configuration.
result Control the privacy loss by adjusting the sanitization degree.
BlurNet defends against adversarial attacks by filtering feature maps.
problem Adversarial attacks on deep neural networks, especially for image classification.
method BlurNet introduces a depthwise convolution layer with standard blur kernels after the first layer to filter high frequency noise.
result The defense reduces the success rate of adversarial attacks from 90% to 20% with total variation regularization.
The paper analyzes transportation dynamics of denoising autoencoders.
problem Analyzing the behavior of deep neural networks.
method Regarding feature maps as transport maps and analyzing transportation dynamics.
result The infinitely deep DAE transports mass to decrease entropy of data distribution.
Proposes FBFAN to defend against adversarial attacks by learning semantic features.
problem Vulnerability of deep neural networks to adversarial attacks.
method Featurized Bidirectional Generative Adversarial Networks (FBGAN) that learns semantic features and filters non-semantic perturbations.
result FBGAN effectively reconstructs adversarial data to denoised data, improving classifier performance.
The paper provides risk bounds for learning many response functions using linear regression.
problem Learning many response functions from a single dataset.
method Ordinary least squares regression in a high-dimensional feature space.
result Convergence guarantees on worst-case excess prediction risk for infinite response functions with finite VC dimension.
Space-efficient feature maps improve string alignment kernel scalability.
problem String alignment kernels scale poorly with quadratic complexity, limiting large-scale applications.
method Presented SFMEDM, a space-efficient feature map for edit distance with moves using metric embedding and random Fourier features.
result Demonstrated superior performance of SFMEDM in prediction accuracy, scalability, and computation efficiency.
New method selects variables for nonlinear regression in large datasets.
problem Nonlinear regression with large-scale datasets.
method Kernel-based variable selection with random features.
result Outstanding performance on large-scale synthetic and real datasets.
Featurization improves density ratio estimation for complex data.
problem Difficulty in estimating density ratios for high-dimensional, different distributions.
method Invertible generative model to map distributions into a common feature space.
result Improved accuracy in density ratio estimation through feature space.
In this article, we extend the conventional framework of convolutional-Restricted-Boltzmann-Machine to learn highly abstract features among abitrary number of time related input maps by constructing a layer of multiplicative units, which capture the relations among inputs. In many cases, more than two maps are strongly…
A new Gaussian process framework uses neural feature maps for scalable, accurate inference.
problem Efficient and accurate Gaussian process inference for diverse data types.
method Neural feature maps to construct expressive kernels, with theoretical guarantees and practical scalability.
result The approach outperforms existing methods in accuracy and efficiency across various data modalities.
D2KE converts distance metrics to kernels for machine learning.
problem Machine learning with structured inputs often lacks vector representations.
method Proposes a framework to derive kernels from dissimilarity measures.
result Functions in the RKHS are Lipschitz-continuous with respect to the distance metric.
Characterizes test error in learning with deep, structured feature maps.
problem Characterizing test error in learning with deep, structured feature maps.
method Asymptotic analysis of feature covariance and population covariance.
result Closed-form formula for feature covariance in Gaussian rainbow neural networks.
DenseNets improve accuracy and efficiency in convolutional networks.
problem Improving accuracy and efficiency in deep convolutional networks.
method Introducing Dense Convolutional Networks (DenseNet) with direct connections between all layers.
result DenseNets achieve significant improvements over state-of-the-art networks on object recognition benchmarks.
Random Forest proximity distances reveal feature contributions in black-box models.
problem Understanding feature contributions in complex, opaque machine learning models.
method Observing changes in input affecting proximity distances and instance movement in decision space.
result Each feature's independent contribution to model decisions can be calculated and analyzed.
Input-cell attention improves RNN saliency detection over time.
problem Vanishing saliency in RNNs makes them unreliable for detecting important features at arbitrary time intervals.
method Proposes input-cell attention, a novel RNN cell structure that uses a fixed-size matrix embedding to attend to different inputs from current or previous time steps.
result Input-cell attention produces a saliency map that can detect important features regardless of their occurrence in time.
New method improves interpretability of fMRI decoding models.
problem Uninterpretable deep neural networks in fMRI decoding.
method Adversarial training to make DNNs robust to noise and improved saliency map methods.
result Saliency maps from adversarial-trained DNNs are more interpretable than those from other methods.
Interprets neural network classifiers for categorical inputs.
problem Neural networks' interpretability in human-sensitive applications.
method Mapping to physical energy model, expansion of neural network layers.
result Each layer's contribution to classification can be analyzed.
Transformers can scale both context and task, but MLPs can only scale task.
problem Understanding and scaling In-Context Learning in transformers.
method Simplified transformer architecture, feature map, and MLP combination.
result Simplified transformer can perform ICL and context-scaling but not task-scaling.
NullSpaceNet maps inputs to a joint-nullspace for clearer class separability.
problem Class separability and interpretability in image classification.
method NullSpaceNet maps inputs to a joint-nullspace, collapsing same-class inputs and separating different classes.
result NullSpaceNet achieves superior performance with reduced parameters and time.
Centroid Transformers reduce memory and computation by summarizing inputs into centroids.
problem Efficiently summarize inputs with reduced memory and computation.
method Generalizes self-attention to map N inputs to M centroids (M ≤ N), reducing complexity.
result Centroid Transformers reduce memory and computation while preserving key information.
Inference for latent feature models is inherently difficult as the inference space grows exponentially with the size of the input data and number of latent features. In this work, we use Kurihara & Welling (2008)'s maximization-expectation framework to perform approximate MAP inference for linear-Gaussian latent featur…
PML-GAN tackles noisy multi-label annotations using adversarial learning.
problem Learning multi-label models from noisy, overcomplete annotations.
method PML-GAN uses a disambiguation network and a generative adversarial network to map noisy labels to clean labels and data samples.
result PML-GAN achieves state-of-the-art performance on partial multi-label learning datasets.
Paper introduces a new model to handle multi-task learning across different input domains.
problem Learning correlated tasks across varying input domains.
method Develops a novel heterogeneous stochastic variational linear model of coregionalization (HSVLMC) for multi-task learning.
result The proposed model outperforms existing models in diverse multi-task scenarios.
Learning tasks such as those involving genomic data often poses a serious challenge: the number of input features can be orders of magnitude larger than the number of training examples, making it difficult to avoid overfitting, even when using the known regularization techniques. We focus here on tasks in which the inp…
Transforms improve CNNs' invariance to image transformations.
problem Current CNN models lack robustness to spatial transformations.
method Randomly transform feature maps during training to learn invariant representations.
result Significant improvements on benchmark tasks, including image recognition and retrieval.
Training shapes the geometry of neural network feature maps, revealing local area magnification.
problem Understanding how training affects the geometric structure of neural network feature maps.
method Analyzing the Riemannian geometry induced by neural network feature maps at infinite width and after training.
result Training breaks the symmetry of the geometry induced by random neural network feature maps, magnifying local areas along decision boundaries.
LMGPs extend GPs to handle mixed data, offering better accuracy and interpretability.
problem Handling mixed data types (quantitative and qualitative) in metamodeling.
method Introduce LMGPs that learn a latent manifold for qualitative inputs, using a low-rank linear map.
result LMGPs outperform existing methods in accuracy and versatility.
WassersteinGrad improves weather forecasting explanations by addressing geometric misalignment issues.
problem Improving explainability of autoregressive neural predictions on dynamic physical fields.
method WassersteinGrad, a geometric consensus method for averaged perturbed attribution maps.
result WassersteinGrad provides more accurate explanations for weather forecasting models.
Many-to-Many VTN improves voice conversion across multiple speakers.
problem Voice conversion across multiple speakers.
method Sequence-to-sequence learning framework with many-to-many VTN architecture.
result Improved sound quality and speaker similarity compared to baseline methods.
Random Fourier features improve tabular deep learning convergence.
problem Tabular deep learning convergence issues.
method Random Fourier projections as a pre-processing step, projecting inputs into a fixed feature space.
result Random Fourier pre-processing accelerates tabular deep learning convergence.
A new model classifies multi-lead ECGs better than single-channel models.
problem Classifying multi-lead ECGs for accurate diagnosis.
method Channel-wise attention mechanism in CNNs for multi-lead ECG classification.
result The model achieves better sensitivity and precision than plain ResNet models.
Proposes a new kernel technique for tensor data in SVM.
problem Handling tensorial data in machine learning.
method Kernelized support tensor train machine for image classification.
result Tensorizes the standard SVM on its input structure and kernel mapping scheme.
Translating or rotating an input image should not affect the results of many computer vision tasks. Convolutional neural networks (CNNs) are already translation equivariant: input image translations produce proportionate feature map translations. This is not the case for rotations. Global rotation equivariance is typic…
Framework combines random features with CDEs for efficient time-series learning.
problem Efficient training of time-series models with strong inductive bias.
method Random Fourier CDEs and Random Rough DEs using continuous-time reservoirs and log-ODE discretization.
result Unified perspective on random-feature reservoirs and path-signature theory.
Paper proposes using pairwise feature comparisons to infer modification costs for user recourse.
problem Learning and inferring user preferences for modifying features in black-box models.
method Bradley-Terry model for inferring feature-wise costs from non-exhaustive human comparison surveys.
result Non-exhaustive human surveys can efficiently learn feature costs, enabling recourse finding.
A new modal autoencoder improves feature extraction by constraining the decoder.
problem Improving autoencoder performance through regularization.
method Proposed a modal autoencoder (MAE) by orthogonalizing the readout weight matrix.
result Extracted functionally independent features that perform better in classification tasks.
This paper simplifies deep learning networks by mapping them to a linear function of a feature map.
problem Understanding how weights in deep networks coordinate across layers and generalize.
method Reparameterizes DNNs as a linear function of a feature map, transforming depth-dependencies into tensor products.
result Develops sample compression representation of neural networks in terms of support vectors, showing sample complexity of O(ns/epsilon).
New method learns shape correspondences robustly from raw geometry.
problem Inaccurate and poor generalization of shape correspondences.
method Learning-based approach with feature-extraction network and functional map representation.
result Robust and accurate shape correspondence learning with less training data.
Neural network synthesizes percussive sounds with adjustable timbral features.
problem Control over high-level timbral characteristics of percussive sounds.
method Feedforward convolutional neural network mapping input parameters to waveform.
result Changing input parameters produces a waveform congruent with desired characteristics.
New hypothesis and method reduce noise in saliency maps.
problem Noisy saliency maps from deep neural networks.
method Proposed a new hypothesis and Rectified Gradient method.
result Rectified Gradient effectively reduces saliency map noise.
New framework analyzes temporal features in state space models.
problem Understanding temporal dependencies in data streams.
method Proposes a framework for rigorous analysis of state representations in ESNs, using temporal feature spaces and kernel machines.
result Phase transition in kernel richness for cycle reservoir topology.
A new model improves uncertainty estimation in deep learning.
problem Deep Kernel Learning (DKL) produces unreliable uncertainty estimates.
method Proposed a bi-Lipschitz constraint to preserve distances in feature space.
result DUE model outperforms previous DKL and other methods in uncertainty quality.