GrateTile optimizes CNN feature map storage for efficient data access.
problem Efficient storage and access of sparse CNN feature maps.
method Divides feature maps into uneven-sized subtensors, compresses and stores them in a compressed yet accessible format.
result Average 55% DRAM bandwidth reduction with minimal indexing overhead.
AEN-SAEs address feature starvation in sparse autoencoders by stabilizing the geometric alignment of sparse coding.
problem Feature starvation in sparse autoencoders, leading to unstable and misaligned representations.
method Adaptive Elastic Net SAEs (AEN-SAEs) combine ℓ2 and ℓ1 terms to stabilize the sparse coding map and control feature interactions. result AEN-SAEs mitigate feature starvation without heuristic resampling, maintaining competitive reconstruction abilities.
ERM performs well in feature learning with minimal feature maps.
problem Empirical risk minimization in feature learning with square loss.
method Asymptotic and non-asymptotic analysis of ERM performance.
result Excess risk quantiles of ERM match those of oracle procedure under certain conditions.
A new kernel, Isolation Kernel, simplifies large scale online kernel learning without sacrificing accuracy.
problem Building efficient and scalable kernel-based models from large datasets with high accuracy.
method Introducing Isolation Kernel, which creates an exact, sparse, and finite-dimensional feature map of a kernel, allowing for efficient large scale online kernel learning without accuracy loss.
result Large scale online kernel learning can be achieved efficiently and accurately using Isolation Kernel.
A new estimator learns sparse linear models with context-dependent coefficients.
problem Sparse linear models lack flexibility compared to deep neural networks for handling feature groups.
method Contextual lasso estimator using a deep neural network with lasso regularization.
result Learned models can be sparser than standard lasso without sacrificing predictive power.
New method discovers concepts in hidden feature layers using sparse subspace clustering.
problem Local attribution methods fail to identify coherent model behavior across samples.
method Sparse Subspace Clustering (SSCC) for concept discovery.
result Empirically validated method for various image classification tasks.
Random sinusoidal features are a popular approach for speeding up kernel-based inference in large datasets. Prior to the inference stage, the approach suggests performing dimensionality reduction by first multiplying each data vector by a random Gaussian matrix, and then computing an element-wise sinusoid. Theoretical …
We develop a 2D travel time tomography method which regularizes the inversion by modeling groups of slowness pixels from discrete slowness maps, called patches, as sparse linear combinations of atoms from a dictionary. We propose to use dictionary learning during the inversion to adapt dictionaries to specific slowness…
Method learns dynamics from sparse, irregular feature data.
problem Learn system dynamics from sparse, irregularly sampled feature time series.
method Formulates as high-dimensional linear regression using signatures.
result Oracle bound on prediction error with explicit sampling dependencies.
The paper learns compact implicit surface maps from streaming data using an ensemble of sparse Gaussian processes.
problem Creating compact and accurate implicit surface maps from streaming range data.
method An ensemble of sparse Gaussian process experts, incrementally adjusted, trades-off between model complexity and prediction error.
result The approach learns compact and accurate implicit surface models comparable to or better than exact GP regression with subsampled data.
Sparse mapping has been a key methodology in many high-dimensional scientific problems. When multiple tasks share the set of relevant features, learning them jointly in a group drastically improves the quality of relevant feature selection. However, in practice this technique is used limitedly since such grouping infor…
Graph matching with feature vectors is solved using a two-layer graph neural network.
problem Graph matching in the presence of sparse binary features.
method Two-layer graph neural network with graph structure.
result Graph neural network can recover correct mapping with high probability under certain conditions.
A new GP model uses spherical harmonics for faster inference.
problem Efficiently fitting large datasets with Gaussian processes.
method Sparse Gaussian processes with spherical harmonic features.
result Significant speed-up in inference for large datasets.
New algorithm learns sparse linear MDPs with polynomial interactions, improving sample complexity.
problem Learning optimal policies in sparse linear MDPs with limited interactions and unknown features.
method Developed a polynomial-time algorithm using feature selection and emulator for sparse linear MDPs.
result First polynomial-time algorithm for learning near-optimal policies in k-sparse linear MDPs.
New model estimates sparse transport maps for high-dimensional data.
problem Estimating optimal transport maps in high-dimensional spaces.
method Proposes a new model using a family of translation invariant costs and sparsity-inducing norms.
result Sparse transport maps that apply Occam's razor to reduce complexity.
Method learns feature map between source and target domains for high-dimensional regression with missing features.
problem High-dimensional regression with differing feature sets in target and source domains.
method First learns a feature map between missing and observed features using source data, then imputes missing features in target domain, and performs two-step transfer learning for penalized regression.
result Developed upper bounds on estimation and prediction errors for HTL, showing dependence on model complexity, sample size, feature map quality, and domain differences.
Proposes HBGNN for better recommendation systems using graph neural networks.
problem Sparse structured data in recommendation systems lacking feature richness.
method Hierarchical BiGraph Neural Network (HBGNN) using bigraph framework.
result Competitive performance compared to current methods.
Nonnegative matrix factorization (NMF) with group sparsity constraints is formulated as a probabilistic graphical model and, assuming some observed data have been generated by the model, a feasible variational Bayesian algorithm is derived for learning model parameters. When used in a supervised learning scenario, NMF …
Nonlinear kernels can be approximated using finite-dimensional feature maps for efficient risk minimization. Due to the inherent trade-off between the dimension of the (mapped) feature space and the approximation accuracy, the key problem is to identify promising (explicit) features leading to a satisfactory out-of-sam…
SeqFM models dynamic and sequential features for better predictive analytics.
problem Inadequate handling of sequential dependencies in existing FM-based models.
method Introduces SeqFM, a novel model that incorporates multi-view self-attention to model static, dynamic, and their interactions.
result SeqFM outperforms existing models in ranking, classification, and regression tasks on six large-scale datasets.
L1 regularized logistic regression has now become a workhorse of data mining and bioinformatics: it is widely used for many classification problems, particularly ones with many features. However, L1 regularization typically selects too many features and that so-called false positives are unavoidable. In this pape…
We consider a transfer-learning problem by using the parameter transfer approach, where a suitable parameter of feature mapping is learned through one task and applied to another objective task. Then, we introduce the notion of the local stability and parameter transfer learnability of parametric feature mapping,and th…
Adaptive sparseness enhances robust regression using MCC and ARD.
problem Developing a robust regression method with adaptive sparseness.
method Integrating MCC with ARD in a Bayesian framework using variational Bayesian inference.
result MCC-ARD regression outperforms existing methods in prediction and feature selection.
Study analyzes feedback complexity for sparse feature retrieval in deep networks.
problem Learning sparse superposed features with feedback.
method Analysis of feedback complexity in sparse settings, including triplet comparisons.
result Establishes tight bounds and strong upper bounds for feature recovery.
Sparse random features improve accuracy in data-scarce settings.
problem Limited accuracy of random feature methods in data-scarce applications.
method Sparse random feature expansion using compressive sensing.
result Improved generalization bounds for sparse random features.
For the problem of multi-class linear classification and feature selection, we propose approximate message passing approaches to sparse multinomial logistic regression (MLR). First, we propose two algorithms based on the Hybrid Generalized Approximate Message Passing (HyGAMP) framework: one finds the maximum a posterio…
In compressed sensing, we wish to reconstruct a sparse signal x from observed data y. In sparse coding, on the other hand, we wish to find a representation of an observed signal y as a sparse linear combination, with coefficients x, of elements from an overcomplete dictionary. While many algorithms are competit…
HARFE approximates sparse additive functions using random features and ridge regression.
problem Approximating high-dimensional sparse additive functions.
method Hard-ridge random feature expansion with sparse ridge regression and hard-thresholding pursuit.
result HARFE method converges with a given error bound and achieves lower error than other algorithms.
The paper tackles learning varying DAG structures based on contextual features.
problem Learning a single DAG for the entire population from observational data.
method A neural network that maps contextual features to a weighted adjacency matrix of a DAG, with a projection layer to ensure acyclicity.
result The new approach can recover context-specific DAGs where existing methods fail.
We present a supervised-learning algorithm from graph data (a set of graphs) for arbitrary twice-differentiable loss functions and sparse linear models over all possible subgraph features. To date, it has been shown that under all possible subgraph features, several types of sparse learning, such as Adaboost, LPBoost, …
Feature hashing and other random projection schemes are commonly used to reduce the dimensionality of feature vectors. The goal is to efficiently project a high-dimensional feature vector living in Rn into a much lower-dimensional space Rm, while approximately preserving Euclidean norm. These sc…
Sparse support vector machine (SVM) is a popular classification technique that can simultaneously learn a small set of the most interpretable features and identify the support vectors. It has achieved great successes in many real-world applications. However, for large-scale problems involving a huge number of samples a…
New entropy-based objective for sparse coding improves learning.
problem Sparse coding with probabilistic priors and non-Gaussian observables.
method Derive a solely entropy-based learning objective for sparse coding parameters.
result Fully analytical ELBO objective for sparse coding with non-trivial posterior approximations.
Improves sparse recovery with non-linear Fourier features.
problem Sparse recovery challenges with non-linear Fourier features.
method Characterizes sufficient data points for perfect recovery.
result Sufficient data points depend on kernel matrix.
Zero-shot learning (ZSL) is a framework to classify images belonging to unseen classes based on solely semantic information about these unseen classes. In this paper, we propose a new ZSL algorithm using coupled dictionary learning. The core idea is that the visual features and the semantic attributes of an image can s…
DFR reduces the computational cost of sparse-group lasso and adaptive sparse-group lasso.
problem Sparse-group lasso's computational expense and need for tuning.
method Dual Feature Reduction (DFR) using strong screening rules and dual norms.
result DFR drastically reduces computational cost without affecting solution optimality.
FI-GNNs learn expressive node representations from sparse features.
problem Sparse and high-dimensional node features limit GNN performance.
method Plug-and-play GNN framework that highlights informative feature interactions.
result FI-GNNs learn highly expressive node representations on feature-sparse graphs.
Sparse GEMINI selects relevant features for clustering without assumptions.
problem Feature selection in clustering with relevant clusters and variables.
method Discriminative clustering model maximizing GEMINI with l1 penalty.
result Sparse GEMINI selects relevant subsets of variables without prior hypotheses.
A new method selects features efficiently for high-dimensional data.
problem High computational costs and memory requirements in high-dimensional data.
method QuickSelection uses the strength of neurons in sparse autoencoders to select features.
result QuickSelection achieves the best trade-off of accuracy, speed, and memory usage.
A new method uses Gaussian Processes for feature-based nonrigid image registration.
problem Estimating dense displacement fields for nonrigid image registration.
method Using Gaussian Processes to estimate both dense displacement field and uncertainty map.
result GP-based interpolation performs similarly to state-of-the-art B-spline interpolation.
ControlBurn selects few features from tree ensembles for better model interpretability.
problem Improving model interpretability in machine learning models.
method Sparse tree ensembles with lasso optimization.
result ControlBurn selects feature-sparse subsets for better model interpretability.
Multi-task sparse feature learning aims to improve the generalization performance by exploiting the shared features among tasks. It has been successfully applied to many applications including computer vision and biomedical informatics. Most of the existing multi-task sparse feature learning algorithms are formulated a…
SANs use sparse activation functions to compress data representations.
problem Learning meaningful features without considering compression.
method Introduce φ metric, define activation functions, and present SANs.
result SANs achieve small description length and interpretable kernels.
A new method scales sparse machine learning to ultra-high dimensional problems.
problem Sparse and interpretable machine learning in ultra-high dimensional data.
method Two-phase approach: backbone set determination followed by reduced problem solving.
result The backbone set contains truly relevant features with high probability.
In text mining, information retrieval, and machine learning, text documents are commonly represented through variants of sparse Bag of Words (sBoW) vectors (e.g. TF-IDF). Although simple and intuitive, sBoW style representations suffer from their inherent over-sparsity and fail to capture word-level synonymy and polyse…
A novel nonstationary permanental process relaxes kernel constraints and captures complex data patterns.
problem Limitations of existing permanental processes in terms of kernel types and stationarity.
method Sparse spectral representation of nonstationary kernels and hierarchical stacking of spectral feature mappings.
result Enhanced model expressiveness and reduced computational complexity.
New method uses sparse random features for crashworthiness analysis.
problem Efficient surrogate modelling for uncertainty quantification.
method Sparse Random Features combined with self-supervised dimensionality reduction.
result Superiority over state-of-the-art techniques in crashworthiness analysis.
Fast classification for sparse models, even with correlated features.
problem Sparse classification with many correlated features.
method Linear and quadratic surrogate cuts, priority queue, and analytical solution for exponential loss.
result 2 to 5 times faster than previous approaches, interpretable models with comparable accuracy.