Sparse text alignments learned via optimal transport improve model explainability.
problem Building self-explaining models by selecting relevant text pieces.
method Employing optimal transport to find minimal cost alignments, introducing constrained variants for sparsity.
result Sparse and interpretable alignments achieved, preserving prediction accuracy.
Detecting correlated trees helps align sparse graphs.
problem Detecting correlation between trees for sparse random graphs.
method MPAlign message-passing algorithm for graph alignment.
result MPAlign succeeds in polynomial time for partial alignment.
Early alignment in neural networks leads to sparse representations but hinders convergence.
problem The implicit bias of gradient descent during early training phases.
method Quantitative description of early alignment phase in small initialisation, one hidden layer networks.
result Early alignment induces a sparse representation but also hinders convergence to global minima.
Proposes a method for coarse graph alignment using sparse partial least squares.
problem Aligning graphs with community structures when there's no natural one-to-one mapping.
method Sparse partial least squares method incorporating observed graph structures and imposing sparsity.
result Demonstrates effectiveness in simulations.
BO method identifies sparse subspaces for efficient high-dimensional optimization.
problem Efficient optimization of high-dimensional black-box functions.
method Sparse Gaussian process surrogate models on axis-aligned subspaces with Hamiltonian Monte Carlo inference.
result SAASBO achieves excellent performance on synthetic and real-world problems.
Align-RUDDER improves reinforcement learning with few demonstrations by redistributing rewards.
problem Learning complex tasks with sparse and delayed rewards using few demonstrations.
method Align-RUDDER uses a profile model for reward redistribution based on multiple sequence alignment of demonstrations.
result Align-RUDDER outperforms competitors on complex tasks with few demonstrations.
An efficient algorithm for aligning diffusion trees to networks with information asymmetry.
problem Aligning diffusion trees to networks with information asymmetry.
method Tree correlation tests for extracting alignment information.
result Explicit lower bounds on the probability of correct matches for each vertex on the diffusion tree.
New estimator corrects bias in CKA for sparsely sampled neurons.
problem Bias in CKA for sparsely sampled neurons.
method Novel estimator that corrects for input and feature sampling.
result Reliable model-to-brain alignment with sparsely sampled neurons.
Bayesian framework proves thresholds for multi-graph alignment feasibility.
problem Determining when multi-graph alignment is statistically possible.
method Developed a Bayesian estimation framework over metric spaces.
result Identified thresholds for Gaussian and sparse Erdős-Rényi models.
We propose to align distributional data from the perspective of Wasserstein means. We raise the problem of regularizing Wasserstein means and propose several terms tailored to tackle different problems. Our formulation is based on the variational transportation to distribute a sparse discrete measure into the target do…
Recent advances in deep neural networks (DNNs) owe their success to training algorithms that use backpropagation and gradient-descent. Backpropagation, while highly effective on von Neumann architectures, becomes inefficient when scaling to large networks. Commonly referred to as the weight transport problem, each neur…
We introduce a new method for sparse principal component analysis, based on the aggregation of eigenvector information from carefully-selected axis-aligned random projections of the sample covariance matrix. Unlike most alternative approaches, our algorithm is non-iterative, so is not vulnerable to a bad choice of init…
This paper improves GNN robustness by aligning feature and adjacency matrix learning.
problem Improving robustness of graph neural networks (GNN) in noisy graph data.
method Proposes a novel regularized GSL approach that aligns feature information and graph information, incorporating sparse dimensional reduction.
result Demonstrates superior performance in noisy graph structures compared to competitive baselines.
Decision forests, including Random Forests and Gradient Boosting Trees, have recently demonstrated state-of-the-art performance in a variety of machine learning settings. Decision forests are typically ensembles of axis-aligned decision trees; that is, trees that split only along feature dimensions. In contrast, many r…
Regularization improves stability and consistency of sparse autoencoders.
problem Varying features across random seeds and training choices in SAEs.
method Added L1 or L2 penalties on encoder and decoder weights.
result L2 regularization increases cross-seed feature consistency.
Researchers prove it's impossible to partially recover graph alignments in certain conditions.
problem Recovering vertex correspondence between two random graphs with correlated edges.
method Used the probabilistic method to build automorphisms between tree components of a subcritical Erdös-Rényi graph.
result Proved an impossibility result for partial recovery in the sparse regime with constant average degree and correlation.
This work optimizes signal estimation for sparse MRA with collision-free signals.
problem Recovering an unknown signal from repeated observations under cyclic isometries with high noise.
method Investigates minimax optimality for collision-free signals in the MRA model.
result The minimax optimal rate of estimation is \( \sigma^2/\sqrt{n} \) for sparse MRA.
DET unifies geometric and functional alignment for high-dimensional scientific data.
problem Challenges in nonrigid registration for high-dimensional, irregular data.
method Domain Elastic Transform (DET) treats data as functions on irregular domains, using a Bayesian framework for elastic motion registration.
result DET achieves 92% topological preservation on MERFISH data and successfully registers whole-embryo Stereo-seq atlases.
Pairwise ranking aligns subjective clinical evaluations with objective indicators.
problem Aligning subjective clinical evaluations with objective indicators for improved diagnosis.
method Pairwise ranking methods to align subjective evaluations with objective indicators.
result The resulting score improves classification accuracy and provides a nuanced severity assessment.
Two-dimensional embeddings remain the dominant approach to visualize high dimensional data. The choice of embeddings ranges from highly non-linear ones, which can capture complex relationships but are difficult to interpret quantitatively, to axis-aligned projections, which are easy to interpret but are limited to biva…
The paper improves DFA for CNN and RNN training to match BP accuracy.
problem Low accuracy in CNN and RNN training using DFA.
method Divide network into modules, apply DFA within, use sparse backward weight, and incorporate dilated convolution and sparse matrix multiplication.
result Achieves BP-level accuracy in CNN and RNN training.
Jointly learns feature and sample relevancies for robust sparse recovery.
problem Sparse recovery sensitivity to data contaminants like outliers or misspecified noise.
method Jointly learns feature and sample relevancies via marginal likelihood optimization.
result Consistent sparse and robust prediction models across diverse tasks.
This work combines deep learning and sparse coding for CT image reconstruction.
problem Improving image quality in low-dose CT scans.
method Sparse signal representation using learned dictionaries, inspired by variational autoencoders and deep learning techniques.
result Regularization with learned dictionaries achieves competitive performance in CT reconstruction.
Neural networks have been proven to be vulnerable to a variety of adversarial attacks. From a safety perspective, highly sparse adversarial attacks are particularly dangerous. On the other hand the pixelwise perturbations of sparse attacks are typically large and thus can be potentially detected. We propose a new black…
GrateTile optimizes CNN feature map storage for efficient data access.
problem Efficient storage and access of sparse CNN feature maps.
method Divides feature maps into uneven-sized subtensors, compresses and stores them in a compressed yet accessible format.
result Average 55% DRAM bandwidth reduction with minimal indexing overhead.
New findings show sparse signals in MRA model require fewer measurements than previously thought.
problem Learning an unknown signal from repeated noisy images under group actions.
method Enhanced probabilistic method and analysis of uniform uncertainty principles.
result Sparse signals exhibit intermediate σ4 sample complexity, improving over traditional σ2. AEN-SAEs address feature starvation in sparse autoencoders by stabilizing the geometric alignment of sparse coding.
problem Feature starvation in sparse autoencoders, leading to unstable and misaligned representations.
method Adaptive Elastic Net SAEs (AEN-SAEs) combine ℓ2 and ℓ1 terms to stabilize the sparse coding map and control feature interactions. result AEN-SAEs mitigate feature starvation without heuristic resampling, maintaining competitive reconstruction abilities.
We introduce SparseVM, a method that registers clinical-quality 3D MR scans both faster and more accurately than previously possible. Deformable alignment, or registration, of clinical scans is a fundamental task for many clinical neuroscience studies. However, most registration algorithms are designed for high-resolut…
Semi-supervised method enhances recommendation from sparse data.
problem Data sparsity in implicit feedbacks and lack of negative samples.
method Domain adaptation using textual features to align latent spaces.
result Our models significantly outperform existing models on real-world datasets.
SAGE enhances reinforcement learning by injecting hints to prevent model stagnation.
problem Sparse rewards cause large language models to stall under relative policy optimization.
method SAGE injects privileged hints during training to increase within-group outcome diversity.
result SAGE consistently outperforms GRPO on 6 benchmarks with LLMs, achieving significant improvements.
Weibull weight-scale parameter λ evolves during AdamW training, with alignment, injection, and decay forces driving its growth and relaxation.
problem Understanding the evolution of the Weibull weight-scale parameter λ during AdamW training. method Deriving a leading-order three-force decomposition of the squared weight norm from AdamW updates.
result The alignment force dominates the rise phase, contributing 88-94% of the absolute force budget across four random seeds.
NCL improves interpretability of deep features by enforcing non-negativity.
problem Lack of interpretability in deep representations.
method Non-negative Contrastive Learning (NCL) using non-negativity constraints.
result NCL outperforms standard contrastive learning in feature disentanglement and selection.
This work explores feature learning tradeoffs in neural networks.
problem Resource tradeoffs in neural feature learning.
method Theoretical and experimental investigation of offline sparse parity learning.
result Width improves sample efficiency in sparse feature learning.
VolNP learns IVS from sparse quotes via meta-learning and SABR priors.
problem Reconstructing implied volatility surfaces from sparse option quotes.
method Meta-learning Neural Process with SABR-induced priors.
result VolNP outperforms SABR, SSVI, and Gaussian process on SPX options.
Self-distillation improves constrained language generation by aligning models with target distributions.
problem Sparse and uninformative reward signals in constrained generation settings.
method Iteratively refining the base model through self-distillation, incorporating learned twist functions and proposals.
result Substantial gains in generation quality through improved model alignment with target distributions.
We present a method for synthesizing a frontal, neutral-expression image of a person's face given an input face photograph. This is achieved by learning to generate facial landmarks and textures from features extracted from a facial-recognition network. Unlike previous approaches, our encoding feature vector is largely…
CtrlNS learns latent factors and distribution shifts from sparse transitions without prior knowledge.
problem Lack of prior knowledge of domain variables limits causal temporal representation learning.
method Sparse transition assumption and identifiability results from theoretical perspective.
result Effective in identifying distribution shifts and latent factors without prior knowledge.
Robust high-dimensional data processing has witnessed an exciting development in recent years, as theoretical results have shown that it is possible using convex programming to optimize data fit to a low-rank component plus a sparse outlier component. This problem is also known as Robust PCA, and it has found applicati…
New CRM models for sparse networks with linear edge growth.
problem Modeling extremely sparse networks with tractable properties.
method Introduced a new class of CRMs with index of variation α∈(0,1] based on mixtures of stable or generalized gamma processes.
result Models produce networks with near-linear edge growth, aligning with empirical evidence.
For low-dimensional data sets with a large amount of data points, standard kernel methods are usually not feasible for regression anymore. Besides simple linear models or involved heuristic deep learning models, grid-based discretizations of larger (kernel) model classes lead to algorithms, which naturally scale linear…
LSDAT reduces query efficiency for decision-based adversarial attacks.
problem Improving query efficiency for decision-based adversarial attacks.
method Low-rank and sparse decomposition (LSD) to craft perturbations.
result LSDAT achieves superior fooling rates with fewer queries.
We present a novel method for exact hierarchical sparse polynomial regression. Our regressor is that degree r polynomial which depends on at most k inputs, counting at most ℓ monomial terms, which minimizes the sum of the squares of its prediction errors. The previous hierarchical sparse specification aligns w…
Automates feature selection and weighting in molecular systems.
problem Optimal feature selection and alignment in molecular systems.
method Differentiable Information Imbalance (DII) method for automated feature ranking and scaling.
result Automated feature selection and scaling that preserves information content and interpretability.
Study compares L1 and VG sparsity priors in inverse problems.
problem Sparse regularization in inverse problems with incomplete or corrupted measurements.
method Compared L1 regularization with Variational Garrote (VG), a probabilistic method approximating L0 sparsity.
result VG often achieves lower minimum generalization error and improved stability in strongly underdetermined regimes.
This paper extends neural collapse to regression problems, revealing key features and structures.
problem Understanding the structure learned by deep neural networks in regression tasks.
method Established Neural Regression Collapse (NRC) across different models, analyzing feature and weight alignments.
result Deep neural regression models exhibit a collapsed feature space, aligning with target dimensions and covariances.
Reinforcement learning agents need exploratory behaviors to escape from local optima. These behaviors may include both immediate dithering perturbation and temporally consistent exploration. To achieve these, a stochastic policy model that is inherently consistent through a period of time is in desire, especially for t…
New approach to learning kernels from data using AIT principles.
problem Learning kernels from data in machine learning.
method Sparse Kernel Flows method based on AIT principles.
result Sparse Kernel Flows aligns with MDL principle and offers a robust theoretical foundation.
Probabilistic NDVI forecasting from sparse satellite data.
problem Challenges in short-term NDVI forecasting due to sparse and irregular satellite data.
method Probabilistic forecasting framework using historical NDVI and meteorological observations, with temporal-distance weighted quantile loss and extreme-weather feature engineering.
result The proposed method outperforms baselines on pointwise and probabilistic evaluation metrics.