The paper improves ranking by integrating covariates and sparse intrinsic scores.
problem Ranking items with incomplete preference scores explained by covariates.
method Extends BTL model with covariate information and sparse intrinsic scores, using penalized MLE.
result Developed debiased estimator for penalized MLE with distributional properties.
A method to train multi-agent reinforcement learning models without intrinsic rewards.
problem Training multi-agent reinforcement learning models with sparse rewards is challenging.
method A learning-based exploration strategy using variational graph autoencoder to generate initial states.
result The method improves the training and performance of multi-agent reinforcement learning models.
Classifies intrinsically linked tournaments by their score sequences.
problem Classifying intrinsically linked tournaments using their score sequences.
method Examining the score sequences of tournaments and identifying linkless sequences.
result The vast majority of score sequences for 8-vertex tournaments are linkless.
Deep networks can approximate score functions in high-dimensional graphical models efficiently.
problem Approximation efficiency of score functions by deep neural networks in high-dimensional graphical models like Markov random fields.
method Variational inference denoising algorithms and efficient neural network representation.
result Efficient sample complexity bound for diffusion-based generative modeling when score functions are learned by deep neural networks.
New score matching method estimates local intrinsic dimension efficiently.
problem Quantifying the local intrinsic dimension of complex data.
method Denoising score matching loss and equivalent implicit score matching loss.
result Denoising score matching loss is a highly competitive and scalable LID estimator.
The paper analyzes reflected diffusion models on hypercube data.
problem Challenges in modeling bounded domains with low-dimensional data.
method Employed an infinite series expansion of transition densities to bound the score function and its approximation.
result Established convergence rates for generative algorithm adapting to intrinsic dimensionality.
Study improves denoising score matching under relaxed manifold assumptions.
problem Improving denoising score matching under relaxed manifold assumptions.
method Model density with nonparametric Gaussian mixtures, relax manifold assumption, derive non-asymptotic bounds.
result Non-asymptotic bounds on approximation and generalization errors, rates of convergence determined by intrinsic dimension.
Sparse coding, which represents a data point as a sparse reconstruction code with regard to a dictionary, has been a popular data representation method. Meanwhile, in database retrieval problems, learning the ranking scores from data points plays an important role. Up to now, these two problems have always been conside…
Study intrinsic motivation for synergistic tasks in reinforcement learning.
problem Sparse-reward synergistic tasks where multiple agents must work together.
method Propose incentivizing actions that affect the world in ways not achievable individually, using either true states or a dynamics model.
result Our approach yields more efficient learning than typical methods.
Bayesian neural networks achieve optimal posterior contraction rates in Besov spaces with intrinsic dimensionality.
problem High-dimensional structured estimation problems with unknown smoothness levels.
method Sparse Bayesian neural networks with either sparse or continuous shrinkage priors.
result Optimal posterior contraction rates are achieved, adapting to the unknown smoothness level of the true function.
FLoE adapts LLMs by selectively deploying LoRA adapters based on layer importance and task requirements.
problem Uniform LoRA deployment across all layers leads to inefficient and redundant parameter allocation.
method FLoE uses Fisher information to dynamically identify task-critical layers and optimizes LoRA ranks.
result FLoE achieves significant efficiency-accuracy trade-offs, especially in resource-constrained environments.
Study shows how diffusion models learn on low-dimensional manifolds.
problem Learning efficiency of diffusion models on manifolds.
method Analyzes denoising score matching with random feature neural networks.
result Sample complexity scales linearly with intrinsic dimension, not ambient dimension.
This work improves SINDy-type algorithms for system identification using score-guided dictionary selection.
problem Improving accuracy and interpretability in dynamical system identification.
method Score-guided library selection to refine dictionary terms in sparse regression.
result Score-guided methods enhance SINDy's robustness in discovering governing equations.
Exploration in sparse reward reinforcement learning remains an open challenge. Many state-of-the-art methods use intrinsic motivation to complement the sparse extrinsic reward signal, giving the agent more opportunities to receive feedback during exploration. Commonly these signals are added as bonus rewards, which res…
A new method for learning manifolds efficiently using canonical basis functions.
problem Learning manifolds in high-dimensional data with efficient and distinct latent dimensions.
method Proposes a novel optimization objective to enforce a transformation matrix with a few prominent and non-degenerate basis functions.
result Demonstrates that minimizing the off-diagonal manifold metric elements ℓ 1 \ell_1 ℓ 1 -norm results in a more efficient latent space representation. DPA autoencoders learn data distribution and intrinsic dimensionality with guarantees.
problem Learning data distribution and intrinsic dimensionality in unsupervised learning.
method Combines distributionally correct reconstruction with principal-component-like interpretability.
result Exact theoretical guarantees on disentangling factors of variation and intrinsic dimensionality.
High-dimensional models trained on smooth manifolds achieve optimal rates in Wasserstein metrics.
problem Training score-based generative models on complex, low-dimensional manifolds.
method Proves optimal rates for SGMs on smooth manifolds, separating into noise regimes and using ReLU nearest-projection coordinates.
result Optimal intrinsic Wasserstein rates are achieved, with polynomial ambient dependence for families with controlled geometry and density.
AMIGo uses adversarial intrinsic goals to teach RL agents new skills.
problem Learning in sparse reward environments.
method Adversarial intrinsic goals to generate a curriculum for a student policy.
result AMIGo enables agents to learn new skills without extrinsic rewards.
Exploration is a difficult challenge in reinforcement learning and is of prime importance in sparse reward environments. However, many of the state of the art deep reinforcement learning algorithms, that rely on epsilon-greedy, fail on these environments. In such cases, empowerment can serve as an intrinsic reward sign…
DFSOS improves sparse discriminant analysis for high-dimensional data.
problem Sparse discriminant analysis in high-dimensional settings with feature selection.
method Deflation-Free Sparse Optimal Scoring (DFSOS) using Bregman iteration and orthogonality-constrained optimization.
result DFSOS achieves comparable or better classification accuracy than deflation-based methods.
DSM on manifolds removes singularities and computes small-noise expansions.
problem DSM on manifolds with singular noise.
method Rao-Blackwellized score matching, nearest-point projection, intrinsic Riemannian score.
result Canonical target equals intrinsic Riemannian score up to a small correction.
LD-EnSF speeds up data assimilation with sparse observations.
problem Efficiently assimilate sparse and noisy data into complex dynamical systems.
method LD-EnSF uses latent dynamics networks and history-aware LSTM encoders to process sparse observations without full-space simulations.
result Achieves significant speedups over existing methods while maintaining high accuracy.
Develops deep generative models for stratified learning.
problem Challenges in learning distributions on stratified spaces with varying dimensions and singularities.
method Two generative frameworks: sieve maximum likelihood and diffusion-based.
result Establishes convergence rates and consistency for estimating intrinsic dimensions and number of strata.
SSDMs generate quantum states directly, outperforming classical methods.
problem Generating pure-state quantum representations efficiently.
method Score-based generative model on complex projective manifold.
result SSDMs match target pure-state ensembles by orders of magnitude.
Learning goal-directed behavior in environments with sparse feedback is a major challenge for reinforcement learning algorithms. The primary difficulty arises due to insufficient exploration, resulting in an agent being unable to learn robust value functions. Intrinsically motivated agents can explore new behavior for …
This paper addresses the problem of inferring sparse causal networks modeled by multivariate auto-regressive (MAR) processes. Conditions are derived under which the Group Lasso (gLasso) procedure consistently estimates sparse network structure. The key condition involves a "false connection score." In particular, we sh…
A new sparse benchmark metabench identifies key abilities from large benchmarks.
problem Redundancy and compression in existing benchmarks.
method Data from 5000+ LLMs to identify most informative items, distilling a sparse benchmark.
result Sparse benchmark metabench captures underlying abilities with high accuracy.
We introduce Supersparse Linear Integer Models (SLIM) as a tool to create scoring systems for binary classification. We derive theoretical bounds on the true risk of SLIM scoring systems, and present experimental results to show that SLIM scoring systems are accurate, sparse, and interpretable classification models.
Sparse-RS framework efficiently attacks models with sparse perturbations.
problem Efficiently attacking models with sparse perturbations in the black-box setting.
method Random search-based Sparse-RS framework for score-based attacks.
result Sparse-RS achieves state-of-the-art success rate and query efficiency.
Capsule Networks improve autonomous navigation in sparse environments.
problem Challenges in reinforcement learning for sparse reward environments.
method Caps-EM pairs CapsNets with Advantage Actor Critic, using fewer parameters.
result Caps-EM achieves significant time improvements and fewer parameters compared to competing methods.
The paper improves the probability flow ODE sampler for faster sampling of natural images.
problem Improving the convergence rate of the probability flow ODE sampler.
method Adapting the probability flow ODE sampler to exploit intrinsic low-dimensional structures in natural image data.
result Achieves a dimension-free convergence rate of O ( k / T ) O(k/T) O ( k / T ) in total variation distance, improving upon existing results. A new model for sparse networks improves consistency of score estimators.
problem Statistical inference in large sparse networks.
method General pairwise comparison model with flexible parametrization.
result Maximum likelihood estimator is uniformly consistent under sparse conditions.
Improved score matching methods for estimating score functions and Hessians without high dimensionality.
problem Estimating score functions and Hessians efficiently in high-dimensional data.
method Implicit score matching and denoising score matching, leveraging Gagliardo-Nirenberg inequalities.
result Achieves convergence rates similar to denoising score matching and estimates Hessians without dimensionality issues.
Diffusion models learn multi-modal distributions with optimal efficiency.
problem Learning high-dimensional distributions with low-dimensional multi-modal structures.
method Score-based diffusion models, focusing on subgaussian distributions within subspaces.
result Diffusion models require O ~ ( ε − k ∨ 2 ) \widetilde{O}(\varepsilon^{-k \vee 2}) O ( ε − k ∨ 2 ) samples for 1-Wasserstein ε \varepsilon ε error, improving over prior guarantees. A method uses ITD and XGBoost for precise power transformer fault diagnosis.
problem Fault diagnosis of power transformers using DGA data.
method Ranking DGA parameters by skewness, extracting ITD features, and using an XGBoost classifier.
result The method achieves over 95% accuracy in classification.
Proposes MSS to identify causal structure from heterogeneous environments.
problem Distribution shifts between environments violate i.i.d. data assumption.
method Sparse mechanism shift hypothesis, score-based approach.
result Identifies entire causal structure with high probability.
MASF improves score-based filters for high-dimensional nonlinear systems with spatially sparse measurements.
problem Challenges in data assimilation for nonlinear, high-dimensional systems with spatially sparse measurements.
method Developed a forward process tailored for filtering that transforms the system state toward the measurement space, enabling a theoretically sound formulation of the likelihood score.
result MASF shows improved performance over existing score-based filters and ensemble-type Kalman filters, achieving up to a 28.2× wall-clock speedup.
Solving tasks with sparse rewards is one of the most important challenges in reinforcement learning. In the single-agent setting, this challenge is addressed by introducing intrinsic rewards that motivate agents to explore unseen regions of their state spaces; however, applying these techniques naively to the multi-age…
New algorithms estimate matrix leverage scores using rank revealing and randomization.
problem Estimating leverage scores for matrices of arbitrary rank.
method Combining rank revealing methods with randomized dimensionality reduction.
result Effective estimators for leverage scores, even in rank deficient cases.
This work uses model uncertainty for efficient exploration in sparse reward environments.
problem Challenging exploration in sparse reward reinforcement learning environments.
method Implicit generative modeling approach to estimate Bayesian uncertainty of the agent's belief of the environment dynamics.
result Our implicit generative model consistently outperforms competing approaches in data efficiency for exploration.
A grand challenge in reinforcement learning is intelligent exploration, especially when rewards are sparse or deceptive. Two Atari games serve as benchmarks for such hard-exploration domains: Montezuma's Revenge and Pitfall. On both games, current RL algorithms perform poorly, even those with intrinsic motivation, whic…
Diffusion models converge linearly to complex data manifolds.
problem Sampling from high-dimensional complex data distributions.
method Score-matching generative models with novel integration scheme.
result Linear convergence in KL divergence to intrinsic dimension d d d . SAEs struggle with curved activation manifolds, revealing layer-dependent scaling laws.
problem Sparse autoencoders' reconstruction error varies across layers, not fitting existing scaling laws.
method Cross-layer study of 844 SAE checkpoints, fitting and regressing on manifold geometry.
result Manifold geometry predicts layer-dependent width exponents in SAEs, with transferable coefficients.
Sparse learning has been shown to be effective in solving many real-world problems. Finding sparse representations is a fundamentally important topic in many fields of science including signal processing, computer vision, genome study and medical imaging. One important issue in applying sparse representation is to find…
Dealing with sparse rewards is a longstanding challenge in reinforcement learning. The recent use of hindsight methods have achieved success on a variety of sparse-reward tasks, but they fail on complex tasks such as stacking multiple blocks with a robot arm in simulation. Curiosity-driven exploration using the predict…
New method scores DAGs by identifying unobserved confounding.
problem Unobserved confounding complicates causal discovery.
method Score-based causal discovery algorithm that accounts for unobserved confounding.
result Sparse linear Gaussian DAGs can be recovered from observed data.
Improved assessment of knee osteoarthritis using geodesic B-score.
problem Need for automatic, reader-independent measures of osteoarthritis clinical outcomes.
method Derive a geodesic B-score for Riemannian shape spaces, develop efficient algorithm for large shape populations.
result Geodesic B-score exhibits improved discrimination ability over Euclidean B-score.
INK scores improve OOD detection for classifiers.
problem Detecting out-of-distribution inputs for classification models.
method INK scores operate on constrained latent embeddings modeled as a mixture of hyperspherical embeddings, optimizing in modern neural networks.
result INK establishes a new state-of-the-art in OOD detection.