In this paper we introduce a new optimization formulation for sparse regression and compressed sensing, called CLOT (Combined L-One and Two), wherein the regularizer is a convex combination of the - and -norms. This formulation differs from the Elastic Net (EN) formulation, in which the regularizer is a…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,695 papers · 148 categories
Trend · papers per month
48 results for “sparse effects”
Multi-task learning models using Gaussian processes (GP) have been developed and successfully applied in various applications. The main difficulty with this approach is the computational cost of inference using the union of examples from all tasks. Therefore sparse solutions, that avoid using the entire data directly a…
New algorithm clusters sparse data effectively.
problem Challenges in clustering sparse data.
method Deterministic Information Bottleneck framework for joint feature weighting and clustering.
result Demonstrated effectiveness on real-world genomics data.
Develops HMRL for sparse reward RL problems, improving meta policy efficiency and transferability.
problem Difficulty in learning meta policies for sparse reward RL problems.
method Hyper-Meta RL framework with cross-environment meta state embedding and shaped meta reward.
result Improves meta policy generalization and efficiency for sparse reward RL problems.
We investigate sparse representations for control in reinforcement learning. While these representations are widely used in computer vision, their prevalence in reinforcement learning is limited to sparse coding where extracting representations for new data can be computationally intensive. Here, we begin by demonstrat…
The paper proposes a method to identify subgroups with different treatment effects in time-to-event data.
problem Identifying subgroups with differential treatment effects in time-to-event data.
method A mixture model with structured sparsity regularization and novel inference procedure.
result The method effectively recovers sparse phenotypes across real-world clinical studies.
The paper shows how instrumental variables can help identify sparse causal effects in linear models.
problem Identifying sparse causal effects in linear models with limited instruments.
method Conditions and graphical criteria for identifiability, spaceIV estimator.
result Causal effects can be identified from observed distributions with sparse effects and limited instruments.
Paper introduces ENZ to measure significant coefficients in sparse recovery, improving over classical methods.
problem Numerical noise creates long tails of negligible coefficients in sparse recovery.
method Entropy-based notion of effective sparsity (ENZ) to measure significant coefficients, proving stability under restricted isometry condition.
result ENZ decomposes into support cardinality and efficiency factor, providing a precise measure of sparsity.
Screening is an effective technique for speeding up the training process of a sparse learning model by removing the features that are guaranteed to be inactive the process. In this paper, we present a efficient screening technique for sparse support vector machine based on variational inequality. The technique is both …
MLP-Mixer achieves better performance through sparsity and wider architecture.
problem Understanding why MLP-Mixer outperforms conventional MLPs.
method Revealed sparseness as a key mechanism, showed effective expression as wider MLP, demonstrated quantitative similarities, and applied a guiding principle.
result MLP-Mixer's performance improvement through sparseness and wider architecture.
Efficiently computes sparse signature coefficients using kernels.
problem Lack of efficient methods for sparse signature coefficients.
method Signature kernels and PDE-based methods.
result Sparse groups of signature coefficients can be isolated effectively.
We develop a sparse representation method for neural network uncertainty.
problem Estimating model uncertainty in neural networks.
method Sparse representation of model uncertainty using inverse Multivariate Normal Distribution (MND), with a novel sparsification algorithm and analytical sampler.
result The information form of neural networks can be effectively applied for model uncertainty representation, showing competitive performance.
Sparse alpha-norm regularization has many data-rich applications in Marketing and Economics. Alpha-norm, in contrast to lasso and ridge regularization, jumps to a sparse solution. This feature is attractive for ultra high-dimensional problems that occur in demand estimation and forecasting. The alpha-norm objective is …
Neural network learning is usually time-consuming since backpropagation needs to compute full gradients and backpropagate them across multiple layers. Despite its success of existing works in accelerating propagation through sparseness, the relevant theoretical characteristics remain under-researched and empirical stud…
Dynamic Sparse Training: Find Efficient Sparse Network From Scratch With Trainable Masked Layerscs.LG
Dynamic Sparse Training finds efficient sparse networks from scratch.
problem Finding efficient sparse neural networks.
method Jointly optimizes network parameters and sparsity with trainable thresholds.
result Achieves state-of-the-art performance with minimal performance loss.
New method improves robust sparse association estimation.
problem Outliers in high-dimensional data.
method Splitting robust estimation into optimization phases, using augmented Lagrangian and adaptive gradient descent.
result Improved precision over existing methods.
Stochastic Sparse Subspace Clustering improves subspace clustering by reducing over-segmentation through dropout.
problem Over-segmentation in subspace clustering.
method Introducing dropout regularization to enforce denser connections between points from the same subspace.
result Stochastic Sparse Subspace Clustering effectively handles large datasets and reduces over-segmentation.
Many Experiments, Few Repetitions, Unpaired Data, and Sparse Effects: Is Causal Inference Possible?stat.ML
Study on estimating causal effects with limited data and multiple environments.
problem Estimating causal effects under hidden confounding with unpaired data and sparse effects.
method Instrumental variable (IV) regression with cross-fold sample splitting and -regularized estimation.
result Proposed GMM-type estimator is consistent as the number of environments grows.
Parsimonious Dynamic Mode Decomposition selects sparse modes robustly.
problem Manual tuning of sparsity parameters in traditional DMD.
method Time-delay embedding and Orthogonal Matching Pursuit.
result Autonomously determines optimally sparse subset of modes.
This paper considers the sparse eigenvalue problem, which is to extract dominant (largest) sparse eigenvectors with at most non-zero components. We propose a simple yet effective solution called truncated power method that can approximately solve the underlying nonconvex optimization problem. A strong sparse recove…
Improves treatment effect estimation by reducing sample size needed.
problem Estimating causal treatment effects from observational data requires many covariates, increasing sample size.
method Proposes a nonconvex joint sparsity regularization objective function to recover a sparse subset of covariates.
result Improves sample complexity to scale with the size of the sparse subset and log of the total covariates.
Paper proposes sparse classification method for high-dimensional data.
problem Sparse classification in high-dimensional data with positive-confidence samples.
method Developed a novel sparse-penalization framework using L1, SCAD, and MCP penalties for convex and non-convex shrinkage.
result Proved near minimax-optimal sparse recovery rates under Restricted Strong Convexity condition.
Modelling Cellular Perturbations with the Sparse Additive Mechanism Shift Variational Autoencoderstat.ML
SAMS-VAE models cellular perturbations using sparse additive mechanisms.
problem Modeling effects of diverse interventions on cells.
method Sparse Additive Mechanism Shift Variational Autoencoder (SAMS-VAE).
result SAMS-VAE identifies disentangled, perturbation-specific latent subspaces.
Sparse coding has shown its power as an effective data representation method. However, up to now, all the sparse coding approaches are limited within the single domain learning problem. In this paper, we extend the sparse coding to cross domain learning problem, which tries to learn from a source domain to a target dom…
Unsupervised methods have proven effective for discriminative tasks in a single-modality scenario. In this paper, we present a multimodal framework for learning sparse representations that can capture semantic correlation between modalities. The framework can model relationships at a higher level by forcing the shared …
Probabilistic NDVI forecasting from sparse satellite data.
problem Challenges in short-term NDVI forecasting due to sparse and irregular satellite data.
method Probabilistic forecasting framework using historical NDVI and meteorological observations, with temporal-distance weighted quantile loss and extreme-weather feature engineering.
result The proposed method outperforms baselines on pointwise and probabilistic evaluation metrics.
Inducing sparseness while training neural networks has been shown to yield models with a lower memory footprint but similar effectiveness to dense models. However, sparseness is typically induced starting from a dense model, and thus this advantage does not hold during training. We propose techniques to enforce sparsen…
Pruning at initialization fails to find sparse subnetworks, revealing information-theoretic barriers.
problem Difficulty in finding sparse subnetworks without training the full model.
method Analysis of effective parameter count and mutual information between sparsity mask and data.
result Pruning at initialization cannot find sparse subnetworks due to high mutual information.
A framework helps reinforcement learning agents understand and decompose tasks from human demonstrations.
problem Sparse-reward tasks where demonstrations are used as sources of causal knowledge.
method Develops causal models through observation and reasons from this knowledge to decompose tasks.
result A basic implementation of Reasoning from Demonstration (RfD) is effective in sparse-reward tasks.
A new reward shaping method balances learning efficiency and effectiveness for robot manipulation.
problem Efficient and effective learning in robot manipulations with system uncertainty.
method Dense2Sparse reward shaping method combining dense and sparse rewards.
result Dense2Sparse method achieves higher expected reward and better system uncertainty tolerance.
Dealing with sparse rewards is a longstanding challenge in reinforcement learning. The recent use of hindsight methods have achieved success on a variety of sparse-reward tasks, but they fail on complex tasks such as stacking multiple blocks with a robot arm in simulation. Curiosity-driven exploration using the predict…
Sparse oblique decision tree improves security rules for renewable power systems.
problem Identifying secure operating conditions in power systems with high renewable energy.
method Sparse weighted oblique decision tree to learn and embed linear security rules.
result The method significantly increases secure states and reduces solution time.
Powerpropagation makes neural networks inherently sparse.
problem Training sparse neural networks to reduce computational footprint and model size.
method Introduces a new weight-parameterisation technique exploiting gradient descent dynamics.
result Models trained with Powerpropagation have a higher density of zero weights, allowing for more efficient pruning.
LD-EnSF speeds up data assimilation with sparse observations.
problem Efficiently assimilate sparse and noisy data into complex dynamical systems.
method LD-EnSF uses latent dynamics networks and history-aware LSTM encoders to process sparse observations without full-space simulations.
result Achieves significant speedups over existing methods while maintaining high accuracy.
New approach uses unlabeled prior data to accelerate exploration in sparse reward tasks.
problem Sparse reward tasks in reinforcement learning.
method Learn reward model from online experience, label prior data, and use concurrently.
result Rapid exploration in challenging sparse-reward domains.
SDAMI enhances interpretable high-dimensional regression with sparse deep learning and footprint principle.
problem Personalized models for small samples and high-dimensional features with interpretability.
method Sparse Deep Additive Model with Interactions (SDAMI) combining sparsity-driven feature selection and deep subnetworks.
result SDAMI successfully identifies pure interactions with near-zero false positive rates.
This work introduces a method to compare sparse neural network topologies using graph theory.
problem Comparing and understanding sparse neural network topologies, especially during training.
method Introducing Neural Network Sparse Topology Distance (NNSTD) to measure distances between different sparse neural networks.
result Sparse neural networks can outperform over-parameterized models without further structure optimization.
We present a robust alternative to principal component analysis (PCA) --- called elliptical component analysis (ECA) --- for analyzing high dimensional, elliptically distributed data. ECA estimates the eigenspace of the covariance matrix of the elliptical data. To cope with heavy-tailed elliptical distributions, a mult…
Nonnegative CANDECOMP/PARAFAC (NCP) decomposition is an important tool to process nonnegative tensor. Sometimes, additional sparse regularization is needed to extract meaningful nonnegative and sparse components. Thus, an optimization method for NCP that can impose sparsity efficiently is required. In this paper, we co…
Solving Statistical Mechanics on Sparse Graphs with Feedback Set Variational Autoregressive Networkscond-mat.stat-mech
We propose a method for solving statistical mechanics problems defined on sparse graphs. It extracts a small Feedback Vertex Set (FVS) from the sparse graph, converting the sparse system to a much smaller system with many-body and dense interactions with an effective energy on every configuration of the FVS, then learn…
Several Approximation Algorithms for Sparse Best Rank-1 Approximation to Higher-Order Tensorsmath.NA
Four algorithms improve sparse tensor BR1Approx with theoretical guarantees.
problem Sparse tensor best rank-1 approximation.
method Four approximation algorithms exploiting multilinearity and sparsity.
result Theoretical worst-case approximation lower bounds for all algorithms.
New model leads to optimal test loss in sparse linear regression.
problem Sparse linear regression with low test loss despite interpolating training data.
method Developed a new parametrization of the model that combines benefits of ℓ1 and ℓ2 norms.
result Training via gradient descent leads to an interpolator with near-optimal test loss.
New sparse penalty improves biclustering for gene expression data.
problem Identifying significant clusters in gene expression data.
method Prenet penalty applied to SSVD for biclustering.
result Mixed Prenet penalty effectively clusters non-overlapped data.
Convolutional sparse representations are a form of sparse representation with a dictionary that has a structure that is equivalent to convolution with a set of linear filters. While effective algorithms have recently been developed for the convolutional sparse coding problem, the corresponding dictionary learning probl…
This letter presents a novel Block Bayesian Hypothesis Testing Algorithm (Block-BHTA) for reconstructing block sparse signals with unknown block structures. The Block-BHTA comprises the detection and recovery of the supports, and the estimation of the amplitudes of the block sparse signal. The support detection and rec…
This paper presents a novel Block Iterative Bayesian Algorithm (Block-IBA) for reconstructing block-sparse signals with unknown block structures. Unlike the existing algorithms for block sparse signal recovery which assume the cluster structure of the nonzero elements of the unknown signal to be independent and identic…
Gradient descent with early stopping achieves optimal sparse recovery.
problem Sparse regression with gradient descent and early stopping.
method Gradient descent on depth-N networks with early stopping.
result Implicit sparse regularization occurs with early stopping for general depth N.
Recent breakthrough results in compressed sensing (CS) have established that many high dimensional objects can be accurately recovered from a relatively small number of non- adaptive linear projection observations, provided that the objects possess a sparse representation in some basis. Subsequent efforts have shown th…