This paper presents Sparse Partitioning, a Bayesian method for identifying predictors that either individually or in combination with others affect a response variable. The method is designed for regression problems involving binary or tertiary predictors and allows the number of predictors to exceed the size of the sa…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,742 papers · 148 categories
Trend · papers per month
48 results for “sparse influence”
New framework IDOL identifies latent causal processes with instantaneous relations from time series data.
problem Identifying latent causal processes with instantaneous relations from time series data.
method Sparse influence constraint and variational inference architecture with sparsity regularization.
result Our method can identify latent causal processes with instantaneous relations.
We consider inference about a scalar parameter under a non-parametric model based on a one-step estimator computed as a plug in estimator plus the empirical mean of an estimator of the parameter's influence function. We focus on a class of parameters that have influence function which depends on two infinite dimensiona…
Improved scalability and interpretability in training data attribution.
problem Identifying which training data drives specific behaviors, especially unintended ones.
method Leveraging interpretable structures within the model to attribute model behavior to semantic directions, not individual test examples.
result Simple probe-based attribution methods are first-order approximations of Concept Influence that achieve comparable performance while being over an order-of-magnitude faster.
Intrinsically motivated reinforcement learning aims to address the exploration challenge for sparse-reward tasks. However, the study of exploration methods in transition-dependent multi-agent settings is largely absent from the literature. We aim to take a step towards solving this problem. We present two exploration m…
Study improves carbon price forecasting using quantile regression and feature selection.
problem Accurately predicting carbon prices influenced by geopolitical, social, and economic factors.
method Collect and analyze various influencing factors, select significant features, and use Sparse Quantile Group Lasso and Adaptive Sparse Quantile Group Lasso for robust predictions.
result Proposed methods outperform existing ones and provide a complete profile of future carbon prices.
There has been a lot of work fitting Ising models to multivariate binary data in order to understand the conditional dependency relationships between the variables. However, additional covariates are frequently recorded together with the binary data, and may influence the dependence relationships. Motivated by such a d…
Sparse APCA identifies sparse factors in financial returns over time.
problem Analyzing co-movements of high-dimensional panel data over time.
method Sparse asymptotic PCA with truncated power method for sparse factors and sequential deflation for multi-factor cases.
result Identification of nine risk factors influencing the S&P 500 stock market.
Jointly learns feature and sample relevancies for robust sparse recovery.
problem Sparse recovery sensitivity to data contaminants like outliers or misspecified noise.
method Jointly learns feature and sample relevancies via marginal likelihood optimization.
result Consistent sparse and robust prediction models across diverse tasks.
SnAp approximates RTRL for online training of sparse recurrent networks.
problem Training large sparse recurrent networks online is computationally expensive.
method Sparse n-step Approximation (SnAp) of the RTRL influence matrix.
result SnAp with n=2 remains tractable for highly sparse networks and outperforms backpropagation through time.
New pruning methods improve dynamic sparse training performance.
problem Improving dynamic sparse training performance.
method Design and empirical analysis of pruning criteria.
result Most pruning methods yield similar results, but magnitude-based pruning performs best in low-density regimes.
When training data is sparse, more domain knowledge must be incorporated into the learning algorithm in order to reduce the effective size of the hypothesis space. This paper builds on previous work in which knowledge about qualitative monotonicities was formally represented and incorporated into learning algorithms (e…
This paper addresses the problem of inferring sparse causal networks modeled by multivariate auto-regressive (MAR) processes. Conditions are derived under which the Group Lasso (gLasso) procedure consistently estimates sparse network structure. The key condition involves a "false connection score." In particular, we sh…
Unsupervised exploration and representation learning become increasingly important when learning in diverse and sparse environments. The information-theoretic principle of empowerment formalizes an unsupervised exploration objective through an agent trying to maximize its influence on the future states of its environme…
We propose a dynamic edge exchangeable network model that can capture sparse connections observed in real temporal networks, in contrast to existing models which are dense. The model achieved superior link prediction accuracy on multiple data sets when compared to a dynamic variant of the blockmodel, and is able to ext…
This paper develops an active sensing method to estimate the relative weight (or trust) agents place on their neighbors' information in a social network. The model used for the regression is based on the steady state equation in the linear DeGroot model under the influence of stubborn agents, i.e., agents whose opinion…
This paper explores efficient neural networks by identifying sparse structures.
problem Reducing computational complexity in neural networks without sacrificing accuracy.
method Identifying and utilizing sparse structures in neural network weights and activations.
result Large and sparse models are more beneficial for practical problems.
Domain adaptation framework identifies latent variables for target distribution identifiability.
problem Unsupervised domain adaptation without identifiable joint distribution of features and labels.
method Formulated latent variable model with invariant and changing components, constrained domain shift to influence only changing components.
result Joint distribution of data and labels in target domain is identifiable under mild conditions.
Large SGD step sizes lead to sparse feature learning in neural networks.
problem Sparse feature learning in neural networks with large step sizes.
method Empirical observations and theoretical analysis of SGD dynamics.
result Large step sizes induce implicit regularization leading to sparse predictors.
New framework detects directional influence in multivariate time series.
problem Detecting directional influence in multivariate time series.
method Order-constrained spectral non-invariance.
result Unique diagnostic functional for directional influence.
Paper proposes an algorithm for robust estimation using Huber's criterion.
problem Non-convexity and non-robustness of joint maximum likelihood estimation.
method Block-wise minimization majorization framework with data-adaptive step sizes.
result Improved convergence and robustness in sparse learning.
In this paper we study the problem of exact recovery of the pure-strategy Nash equilibria (PSNE) set of a graphical game from noisy observations of joint actions of the players alone. We consider sparse linear influence games --- a parametric class of graphical games with linear payoffs, and represented by directed gra…
PEAR dynamically reconfigures agent roles to prevent persistent biases in multi-agent debates.
problem Persistent positional biases and sensitivity to role assignments in fixed topologies.
method Dynamic reconfiguration of agent roles and sparse topologies based on evolving agent states.
result Significantly improves average accuracy over debate baselines across multiple reasoning benchmarks.
Sparse rewards with different magnitudes speed up learning in model-based reinforcement learning.
problem Speeding up learning in model-based reinforcement learning methods.
method Add uniformly sampled reward values to sparse binary rewards during training.
result Training can be more efficient and faster with less variability.
We consider a distributed learning setup where a sparse signal is estimated over a network. Our main interest is to save communication resource for information exchange over the network and reduce processing time. Each node of the network uses a convex optimization based algorithm that provides a locally optimum soluti…
Proposes a novel neural architecture for sparse coding using learned greedy pursuit.
problem Lack of interpretability in neural network architectures for sparse coding.
method Unfolded and learned version of Orthogonal Matching Pursuit (OMP) algorithm.
result Demonstrates flexibility and efficiency of the Learned Greedy Method (LGM) in various experiments.
Bayesian modeling predicts hydroxide ion conductivity in polymer membranes.
problem Quantitative relationship between hydrophilic domain size and hydroxide ion conductivity in polymer membranes is unknown.
method Bayesian sparse modeling applied to copolymer composition data.
result Composition-derived features are identified as critical for predicting hydroxide ion conductivity.
The estimation of advantage is crucial for a number of reinforcement learning algorithms, as it directly influences the choices of future paths. In this work, we propose a family of estimates based on the order statistics over the path ensemble, which allows one to flexibly drive the learning process, towards or agains…
The paper studies how regularization parameters affect sparsity in deep neural networks.
problem Reducing the complexity of deep neural networks by promoting sparsity.
method Derives -norm sparsity-promoting models, characterizes sparsity levels, and develops algorithms for selecting optimal regularization parameters.
result Developed algorithms to select regularization parameters for desired sparsity levels in neural networks.
In many applications of finance, biology and sociology, complex systems involve entities interacting with each other. These processes have the peculiarity of evolving over time and of comprising latent factors, which influence the system without being explicitly measured. In this work we present latent variable time-va…
Sparse Conformal Predictorsmath.ST
Conformal predictors, introduced by Vovk et al. (2005), serve to build prediction intervals by exploiting a notion of conformity of the new data point with previously observed data. In the present paper, we propose a novel method for constructing prediction intervals for the response variable in multivariate linear mod…
Sparsity via Hyperpriors: A Theoretical and Algorithmic Study under Empirical Bayes Frameworkstat.ML
The paper studies sparsity in EBF with hyperpriors and proposes a PALM algorithm.
problem Promoting sparsity in sparse learning problems.
method Empirical Bayes framework, hyperpriors, proximal alternating linearized minimization (PALM) algorithm.
result Appropriate hyperpriors can significantly enhance sparsity and restoration accuracy.
New conditions show proxy data can improve policy learning in sparse expert data contexts.
problem Improving sample complexity of policy learning with sparse expert data.
method Sufficient conditions for proxy data to improve sample complexity, adaptable to LLMs.
result Provable improvement in sample complexity for policy learning with proxy data.
This research improves neural network performance with adaptive activation functions in sparse data settings.
problem Limited data availability in scientific and engineering problems.
method Investigation of two types of adaptive activation functions with individual trainable parameters.
result Adaptive activation functions, especially with individual trainable parameters, enhance prediction accuracy and confidence in sparse data settings.
New SVM model balances sparsity and robustness in noisy data.
problem Noise sensitivity and lack of sparsity in traditional SVM models.
method Combines elastic net loss with robust loss framework, integrates with SVM, uses half-quadratic algorithm.
result Proves sparsity and robustness, outperforms traditional SVMs in noisy environments.
Prototype model improves model auditing and understanding.
problem Auditing and understanding modern language models is expensive and approximate.
method Introduced a sparse, non-negative mixture of learned prototypes trained with clustering objectives.
result Prototype models either surpass or remain within 2.5 percentage points of dense baselines on downstream tasks.
This paper addresses the problem of scalable optimization for L1-regularized conditional Gaussian graphical models. Conditional Gaussian graphical models generalize the well-known Gaussian graphical models to conditional distributions to model the output network influenced by conditioning input variables. While highly …
Matrix factorization is a key component of collaborative filtering-based recommendation systems because it allows us to complete sparse user-by-item ratings matrices under a low-rank assumption that encodes the belief that similar users give similar ratings and that similar items garner similar ratings. This paradigm h…
Recommender systems leverage product and community information to target products to consumers. Researchers have developed collaborative recommenders, content-based recommenders, and (largely ad-hoc) hybrid systems. We propose a unified probabilistic framework for merging collaborative and content-based recommendations…
Study reveals benign overfitting in time series models with over-parameterization.
problem Analyzing over-parameterized linear models with dependent time-series data.
method Developed an estimator using interpolation and derived non-asymptotic risk bounds.
result Risk bound is influenced by the coherence of temporal covariance matrices at different time steps.
The paper analyzes how adversarial attacks affect sparse regression models.
problem Effects of adversarial attacks on sparse regression models.
method Primal-dual witness paradigm to analyze support of estimated regression parameter vector.
result Adversaries can influence sample complexity by corrupting irrelevant features.
Model predicts phytoplankton subpopulations based on environmental factors.
problem Predicting phytoplankton dynamics under changing environmental conditions.
method Sparse mixture of multivariate regressions model.
result Identifies environmental covariates influencing phytoplankton subpopulations.
Establishes statistical and computational bounds for influence diagnostics.
problem Identifying influential datapoints or subsets in machine learning models.
method Finite-sample statistical bounds and computational complexity for influence functions and approximate maximum influence perturbations.
result Established statistical and computational guarantees for influence diagnostics.
Neural networks learn task-specific features, influenced by nonlinearity.
problem Understanding the nature of task-dependent feature learning in neural networks.
method Investigation of fully-connected, wide neural networks using Bayesian framework.
result The nature of internal representations depends on neuronal nonlinearity, leading to analog, redundant, or sparse coding schemes.
Dynamic Influence Tracker measures changing sample importance during model training.
problem Static influence measurements during training overlook how sample importance varies over time.
method Dynamic Influence Tracker (DIT) captures time-varying sample influence across arbitrary time windows.
result DIT reveals distinct learning phases with shifting priorities and detects corrupted samples more efficiently.
Previous work has shown that popular trending events are important external factors which pose significant influence on user search behavior and also provided a way to computationally model this influence. However, their problem formulation was based on the strong assumption that each event poses its influence independ…
We address the problem of influence maximization when the social network is accompanied by diffusion cascades. In prior works, such information is used to compute influence probabilities, which is utilized by stochastic diffusion models in influence maximization. Motivated by the recent criticism on the effectiveness o…
RelatIF selects more intuitive training examples for explaining model predictions.
problem Influence functions identify outliers as explanatory examples, leading to poor explanations.
method RelatIF separates global and local influence, optimizing for local relative to global effects.
result Examples selected by RelatIF are more intuitive than those from influence functions.