SIP corrects model bias in Bayesian ML.
problem Model selection biases predictions in Bayesian ML.
method Sparse Implicit Processes (SIP) for flexible, trainable predictions.
result SIP provides better predictive distributions than initial models.
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,695 papers · 148 categories
Trend · papers per month
48 results for “Sparse Implicit Processes”
New model leads to optimal test loss in sparse linear regression.
problem Sparse linear regression with low test loss despite interpolating training data.
method Developed a new parametrization of the model that combines benefits of ℓ1 and ℓ2 norms.
result Training via gradient descent leads to an interpolator with near-optimal test loss.
The paper learns compact implicit surface maps from streaming data using an ensemble of sparse Gaussian processes.
problem Creating compact and accurate implicit surface maps from streaming range data.
method An ensemble of sparse Gaussian process experts, incrementally adjusted, trades-off between model complexity and prediction error.
result The approach learns compact and accurate implicit surface models comparable to or better than exact GP regression with subsampled data.
A new Gaussian process regression method infers implicit manifold structure from data.
problem Scaling Gaussian process regression to high-dimensional data.
method Proposes a fully differentiable Gaussian process regression technique that infers implicit manifold structure from data.
result Improves predictive performance and calibration of standard Gaussian process regression in high-dimensional settings.
Gradient descent with early stopping achieves optimal sparse recovery.
problem Sparse regression with gradient descent and early stopping.
method Gradient descent on depth-N networks with early stopping.
result Implicit sparse regularization occurs with early stopping for general depth N.
New method tunes prior IP to data for flexible predictive distributions.
problem Challenges in approximate inference for large models with high parameter dependencies.
method Inducing-point representation of prior IP to approximate posterior process.
result Scalable method that tunes prior IP to data and provides accurate non-Gaussian predictive distributions.
This work improves distribution recovery from sparse data using Random Forest implicit regularization.
problem Distribution recovery from limited statistics.
method Closed-form estimator for scaled beta distributions, using composite quantile and moment matching.
result Improved classification accuracy through closed-form distribution recovery and implicit regularization.
Tuning-Free Structured Sparse Recovery of Multiple Measurement Vectors using Implicit Regularizationcs.LG
A tuning-free method recovers jointly sparse signals in MMV using implicit regularization.
problem Recovering jointly sparse signals in MMV with minimal tuning or prior knowledge.
method Reparameterizes MMV estimation matrix into decoupled factors and applies gradient descent to a least-squares objective.
result Gradient descent dynamics exhibit a momentum-like effect, converging towards an idealized row-sparse solution.
SINDy-PI robustly identifies implicit dynamics from noisy data.
problem Accurately modeling nonlinear dynamics from noisy data.
method Parallel, implicit SINDy algorithm with multiple optimization algorithms and model selection.
result Significantly more noise robust than previous SINDy approaches.
Gradient descent implicitly favors group sparsity in neural networks.
problem Understanding implicit regularization in neural networks for structured sparsity.
method Novel neural reparameterization for diagonally grouped linear networks.
result Gradient descent without explicit regularization biases towards group sparsity.
We present a sparse grid high-order alternating direction implicit (ADI) scheme for option pricing in stochastic volatility models. The scheme is second-order in time and fourth-order in space. Numerical experiments confirm the computational efficiency gains achieved by the sparse grid combination technique.
(S)GD over Diagonal Linear Networks: Implicit Regularisation, Large Stepsizes and Edge of Stabilitycs.LG
Study on GD and SGD over diagonal networks, focusing on stepsizes and regularisation.
problem Understanding the impact of stochasticity and large stepsizes on gradient descent and SGD solutions.
method Investigation of GD and SGD over diagonal linear networks with macroscopic stepsizes, proving convergence and characterizing solutions.
result Large stepsizes consistently benefit SGD for sparse regression problems, but can hinder GD recovery of sparse solutions, especially in the edge of stability regime.
This work explores non-negative low-rank matrix factorization based on regularized Poisson models (PF or "Poisson factorization" for short) for recommender systems with implicit-feedback data. The properties of Poisson likelihood allow a shortcut for very fast computations over zero-valued inputs, and oftentimes result…
New method improves deep learning models robustness to label noise.
problem Improving deep learning models' robustness to corrupted labels.
method Sparse over-parameterization and implicit regularization.
result State-of-the-art test accuracy against label noise on various datasets.
VRSMD improves SMD convergence and has implicit regularization.
problem Efficiently estimating models with large datasets.
method Variance reduction in stochastic mirror descent.
result VRSMD converges to the minimum mirror interpolant.
We introduce the implicit processes (IPs), a stochastic process that places implicitly defined multivariate distributions over any finite collections of random variables. IPs are therefore highly flexible implicit priors over functions, with examples including data simulators, Bayesian neural networks and non-linear tr…
Diverse sampling improves kernel methods' performance in sparse regions.
problem Improving kernel methods' performance in sparse regions of datasets.
method Using Determinantal Point Processes (DPP) for sampling diverse landmarks in Nyström approximation.
result Nyström kernel regression with diverse landmarks increases accuracy in sparse regions of the dataset.
Efficiently integrates stiff ODEs with vectorized methods.
problem Stiff systems and sparse training data in ODEs.
method Implicit, vectorized time integration with adjoint method.
result Achieves speed ups of greater than 100x on modern GPUs.
Reduces function approximation dimensions from high to low with sparse data.
problem Function approximation from sparse data.
method Nonlinear Level Set Learning (NLL) with geometric information.
result Reduces input dimension to theoretical lower bound with minor accuracy loss.
Noise in SGD affects overparameterized models, favoring sparse solutions.
problem Understanding and mitigating implicit bias in SGD with parameter-dependent noise.
method Theoretical analysis of a quadratically-parameterized model with label noise and Gaussian noise.
result SGD with label noise recovers sparse ground-truth solutions, while SGD with Gaussian noise overfits dense solutions.
Many statistical estimators for high-dimensional linear regression are M-estimators, formed through minimizing a data-dependent square loss function plus a regularizer. This work considers a new class of estimators implicitly defined through a discretized gradient dynamic system under overparameterization. We show that…
Multi-label network classification is a well-known task that is being used in a wide variety of web-based and non-web-based domains. It can be formalized as a multi-relational learning task for predicting nodes labels based on their relations within the network. In sparse networks, this prediction task can be very chal…
The study provides a generalization bound for a family of implicit networks.
problem Theoretical understanding of implicit networks' generalization is limited.
method A generalization bound is derived for a family of implicit networks using a covering number argument for Rademacher complexity.
result A theoretical generalization bound is established for implicit networks.
Proposes LDIDPs for efficient sequential data generation from latent dynamical models.
problem Challenges in generating high-fidelity sequential samples from latent dynamical models.
method Utilizes implicit diffusion processes to sample from latent dynamical processes.
result Demonstrates accurate learning of dynamics and efficient generation of high-quality sequential data.
Large SGD step sizes lead to sparse feature learning in neural networks.
problem Sparse feature learning in neural networks with large step sizes.
method Empirical observations and theoretical analysis of SGD dynamics.
result Large step sizes induce implicit regularization leading to sparse predictors.
Implicit Finite-Horizon Approximation and Efficient Optimal Algorithms for Stochastic Shortest Pathcs.LG
New algorithms minimize regret in SSP with optimal sparse updates.
problem Minimizing regret in Stochastic Shortest Path models.
method Implicit finite-horizon approximation for analysis, model-free and model-based algorithms developed.
result Minimax optimal regret for both model-free and model-based algorithms.
Guarantees sparse recovery for neural networks with iterative hard thresholding.
problem Recovering sparse network weights in neural networks.
method Structural properties of sparse network weights and iterative hard thresholding algorithm.
result Simple iterative hard thresholding algorithm recovers sparse network weights exactly using linear memory.
ADSGD method speeds up model identification in sparse optimization.
problem Implicit model identification in sparse optimization problems.
method Accelerated Doubly Stochastic Gradient Method (ADSGD) for faster explicit model identification.
result ADSGD achieves faster explicit model identification and improved algorithm efficiency.
Paper proposes a new method for probabilistic electricity price forecasting.
problem Accurate estimation of forecast uncertainties for optimal decision making.
method Implicit generative ensemble post-processing using an ensemble of point forecasting models.
result Method outperforms well-established model combination benchmarks.
Transformers learn to integrate information from past positions incrementally, specializing heads in distinct patterns.
problem How transformers learn to integrate information from multiple past positions with varying statistical significance.
method High-order Markov chain task, incremental learning, sparse attention patterns, simplified differential equations, stage-wise convergence, early stopping as regularizer.
result Transformers learn to specialize heads in distinct patterns, shifting from competitive to cooperative learning dynamics.
Combining simple elements from the literature, we define a linear model that is geared toward sparse data, in particular implicit feedback data for recommender systems. We show that its training objective has a closed-form solution, and discuss the resulting conceptual insights. Surprisingly, this simple model achieves…
Early alignment in neural networks leads to sparse representations but hinders convergence.
problem The implicit bias of gradient descent during early training phases.
method Quantitative description of early alignment phase in small initialisation, one hidden layer networks.
result Early alignment induces a sparse representation but also hinders convergence to global minima.
This paper improves the scalability of sparse neural network compression.
problem Sparse neural network compression for diverse data modalities.
method State-of-the-art sparsification techniques and meta-learning.
result Meta-learning sparse compression networks achieve new state-of-the-art results.
Semi-supervised method enhances recommendation from sparse data.
problem Data sparsity in implicit feedbacks and lack of negative samples.
method Domain adaptation using textual features to align latent spaces.
result Our models significantly outperform existing models on real-world datasets.
The presence of a sparse "truth" has been a constant assumption in the theoretical analysis of sparse PCA and is often implicit in its methodological development. This naturally raises questions about the properties of sparse PCA methods and how they depend on the assumption of sparsity. Under what conditions can the r…
AIR-Net adapts low-rank regularization dynamically for better image completion.
problem Fixed low-rank regularization limits adaptability to different images.
method AIR-Net uses adaptive and implicit regularization parameterized by a dynamic Laplacian matrix.
result AIR-Net enhances implicit regularization and outperforms fixed methods in non-uniform missing data scenarios.
Paper explores how DPP sampling can implicitly regularize kernel regression.
problem Improving kernel regression by reducing redundancy in data.
method Using Determinantal Point Processes (DPPs) to sample subsets implicitly regularizes ridgeless Kernel Regression.
result Ensemble of ridgeless regressors can be effective for datasets with redundant information.
DVIP improves on IP-based methods by using IPs as priors over latent functions.
problem Limited expressiveness of IP-based models, especially in function space.
method Proposes DVIP, a multi-layer generalization of IPs, and scalable variational inference.
result DVIP outperforms previous IP-based methods and deep GPs in regression and classification tasks.
We investigate implicit regularization schemes for gradient descent methods applied to unpenalized least squares regression to solve the problem of reconstructing a sparse signal from an underdetermined system of linear measurements under the restricted isometry assumption. For a given parametrization yielding a non-co…
Implicit probabilistic models are a flexible class of models defined by a simulation process for data. They form the basis for theories which encompass our understanding of the physical world. Despite this fundamental nature, the use of implicit models remains limited due to challenges in specifying complex latent stru…
DeRegiME forecasts with regime structure, improving probabilistic predictions across various time series.
problem Probabilistic forecasting discards residual uncertainty, and distribution shifts are hard to capture.
method DeRegiME uses a sparse variational Gaussian process with a nonstationary regime-mixing kernel to separate latent uncertainty regimes.
result DeRegiME improves NLPD by 20.3% on average across benchmarks, with gains on CRPS and MSE.
In this paper we consider the cluster estimation problem under the Stochastic Block Model. We show that the semidefinite programming (SDP) formulation for this problem achieves an error rate that decays exponentially in the signal-to-noise ratio. The error bound implies weak recovery in the sparse graph regime with bou…
Increased public interest in healthy lifestyles has motivated the study of algorithms that encourage people to follow a healthy diet. Applying collaborative filtering to build recommendation systems in domains where only implicit feedback is available is also a rapidly growing research area. In this report we combine t…
MLP-Mixer achieves better performance through sparsity and wider architecture.
problem Understanding why MLP-Mixer outperforms conventional MLPs.
method Revealed sparseness as a key mechanism, showed effective expression as wider MLP, demonstrated quantitative similarities, and applied a guiding principle.
result MLP-Mixer's performance improvement through sparseness and wider architecture.
DINo forecasts PDEs with flexible extrapolation and adaptability.
problem Fixed discretizations limit real-world PDE forecasting.
method DINo uses implicit neural representations for continuous-time dynamics.
result DINo outperforms other neural PDE forecasters.
The paper studies implicit regularization in over-parameterized models for high-dimensional data.
problem Understanding implicit regularization in over-parameterized models for high-dimensional data.
method The paper designs regularization-free algorithms for the high-dimensional single index model and provides theoretical guarantees for the induced implicit regularization phenomenon.
result The proposed methods achieve minimax optimal statistical rates of convergence and outperform classical methods with explicit regularization.
A new model shows fairness mechanisms can improve selection utility even without implicit bias.
problem Improving selection fairness without introducing a utility trade-off.
method A model with latent quality and group-dependent variance, comparing fairness mechanisms to group-oblivious selection.
result Demographic parity always increases selection utility, while -rules weakly increase it.
User preferences for items can be inferred from either explicit feedback, such as item ratings, or implicit feedback, such as rental histories. Research in collaborative filtering has concentrated on explicit feedback, resulting in the development of accurate and scalable models. However, since explicit feedback is oft…