Research
On-device research index

arXiv research

A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.

168,742 papers · 148 categories

Trend · papers per month

10.8%21.6%32.5%43.3% · Jun 202019922001200920172026
48 results for data filtering

New method filters large networks from financial data to reveal key subnetworks.

problem Filtering large dimensional networks to isolate key constituents.
method Exploits spectral properties of high-dimensional data networks, tuning for sparsity and consistency.
result Shows method can interpolate between zero and maximal filtering, preserving spectral properties.

Filtering data with a pre-trained model improves multimodal contrastive learning performance.

problem Improving the quality of internet-scale multimodal datasets.
method Characterized the performance of filtered contrastive learning under a bimodal data generation model.
result Data filtering using a pre-trained model reduces contrastive learning error by a factor of η\sqrt{η} in the large ηη regime.

A new SOHP filter improves trend estimation in economic time series.

problem Improving trend estimation in nonlinear economic time series.
method Recursive application of one-sided HP filter on updated cyclical components, combined with an incremental HP filtering algorithm.
result Better performance of SOHP filter compared to other HP-type filters on real economic data.

Gradient filters track moving parameters under noisy data and misspecification.

problem Tracking multidimensional time-varying parameters under noisy observations and model misspecification.
method Gradient-based filters update parameters using the gradient of a postulated objective function, evaluated at either the predicted or updated parameters.
result Novel sufficient conditions for exponential stability of the filtered parameter path, and finite-sample and asymptotic mean squared error bounds.

This work preserves linear invariants in ensemble filters for non-Gaussian data assimilation.

problem Maintaining critical invariants like mass, stoichiometric balance, and charge in non-Gaussian data assimilation.
method Introducing a novel class of nonlinear ensemble filters using measure transport theory.
result Recovery of a constrained Kalman filter for Gaussian settings and combination with regularization techniques.

The Hodrick-Prescott (HP) filter is one of the most widely used econometric methods in applied macroeconomic research. Like all nonparametric methods, the HP filter depends critically on a tuning parameter that controls the degree of smoothing. Yet in contrast to modern nonparametric methods and applied work with these…

2019-05-01abs ↗pdf ↗

A new ML-based filter improves data assimilation for nonlinear systems.

problem Improving data assimilation for nonlinear systems using ensemble methods.
method Developed a machine learning-based conditional mean filter (ML-EnCMF) integrating ANN and linear functions.
result ML-EnCMF outperforms EnKF and likelihood-based EnCMF in nonlinear systems.

In this paper we formally analyse the use of sparse filtering algorithms to perform covariate shift adaptation. We provide a theoretical analysis of sparse filtering by evaluating the conditions required to perform covariate shift adaptation. We prove that sparse filtering can perform adaptation only if the conditional…

2016-07-22abs ↗pdf ↗

AD-EnKFs use machine learning to improve data assimilation in high-dimensional systems.

problem Data assimilation in high-dimensional, unknown dynamics systems.
method Auto-differentiable ensemble Kalman filters blending machine learning and ensemble Kalman filters.
result AD-EnKFs outperform existing methods in the Lorenz-96 model.

This paper proves long-time accuracy of ensemble Kalman filters for chaotic and machine-learned systems.

problem Ensuring long-term accuracy of ensemble Kalman filters for complex dynamical systems.
method Established conditions for long-time accuracy of ensemble Kalman filters for chaotic and machine-learned dynamical systems.
result Ensemble Kalman filters maintain small estimation error over long time horizons for chaotic and machine-learned systems.

BankGCN improves graph convolution networks by handling multi-channel signals with adaptive filter banks.

problem Handling multi-channel graph signals with limited architectures.
method BankGCN decomposes multi-channel signals into subspaces and uses adapted filters for each subspace.
result BankGCN achieves excellent performance in graph classification on benchmark datasets.

Data is said to follow the transform (or analysis) sparsity model if it becomes sparse when acted on by a linear operator called a sparsifying transform. Several algorithms have been designed to learn such a transform directly from data, and data-adaptive sparsifying transforms have demonstrated excellent performance i…

2018-03-06abs ↗pdf ↗

Recent work has suggested enhancing Bloom filters by using a pre-filter, based on applying machine learning to determine a function that models the data set the Bloom filter is meant to represent. Here we model such learned Bloom filters,, with the following outcomes: (1) we clarify what guarantees can and cannot be as…

2019-01-03abs ↗pdf ↗

This paper learns state, dynamics, and filtering algorithms together for data assimilation.

problem Costly parameter tuning and inaccurate dynamics models hinder data assimilation algorithms.
method Auto-differentiable data assimilation framework that learns state, dynamics, and parameters via gradient-based optimization.
result Several data assimilation methods can be learned or tuned within this framework.

A novel approach is put forth that utilizes data similarity, quantified on a graph, to improve upon the reconstruction performance of principal component analysis. The tasks of data dimensionality reduction and reconstruction are formulated as graph filtering operations, that enable the exploitation of data node connec…

2018-09-25abs ↗pdf ↗

This paper presents a fast Bayesian filtering technique for state estimation.

problem Bottleneck in Bayesian inference for state estimation from noisy sensor data.
method Processor-native uncertainty tracking for uncertainty propagation and inference.
result Deterministic approximate filtering with up to 805x speedup and competitive accuracy.

Graph neural networks leverage graph filters to learn from network data.

problem Learning from network data with graph structure.
method Characterize graph neural networks using graph signal processing and graph convolutional filters.
result Graph neural networks have permutation equivariance and stability to topology changes.

Paper uses averaging from many particle filters to approximate posterior predictive distributions.

problem Approximating posterior predictive distributions efficiently and accurately.
method Particle swarm filter algorithm that averages many particle filter approximations.
result Law of large numbers and central limit theorem support the method's effectiveness.

Proposes CE-BASS for robust Kalman filtering with innovative and additive outliers.

problem Robustness to both innovative and additive outliers in Kalman filtering.
method Particle mixture Kalman filter with re-sampling of past states.
result CE-BASS efficiently handles multi-modality and trend changes in hidden state distributions.

This paper explores how balancing and filtering techniques affect predictive multiplicity in machine learning models.

problem Predictive multiplicity due to Rashomon effect in high-stakes environments.
method Investigates the impact of balancing and filtering techniques on predictive multiplicity using 21 real-world datasets.
result Data-centric AI strategies can mitigate predictive multiplicity, but preprocessing methods may introduce it.

The paper develops a computational method for efficient online filtering of diffusion processes.

problem Online filtering of discretely observed nonlinear diffusion processes.
method The approach involves Doob's hh-transforms approximated by solving backward Kolmogorov equations using nonlinear Feynman-Kac formulas and neural networks.
result The proposed method can be orders of magnitude more efficient than state-of-the-art particle filters.

A new ensemble filter uses transport maps and MMD optimization for high-dimensional data assimilation.

problem High-dimensional data assimilation challenges in ensemble filtering.
method Optimized Maximum Mean Discrepancy (MMD) for transport map construction.
result Significant improvement in robustness and posterior approximation.

Novel method uses Bayesian filters and PCRLB for state estimation of option prices.

problem Estimating unobserved latent variables from option prices.
method Posterior Cramer-Rao Lower Bound (PCRLB) based adaptive state estimation using various Bayesian filters.
result Proposed method outperforms individual filters and improves forecasting.

This paper presents a new approach for filter design based on stochastic distances and tests between distributions. A window is defined around each pixel, overlapping samples are compared and only those which pass a goodness-of-fit test are used to compute the filtered value. The technique is applied to intensity SAR d…

2013-08-29abs ↗pdf ↗

Graph-based methods have been demonstrated as one of the most effective approaches for semi-supervised learning, as they can exploit the connectivity patterns between labeled and unlabeled data samples to improve learning performance. However, existing graph-based methods either are limited in their ability to jointly …

2019-01-28abs ↗pdf ↗

Adaptive filters are applied in several electronic and communication devices like smartphones, advanced headphones, DSP chips, smart antenna, and teleconference systems. Also, they have application in many areas such as system identification, channel equalization, noise reduction, echo cancellation, interference cancel…

2019-09-06abs ↗pdf ↗

Filters in a Convolutional Neural Network (CNN) contain model parameters learned from enormous amounts of data. In this paper, we suggest to decompose convolutional filters in CNN as a truncated expansion with pre-fixed bases, namely the Decomposed Convolutional Filters network (DCFNet), where the expansion coefficient…

2018-02-12abs ↗pdf ↗

Recursive KalmanNet combines neural networks with Kalman filters for precise state estimation.

problem State estimation in systems with noisy measurements and non-Gaussian noise.
method Recursive KalmanNet uses a recurrent neural network to estimate states with consistent error covariance, optimizing for Gaussian negative log-likelihood.
result Recursive KalmanNet outperforms conventional Kalman filters and deep learning-based estimators in non-Gaussian noise conditions.

A new flow-based Bayesian filter tackles high-dimensional nonlinear stochastic systems.

problem Bayesian filtering for high-dimensional nonlinear systems is challenging due to non-Gaussian distributions and computational limitations.
method Integrates normalizing flows to construct a latent linear state-space model with efficient density estimation and sampling.
result Demonstrates superior accuracy and efficiency in numerical experiments.

Volatility dynamics of wavelet - filtered stock price time series is studied. Using the universal thresholding method of wavelet filtering and a principle of minimal linear autocorrelation of noise component we find that the quantitative characteristics of volatility dynamics of denoised series are noticeably different…

2006-12-18abs ↗pdf ↗