arXiv research

This paper presents a fast and robust algorithm for trend filtering, a recently developed nonparametric regression tool. It has been shown that, for estimating functions whose derivatives are of bounded variation, trend filtering achieves the minimax optimal error rate, while other popular methods like smoothing spline…

A new SOHP filter improves trend estimation in economic time series.

Extracting the underlying trend signal is a crucial step to facilitate time series analysis like forecasting and anomaly detection. Besides noise signal, time series can contain not only outliers but also abrupt trend changes in real-world scenarios. To deal with these challenges, we propose a robust trend filtering al…

Kronecker trend filtering improves lattice data smoothing.

In this article, we discuss various implementation of L1 filtering in order to detect some properties of noisy signals. This filter consists of using a L1 penalty condition in order to obtain the filtered signal composed by a set of straight trends or steps. This penalty condition, which determines the number of breaks…

We introduce a family of adaptive estimators on graphs, based on penalizing the $\ell_1$ norm of discrete graph differences. This generalizes the idea of trend filtering [Kim et al. (2009), Tibshirani (2014)], used for univariate nonparametric regression, to graphs. Analogous to the univariate case, graph trend filteri…

We study trend filtering, a recently proposed tool of Kim et al. [SIAM Rev. 51 (2009) 339-360] for nonparametric regression. The trend filtering estimate is defined as the minimizer of a penalized least squares criterion, in which the penalty term sums the absolute $k$th order discrete derivatives over the input points…

Identifying the unknown underlying trend of a given noisy signal is extremely useful for a wide range of applications. The number of potential trends might be exponential, which can be computationally exhaustive even for short signals. Another challenge, is the presence of abrupt changes and outliers at unknown times w…

We study additive models built with trend filtering, i.e., additive models whose components are each regularized by the (discrete) total variation of their $k$th (discrete) derivative, for a chosen integer $k \geq 0$. This results in $k$th degree piecewise polynomial components, (e.g., $k=0$ gives piecewise constant co…

In this paper we study the estimation of changing trends in time-series using $\ell_1$ trend filtering. This method generalizes 1D Total Variation (TV) denoising for detection of step changes in means to detecting changes in trends, and it relies on a convex optimization problem for which there are very efficient numer…

The Hodrick-Prescott (HP) filter is one of the most widely used econometric methods in applied macroeconomic research. Like all nonparametric methods, the HP filter depends critically on a tuning parameter that controls the degree of smoothing. Yet in contrast to modern nonparametric methods and applied work with these…

Boosting improves trend detection in financial data.

The paper develops a cross-validation method for improving signal denoising techniques.

Proposes CE-BASS for robust Kalman filtering with innovative and additive outliers.

Proposes a new graph trend filtering model for inhomogeneous graph signals.

In this paper, we consider a stochastic asset price model where the trend is an unobservable Ornstein Uhlenbeck process. We first review some classical results from Kalman filtering. Expectedly, the choice of the parameters is crucial to put it into practice. For this purpose, we obtain the likelihood in closed form, a…

TASC improves synthetic control for time-series data with trends.

In this paper, we revisit the Kalman filter theory. After giving the intuition on a simplified financial markets example, we revisit the maths underlying it. We then show that Kalman filter can be presented in a very different fashion using graphical models. This enables us to establish the connection between Kalman fi…

The vast majority of the neural network literature focuses on predicting point values for a given set of response variables, conditioned on a feature vector. In many cases we need to model the full joint conditional distribution over the response variables rather than simply making point predictions. In this paper, we …

Model accurately gates ocean microbes from high-frequency flow cytometry data.

This work studies the denoising of piecewise smooth graph signals that exhibit inhomogeneous levels of smoothness over a graph, where the value at each node can be vector-valued. We extend the graph trend filtering framework to denoising vector-valued graph signals with a family of non-convex regularizers, which exhibi…

Study examines how COVID-19 affects bond yields using network filtering methods.

A new method for joint noise removal and trend estimation from sparse signals.

Decomposing complex time series into trend, seasonality, and remainder components is an important task to facilitate time series anomaly detection and forecasting. Although numerous methods have been proposed, there are still many time series characteristics exhibiting in real-world data which are not addressed properl…

Trend and Value are pervasive anomalies, common to all financial markets. We address the problem of their co-existence and interaction within the framework of Heterogeneous Agent Based Models (HABM). More specifically, we extend the Chiarella (1992) model by adding noise traders and a non-linear demand of fundamentalis…

Increased public interest in healthy lifestyles has motivated the study of algorithms that encourage people to follow a healthy diet. Applying collaborative filtering to build recommendation systems in domains where only implicit feedback is available is also a rapidly growing research area. In this report we combine t…

There has been a recent trend in training neural networks to replace data structures that have been crafted by hand, with an aim for faster execution, better accuracy, or greater compression. In this setting, a neural data structure is instantiated by training a network over many epochs of its inputs until convergence.…

Have you ever felt miserable because of a sudden whipsaw in the price that triggered an unfortunate trade? In an attempt to remove this noise, technical analysts have used various types of moving averages (simple, exponential, adaptive one or using Nyquist criterion). These tools may have performed decently but we show…

This paper studies the question of filtering and maximizing terminal wealth from expected utility in a partially information stochastic volatility models. The special features is that the only information available to the investor is the one generated by the asset prices, and the unobservable processes will be modeled …

The filtering-clustering models, including trend filtering and convex clustering, have become an important source of ideas and modeling tools in machine learning and related fields. The statistical guarantee of optimal solutions in these models has been extensively studied yet the investigations on the computational as…

Podcast recommendations improved by analyzing user listening paths.

Kalman filtering and smoothing algorithms are used in many areas, including tracking and navigation, medical applications, and financial trend filtering. One of the basic assumptions required to apply the Kalman smoothing framework is that error covariance matrices are known and given. In this paper, we study a general…

Solves inventory control with unknown demand trend using singular control.

The paper analyzes MACD using operator theory.

A method to split a data point into two parts that individually cannot reconstruct the whole, but together can.

In this paper, we propose a modified Levy jump diffusion model with market sentiment memory for stock prices, where the market sentiment comes from data mining implementation using Tweets on Twitter. We take the market sentiment process, which has memory, as the signal of Levy jumps in the stock price. An online learni…

Plants monitor their surrounding environment and control their physiological functions by producing an electrical response. We recorded electrical signals from different plants by exposing them to Sodium Chloride (NaCl), Ozone (O3) and Sulfuric Acid (H2SO4) under laboratory conditions. After applying pre-processing tec…

We study a novel spline-like basis, which we name the "falling factorial basis", bearing many similarities to the classic truncated power basis. The advantage of the falling factorial basis is that it enables rapid, linear-time computations in basis matrix multiplication and basis matrix inversion. The falling factoria…

This paper debiases KG embeddings by filtering sensitive attributes.

Paper uses NLP and IRT to score ESG factors from news articles.

In the past decade, tracking health trends using social media data has shown great promise, due to a powerful combination of massive adoption of social media around the world, and increasingly potent hardware and software that enables us to work with these new big data streams. At the same time, many challenging proble…

Large and stable indices of the world wide stock markets such as NYSE and SP 500 together with NASDAQ -- the index representing markets of new trends, and WIG -- the index of the local stock market of Eastern Europe, are considered. Due to the relation between artificial insymmetrised patterns (AIP) and time series, st…

We consider efficient implementations of the generalized lasso dual path algorithm of Tibshirani and Taylor (2011). We first describe a generic approach that covers any penalty matrix D and any (full column rank) matrix X of predictor variables. We then describe fast implementations for the special cases of trend filte…

The study establishes risk bounds for distributional regression estimators.

We present an approach based on machine learning (ML) to distinguish eruption and precursory signals of Chimayó geyser (New Mexico, USA) under noisy environments. This geyser can be considered as a natural analog of $\mathrm{CO}_2$ intrusion into shallow water aquifers. By studying this geyser, we can understand upwell…

Paper proposes a forecasting model combining autoregressive models with spectral attention.

Hybrid AI system combines technical, sentiment analysis for adaptive equity trading.

We establish adaptive results for trend filtering: least squares estimation with a penalty on the total variation of $(k-1)^{\rm th}$ order differences. Our approach is based on combining a general oracle inequality for the $\ell_1$-penalized least squares estimator with "interpolating vectors" to upper-bound the "effe…