We study trend filtering, a recently proposed tool of Kim et al. [SIAM Rev. 51 (2009) 339-360] for nonparametric regression. The trend filtering estimate is defined as the minimizer of a penalized least squares criterion, in which the penalty term sums the absolute th order discrete derivatives over the input points…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,695 papers · 148 categories
Trend · papers per month
48 results for “Locally Adaptive Regression Splines”
Bayesian nonparametric LABS model adapts to function smoothness in Besov spaces.
problem Estimating functions with unknown smoothness in Besov spaces.
method Lévy Adaptive B-spline (LABS) regression model with automatic smoothness adaptation.
result LABS posterior contracts around true function in Besov classes at nearly minimax-optimal rates.
The paper develops a new method for estimating non-parametric regression functions with spatio-temporal dependencies.
problem Estimating non-parametric regression functions with spatio-temporal dependencies.
method Locally Adaptive Regression Splines (LARS) with ADMM algorithm.
result The method shows superior performance compared to existing techniques.
Improves BN graph learning with splines for scalability.
problem Learning accurate BN graph structures from data.
method Score-and-search approach with MARS for CPD modeling.
result Improves BN graph accuracy and scalability.
SMART: A Flexible Approach to Regression using Spline-Based Multivariate Adaptive Regression Treesstat.ML
SMART combines decision trees and MARS for better regression modeling.
problem High variance in decision trees for continuous relationships, poor performance in MARS for discontinuities.
method SMART uses a decision tree to identify subsets with distinct continuous relationships, then applies MARS to fit these relationships independently.
result SMART improves regression performance over state-of-the-art methods in capturing discontinuities and continuous relationships.
A new method evolves point clouds using B-splines for smooth surfaces.
problem Evolution of smooth surfaces from discrete point clouds.
method Adaptive Lagrangian B-spline framework for geometric evolution.
result Efficient and accurate reproduction of surface evolution phenomena.
Highly Adaptive Ridgestat.ML
HAR regression improves performance on small datasets.
problem Small datasets with complex functions.
method Data-adaptive kernel ridge regression using tensor-product spline basis.
result Achieves convergence rate for right-continuous functions.
We extend the adaptive regression spline model by incorporating saturation, the natural requirement that a function extend as a constant outside a certain range. We fit saturating splines to data using a convex optimization problem over a space of measures, which we solve using an efficient algorithm based on the condi…
Locally-verifiable conditions ensure exactness of spline discrete de Rham complex.
problem Ensuring cohomological equivalence of spline discrete complex to continuous de Rham complex.
method Theoretical analysis and locally-verifiable sufficient conditions for exactness.
result Locally-verifiable conditions guarantee exactness of hierarchical B-spline discrete de Rham complex.
Dimension Reduction and MARSstat.ME
Improves MARS for nonparametric multivariate regression with dimension reduction.
problem High number of basis functions in MARS for high-order interactions.
method Linear combinations of covariates for dimension reduction, facilitating gradient calculation and eigen-analysis for estimation.
result Asymptotic theory and numerical studies show improved performance over MARS.
Adaptive Bayesian Multivariate Spline Knot Inference with Prior Specifications on Model Complexitystat.ME
Bayesian method for knot inference in multivariate spline regression.
problem Inference on knot locations in multivariate spline regression due to non-differentiability and varying dimensions.
method Fully Bayesian approach with a new prior on knot number and analytic formula for normal model, extended Bayesian information criterion for non-normal cases, reversible jump Markov chain Monte Carlo.
result Demonstrated superior performance in function fitting with jumping discontinuity.
A new model approximates complex functions in parameter space.
problem Complex and nonlinear functional regression problems.
method Mapping-to-Parameter function model with B-spline free knot placement.
result Robust knot placement algorithms improve model performance.
NQE uses quantile regression for fast SBI with cubic Hermite splines.
problem Efficient Bayesian inference for complex models with limited data.
method Neural Quantile Estimation (NQE) learns quantiles autoregressively and interpolates them using cubic Hermite splines.
result NQE achieves state-of-the-art performance on various benchmark problems.
Unified approach for interpretable regression with flexible modeling.
problem Combining predictive adaptivity with interpretability in heterogeneous data.
method Combining random Fourier features, spectral feature map, principal component analysis, Gaussian mixture model, and cluster-specific generalized additive models.
result Consistently improves upon classical and black-box models across benchmark datasets.
This paper presents an efficient algorithm for evolving point cloud data on smooth manifolds using B-Splines.
problem Evolution of point cloud data on smooth manifolds in higher dimensions.
method Lagrangian approach using adaptive B-Spline interpolation.
result Demonstrates the convergence of geometric quantities and the effectiveness of the approach.
Deep neural networks (DNNs) generate much richer function spaces than shallow networks. Since the function spaces induced by shallow networks have several approximation theoretic drawbacks, this explains, however, not necessarily the success of deep networks. In this article we take another route by comparing the expre…
This paper develops a new method for constructing splines on Lie groups using Poisson equation solutions.
problem Existing methods for constructing splines on Lie groups have limitations and assumptions that may not reflect actual curves.
method The paper introduces a new approach using solutions of the Poisson equation on Lie groups to construct splines.
result The new method allows for global splines with arbitrary initial conditions, improving curve reconstruction.
Prediction of dynamical time series with additive noise using support vector machines or kernel based regression has been proved to be consistent for certain classes of discrete dynamical systems. Consistency implies that these methods are effective at computing the expected value of a point at a future time given the …
Study prenatal PM2.5 exposure and 4th grade reading scores, identifying critical windows of susceptibility.
problem Understanding the impact of prenatal PM2.5 exposure on educational outcomes.
method Developed a locally adaptive Bayesian regression model with B-spline basis expansion and dynamic shrinkage priors.
result Prenatal PM2.5 exposure during early and late pregnancy is most adverse for 4th grade reading scores.
Deep P-Spline automates DNN structure selection for complex regression problems.
problem Challenges in selecting optimal network structures for DNNs.
method Linking neuron selection to knot placement in basis expansion techniques, introducing a difference penalty for automated knot selection.
result Deep P-Spline extends model class and forms a latent variable modeling framework with theoretical guarantees.
We consider the generic regularized optimization problem . Efron, Hastie, Johnstone and Tibshirani [Ann. Statist. 32 (2004) 407--499] have shown that for the LASSO--that is, if is squared error loss and is the norm of --the opti…
A new modeling framework CSN simplifies and interprets machine learning models.
problem Complexity and interpretability issues in machine learning models.
method Combines spline transformation and cross-network to create CSN.
result CSN is as performant and interpretable as XGBoost and FCNN.
Graph poly-Laplacian method improves regression accuracy.
problem Regression with noisy labels on graphs.
method Graph poly-Laplacian regularization for non-parametric regression.
result Rate of convergence matches known results for smoothing splines.
Kolmogorov-Arnold Networks achieve optimal convergence rates in nonparametric regression.
problem Nonparametric function approximation in multivariate settings.
method Structured additive and multiplicative KANs using B-splines.
result Achieve minimax-optimal convergence rate for Sobolev space functions.
Quantum walks blend patterns into splines when averaged.
problem Understanding the asymptotic patterns of quantum random walks.
method Averaging over quantum coins using the Haar measure.
result Patterns blend into splines, showing a unified behavior.
This work relates the framework of model-based clustering for spatial functional data where the data are surfaces. We first introduce a Bayesian spatial spline regression model with mixed-effects (BSSR) for modeling spatial function data. The BSSR model is based on Nodal basis functions for spatial regression and accom…
We use splines and the Sasaki metric to analyze and compare manifold-valued trajectories.
problem Analyzing and comparing trajectories on Riemannian manifolds.
method Riemannian hierarchical model, Bézier splines, Sasaki metric.
result Spline-based approaches outperform state-of-the-art methods in intensity classification of trajectories.
Neural networks require a careful design in order to perform properly on a given task. In particular, selecting a good activation function (possibly in a data-dependent fashion) is a crucial step, which remains an open problem in the research community. Despite a large amount of investigations, most current implementat…
Gradient descent training of neural networks leads to solutions close to natural cubic splines.
problem Understanding the implicit bias of gradient descent in neural networks.
method Analysis of gradient descent training for wide neural networks, focusing on the curvature penalty and initialization schemes.
result The solutions of gradient descent training are polyharmonic splines for certain initialization schemes.
Develops a new method to model overlapping asymmetric datasets effectively.
problem Handling overlapping asymmetric datasets in data science.
method Twice penalized P-Spline approximation method.
result Improves model fit by over 65% in a real-life dataset.
This paper tackles the problem of selecting among several linear estimators in non-parametric regression; this includes model selection for linear regression, the choice of a regularization parameter in kernel ridge regression, spline smoothing or locally weighted regression, and the choice of a kernel in multiple kern…
New approach to adaptively select bandwidths in nonparametric regression.
problem Adaptive bandwidth selection in nonparametric regression.
method Inspired by -norms of interval projections, introduces a new bandwidth selection procedure.
result Obtains non-asymptotic risk bounds for local polynomial regression methods that adapt to local Hölder exponent.
This paper is devoted to the application of B-splines to volatility modeling, specifically the calibration of the leverage function in stochastic local volatility models and the parameterization of an arbitrage-free implied volatility surface calibrated to sparse option data. We use an extension of classical B-splines …
Adaptive algorithm improves nonlinear data assimilation for non-Gaussian systems.
problem Challenges of non-Gaussian statistics in data assimilation.
method Triangular measure transport with P-spline basis functions and an information criterion.
result Automatic selection of parsimonious parametrization for efficient adaptation.
Locally adaptive interpretable regression improves linear regression's predictability.
problem Linear regression's predictability is limited; it lacks adaptability.
method Locally adaptive interpretable regression (LoAIR) uses neural networks to predict percentile of a Gaussian distribution for regression coefficients.
result LoAIR achieves comparable or better predictive performance than state-of-the-art baselines.
This paper aims to solve a basic problem in distributed statistical inference: how many machines can we use in parallel computing? In kernel ridge regression, we address this question in two important settings: nonparametric estimation and hypothesis testing. Specifically, we find a range for the number of machines und…
Adversarial online nonparametric regression achieves optimal rates with locally adaptive learning.
problem Adversarial online nonparametric regression with general convex losses.
method Parameter-free learning algorithm leveraging chaining trees to compete against H{ö}lder functions, dynamically tracking and adapting to local smoothness variations.
result First computationally efficient algorithm with locally adaptive optimal rates for online regression in an adversarial setting.
This paper introduces a spline-based method for nonparametric ADVI that handles complex posterior distributions.
problem Learning complex posterior distributions with skewness, multimodality, and bounded support.
method Develops a spline-based nonparametric approximation approach for ADVI.
result Establishes the asymptotic consistency of the derived lower bound for importance weighted autoencoder.
Proposes a continuous, differentiable model from local adaptive models.
problem Inadequate continuity and differentiability in over-parameterized models.
method A global continuous and differentiable model constructed from weighted averages of locally learned models.
result Achieves faster statistical convergence and improved performance in various settings.
We consider the problem of non-parametric regression with a potentially large number of covariates. We propose a convex, penalized estimation framework that is particularly well-suited for high-dimensional sparse additive models. The proposed approach combines appealing features of finite basis representation and smoot…
In this paper, we consider one dimensional (shallow) ReLU neural networks in which weights are chosen randomly and only the terminal layer is trained. First, we mathematically show that for such networks L2-regularized regression corresponds in function space to regularizing the estimate's second derivative for fairly …
We propose to optimize the activation functions of a deep neural network by adding a corresponding functional regularization to the cost function. We justify the use of a second-order total-variation criterion. This allows us to derive a general representer theorem for deep neural networks that makes a direct connectio…
TensorSketch is an oblivious linear sketch introduced in Pagh'13 and later used in Pham, Pagh'13 in the context of SVMs for polynomial kernels. It was shown in Avron, Nguyen, Woodruff'14 that TensorSketch provides a subspace embedding, and therefore can be used for canonical correlation analysis, low rank approximation…
Kolmogorov-Arnold Networks enable ultrafast online learning with fixed-point quantization.
problem Efficient online learning for high-frequency systems with strict memory constraints.
method Fixed-point online training on FPGAs exploiting B-spline locality in KANs.
result Kolmogorov-Arnold Networks are more efficient and expressive than MLPs for low-latency tasks.
ABHT boosts regression by filtering regions with different smoothness.
problem Improving regression performance through local adaptivity.
method Gradient boosting with adaptive histogram transform.
result ABHT converges faster than PEHT in Hölder continuous spaces.
Adaptive RBF-KAN: A Comparative Evaluation of Dynamic Shape Parameters in Kolmogorov-Arnold Networksstat.ML
Adaptive RBF-KAN improves KANs by dynamically adjusting kernel parameters.
problem Efficiently approximating multivariate functions using learnable univariate edge functions.
method Integrates LOOCV-based kernel scale estimation with adaptive kernel learning.
result Adaptive RBF-KAN outperforms fixed kernel KANs on various benchmark functions.
TPBS models improve robustness to overfitting with localized Dirichlet energy regularization.
problem Global Dirichlet energy-based regularization fails for TPBS models due to perfect interpolation.
method Propose local Dirichlet energy regularization and two inference estimators.
result TPBS models outperform neural networks in overfitting regimes and maintain competitive performance otherwise.
Unified analysis of multi-task functional linear regression with manifold and composite penalties.
problem Estimating slope functions from functional data with multi-task learning.
method Penalized splines with manifold constraint and composite quadratic penalty.
result Unified convergence upper bound and phase transition behaviors for estimators.