Develops EM algorithm for analyzing multi-curve data with switching nonparametric regression models.
problem Analyzing multi-curve data with switching latent state processes.
method Switching nonparametric regression models and an EM algorithm for parameter estimation.
result Frequentist properties of parameter estimates validated through simulation studies and real data application.
Many complex dynamical phenomena can be effectively modeled by a system that switches among a set of conditionally linear dynamical modes. We consider two such models: the switching linear dynamical system (SLDS) and the switching vector autoregressive (VAR) process. Our Bayesian nonparametric approach utilizes a hiera…
The paper derives error bounds for piecewise smooth and switching regression models.
problem Regression problems with target functions switching between different modes.
method Derives generalization error bounds using Rademacher complexities and chaining arguments.
result Error bounds with radical dependency on the number of modes for piecewise smooth regression, and linear dependency for switching regression.
Technical note on switching linear regression complexity.
problem Minimizing error in switching linear regression models.
method Extended recent results on computational complexity.
result Problem is NP-hard but admits a polynomial-time algorithm.
Bayesian method clusters time series with varying dynamics.
problem Modeling and clustering time series with unknown number of clusters and dynamics.
method Hierarchical Dirichlet process and Gaussian process for modeling time series patterns and variations.
result Efficiently clusters time series with varying dynamics without unnecessary proliferation of clusters.
Robots learn movement libraries by segmenting complex trajectories.
problem Segmenting complex robot movement demonstrations for library building.
method Model trajectories as Switching Linear Dynamical Systems and infer segmentation using a nonparametric Bayesian approach.
result Robots can learn movement libraries more effectively by segmenting demonstrations.
We link disjoint longitudinal data for rare disease patients using latent representations and mixed-effects regression.
problem Analyzing treatment switches in rare diseases with limited data and changing measurement instruments.
method We embed item values into a shared latent space using variational autoencoders and apply mixed-effects regression to quantify treatment effects.
result Our approach allows for statistical inference and quantifies the impact of treatment switches in spinal muscular atrophy.
Optimal nonparametric regression estimator adapts to unknown smoothness.
problem Nonparametric regression with unknown smoothness.
method Constructs an interpolating estimator that adapts to unknown smoothness.
result Minimax optimal rates achieved on Hölder classes.
Estimates heterogeneous dependencies in Gaussian graphical models.
problem Complex dependence structure of high-dimensional data from different resources.
method Nonparametric mixture of Gaussian graphical models with unified penalized likelihood approach and efficient EM algorithm.
result Effective estimation of nonparametric functional and heterogeneous graphical parameters.
Study uniform consistency in nonparametric mixture models and mixed regression.
problem Uniform consistency in nonparametric mixture models and mixed regression models.
method Construct uniformly consistent estimators under general conditions, develop novel technical tools.
result Prove uniform consistency results for nonparametric mixtures and mixed regression models.
We consider the problem of speaker diarization, the problem of segmenting an audio recording of a meeting into temporal segments corresponding to individual speakers. The problem is rendered particularly difficult by the fact that we are not allowed to assume knowledge of the number of people participating in the meeti…
Markov Chain Monte Carlo is repeatedly used to analyze the properties of intractable distributions in a convenient way. In this paper we derive conditions for geometric ergodicity of a general class of nonparametric stochastic volatility models with skewness driven by hidden Markov Chain with switching.
Data interpolation can achieve optimal rates in nonparametric regression and prediction.
problem Achieving optimal rates in nonparametric regression and prediction.
method Interpolating the training data to achieve optimal rates.
result Interpolating the training data can achieve optimal rates for nonparametric regression and prediction.
A debiasing method improves nonparametric regression's statistical properties.
problem Lack of theoretical guarantees for modern nonparametric regression methods.
method Model-free debiasing method incorporating a correction term.
result Debiased estimator satisfies pointwise and uniform risk convergence, asymptotic normality.
A new approach switches between simple and complex models to handle concept drifts in regression tasks.
problem Handling concept drifts in regression models to maintain accurate predictions over time.
method Error Intersection Approach: switches between simple and complex models based on drift detection.
result The Error Intersection Approach significantly outperforms baselines in handling concept drifts in a real-world taxi demand dataset.
Study on online nonparametric regression using Sobolev kernel methods.
problem Adversarial nonparametric regression in high dimensions.
method Online kernelized ridge regression with Sobolev kernel analysis.
result Upper bounds on regret for Sobolev space classes, revealing optimality in certain cases.
The study explores nonparametric regression with shape constraints using least squares estimation.
problem Nonparametric regression under shape constraints.
method Least squares estimation (LSE) with focus on isotonic, unimodal, convex, and additive shape-restricted regression.
result Adaptive nature of the LSE and its risk behavior, with pointwise limiting distribution theory for isotonic regression.
Flexible nonparametric models for density regression using NCoRM mixtures.
problem Density regression problems.
method Normalized compound random measure mixture models with a novel Metropolis-Hastings sampler.
result Illustrated on density regression problems, the approach makes unbiased estimation of Laplace functionals possible.
OBD algorithm optimizes online convex optimization with strong convexity and switching costs.
problem Online convex optimization with strong convexity and switching costs.
method Online Balanced Descent (OBD) algorithm for m-strongly convex costs with near-optimal dynamic regret and per-round accuracy for ε-smooth sequences. result OBD achieves a competitive ratio of 3+O(1/m) for m-strongly convex costs. Study shows time-varying stock returns across economic states.
problem Equity premium predictability varies by economic state.
method State-switching predictive regression using yield curve slope.
result The Aligned Economic Index improves stock return prediction.
Improved nonparametric regression with debiasing for root-n consistency.
problem Challenges in achieving root-n consistency and normal distribution for nonparametric estimators.
method Debiasing technique by adding a correction term to nonparametric estimators.
result Achieves root-n consistency and asymptotic normality.
Improves tensor regression convergence rate with Bayesian estimation.
problem Nonlinear tensor regression in high dimensions.
method Low-rank tensor decomposition and Bayesian Gaussian process estimation.
result Significantly improved convergence rate compared to naive methods.
Paper proposes efficient data-driven confidence bands for distributed nonparametric regression.
problem High computational complexity in nonparametric regression for large datasets.
method Divide and conquer approach, averaging local estimates, with data-driven uncertainty quantification.
result Frequentist L2-confidence bands and minimax-optimal high-probability bounds for averaged estimator. Transformers can efficiently approximate nonparametric regression with minimal parameters and sequences.
problem Efficiently approximating nonparametric regression functions with transformers.
method Kernel-weighted polynomial basis and gradient descent.
result Achieves minimax optimal rate of convergence with fewer parameters and sequences.
A novel deep bootstrap framework for nonparametric regression using conditional diffusion models.
problem Nonparametric regression with efficient sampling and accurate estimation.
method Conditional diffusion model for learning conditional distributions, integrating sampling and regression into a unified generative framework.
result Established optimal convergence rates in the Wasserstein distance and convergence guarantees for the bootstrap procedure.
Accelerates nonparametric estimation to near-linear time.
problem Quadratic time complexity in local polynomial regression.
method Novel use of binary indexed trees for multi-dimensional data.
result Near-linear time complexity in computation.
A new interpolation-based method for nonparametric regression.
problem Nonparametric regression challenges in computational complexity and experimental design.
method Reconstruction approach using interpolators and regularized least squares.
result Effective surrogates for complex methods with reduced computational burden.
Adversarial online nonparametric regression achieves optimal rates with locally adaptive learning.
problem Adversarial online nonparametric regression with general convex losses.
method Parameter-free learning algorithm leveraging chaining trees to compete against H{ö}lder functions, dynamically tracking and adapting to local smoothness variations.
result First computationally efficient algorithm with locally adaptive optimal rates for online regression in an adversarial setting.
Breaks down complex nonlinear dynamics into simpler components.
problem Control of nonlinear dynamical systems remains challenging.
method Inspired by hybrid switching systems, decomposes dynamics into simpler stochastic switching linear dynamical systems.
result Extracts hierarchies of Markovian and auto-regressive locally linear controllers from nonlinear experts.
Procedure groups nonparametric regression curves automatically.
problem Determining groups of nonparametric regression curves when curves are numerous.
method Automatic selection of group number through testing procedure.
result Groups of nonparametric regression curves exist in tunnel geometry.
DSARF models complex spatio-temporal data with deep switching auto-regressive factors.
problem Forecasting complex spatio-temporal data with recurring patterns.
method Deep switching auto-regressive factorization (DSARF) with stochastic variational inference.
result DSARF outperforms state-of-the-art methods in long- and short-term prediction accuracy.
Method constructs nonparametric prediction intervals with finite-sample guarantees.
problem Nonparametric instrumental variable regression with finite-sample coverage.
method Conformal inference framework applied to NPIV, combining with various estimators.
result Distribution-free, finite-sample coverage over chosen IV shifts.
Deep neural networks with ReLU activation achieve optimal nonparametric regression rates.
problem Nonparametric regression with general composition assumptions.
method Sparsely connected deep neural networks with ReLU activation function.
result Achieve minimax rates of convergence under general composition assumption.
Study shows rates for Laplacian-eigenmap methods in nonparametric regression.
problem Minimizing error in nonparametric regression using Laplacian-eigenmap.
method Adaptive and non-adaptive minimax rates using Sobolev space constraints.
result Extends minimax rates to various weighted Laplacian matrices.
Develops a sparse Bayesian model for high-dimensional data.
problem Sparse nonparametric regression with complex interactions.
method Sparse additive Gaussian process with soft interactions.
result Excellent performance in simulated and real data.
Develops abstention procedure for nonparametric regression via variance testing.
problem Prediction with selective abstention in error-critical machine learning.
method Nonparametric heteroskedastic regression via testing hypothesis on conditional variance.
result Non-asymptotic risk bounds and convergence regimes for the estimator.
Researchers develop a method to control nonlinear systems with Koopman operator regression.
problem Controlling nonlinear systems with finite action spaces.
method Koopman operator regression for dynamics estimation and model predictive control for control.
result The method yields a linear switching predictive model for control.
New similarity measure for covariate shift improves nonparametric regression rates.
problem Improving nonparametric regression under covariate shift.
method Introducing a new similarity measure based on probability ratios.
result Shows a sharper rate of convergence compared to transfer exponent.
New method screens important covariates in ultrahigh-dimensional data.
problem Handling ultrahigh-dimensional data for regression analysis.
method Favored smoothing bandwidth screening followed by iterative recovery.
result Screening method proves model selection consistency.
Neural networks improve nonparametric regression with measurement errors.
problem Nonparametric regression with measurement errors.
method Proposes a neural network design using FNN, normalizing flow, and inference network.
result Neural network approach is more flexible and superior or comparable to classical methods.
NTK neural networks are robust to adversarial attacks in nonparametric regression.
problem Adversarial robustness of neural networks in nonparametric regression.
method Gradient flow with early stopping for NTK neural networks, proving robustness in Sobolev spaces.
result NTK neural networks achieve optimal adversarial robustness rates in Sobolev spaces.
Develops nonparametric regression for non-smooth functions using fractional Laplacian.
problem Non-smooth regression functions in high dimensions.
method Fractional Laplacian eigenmaps for L2-fractional Sobolev spaces. result Upper bound on estimation error of $n^{- rac{2s}{2s+d}}$.
Localized transfer learning improves nonparametric regression performance.
problem Improving nonparametric regression performance on target tasks.
method Localized transfer learning framework that models heterogeneity and partition covariate space into cells.
result Sharp minimax rates show local transfer mitigates the curse of dimensionality.
Improves MARS for nonparametric multivariate regression with dimension reduction.
problem High number of basis functions in MARS for high-order interactions.
method Linear combinations of covariates for dimension reduction, facilitating gradient calculation and eigen-analysis for estimation.
result Asymptotic theory and numerical studies show improved performance over MARS.
Paper extends nonparametric regression bounds for dependent β-mixing samples.
problem Analyzing error in nonparametric regression with dependent data.
method Extends uniform deviation inequalities from independent to dependent β-mixing samples. result Derives generalization bounds for nonparametric regression with dependent data.
Bayesian methods estimate regression functions on submanifolds using graph Laplacian eigenbasis.
problem Estimating regression functions on unknown smooth submanifolds.
method Random geometric graph structure, Bayesian priors based on random basis expansion in graph Laplacian eigenbasis.
result Posterior contraction rates are minimax optimal for any positive smoothness index.
New approach to adaptively select bandwidths in nonparametric regression.
problem Adaptive bandwidth selection in nonparametric regression.
method Inspired by ℓ2-norms of interval projections, introduces a new bandwidth selection procedure. result Obtains non-asymptotic risk bounds for local polynomial regression methods that adapt to local Hölder exponent.
Paper develops a privacy-preserving nonparametric regression method.
problem Nonparametric regression with local differential privacy constraints.
method Privatised discretisation and Laplace noise applied to feature vectors and responses.
result Strongly universally consistent estimator for regression and classification.