A method for identifying NPWARX models with arbitrary domains using probabilistic mixture models.
problem Identifying hybrid system models with discontinuous maps.
method Probabilistic mixture model with a neural network for nonlinear partitioning and Expectation Maximization for parameter estimation.
result Demonstrated on a nonlinear piece-wise problem with discontinuous maps.
Auto-regressive models improve smoothing efficiency with exponentially tapered windows.
problem Improving time-series smoothing efficiency.
method An auto-regressive formulation for time-series smoothing.
result Auto-regressive models result in moving means with exponentially tapered windows.
PARC uses piecewise linear predictors for regression and classification.
problem Multivariate regression and classification problems.
method Alternates between ridge and softmax regression, and cluster assignment based on accuracy and separability.
result Converges to a local minimum in a finite number of steps.
New GP model estimates piecewise continuous functions.
problem Piecewise continuous regression functions in scientific and engineering applications.
method Local Gaussian process model with partitioned local data and joint estimation of boundaries.
result Superior performance over conventional GP models in estimating piecewise regression functions.
Proof of Gaussian ML estimator consistency in linear auto-regressive models.
problem Consistency of Gaussian maximum likelihood estimator in linear auto-regressive models.
method Information-theoretic proof without stability assumptions.
result Nearly optimal non-asymptotic rates for parameter recovery.
Proposes isotonic recalibration for insurance pricing to ensure auto-calibration under low signal-to-noise ratio.
problem Ensuring auto-calibration in insurance pricing models to prevent cross-financing.
method Applies isotonic recalibration to regression models to achieve auto-calibration.
result Isotonically recalibrated regression functions have low complexity under low signal-to-noise ratio.
Paper proposes algorithms to accurately identify breakpoints in piecewise regression.
problem Identifying accurate breakpoints in piecewise regression for better data fitting.
method Proposes novel greedy algorithms to minimize error and determine optimal breakpoints.
result The proposed algorithms outperform existing methods in accuracy and efficiency.
CAFLOW uses auto-regressive flows to translate images efficiently.
problem Image-to-image translation tasks.
method Transforms conditioning image into latent encodings using normalizing flows, models conditional distribution with auto-regressive distributions.
result Outperforms former conditional flow designs.
Auto-regressive models learn latent states from partially observed linear dynamical systems.
problem Understanding how auto-regressive models learn latent representations from partially observed linear dynamical systems.
method Empirical risk minimization on partially observed linear dynamical systems.
result Two-layer linear auto-regressive models learn to approximate Kalman filtering, coinciding with optimal state estimates.
New method uses DC functions for piecewise linear regression.
problem Regression with piecewise linear constraints.
method Estimates piecewise linear convex functions using a difference of convex functions.
result Method achieves close to minimax statistical risk and comparable performance to existing methods.
Proposes adaptive ridge regression for functional linear models with piecewise shapes.
problem Functional linear regression with unknown coefficient function.
method Adaptive piecewise function template with L2 penalization. result Improves predictive power and interpretability compared to standard methods.
Gini index needs auto-calibration for consistent decision-making.
problem Gini index's inconsistency in decision-making.
method Restrict Gini index to auto-calibrated regression models.
result Gini index becomes strictly consistent with auto-calibration.
VEST automates feature engineering for time series forecasting.
problem Challenges in time series forecasting with improved performance.
method VEST combines auto-regression with statistical summarization of recent past dynamics.
result VEST significantly improves forecasting performance.
The fused lasso is analyzed for high-dimensional piecewise-constant regression coefficients.
problem Estimation of high-dimensional piecewise-constant regression coefficients.
method Formulated a restricted isometry condition for the fused lasso estimator and derived estimation bounds.
result The estimation error can be dominated by either the lasso or the fused lasso rate, depending on the number of non-zero coefficients and piece-wise constant segments.
We present a detailed analysis of the class of regression decision tree algorithms which employ a regulized piecewise-linear node-splitting criterion and have regularized linear models at the leaves. From a theoretic standpoint, based on Rademacher complexity framework, we present new high-probability upper bounds for …
TraDE uses self-attention for better density estimation of tabular and image data.
problem Improving density estimation for tabular and image data.
method Self-attention-based architecture trained with a penalized maximum likelihood objective.
result TraDE produces significantly better density estimates than existing methods.
Paper presents ABGD for efficient piecewise linear regression in high dimensions.
problem Efficiently solving piecewise linear regression in high-dimensional spaces.
method Parametrizes piecewise linear functions as difference of max-affine functions, using ABGD algorithm.
result ABGD converges linearly to an ε-accurate estimate with optimal sample complexity.
In this paper, non-linear time series models are used to describe volatility in financial time series data. To describe volatility, two of the non-linear time series are combined into form TAR (Threshold Auto-Regressive Model) with AARCH (Asymmetric Auto-Regressive Conditional Heteroskedasticity) error term and its par…
Cactus improves auto-regressive decoding speed without sacrificing quality.
problem Accelerating auto-regressive decoding while maintaining output quality.
method Formalizes speculative sampling as constrained optimization and proposes Cactus for controlled divergence from the verifier distribution.
result Empirically validated effectiveness across various benchmarks.
The problem of subgroups is ubiquitous in scientific research (ex. disease heterogeneity, spatial distributions in ecology...), and piecewise regression is one way to deal with this phenomenon. Morse-Smale regression offers a way to partition the regression function based on level sets of a defined function and that fu…
This paper introduces a novel mixture model-based approach for simultaneous clustering and optimal segmentation of functional data which are curves presenting regime changes. The proposed model consists in a finite mixture of piecewise polynomial regression models. Each piecewise polynomial regression model is associat…
Paper reduces vocabulary losslessly for language model cooperation.
problem Language models struggle to cooperate with different tokenizations.
method Established a theoretical framework for lossless vocabulary reduction.
result Efficiently converts models with different tokenizations to cooperate with maximal common vocabulary.
The paper proposes a new auto-regressive model for multivariate distributional time series.
problem Statistical analysis of multivariate time series of probability measures.
method Wasserstein space, auto-regressive model, iterated random function systems.
result Consistent estimator for auto-regressive coefficients with sparse structure.
Efficiently recovers piecewise linear functions from noisy samples.
problem Recovering a piecewise linear function from noisy samples with unknown segmentation.
method Iterative merging approach for multidimensional segmented regression.
result First sample and computationally efficient algorithm in any fixed dimension.
AutoML serves as the bridge between varying levels of expertise when designing machine learning systems and expedites the data science process. A wide range of techniques is taken to address this, however there does not exist an objective comparison of these techniques. We present a benchmark of current open source Aut…
DSARF models complex spatio-temporal data with deep switching auto-regressive factors.
problem Forecasting complex spatio-temporal data with recurring patterns.
method Deep switching auto-regressive factorization (DSARF) with stochastic variational inference.
result DSARF outperforms state-of-the-art methods in long- and short-term prediction accuracy.
VAR-GPs solve continual learning by updating posteriors sequentially.
problem Catastrophic forgetting in sequential learning tasks.
method Sparse inducing point approximations and auto-regressive variational distribution.
result VAR-GPs prevent catastrophic forgetting and outperform baselines.
XGBoost is often presented as the algorithm that wins every ML competition. Surprisingly, this is true even though predictions are piecewise constant. This might be justified in high dimensional input spaces, but when the number of features is low, a piecewise linear model is likely to perform better. XGBoost was exten…
In this paper, a multi-layer architecture (in a hierarchical fashion) by stacking various Kernel Ridge Regression (KRR) based Auto-Encoder for one-class classification is proposed and is referred as MKOC. MKOC has many layers of Auto-Encoders to project the input features into new feature space and the last layer was r…
We consider the generic regularized optimization problem β^(λ)=argminβL(y,Xβ)+λJ(β). Efron, Hastie, Johnstone and Tibshirani [Ann. Statist. 32 (2004) 407--499] have shown that for the LASSO--that is, if L is squared error loss and J(β)=∥β∥1 is the ℓ1 norm of β--the opti…
Boosting trees can test necessary conditions for regression model calibration.
problem Testing calibration and auto-calibration in regression models.
method Using boosting trees to test calibration and auto-calibration.
result Boosting trees prove to be very powerful in testing calibration and auto-calibration in large insurance datasets.
BMBO-DARN optimizes expensive functions with varying fidelities.
problem Optimizing expensive, multi-fidelity functions efficiently.
method Batch Multi-fidelity Bayesian Optimization with Deep Auto-Regressive Networks.
result BMBO-DARN improves surrogate learning and optimization performance.
Improved text generation with constraints using discrete auto-regressive biasing.
problem Balancing fluency and constraint satisfaction in LLM outputs.
method Discrete Auto-regressive Biasing, leveraging gradients in discrete text space.
result Significantly improved constraint satisfaction with comparable fluency.
We show that a generative random field model, which we call generative ConvNet, can be derived from the commonly used discriminative ConvNet, by assuming a ConvNet for multi-category classification and assuming one of the categories is a base category generated by a reference distribution. If we further assume that the…
Efficiently finds sparse solutions to max-plus equations for convex regression.
problem Finding sparse solutions to max-plus equations for convex multivariate regression.
method Polynomial-time algorithm for sparse approximate solutions.
result Optimal piecewise-linear fitting with minimum number of regions.
BART and MOTR-BART improve tree-based predictions with local linear models.
problem Non-linearity and high-order interactions in data.
method Bayesian Additive Regression Trees (BART) and Model Trees BART (MOTR-BART) using piecewise linear functions.
result MOTR-BART achieves equal or better performance with fewer trees than BART.
A new method estimates expectations from subtractive mixture models without sampling.
problem Estimating expectations from multimodal distributions using SMMs.
method Difference representation of SMMs to create unbiased IS estimator (ΔextEx). result Demonstrates that ΔextEx can achieve comparable estimation quality to auto-regressive sampling but is faster. Proposed by Donoho (1997), Dyadic CART is a nonparametric regression method which computes a globally optimal dyadic decision tree and fits piecewise constant functions in two dimensions. In this article we define and study Dyadic CART and a closely related estimator, namely Optimal Regression Tree (ORT), in the contex…
New algorithm reduces dynamic regret for noisy gradient feedback with piecewise polynomial comparators.
problem Online estimation of piecewise polynomial trends with noisy feedback.
method Introduces variational constraint for piecewise polynomial comparators, designs adaptive algorithm.
result Achieves nearly optimal dynamic regret of $ ilde{O}(n^{ rac{1}{2k+3}}C_n^{ rac{2}{2k+3}})$.
MDMs train to decode tokens in a random order, which affects performance; we show they can be optimized for a favorable order.
problem Performance of MDMs is affected by the random order in which tokens are decoded.
method We show that MDMs can be optimized for a favorable decoding order by equipping their continuous-time variational objective with multivariate noise schedules.
result MDMs can be decomposed into a weighted auto-regressive losses over orders, making them auto-regressive models with learnable orders.
A simple method reduces bias in LLM auto-evaluators by controlling output length.
problem Bias in LLM auto-annotators, particularly length bias.
method Simple regression analysis to control for length difference and other mediators.
result Improved robustness and increased correlation with human preferences.
New estimator achieves minimax optimal risk in transfer learning.
problem Nonparametric regression with transfer learning.
method Confidence thresholding estimator and data-driven adaptive algorithm.
result Adaptive algorithm achieves minimax risk up to a logarithmic factor.
Auto-regressive diffusion models improve capturing conditional dependence in data.
problem Vanilla diffusion models struggle to capture important, high-level relationships in real-world data.
method Developed auto-regressive diffusion models to better capture conditional dependence structures.
result AR diffusion models produce samples with a reduced gap in approximating the data conditional distribution.
The paper deals with regression problems, in which the nonsmooth target is assumed to switch between different operating modes. Specifically, piecewise smooth (PWS) regression considers target functions switching deterministically via a partition of the input space, while switching regression considers arbitrary switch…
Paper introduces ℓ-DER for regression tasks using morphological operators and convex-concave procedure.
problem Developing a universal approximator for regression tasks.
method Introduces ℓ-DER model, trains it using a convex-concave procedure (CCP) to minimize least-squares. result Outperforms other hybrid morphological models and state-of-the-art approaches.
A new method improves text generation quality and diversity.
problem Exposure bias in Maximum Likelihood Estimation for text generation.
method ψ-MLE, a new training scheme based on density ratio estimation.
result ψ-MLE outperforms Maximum Likelihood Estimation and other models in text generation quality and diversity.
Improves naturalness in TTS samples using quantized VAE and auto-regressive prosody.
problem Discontinuous and unnatural speech from standard VAE priors.
method Discretized latent features using vector quantization (VQ), and separately trained autoregressive (AR) prior model.
result Significantly improves naturalness in random sample generation.
In this paper, we investigate adaptive nonlinear regression and introduce tree based piecewise linear regression algorithms that are highly efficient and provide significantly improved performance with guaranteed upper bounds in an individual sequence manner. We use a tree notion in order to partition the space of regr…