Optimizes piecewise local-linear approximations for global model understanding.
problem Global model behavior interpretation for black-box models.
method Dynamic programming framework for piecewise local-linear approximations with fidelity guarantees.
result Polynomial time algorithm for optimal clustering.
New GP model estimates piecewise continuous functions.
problem Piecewise continuous regression functions in scientific and engineering applications.
method Local Gaussian process model with partitioned local data and joint estimation of boundaries.
result Superior performance over conventional GP models in estimating piecewise regression functions.
BART and MOTR-BART improve tree-based predictions with local linear models.
problem Non-linearity and high-order interactions in data.
method Bayesian Additive Regression Trees (BART) and Model Trees BART (MOTR-BART) using piecewise linear functions.
result MOTR-BART achieves equal or better performance with fewer trees than BART.
Proposes a method to make deep networks' derivatives more stable.
problem Making deep networks' derivatives more stable over larger regions.
method A learning problem to encourage stable derivatives, with an inference step and optimization step.
result Proposes a novel relaxation to scale the algorithm to realistic models.
SyMPLER improves time series forecasting in nonstationary environments with explainable models.
problem Nonstationary time series forecasting with limited interpretability.
method Dynamic piecewise-linear approximations based on Statistical Learning Theory generalization bounds.
result SyMPLER achieves comparable performance to black-box and explainable models while maintaining interpretability.
XGBoost is often presented as the algorithm that wins every ML competition. Surprisingly, this is true even though predictions are piecewise constant. This might be justified in high dimensional input spaces, but when the number of features is low, a piecewise linear model is likely to perform better. XGBoost was exten…
Deep Jump Gaussian Processes model high-dimensional piecewise functions.
problem Modeling high-dimensional piecewise continuous functions with limited accuracy.
method Integrates region-specific locally linear projections with Jump Gaussian Processes (JGP) to capture local low-dimensional subspace structures.
result DJGP achieves superior predictive accuracy and more reliable uncertainty quantification compared to existing methods.
A new tree-based model for multivariate responses interprets piecewise linear regimes.
problem Recovering piecewise multivariate linear regimes in complex data.
method Twoblock clustering trees with coskewness-based dimension reduction.
result Recovery of piecewise linear regimes in data.
A method for identifying NPWARX models with arbitrary domains using probabilistic mixture models.
problem Identifying hybrid system models with discontinuous maps.
method Probabilistic mixture model with a neural network for nonlinear partitioning and Expectation Maximization for parameter estimation.
result Demonstrated on a nonlinear piece-wise problem with discontinuous maps.
New classifier combines locally linear kernels for fast and accurate non-linear classification.
problem Developing a fast and accurate non-linear classifier.
method Combines locally linear classifiers using a ℓ1 Multiple Kernel Learning (MKL) problem with scalable MKL training for streaming kernels. result The resulting classifier achieves high accuracy with fast inference time.
Paper analyzes nLasso for localized linear regression in networked data.
problem Learning sparse linear models from networked data.
method Extends nLasso to network models using convex optimization.
result Sufficient condition for nLasso to accurately learn localized linear regression.
We introduce the Locally Linear Latent Variable Model (LL-LVM), a probabilistic model for non-linear manifold discovery that describes a joint distribution over observations, their manifold coordinates and locally linear maps conditioned on a set of neighbourhood relationships. The model allows straightforward variatio…
This paper discusses topological and locally linear actions of finite groups on S4. Local linearity of the orientation preserving actions on S4 forces the group to be a subgroup of SO(5). On the other hand, orientation reversing topological actions of "exotic" groups G (i.e. G⊂O(5)) on S4 are …
Paper introduces LLISE for image structure learning using SSIM.
problem Image quality assessment using MSE or ℓ2 norm is not promising. method Locally Linear Image Structural Embedding (LLISE) using SSIM.
result LLISE captures image structure features and discriminates distortions.
This paper simplifies deep ReLU networks into local linear models for better interpretability.
problem Limited transparency and interpretability of deep neural networks, especially ReLU networks.
method Local linear representation and equivalent set of local linear models (LLMs).
result Simplified deep ReLU networks for better interpretability and diagnostics.
Study proposes Local Linear Encoding for better feature discretization.
problem Improving feature discretization for numeric data.
method Theoretical analysis and Local Linear Encoding (LLE) method.
result LLE outperforms conventional methods with fewer parameters.
Local Linear Forests improve random forests for smooth signals and causal inference.
problem Random forests struggle with smooth signals and poor predictive performance in smooth effects.
method Pairing forest kernel with local linear regression adjustment.
result Improves asymptotic rates of convergence and accuracy on real and simulated data.
A new method simplifies HLLE for better robustness.
problem Improving robustness of Hessian locally linear embedding.
method Replacing Hessian with arbitrary weights and modifying manifold dimension.
result Achieved a new LLE-type method called tangential LLE.
LLE produces unwanted results without regularization, which can be prevented with regularization.
problem LLE's inherent unwanted results without regularization.
method Mathematical proof and numerical examples of regularization effectiveness.
result Regularization prevents unwanted results in LLE.
Locally Linear Embedding improves psychiatric diagnosis accuracy from fMRI data.
problem Improving psychiatric diagnosis accuracy from fMRI data.
method Locally Linear Embedding of BOLD time-series data to optimise feature selection using LOOCV.
result Embedded fMRI gave highly diagnostic performances (> 80%) on eleven publicly-available datasets.
This paper analyzes a simplified strategy for nonlinear control using local linear models and iLQR updates.
problem Nonlinear policy optimization in control systems.
method Iterative estimation of local linear models and iLQR-like policy updates.
result Demonstrates polynomial sample complexity and overcomes exponential problem horizon dependence.
The paper develops predictors for functional data on manifolds.
problem Functional data prediction on time-varying manifolds.
method Least-squares local linear Fréchet curve predictor and weighted Fréchet mean approach.
result Asymptotical optimality of the proposed predictors.
In this paper, we propose and study random maxout features, which are constructed by first projecting the input data onto sets of randomly generated vectors with Gaussian elements, and then outputing the maximum projection value for each set. We show that the resulting random feature map, when used in conjunction with …
New insights into continual learning for deep models, showing convergence issues but local linear solutions.
problem Challenges in continual learning for homogeneous deep models.
method Sequential projections onto task margin sets, leveraging nonconvex projection theory.
result Local linear convergence under certain conditions for homogeneous deep networks.
Survey of Locally Linear Embedding and its variants.
problem Representing high-dimensional data in a lower-dimensional space while preserving local structure.
method Explains various LLE and variant methods, including kernel LLE, inverse LLE, feature fusion, out-of-sample embedding, incremental LLE, landmark LLE, supervised LLE, robust LLE, fusion with other methods, and weighted LLE.
result Comprehensive overview of LLE and its variants.
Cyclic coordinate descent identifies models in finite time and converges linearly.
problem Model identification in composite nonsmooth optimization problems.
method Cyclic coordinate descent for a wide class of functions.
result Explicit local linear convergence rates for coordinate descent.
This paper improves Bayesian neural nets by using local linearization.
problem Underfitting in Bayesian neural networks.
method Local linearization of Bayesian neural networks to create a generalized linear model (GLM) for predictions.
result The GLM predictive resolves common underfitting problems of the Laplace approximation.
S.Bauer and M.Furuta defined a stable cohomotopy refinement of the Seiberg-Witten invariants. In this paper, we prove a vanishing theorem of Bauer-Furuta invariants for 4-manifolds with smooth Z/2-actions. As an application, we give a constraint on smooth Z/2-actions on homotopy K3#K3, and construct a nonsmoothable loc…
We investigate finite-time decoupled convergence in nonlinear two-time-scale stochastic approximation.
problem Achieving decoupled convergence in nonlinear two-time-scale stochastic approximation.
method Nested local linearity assumption, suitable step size selection, convergence analysis of matrix cross term, fourth-order moment convergence rates.
result Finite-time decoupled convergence rates can be achieved in nonlinear two-time-scale stochastic approximation with proper step size selection.
Policy gradient converges linearly with Hadamard parameterization in tabular settings.
problem Convergence of policy gradient methods under Hadamard parameterization.
method Studied convergence rate and established linear convergence after k0 iterations. result Algorithm converges linearly with rate $O( rac{1}{k})$ and faster locally after k0. This paper develops a new method to model treatment effects that are heterogeneous across different quantiles.
problem Modeling treatment effects that vary across different quantiles of the outcome distribution.
method The paper combines quantile classification with local polynomial estimation to build a decision tree and forest.
result The proposed QLPRT and QLPRF methods provide a new way to estimate and infer heterogeneous treatment effects.
Local Linear embedding (LLE) is a popular dimension reduction method. In this paper, we first show LLE with nonnegative constraint is equivalent to the widely used Laplacian embedding. We further propose to iterate the two steps in LLE repeatedly to improve the results. Thirdly, we relax the kNN constraint of LLE and p…
We show LLMs can be locally linear, enabling better control of activations.
problem Suboptimal control of LLM activations during generation.
method Model LLM inference as a linear dynamical system, compute feedback controllers using Jacobians, and adapt classical control theory.
result Robust, fine-grained control of LLM activations across models and tasks.
Following the programme set out in Part I of this work, we develop a conceptual higher order differential calculus. The '' local linear algebra '' defined in Part I is generalized by '' higher order local linear algebra ''. The underlying combinatorial object of such higher algebra is the natural n-dimensional hyper-cu…
In this paper we implement a Local Linear Regression Ensemble Committee (LOLREC) to predict 1-day-ahead returns of 453 assets form the S&P500. The estimates and the historical returns of the committees are used to compute the weights of the portfolio from the 453 stock. The proposed method outperforms benchmark portfol…
We introduce Embed to Control (E2C), a method for model learning and control of non-linear dynamical systems from raw pixel images. E2C consists of a deep generative model, belonging to the family of variational autoencoders, that learns to generate image trajectories from a latent space in which the dynamics is constr…
Covariance in physics and CNNs share similarities, with simple assumptions uniquely determining convolution forms.
problem Understanding the similarities between physics and CNNs using covariance.
method Examined similarities and differences, and demonstrated that simple assumptions lead to unique convolution forms.
result Simple assumptions of covariance, locality, linearity, and weight sharing uniquely determine convolution forms.
Generative LLE modifies LLE to generate stochastic embeddings.
problem Nonlinear dimensionality reduction and manifold learning.
method Generative LLE modifies LLE by using stochastic linear reconstruction.
result Generative LLE can generate various LLE embeddings stochastically.
We use partial actions, as formalized by Exel, to construct various commensurating actions. We use this in the context of groups piecewise preserving a geometric structure, and we interpret the transfixing property of these commensurating actions as the existence of a model for which the group acts preserving the geome…
Study shows DNNs can recover functions with fewer samples than model parameters at overparameterization.
problem Determining reliable function recovery in overparameterized deep neural networks.
method Introducing 'local linear recovery' (LLR) and proving upper bounds on sample sizes for recovery.
result Upper bounds on optimistic sample sizes for function recovery in overparameterized DNNs are achieved.
NGSLL combines DNN accuracy with linear model interpretability.
problem Combining high accuracy of DNNs with interpretability of linear models.
method Neural generators of sparse local linear models (NGSLL) using DNNs to approximate non-linear functions.
result Effective in real-world datasets, achieving high predictive performance and interpretability.
Investigates stability of piecewise flat Ricci flow using analysis and simulations.
problem Stability of piecewise flat Ricci flow.
method Linear stability analysis and numerical simulations.
result Adaptations avoided numerical instability and led to convergence to smooth solutions.
This work connects LLE, factor analysis, and probabilistic PCA through a stochastic perspective.
problem Exploring the theoretical connection between LLE, factor analysis, and probabilistic PCA.
method Solving the stochastic linear reconstruction of LLE using expectation maximization.
result LLE, factor analysis, and probabilistic PCA are shown to be connected through a stochastic perspective.
Neural networks can represent complex piecewise functions efficiently.
problem Representing continuous piecewise affine functions with neural networks.
method Two hidden layers with ReLU activation, O(p) neurons for p pieces. result CPA functions can be represented by a neural network with linear size.
Folded concave penalization methods have been shown to enjoy the strong oracle property for high-dimensional sparse estimation. However, a folded concave penalization problem usually has multiple local solutions and the oracle property is established only for one of the unknown local solutions. A challenging fundamenta…
Theorem proves integrability for piecewise-smooth distributions.
problem Integrability of piecewise-smooth distributions.
method Generalizations of Frobenius integrability theorem.
result Sufficient criteria for complete integrability with bi-Lipschitz coordinates.
Simple stochastic Newton and cubic Newton methods with fast convergence.
problem Minimizing large numbers of smooth and strongly convex functions.
method Stochastic Newton and cubic Newton methods with simple local linear-quadratic rates.
result Local linear-quadratic convergence results with fast adaptation to problem's curvature.
Piecewise flat approximations for curvature in Euclidean and non-Euclidean spaces.
problem Approximating local extrinsic curvature on discrete manifolds.
method Constructing discrete curvature forms on piecewise flat manifolds, using weighted sums of hinge angles.
result Converges to smooth curvature values as mesh refinement occurs, favorably comparing with other discrete approaches.