We consider the problem of learning a forest of nonlinear decision rules with general loss functions. The standard methods employ boosted decision trees such as Adaboost for exponential loss and Friedman's gradient boosting for general loss. In contrast to these traditional boosting algorithms that treat a tree learner…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
169,051 papers · 148 categories
Trend · papers per month
48 results for “greedy boosting”
Proposes ContSup to boost local learning by supplying context between isolated modules.
problem Local learning's performance degrades with more isolated modules.
method Theoretical analysis and ContSup scheme to supply context between modules.
result Significant performance improvement with minimal overhead.
Slow-Growing Treesstat.ML
A single slow-growing tree matches Random Forest's performance.
problem Matching Random Forest's performance with a single tree.
method SGT uses a learning rate to tame CART's greedy algorithm, improving on greedy ML algorithms.
result SGT and tree ensembles like Booging, BT, and RF improve performance.
GrowNet uses shallow neural networks for gradient boosting, outperforming existing methods.
problem Improving gradient boosting performance through shallow neural networks.
method Unified gradient boosting framework with shallow neural networks as weak learners, incorporating corrective steps.
result GrowNet outperformed state-of-the-art boosting methods in classification, regression, and learning to rank tasks.
In this paper, we derive a novel probabilistic model of boosting as a Product of Experts. We re-derive the boosting algorithm as a greedy incremental model selection procedure which ensures that addition of new experts to the ensemble does not decrease the likelihood of the data. These learning rules lead to a generic …
A new method for automatic gradient tree boosting using information theory.
problem Automatic selection of tree complexity and number in gradient boosting.
method Optimism of greedy leaf splitting procedure modeled as a Cox-Ingersoll-Ross process, leading to an information criterion for model selection.
result The method achieves significant speedups (10-1400) compared to xgboost without sacrificing predictive power.
RFRBoost uses random features to boost deep residual neural networks, improving performance and computational efficiency.
problem Improving performance of deep residual neural networks (RFNNs) while preserving convex optimization benefits.
method Random Feature Representation Boosting (RFRBoost) using boosting theory and random features at each layer.
result RFRBoost significantly outperforms RFNNs and end-to-end trained MLP ResNets in small- to medium-scale tabular datasets.
Boosted embeddings improve time series forecasting models.
problem Improving time series forecasting accuracy.
method Gradient boosting with DNN weak learners and incremental weight updates.
result DeepGB model outperforms state-of-the-art models.
Greedy convex combination of models outperforms baselines.
problem Overfitting and underfitting in convex combinations of models.
method Greedy learning of convex combinations with early stopping.
result Greedy approach is competitive or better than boosting and random forests.
Boosting is one of the most significant developments in machine learning. This paper studies the rate of convergence of Boosting, which is tailored for regression, in a high-dimensional setting. Moreover, we introduce so-called \textquotedblleft post-Boosting\textquotedblright. This is a post-selection estimator w…
Boosted decision trees enjoy popularity in a variety of applications; however, for large-scale datasets, the cost of training a decision tree in each round can be prohibitively expensive. Inspired by ideas from the multi-arm bandit literature, we develop a highly efficient algorithm for computing exact greedy-optimal d…
Decentralized learning of personalized models and collaboration graphs without central coordination.
problem Training personalized models and collaboration graphs in a decentralized manner without a central coordinator.
method Alternates between training models given the graph and updating the graph given the models, using peer-to-peer exchanges.
result Communication-efficient approach that avoids exchanging personal data, with benefits demonstrated on synthetic and real datasets.
Proposes an algorithm for semi-supervised learning with budget constraints.
problem Learning classifiers within test-time budget constraints with limited labeled data.
method Leverages unlabeled data through Laplace smoothing and gradient boosted regression trees.
result First algorithm for semi-supervised budgeted learning.
Paper proposes a boosting method with fast learning rates and early stopping.
problem Missing theoretical guarantees for boosting methods in binary classification.
method Fully-corrective gradient boosting with squared hinge loss and ADMM algorithm.
result Derives fast learning rates of and .
Converts GBDT trees to neural networks for online updates.
problem Performance loss in converting GBDT trees to neural networks.
method Converts existing GBDT implementations to neural network architectures, allowing online updates of decision splits.
result Learning bounds for neural network architecture with updated splits.
Approximating a probability density in a tractable manner is a central task in Bayesian statistics. Variational Inference (VI) is a popular technique that achieves tractability by choosing a relatively simple variational family. Borrowing ideas from the classic boosting framework, recent approaches attempt to \emph{boo…
We discuss the relative merits of optimistic and randomized approaches to exploration in reinforcement learning. Optimistic approaches presented in the literature apply an optimistic boost to the value estimate at each state-action pair and select actions that are greedy with respect to the resulting optimistic value f…
Regularization-induced exploration improves contextual bandit performance.
problem Complex reward models in real-world contextual bandits are hard to explore effectively.
method Regularization-induced exploration using stochasticity in cross-validation.
result Regularization-induced exploration leads to reliable exploration in large-scale business environments.
PILOT is a fast algorithm for linear model trees that outperforms existing methods.
problem Fitting linear model trees to large datasets efficiently and accurately.
method Greedy training with boosting and model selection rule.
result PILOT outperforms standard decision trees and other linear model trees on various datasets.
RGBM improves GBM efficiency with randomization.
problem Limited theoretical understanding of Gradient Boosting Machine.
method Randomization scheme to reduce search in weak-learners space.
result RGBM leads to substantial computational gains compared to GBM.
LGB+ improves macroeconomic forecasting by combining linear and tree models.
problem Efficiency in small samples for forecasting with mixed linear and nonlinear dynamics.
method LGB+ is a boosting procedure that evaluates both tree and linear candidates at each step, advancing only the winner. It decomposes forecasts into linear and nonlinear contributions.
result LGB+ delivers strong gains for targets with pronounced autoregressive dynamics or mixed signals.
A method to prevent VAE forgetting across tasks.
problem Catastrophic forgetting in VAEs when learning sequentially.
method Learn an end-to-end approximation of the aggregated posterior as a prior for each task, encouraging diversity among components.
result The method avoids catastrophic forgetting in various datasets.
Greedy algorithm achieves sublinear regret for various distributions.
problem Efficient performance of greedy algorithms in linear contextual bandit problems.
method Introduced Local Anti-Concentration (LAC) condition to ensure sublinear regret.
result Greedy algorithm achieves cumulative expected regret.
Greedy algorithm optimizes consumption habits with power utility.
problem Optimizing lifetime consumption with habit formation under power utility.
method Developed a greedy algorithm using Monte Carlo simulation.
result Greedy solution is a good approximation to the optimal solution.
Greedy policy achieves good results for adaptive submodular problems.
problem Sequential decision making with adaptive stochastic optimization.
method Adaptive submodularity ratio to analyze greedy policy performance.
result Greedy policy achieves approximation guarantees for a broader class of problems.
Greedy AutoAugmentcs.LG
Greedy AutoAugment improves accuracy with less computation.
problem Finding effective data augmentation policies to cover the search space.
method Greedy approach to reduce the number of trials from exponential to linear growth.
result Greedy AutoAugment increases accuracy by 360 times with fewer resources.
We present an information-theoretic framework for sequential adaptive compressed sensing, Info-Greedy Sensing, where measurements are chosen to maximize the extracted information conditioned on the previous measurements. We show that the widely used bisection approach is Info-Greedy for a family of -sparse signals b…
New Classes of the Greedy-Applicable Arm Feature Distributions in the Sparse Linear Bandit Problemcs.LG
New distributions allow greedy arm selection in sparse bandit problems.
problem Sparse contextual bandit problem with sparse parameters and feature distributions.
method Introduced new distribution classes and demonstrated that mixtures of these distributions are also greedy-applicable.
result Greedy algorithm applicable to a wider range of arm feature distributions, including those with origin-asymmetric support.
Simpler ε-greedy with longer action durations improves exploration.
problem Limited exploration capability of ε-greedy in complex domains.
method Temporally extended ε-greedy with repeated actions for random durations.
result Temporally extended ε-greedy outperforms sophisticated methods on various domains.
This paper is a follow up to the previous author's paper on convex optimization. In that paper we began the process of adjusting greedy-type algorithms from nonlinear approximation for finding sparse solutions of convex optimization problems. We modified there three the most popular in nonlinear approximation in Banach…
Introduces greedy feature selection for classifier-dependent feature ranking.
problem Feature selection for classification tasks.
method Greedy feature selection, identifying the most important feature at each step based on the selected classifier.
result Theoretical and numerical benefits of greedy feature selection.
Greedy approach constructs efficient neural networks with ReLU.
problem Constructing efficient neural networks with limited resources.
method Greedy algorithm with integral representation of the network using ridgelet transform.
result Proposed approach constructs efficient shallow neural networks.
Greedy coordinate descent achieves linear convergence for non-smooth composite problems.
problem Optimization of non-smooth composite problems.
method Greedy selection of subgradients for optimization.
result Linear convergence rates independent of problem dimension .
Greedy selection works well in a toy model of independent increments.
problem Iterative selection of maximum-value processes from i.i.d. stochastic processes.
method Fixed greedy selection at each stage.
result Optimal strategy is greedy selection under independent increments.
Greedy policy maximizes information in unknown linear systems.
problem Exploration in unknown linear dynamical systems.
method Online greedy policy maximizing information.
result Competitive performance compared to gradient-based methods.
We consider the high-dimensional sparse linear regression problem of accurately estimating a sparse vector using a small number of linear measurements that are contaminated by noise. It is well known that the standard cadre of computationally tractable sparse regression algorithms---such as the Lasso, Orthogonal Matchi…
This paper proposes and evaluates the k-greedy equivalence search algorithm (KES) for learning Bayesian networks (BNs) from complete data. The main characteristic of KES is that it allows a trade-off between greediness and randomness, thus exploring different good local optima. When greediness is set at maximum, KES co…
Multiple-step lookahead policies have demonstrated high empirical competence in Reinforcement Learning, via the use of Monte Carlo Tree Search or Model Predictive Control. In a recent work \cite{efroni2018beyond}, multiple-step greedy policies and their use in vanilla Policy Iteration algorithms were proposed and analy…
Paper analyzes Greedy-GQ for reinforcement learning with Markovian noise.
problem Analyzing Greedy-GQ for reinforcement learning with Markovian noise.
method Develops finite-sample analysis for Greedy-GQ with linear function approximation under Markovian noise.
result Provides theoretical justification for choosing stepsizes for faster convergence.
New algorithm speeds up learning of graphical models.
problem Learning graphical models with sparse structure efficiently.
method Vertex-greedy score-based algorithm for learning DAGs.
result Polynomial runtime for learning DAG models.
Paper improves greedy algorithm for non-submodular matroid constraints.
problem Maximizing non-submodular functions subject to matroid constraints.
method Developed and analyzed a greedy algorithm with approximation guarantees.
result Greedy algorithm offers approximation factors for matroid constraints.
Improved greedy 2-coordinate updates for optimization problems with constraints.
problem Minimizing smooth functions subject to constraints.
method Exploiting a connection to steepest descent in the 1-norm, we give faster convergence rates and efficient computation.
result Greedy selection converges faster than random selection and can be computed in time.
The contextual bandit literature has traditionally focused on algorithms that address the exploration-exploitation tradeoff. In particular, greedy algorithms that exploit current estimates without any exploration may be sub-optimal in general. However, exploration-free greedy algorithms are desirable in practical setti…
Greedy algorithm nearly outperforms exploration in contextual bandits.
problem Balancing exploration and exploitation in online learning.
method Smoothed analysis of the greedy algorithm in linear contextual bandits.
result Greedy algorithm nearly matches Bayesian regret rate under diversity conditions, with regret at most .
New greedy algorithms improve Bayesian optimisation performance.
problem Optimizing continuous functions with exploration vs exploitation trade-offs.
method Introduced two novel ε-greedy acquisition functions and compared them with conventional methods.
result ε-greedy algorithms generally outperform conventional methods, especially in higher dimensions.
Greedy training of recursive partitioning estimators faces a computational barrier when the true function doesn't satisfy a specific property.
problem Computational inefficiency of greedy training for recursive partitioning estimators.
method Analysis of greedy training for sparse regression functions over binary features.
result Greedy training requires exponential samples when the true function doesn't satisfy a specific property (MSP), but only logarithmic samples when it does.
In this paper we consider the task of estimating the non-zero pattern of the sparse inverse covariance matrix of a zero-mean Gaussian random vector from a set of iid samples. Note that this is also equivalent to recovering the underlying graph structure of a sparse Gaussian Markov Random Field (GMRF). We present two no…
Greedy method adds neurons one by one for better function approximation.
problem Function approximation in neural networks.
method Growing deep neural network by adding one neuron at a time with non-linear activation.
result Accurate approximants for model problems in function approximation.