Boosting improves accuracy by combining weak learners into a voting classifier.
problem Boosting's theoretical performance is sub-optimal, especially for voting classifiers.
method Proposes a randomized boosting algorithm that outputs voting classifiers with a single logarithmic dependency on sample size.
result Randomized boosting achieves a generalization error with a single logarithmic dependency on the sample size.
QR-learner estimates individual treatment effects using external data.
problem Limited power to detect individual treatment effects in randomized trials.
method Model-agnostic learner that estimates conditional average treatment effects (CATE) using external data.
result QR-learner reduces mean squared error and can recover true CATE.
Self-directed learners can minimize mistakes in online classification.
problem Minimizing mistakes in online classification with adaptive prediction order.
method Designing efficient self-directed learners for linear classification.
result Strong separation between worst-order and random-order learning for linear classification.
Study the tradeoffs of bandit feedback in multiclass classification.
problem The price of using bandit feedback in multiclass classification.
method Mistake bound model, analysis of variants, and comparison of learners and adversaries.
result The optimal mistake bound under bandit feedback is at most O(k) times higher than in full information, with a tight bound of O(k). pystacked combines machine learning models for improved predictions.
problem Improving machine learning model performance through stacking.
method Stacked generalization using Python's scikit-learn with various base learners.
result Enhanced predictive models through combining multiple machine learning algorithms.
2DSCNs improve image data analytics by extending SCN to handle spatial information.
problem Limitation of 1D SCNs in preserving spatial information of images.
method Extend SCN to 2DSCNs by stochastically configuring hidden nodes in a matrix-inputs framework.
result 2DSCNs outperform 1D SCNs in image data analytics tasks.
BoostForest combines multiple BoostTree models for improved accuracy.
problem Improving ensemble learning performance.
method BoostTree uses gradient boosting and random cut-points. BoostForest bootstraps training data and randomly samples parameters.
result BoostForest outperforms classical ensemble methods on 35 datasets.
Machine learning improves learning and memory retention by optimizing study sessions.
problem Improving learning and memory retention methods for factual material.
method Large-scale randomized controlled trial with machine learning optimization of study sessions.
result Study sessions optimized with machine learning lead to 67% longer retention and 50% higher return rate.
Tree ensembles such as random forests and boosted trees are accurate but difficult to understand, debug and deploy. In this work, we provide the inTrees (interpretable trees) framework that extracts, measures, prunes and selects rules from a tree ensemble, and calculates frequent variable interactions. An rule-based le…
Novel method to quantify aleatoric uncertainty of treatment effects from observational data.
problem Understanding randomness in treatment effects for medical treatments.
method Partial identification and Neyman-orthogonality to quantify aleatoric uncertainty.
result Developed a novel orthogonal learner (AU-learner) for quantifying aleatoric uncertainty.
The paper proposes a method to assess surrogate heterogeneity in non-randomized data.
problem Lack of methods to evaluate surrogate heterogeneity in non-randomized data.
method Proposes a framework using meta-learners to assess surrogate heterogeneity in real-world data.
result Identifies individuals for whom the surrogate is a valid replacement of the primary outcome.
A new ensemble method using random projections for kNN classification.
problem Improving kNN classification accuracy through ensemble methods.
method Random projection of bootstrap samples into lower dimensions, using extended neighbourhood rule for base learners.
result Enhanced classification accuracy compared to traditional kNN and other ensembles.
Online learners track optimal solutions with constant step-size.
problem Tracking optimal solutions in online learning settings.
method Established a link between steady-state performance and tracking performance using analogies with adaptive filters.
result Inferred tracking performance from steady-state expressions directly.
The paper predicts survival functions using random survival trees and concordance maximization.
problem Predicting conditional survival functions in right-censored data.
method The approach combines regression strategies with random survival trees and maximizes concordance.
result The proposed weighted predictor outperforms the usual survival cobra in terms of concordance.
New survival learners estimate heterogeneous treatment effects from time-to-event data.
problem Estimating HTEs from time-to-event data with censoring outcomes.
method Orthogonal survival learners with theoretical guarantees and custom weighting functions.
result Orthogonal survival learners provide robust and model-agnostic HTE estimation.
The infinitesimal jackknife (IJ) has recently been applied to the random forest to estimate its prediction variance. These theorems were verified under a traditional random forest framework which uses classification and regression trees (CART) and bootstrap resampling. However, random forests using conditional inferenc…
New model considers varying costs in learning, outperforming existing methods.
problem Learning with varying costs in machine learning models.
method Introduces ε-frugal learning that considers both known and unknown costs.
result ε-frugal learners outperform learners with known costs and random sampling.
Random Forests automatically prune a latent 'true' tree, explaining their overfitting without tuning.
problem Difficulty in building bad Random Forests and overfitting without apparent consequences.
method Bootstrap aggregation and model perturbation in Random Forests.
result Randomized ensembles implicitly perform optimal early stopping out-of-sample, explaining overfitting.
Meta-learners estimate CATE from multiple environments with partial identification.
problem Estimating CATE from observational data across multiple environments with violations of causal assumptions.
method Adapt IV literature for partial identification, propose model-agnostic meta-learners.
result Meta-learners effectively estimate CATE bounds across various experiments.
End-to-end kernel learning using generative RFFs for improved performance.
problem Improving kernel learning performance and generalization.
method Develops a generative network via RFFs to implicitly learn the kernel, followed by a linear classifier, jointly trained by ERM.
result Shows superior generalization performance over classical methods in real-world tasks.
The study proves Gaussian universality of deep random features learning.
problem Understanding the test error in deep random features learning.
method Proving Gaussian universality of test error in ridge regression and arbitrary convex losses.
result Sharp asymptotic formula for test error in ridge regression setting.
New insights into how randomization affects greedy model selection.
problem Understanding the impact of feature subsampling on greedy model selection.
method Investigated greedy forward selection with feature subsampling, proving effects on bias and variance.
result Ensembling with feature subsampling reduces both bias and variance, unlike convex base learners.
Online boosting for multiclass classification with limited feedback.
problem Online multiclass classification with bandit feedback.
method Proposed unbiased loss estimate and extended full information boosting algorithms to bandit setting.
result Asymptotic error bounds match full information counterparts, with larger sample complexity due to limited feedback.
LIBRE learns interpretable Boolean rules from data.
problem Creating interpretable classifiers in imbalanced settings.
method Ensemble of weak learners on random feature subsets, combined with a simple union.
result Efficiently balances prediction accuracy and interpretability.
The paper estimates personalized treatment effects in medical settings with competing risks.
problem Estimating treatment effectiveness for specific events in the presence of alternative event types.
method Meta-learners combining Cox regression or random survival forests for risk modeling and elastic net regression or random forests for direct CATE modeling.
result Compared meta-learners in multiple simulation settings, providing practical guidance for model selection.
Meta-learning overfits both model and base learner; meta-augmentation helps.
problem Meta-learning overfits model and base learner.
method Use an information-theoretic framework to describe and demonstrate meta-augmentation.
result Meta-augmentation produces large complementary benefits to meta-regularization.
Optimal weighted random forests improve prediction accuracy.
problem Unequal prediction performance among random forest trees.
method Proposes 1-step and 2-step optimal weighting algorithms.
result Asymptotically optimal in terms of squared loss and risk.
Treeging combines regression trees and kriging for spatial and space-time prediction.
problem Improving spatial and space-time prediction accuracy.
method Combines regression trees with kriging's covariance structures.
result Treeging outperforms kriging and random forest in various scenarios.
Framework for private, noise-tolerant, and efficient learning algorithms.
problem Private and efficient learning of large-margin halfspaces in noisy environments.
method Simple framework using differential privacy and noise tolerance conditions.
result Noise-tolerant and private PAC learners for large-margin halfspaces with sample complexity independent of dimension.
While statistical learning methods have proved powerful tools for predictive modeling, the black-box nature of the models they produce can severely limit their interpretability and the ability to conduct formal inference. However, the natural structure of ensemble learners like bagged trees and random forests has been …
B-Learner provides bounds on CATE under hidden confounding risks.
problem Estimating CATE in the presence of hidden confounding.
method Adapting bounds on average treatment effect to conditional distributional treatment effects.
result B-Learner offers valid, sharp, efficient, and quasi-oracle bounds on CATE.
RaSE ensemble framework improves sparse classification accuracy.
problem Sparse classification challenges in high-dimensional data.
method Random Subspace Ensemble (RaSE) framework with subspace selection via RIC.
result RaSE achieves low misclassification rates and accurate feature ranking.
Ensemble CNNs improve mode classification in smartphone travel surveys.
problem Classifying transportation modes from smartphone travel survey data.
method Developed an ensemble of CNN models with different architectures and hyper-parameters, combined using average voting, majority voting, optimal weights, and a Random Forest meta-learner.
result The ensemble method with Random Forest as meta-learner achieved 91.8% accuracy, surpassing other methods.
Paper proposes IRL methods for limited interaction scenarios.
problem Learning with limited teacher interaction.
method Curriculum Inverse Reinforcement Learning (CIRL) and Self-Paced Inverse Reinforcement Learning (SPIRL).
result Training strategies can accelerate learning compared to random or batch methods.
Optimal adversarial noise algorithms for multiple classifiers using game theory.
problem Designing robust attacks against multiple classifiers.
method Formulating the problem as a two-player, zero-sum game and using Multiplicative Weights Update framework with best response oracles.
result Demonstrated the effectiveness of randomization in adversarial attacks and optimal mixed strategies.
We study the problem of identifying a probability distribution for some given randomly sampled data in the limit, in the context of algorithmic learning theory as proposed recently by Vinanyi and Chater. We show that there exists a computable partial learner for the computable probability measures, while by Bienvenu, M…
Study online learning with set-valued feedback, showing differences between deterministic and randomized approaches.
problem Online learning with set-valued feedback, where labels are sets rather than single labels.
method Introduced new combinatorial dimensions (Set Littlestone and Measure Shattering) to characterize learnability.
result Characterized deterministic and randomized online learnability, and established bounds for various learning settings.
A new method estimates treatment effects across multiple studies considering differences.
problem Estimating treatment effects across multiple studies with varying conditions.
method The multi-study R-learner framework that accounts for between-study heterogeneity.
result The multi-study R-learner is more efficient and normal than existing methods in the presence of heterogeneity.
Efficient algorithms learn non-binary concepts from random counter-examples.
problem Learning non-binary concepts from random counter-examples efficiently.
method Two simple LRC algorithms: deterministic and randomized.
result Both algorithms achieve optimal average learning time of O(log|H|).
RGBM improves GBM efficiency with randomization.
problem Limited theoretical understanding of Gradient Boosting Machine.
method Randomization scheme to reduce search in weak-learners space.
result RGBM leads to substantial computational gains compared to GBM.
New strategies for identifying the best arm in bandits with decreasing variances.
problem Best arm identification in bandits with time-varying variances.
method Two policies: initial wait followed by continuous sampling, and periodic sampling with weighted average.
result Analytical guarantees and simulations show improved performance over existing methods.
The kernel embedding algorithm is an important component for adapting kernel methods to large datasets. Since the algorithm consumes a major computation cost in the testing phase, we propose a novel teacher-learner framework of learning computation-efficient kernel embeddings from specific data. In the framework, the h…
Robust learner finds subspace for MIMs with label noise.
problem Learning Multi-Index Models with label noise under Gaussian distribution.
method Iterative subspace approximation using conditional moments.
result Qualitatively optimal robust learner in SQ model.
This paper analyzes the difficulty of unsupervised domain adaptation using information theory.
problem The challenge of unsupervised domain adaptation under covariate shift.
method Formulates the problem using a distribution π in the ground-truth triples (p, q, f), defines optimal learner performance, and introduces PTLU for quantifying difficulty.
result Characterizes the optimal learner and introduces PTLU as a measure of UDA difficulty.
In this paper we address the problem of pool based active learning, and provide an algorithm, called UPAL, that works by minimizing the unbiased estimator of the risk of a hypothesis in a given hypothesis space. For the space of linear classifiers and the squared loss we show that UPAL is equivalent to an exponentially…
Improved cumulative regret for sequence prediction with limited expert advice.
problem Minimizing cumulative regret in sequence prediction with limited information.
method Convex combination of experts with limited observation, achieving constant regret.
result Strategies achieve constant regret independent of the horizon T, improving over standard bounds.
A deterministic apple tasting learner is developed, confirming a conjecture and providing tight bounds for mistake bounds.
problem Determining the learnability of hypothesis classes in binary online classification with apple tasting feedback.
method Developed a deterministic apple tasting learner and proved tight bounds for mistake bounds.
result Deterministic apple tasting is feasible and provides tight bounds for mistake bounds.
Study improves resilience against adversarial clean-label attacks in real and noisy settings.
problem Ensuring accurate predictions in the presence of adversarial clean-label samples.
method Sequential learning from a stream of i.i.d. data, allowing abstention for uncertain predictions.
result Theoretical analysis and adaptations for the agnostic setting with a clean-label adversary and noise.