EntroPath learns manifold geometry from diffusion paths.
problem Learning geodesic geometry from data graphs with spurious shortcuts.
method Maximum Entropy Path Ensemble Embedding (MERW) with k-step diffusion paths.
result EntroPath converges to squared geodesic distance in the short-time limit.
Unified representation for tree ensembles indexed by nodes
problem Unifying geometric object for tree ensembles indexed by nodes
method KPP indexes feature map by nodes, weighted by path metric
result Unified non-diagonal Gram for prediction, additive attribution, robust radius, and risk bounds
We study the problem of online path learning with non-additive gains, which is a central problem appearing in several applications, including ensemble structured prediction. We present new online algorithms for path learning with non-additive count-based gains for the three settings of full information, semi-bandit and…
Study shows how feature weighting affects neural network regularization.
problem Understanding how feature weighting influences neural network regularization.
method Derived equivalence paths connecting different weighting matrices and ridge regularization levels.
result Ridge estimators trained on weighted features are asymptotically equivalent when evaluated against test vectors.
New algorithms sample from complex path measures using neural networks.
problem Sampling from posterior path measures under a general prior process.
method Combines controlled equilibrium dynamics and optimization in infinite-dimensional probability space.
result The algorithms can be integrated with neural networks for learning target trajectory ensembles.
Ensemble method for fast portfolio valuation and risk management.
problem Dynamic portfolio valuation and risk management from cash flow data.
method Regression trees for dynamic value process learning.
result Fast and accurate estimator with closed-form solution.
The estimation of advantage is crucial for a number of reinforcement learning algorithms, as it directly influences the choices of future paths. In this work, we propose a family of estimates based on the order statistics over the path ensemble, which allows one to flexibly drive the learning process, towards or agains…
We describe parallel Markov chain Monte Carlo methods that propagate a collective ensemble of paths, with local covariance information calculated from neighboring replicas. The use of collective dynamics eliminates multiplicative noise and stabilizes the dynamics thus providing a practical approach to difficult anisotr…
EnKF's update is shown to be similar to Matheron's method in Gaussian process regression.
problem Data assimilation in high-dimensional systems.
method Empirical Matheron update applied to EnKF.
result Ensemble Kalman Filter's update is equivalent to an empirical Matheron update.
Several real problems ranging from text classification to computational biology are characterized by hierarchical multi-label classification tasks. Most of the methods presented in literature focused on tree-structured taxonomies, but only few on taxonomies structured according to a Directed Acyclic Graph (DAG). In thi…
Paper proposes an ensemble-based AIS for multimodal sampling.
problem Sampling from multimodal distributions is challenging.
method Combines AIS with population-based Monte Carlo methods.
result Improves efficiency through ensemble interaction.
We introduce a new deep convolutional neural network, CrescendoNet, by stacking simple building blocks without residual connections. Each Crescendo block contains independent convolution paths with increased depths. The numbers of convolution layers and parameters are only increased linearly in Crescendo blocks. In exp…
Paper discovers simplicial complexes connecting trained models for improved ensembling.
problem Improving robustness and accuracy of deep learning ensembles.
method Identifies mode-connecting simplicial complexes on loss surfaces.
result Efficiently builds simplicial complexes for ensembling, outperforming independent ensembles.
High-capacity neural network ensembles often benefit more from high-capacity models than from increased diversity.
problem The performance of high-capacity neural network ensembles is often harmed by interventions that promote predictive diversity.
method A large-scale study of nearly 600 neural network classification ensembles, examining various interventions and architectures.
result Discouraging predictive diversity can be benign in large-network ensembles, and higher-capacity models often yield better performance than diverse architectures.
We present MILABOT: a deep reinforcement learning chatbot developed by the Montreal Institute for Learning Algorithms (MILA) for the Amazon Alexa Prize competition. MILABOT is capable of conversing with humans on popular small talk topics through both speech and text. The system consists of an ensemble of natural langu…
Our purpose is to explore, in the context of loop ensembles on finite graphs, the relations between combinatorial group theory, loops topology, loop measures, and signatures of discrete paths. We determine the distributions of the loop homotopy class, and of the first and second homologies, defined by the lower central…
ControlBurn selects few features from tree ensembles for better model interpretability.
problem Improving model interpretability in machine learning models.
method Sparse tree ensembles with lasso optimization.
result ControlBurn selects feature-sparse subsets for better model interpretability.
Improved deep hedging with ensemble uncertainty quantification.
problem Uncertainty in deep hedging models hinders their deployment.
method Trained an ensemble of LSTM networks to quantify uncertainty in deep hedging under Heston volatility and proportional transaction costs.
result The ensemble's disagreement provides a strong predictive confidence measure for hedge performance.
We propose a method to extract interpretable rules from tree ensembles.
problem Tree ensembles are accurate but hard to interpret.
method Propose an estimator to extract compact sets of decision rules from tree ensembles.
result Our estimator improves accuracy and reveals useful relationships in the data.
EnFF uses flows to speed up DA in high dimensions.
problem Efficiently assimilating noisy data in high-dimensional systems.
method Flow Matching (FM) for training-free, scalable data assimilation.
result EnFF accelerates DA with improved cost-accuracy tradeoffs and scalability.
Residual networks' depth is mathematically equivalent to expanding an implicit ensemble size.
problem Understanding why deep residual networks are effective.
method Formal analysis of residual networks as ensembles of shallow models.
result Increasing network depth is equivalent to expanding the size of an implicit ensemble, revealing a hierarchical structure.
GER learns particle dynamics from unpaired snapshots using physics-informed GANs.
problem Learning particle dynamics from unpaired snapshots with physics constraints.
method Physics-informed generative model to fit particle ensemble distributions.
result Inferred dynamics of particle ensembles governed by SODEs up to 100 dimensions.
GH-PID uses guided harmonic paths for efficient SOT with interpretable diagnostics.
problem Efficiently solving Stochastic Optimal Transport with hard terminal distributions and soft costs.
method Guided Harmonic Path-Integral Diffusion (GH-PID) framework with low-dimensional guidance.
result GH-PID generates geometry-aware, cost-reducing trajectories that match terminal distributions.
Recent work on mode connectivity in the loss landscape of deep neural networks has demonstrated that the locus of (sub-)optimal weight vectors lies on continuous paths. In this work, we train a neural network that serves as a hypernetwork, mapping a latent vector into high-performance (low-loss) weight vectors, general…
SLEM uses machine learning to improve causal inference from observational data.
problem Improving causal inference from observational data using non-linear relationships.
method Super Learner Equation Modeling integrating machine learning ensembles.
result SLEM provides consistent and unbiased estimates of causal effects.
The exploration mechanism used by a Deep Reinforcement Learning (RL) agent plays a key role in determining its sample efficiency. Thus, improving over random exploration is crucial to solve long-horizon tasks with sparse rewards. We propose to leverage an ensemble of partial solutions as teachers that guide the agent's…
Ant colonies and boosting algorithms both reduce bias and variance through adaptive mechanisms.
problem Understanding the mathematical principles behind ensemble learning and ant colony behavior.
method Developed a formal mapping between AdaBoost's adaptive reweighting and ant recruitment dynamics.
result Proved that the fundamental theorem of weak learnability has a direct analog in colony decision-making.
With an eye toward understanding complexity control in deep learning, we study how infinitesimal regularization or gradient descent optimization lead to margin maximizing solutions in both homogeneous and non-homogeneous models, extending previous work that focused on infinitesimal regularization only in homogeneous mo…
The paper proposes a method to improve random forest classification accuracy by weighting trees based on their decision path reliability.
problem Random forests' uniform voting fails to correct errors in regions where incorrect tree representations outnumber correct ones.
method The paper introduces using the structural pattern of each tree's decision path as an instance-adaptive reliability signal to identify and weight more reliable trees.
result Using the proposed method yields a statistically significant accuracy improvement over RF on 36 binary classification benchmarks.
Auto-encoding is an important task which is typically realized by deep neural networks (DNNs) such as convolutional neural networks (CNN). In this paper, we propose EncoderForest (abbrv. eForest), the first tree ensemble based auto-encoder. We present a procedure for enabling forests to do backward reconstruction by ut…
Current deep neural networks suffer from two problems; first, they are hard to interpret, and second, they suffer from overfitting. There have been many attempts to define interpretability in neural networks, but they typically lack causality or generality. A myriad of regularization techniques have been developed to p…
VisRuler simplifies decision extraction from bagged and boosted trees.
problem Complexity and lack of interpretability in ensemble models.
method Visual analytics tool for selecting robust models, important features, and essential decisions.
result Users successfully extracted and explained decisions from ensemble models.
Cyclic Boosting offers detailed prediction understanding for machine learning models.
problem Complex machine learning models are often black boxes, making individual predictions hard to understand.
method Cyclic Boosting is a novel machine learning algorithm that provides detailed understanding of predictions.
result Cyclic Boosting enables detailed understanding of how predictions are made, even for complex models.
New equivalences found between subsampling and ridge regularization methods.
problem Establishing precise structural and risk equivalences between subsampling and ridge regularization.
method Proved structural and risk equivalences between subsample ridge estimators and different ridge regularization levels and subsample aspect ratios.
result Optimally tuned ridge regression exhibits a monotonic prediction risk in the data aspect ratio.
Gradient flow in a potential energy (or Euclidean action) landscape provides a natural set of paths connecting different saddle points. We apply this method to General Relativity, where gradient flow is Ricci flow, and focus on the example of 4-dimensional Euclidean gravity with boundary S^1 x S^2, representing the can…
New algorithms reduce dynamic regret in online MDPs with changing losses.
problem Online MDPs with adversarial loss changes and known transitions.
method Dynamic regret measure, novel ensemble algorithms for three models.
result Provably optimal dynamic regret bounds for episodic SSP, improved bounds for predictable environments.
Hybrid model combines LSTM and ETS for mid-term electric load forecasting.
problem Mid-term electric load forecasting accuracy.
method Combines LSTM, ETS, and ensemble learning; uses dilated LSTM for long-term relationships.
result High performance and competitiveness compared to classical and machine learning models.
Mean-field neural nets approximate functions using a free energy functional and controlled dynamics.
problem Function approximation by two-layer neural nets in the mean-field regime.
method Phrasing function approximation as global minimization of a free energy functional, examining dynamics in the space of probability measures over weights.
result Characterization of the unique global minimizer and dynamics achieving it, including the Föllmer drift.
Multi-headed ensembles boost model performance with faster training.
problem Limited computational resources hinder ensemble search performance.
method Extend NES to multi-headed ensembles, leveraging end-to-end training and one-shot NAS methods.
result Multi-headed ensemble search finds robust ensembles 3 times faster with comparable performance.
MF-PID uses interacting samples to efficiently transport probability mass.
problem Efficiently transporting probability mass in generative models.
method Introducing Mean-Field Path-Integral Diffusion (MF-PID) where samples become interacting agents.
result MF-PID achieves 19-24% reductions in control energy for demand-response control of energy systems.
Ensemble learning is a methodology that integrates multiple DNN learners for improving prediction performance of individual learners. Diversity is greater when the errors of the ensemble prediction is more uniformly distributed. Greater diversity is highly correlated with the increase in ensemble accuracy. Another attr…
We study the computational complexity of Markov chain Monte Carlo (MCMC) methods for high-dimensional Bayesian linear regression under sparsity constraints. We first show that a Bayesian approach can achieve variable-selection consistency under relatively mild conditions on the design matrix. We then demonstrate that t…
Transforms ensemble predictions to maintain interpretability.
problem Loss of interpretability in deep ensembles.
method Proposes transformation ensembles that aggregate predictions while preserving interpretability.
result Transformation ensembles yield better predictions than individual models and maintain interpretability.
Overparameterized ensembles don't offer generalization benefits over single large models.
problem Theoretical limitations of ensembles in overparameterized settings.
method Using ensembles of random feature (RF) regressors, the paper clarifies how modern ensembles differ from underparameterized counterparts.
result Infinite ensembles of overparameterized RF regressors become pointwise equivalent to single infinite-width RF regressors, and finite width ensembles converge to single models with the same parameter budget.
New method creates diverse neural ensembles for better uncertainty estimation and robustness.
problem Creating more robust neural networks for uncertainty estimation and dataset shift.
method Automatically constructing ensembles with varying architectures.
result Ensembles with varying architectures outperform deep ensembles in accuracy, uncertainty calibration, and robustness.
New method certifies joint adversarial robustness of model ensembles.
problem Ensuring robustness of model ensembles against adversarial attacks.
method Proposes a novel technique to certify joint robustness, building on prior work on single-model robustness certification.
result Demonstrates the effectiveness of certifying joint robustness of ensembles, improving understanding of ensemble defenses.
Embedded ensembles improve neural network performance efficiently.
problem Improving neural network performance with fewer resources.
method Analyzing the wide network limit of gradient descent dynamics using Neural-Tangent-Kernel.
result Embedded ensembles exhibit two regimes: independent and collective, affecting performance.
A framework preserves uncertainty in ensemble distillation.
problem Preserving uncertainty decomposition in ensemble distillation.
method General framework for distilling both regression and classification ensembles, preserving natural uncertainty decomposition.
result Framework maintains decomposition of predictive uncertainty.