A novel gradient-based method optimizes decision trees for complex tasks.
problem Training decision trees with arbitrary differentiable loss functions.
method Gradient-based optimization using first and second derivatives of loss functions.
result Improves accuracy and flexibility in decision tree optimization.
A new method improves model trees for higher accuracy and transparency.
problem Combining transparency and high accuracy in machine learning models.
method Gradient-based split criterion for model trees.
result Significantly higher predictive power with simpler models.
Early stopping avoids validation sets by using gradient statistics.
problem Preventing overfitting in over-expressive models.
method Local gradient statistics for early stopping.
result Early stopping without validation is feasible and effective.
FedForest adapts RF for federated learning, improving performance and efficiency.
problem Adapting RF for federated learning with heterogeneous data.
method FedForest uses a novel splitting procedure to aggregate client statistics, allowing non-parametric personalization.
result FedForest's federated RF achieves performance close to centralized models while being communication-efficient.
Stochastic Gradient Trees learn decision trees incrementally.
problem Learning decision trees using stochastic gradient information.
method Incremental learning setting, soft splits not used, new tree not constructed per update.
result Performs similarly to standard incremental classification trees, outperforms state of the art incremental regression trees, comparable to batch multi-instance learning methods.
MERL uses evolutionary and gradient-based methods to optimize sparse team-based and dense agent-specific rewards in multiagent coordination.
problem Training multiagent reinforcement learning policies on sparse team-based rewards is difficult and relying solely on agent-specific rewards is sub-optimal.
method MERL employs a split-level training platform with an evolutionary algorithm and a gradient-based optimizer, transferring skills between the two processes.
result MERL significantly outperforms state-of-the-art methods on coordination benchmarks.
New method accelerates energetic variational inference using particle dynamics.
problem Efficiently solving variational inference problems with reduced computational cost.
method Particle-based variational inference with implicit scheme, inspired by energy quadratization and operator splitting.
result Significantly reduces computational cost compared to existing methods.
The paper develops statistical inference for gradient flows in optimization.
problem Uncertainty quantification along the entire optimization path.
method Uniform central limit theorem and algorithm-aware covariance estimator.
result Asymptotically valid confidence intervals for target parameter.
Cross-regularization adapts model complexity during training.
problem Manual tuning of model complexity for overfitting prevention.
method Directly adapts regularization parameters through validation gradients during training.
result Organic emergence of architecture-specific regularization during training.
A faster method for estimating effects in large data using fixed-point trees.
problem Estimating heterogeneous effects in large dimensions with computational efficiency.
method Fixed-point approximation to eliminate Jacobian estimation and speed up GRFs.
result Significant computational efficiency improvement without sacrificing statistical accuracy.
Unified view of accelerated and stochastic optimization methods.
problem Optimization challenges in machine learning and physics.
method Unified gradient flow approach to proximal algorithms and their accelerated variants.
result Unified framework for accelerated and stochastic optimization methods.
Review of gradient-based algorithms in statistical inference problems.
problem Understanding the dynamics of gradient-based algorithms in statistical inference.
method Insights from physics of glassy systems.
result Quantitative and qualitative understanding of algorithm performance.
Gradient-based methods improve understanding of deep learning survival models.
problem Limited interpretability of deep learning survival models hinders their adoption.
method Gradient-based explanation methods tailored to survival neural networks.
result Gradient-based methods capture feature effects and temporal dynamics.
Introduces new gradient-based methods for machine learning problems.
problem New challenges in machine learning due to decision-making and multi-agent problems.
method Gradient-based optimization and variational inequalities.
result Shifts focus from pattern recognition to decision-making and multi-agent problems.
New gradient-based method learns causal structures from data.
problem Challenging task in causal structure learning.
method Graph Autoencoder framework for gradient-based optimization.
result Significantly outperforms other gradient-based methods on large causal graphs.
Bayesian Neural Networks are robust to gradient-based attacks in the large-data limit.
problem Vulnerability of deep learning models to adversarial attacks.
method Analysis of adversarial attacks in the large-data, overparametrized limit for Bayesian Neural Networks.
result BNN posteriors are robust to gradient-based adversarial attacks in the limit.
Generative model learns hierarchical structure for better sample diversity.
problem Generating high-quality samples from arbitrary distributions.
method Hierarchical mixtures of generators inspired by hierarchical mixture of experts.
result Hierarchical model generates high-quality and diverse samples.
Gradient-based meta-RL fails with incorrect task distributions, leading to instability and poor performance.
problem Gradient-based meta-RL's sensitivity to task distributions causes instability and poor performance.
method Proposes meta Active Domain Randomization (meta-ADR) to learn task distributions for gradient-based meta-RL.
result Meta-ADR improves stability and generalization of MAML on simulated locomotion and navigation tasks.
Proposes VSGD optimizer combining probabilistic and gradient-based methods.
problem Uncertainty modeling in deep neural networks.
method Combines probabilistic and gradient-based approaches using SVI.
result VSGD outperforms Adam and SGD on image classification tasks.
We propose a technique for increasing the efficiency of gradient-based inference and learning in Bayesian networks with multiple layers of continuous latent vari- ables. We show that, in many cases, it is possible to express such models in an auxiliary form, where continuous latent variables are conditionally determini…
Analyzes gradient-based learning in games, avoiding local Nash equilibria.
problem Analyzes gradient-based learning in competitive games.
method Formulates a general framework and uses dynamical systems theory.
result Characterizes local Nash equilibria avoided by gradient-based learning.
Gradient-based methods can be biased by distributional asymmetries in bivariate categorical data.
problem Gradient-based causal discovery methods can be biased by distributional asymmetries in bivariate categorical data.
method Identified and examined two distributional biases: Marginal Distribution Asymmetry and Marginal Distribution Shift Asymmetry. Employed two simple models to demonstrate and control these biases.
result Gradient-based methods can be biased by distributional asymmetries, and these biases can be controlled.
New method creates universal perturbations to fool neural network interpretations.
problem Vulnerability of gradient-based saliency maps to adversarial perturbations.
method Gradient-based optimization and PCA-based approach to create UPI.
result Existence and successful application of Universal Perturbation for Interpretation (UPI).
Gradient-based MCMC for discrete spaces improves sampling performance.
problem Sampling in discrete spaces using traditional methods is challenging.
method Introduced new discrete Metropolis-Hastings samplers inspired by MALA, with a novel preconditioning technique.
result Demonstrated strong empirical performance across various challenging sampling problems.
Study on generalization in gradient-based meta-learning, showing flatter solutions and coherence between adaptation trajectories.
problem Understanding generalization in gradient-based meta-learning.
method Analysis of objective landscapes, experimental demonstration of solution properties, and empirical evidence on coherence between adaptation trajectories.
result Meta-test solutions become flatter, lower in loss, and further away from the meta-train solution as meta-training progresses, even as generalization starts to degrade.
This work analyzes and compares gradient-based attribution methods for DNNs.
problem Understanding the flow of information in Deep Neural Networks.
method Formal analysis and comparison of four gradient-based attribution methods.
result Unified framework and novel evaluation metric (Sensitivity-n) for comparison.
Gradient-based sampling improves LS problem solving efficiency.
problem Computational difficulties in large sample size data analysis.
method Adaptive sampling dependent on input and output gradients.
result Gradient-based sampling improves statistical efficiency and computational saving.
Automated meta-learning improves model performance on small datasets.
problem Improving machine learning model performance on limited data.
method Gradient-based meta-learning combined with automated neural architecture search.
result Automatically found meta-learner achieved 74.65% accuracy on 5-shot 5-way Mini-ImageNet, 11.54% better than MAML.
Efficiently minimizes regret in non-convex games with gradient-based methods.
problem Computational intractability of standard regret minimization in non-convex games.
method Defining a new notion of regret and using gradient-based optimization methods.
result Achieves optimal regret, leading to convergence to equilibrium.
Gradient-based method extracts slow features from high-dimensional data.
problem Extracting meaningful low-dimensional features from high-dimensional, temporally varying data.
method Power Slow Feature Analysis (PowerSFA) using gradient-based training of differentiable architectures.
result PowerSFA effectively extracts meaningful low-dimensional features in various data types.
GIT uses gradient estimators to target interventions for causal discovery.
problem Challenges in inferring causal structure from observational data.
method GIT uses gradient estimators to target interventions for causal discovery.
result GIT performs on par with competitive baselines, surpassing them in low-data regimes.
Meta learning works well with overparameterized models, a phenomenon called 'benign overfitting'.
problem Understanding why overparameterized models perform well in few-shot learning.
method Analyzed the generalization performance of gradient-based meta learning with an overparameterized meta linear regression model.
result Demonstrated that overparameterized meta learning can still generalize well, a phenomenon called 'benign overfitting'.
Extends gradient-based optimization to spline functions.
problem Limitations of standard differentiable programming methods.
method Derives Jacobian of spline functions and uses it in predictive models.
result Improved performance in various applications.
This work investigates how gradient-based learning performs with structured data, revealing issues and improvements.
problem Gradient-based learning under structured data, particularly with a spiked covariance structure.
method Investigates the effect of a spiked covariance structure on gradient-based feature learning and proposes weight normalization.
result Gradient-based dynamics may fail to recover the true direction in anisotropic settings, but weight normalization can improve performance.
Paper presents an optimization-based attack and defense for graph neural networks.
problem Adversarial robustness of graph neural networks (GNNs).
method Gradient-based attack and optimization-based adversarial training.
result Optimization-based attack can significantly decrease GNN classification performance with minimal edge perturbations.
Improved Hamiltonian Monte Carlo for Bayesian inference reduces variance and improves performance.
problem Efficiently sampling from posterior distributions in Bayesian inference with stochastic gradients.
method Variance reduction techniques applied to Hamiltonian Monte Carlo.
result Theoretical and experimental improvements in convergence and performance compared to variance-reduced Langevin dynamics.
Two gradient-based methods for hyperparameter optimization are introduced, with applications in machine learning.
problem Optimizing hyperparameters for machine learning models.
method Forward and reverse-mode procedures for computing gradients of validation error with respect to hyperparameters.
result Forward-mode procedure suitable for real-time hyperparameter updates, potentially speeding up optimization on large datasets.
Meta-learning technique bypasses data limitations by optimizing latent space.
problem Challenges in gradient-based meta-learning with high-dimensional parameter spaces in low-data regimes.
method Learning a latent generative representation of model parameters and performing meta-learning in this low-dimensional space.
result Achieves state-of-the-art performance on few-shot classification tasks.
This paper surveys gradient-based multi-objective deep learning methods.
problem Balancing multiple conflicting objectives in deep learning models.
method Gradient-based techniques adapted from Multi-Objective Optimization.
result Comprehensive survey of gradient-based multi-objective deep learning algorithms.
Randomized gradient-based ensemble improves prediction accuracy.
problem Improving prediction accuracy in machine learning.
method Randomization and gradient-based aggregation of weakly-correlated estimators.
result The method outperforms existing techniques in terms of increased accuracy.
Deep learning algorithms fail on simple problems.
problem Gradient-based deep learning algorithms fail on simple problems.
method Experiments and theoretical insights.
result Gradient-based algorithms fail on simple problems.
Gradient-based training and pruning for radial basis function networks in materials physics.
problem Interpretable and robust machine learning for materials physics problems.
method Gradient-based training and pruning of radial basis function networks with closed-form optimization criteria.
result Pruned models provide compact and interpretable versions of larger models, offering insights into atom-level migration processes.
TrIM improves gradient-based dimension reduction and regression.
problem Efficiently identifying relevant feature subspace for high-dimensional regression.
method Introduced TrIM forest, an iterative approach using Mondrian forest and EGOP estimate.
result Consistency guarantees and convergence rates for EGOP matrix and random forest estimator.
Gradient-based feature selection for large datasets.
problem Feature selection for large datasets with high-order correlations.
method Iterative mini-batch calculation, discrete-to-continuous relaxation.
result Efficiently finds higher-order feature correlations in both N > D and N < D regimes.
Hybrid method improves sampling from multimodal distributions.
problem Sampling from multimodal posterior distributions efficiently.
method Jump-Diffusion Langevin Dynamics hybrid with Metropolis.
result Calibrated hybrid method outperforms pure methods.
Novel BSG method for efficient stochastic optimization.
problem Efficient optimization of non-convex surfaces in stochastic settings.
method Binary search combined with first order gradient optimization.
result BSG produces more promising results and better generalization than other methods.
Proposes an automatic cyclical scheduling for gradient-based discrete sampling.
problem Gradient-based sampling in high-dimensional models can get stuck in local modes.
method Cyclical step size and balancing schedules with automatic hyperparameter tuning.
result Proves non-asymptotic convergence and inference guarantees for general discrete distributions.
Differentiable pipeline replaces non-differentiable CAE components for shape optimization.
problem Gradient-based optimization is limited by non-differentiable components in CAE workflows.
method Surrogate models replace non-differentiable pipeline components, enabling gradient-based optimization.
result Gradient-based shape optimization possible without differentiable solvers.