New algorithms ensure reproducibility and optimal convergence in convex optimization.
problem Trade-off between reproducibility and convergence rate in convex optimization.
method Regularization-based algorithms for smooth convex minimization and minimax optimization.
result Achieves optimal reproducibility and near-optimal gradient complexity for various oracle settings.
Paper explores RKHS properties for derivative and integral operators.
problem Establishing sufficient conditions for reproducing property in RKHS.
method Establishing reproducing property for combinations of composition operators.
result Provides framework for regularized learning algorithms involving function values, gradients, or operators.
This is a report for reproducibility challenge of NeurlIPS 2019 on the paper Competitive Gradient Descent (Schafer et al., 2019). The paper introduces a novel algorithm for the numerical computation of Nash equilibria of competitive two-player games. It avoids oscillatory and divergent behaviours seen in alternating gr…
This paper extends mirror descent to Banach spaces with reproducing kernels.
problem Optimizing in Banach spaces with reproducing kernels.
method Mirror descent algorithm adapted for Banach spaces with reproducing kernels.
result Mirror descent achieves linear convergence in certain conditions and standard convergence in a constrained setting.
In this paper we solve support vector machines in reproducing kernel Banach spaces with reproducing kernels defined on nonsymmetric domains instead of the traditional methods in reproducing kernel Hilbert spaces. Using the orthogonality of semi-inner-products, we can obtain the explicit representations of the dual (nor…
As reinforcement learning (RL) achieves more success in solving complex tasks, more care is needed to ensure that RL research is reproducible and that algorithms herein can be compared easily and fairly with minimal bias. RL results are, however, notoriously hard to reproduce due to the algorithms' intrinsic variance, …
Unsupervised clustering can reproduce categorization systems if features and metrics are correctly selected.
problem Reproducing expert-provided categorization systems using unsupervised clustering.
method Investigated using toy datasets and real-world fund categorization. Used appropriate feature selection and a supervised Random Forest-based distance metric.
result Unsupervised clustering can reproduce ground truth classes if features and metrics are correctly selected.
DIGEN benchmark provides synthetic datasets for ML algorithm evaluation.
problem Understanding and comparing machine learning algorithms' performance.
method Synthetic datasets generated using 40 mathematical functions to evaluate machine learning algorithms.
result DIGEN resource facilitates understanding why algorithms perform poorly and provides ideas for improvement.
Machine learning algorithms designed to characterize, monitor, and intervene on human health (ML4H) are expected to perform safely and reliably when operating at scale, potentially outside strict human supervision. This requirement warrants a stricter attention to issues of reproducibility than other fields of machine …
Paper proposes a method for early stopping in regression using reproducing kernels.
problem Early stopping for iterative learning algorithms in nonparametric regression.
method Data-driven rule based on minimum discrepancy principle, validated by fixed-point analysis of localized Rademacher complexities.
result The proposed rule is minimax-optimal and performs comparably to cross-validation.
Survey on reproducibility and distortion issues in text clustering and topic modeling.
problem Reproducibility and misleading cluster geometry in unsupervised learning for text categorization.
method Systematic literature review of text clustering and topic modeling from 2011-2022.
result Outliers and initialization issues are significant factors in text clustering and topic modeling.
learn2learn simplifies meta-learning research by providing a library and standardized interfaces.
problem Prototyping and reproducibility issues in meta-learning.
method Developed a library (learn2learn) with common routines and standardized interfaces.
result Fosters a community around standardized software for meta-learning research.
New method improves consistency of reinforcement learning performance evaluations.
problem Inconsistent performance results in reinforcement learning due to flawed evaluation metrics.
method Proposes a new comprehensive evaluation methodology for reinforcement learning algorithms.
result Demonstrates improved reliability of performance measurements for reinforcement learning algorithms.
AdaStop improves statistical testing for Deep RL algorithm comparisons.
problem Statistical reproducibility issues in Deep RL.
method AdaStop, a new statistical test based on multiple group sequential tests.
result AdaStop ensures theoretically sound comparisons of Deep RL algorithms.
New framework promotes reproducible, domain-agnostic reinforcement learning algorithms.
problem Domain-specific, compute-resource-maximizing, and non-reproducible participant solutions in reinforcement learning competitions.
method Submission retraining, domain randomization, desemantization through domain obfuscation, and compute/environment-sample budget limitation.
result Participant submissions are reproducible, non-specific to the competition environment, and sample/resource efficient.
Kernel methods are studied in a mean field limit for high-dimensional data.
problem Analyzing kernel methods in high-dimensional data with many variables.
method Investigation of kernel methods in the mean field limit of interacting particle systems.
result Rigorous mean field limit of kernels and detailed analysis of the limiting reproducing kernel Hilbert space.
We introduce SLM Lab, a software framework for reproducible reinforcement learning (RL) research. SLM Lab implements a number of popular RL algorithms, provides synchronous and asynchronous parallel experiment execution, hyperparameter search, and result analysis. RL algorithms in SLM Lab are implemented in a modular w…
AL methods show inconsistent performance gains over random sampling, highlighting variability in neural network-based approaches.
problem Inconsistent performance of AL methods using neural networks.
method Different types of AL algorithms (uncertainty based, diversity based, and committee based) were tested under identical experimental settings.
result AL methods do not consistently outperform random sampling, revealing variability in performance metrics.
Improves Monte-Carlo simulations for consistent mean and variance.
problem Artificial randomness in running mean calculations.
method Combining running mean and variance with accurate summing.
result Increased accuracy and robustness of Monte-Carlo estimates.
In this paper, we introduce ChainerRL, an open-source deep reinforcement learning (DRL) library built using Python and the Chainer deep learning framework. ChainerRL implements a comprehensive set of DRL algorithms and techniques drawn from state-of-the-art research in the field. To foster reproducible research, and fo…
FairGround offers a diverse dataset corpus for fair ML research.
problem Lack of diverse, well-annotated datasets in fair ML research.
method Unified framework and Python package for reproducible fair ML research.
result Advances reproducibility and generalizability of fair ML research.
Reproduces IVFS for high-dimensional data structure preservation.
problem Preserving high-dimensional data structure in unsupervised feature selection.
method Inspired by random subset method, IVFS maintains data similarity through topological structure.
result IVFS outperforms SPEC and MCFS on most datasets.
Torch-Points3D simplifies 3D deep learning research and reproducibility.
problem Lack of transparency and reproducibility in 3D deep learning research.
method Modular framework with quality-of-life features, standardized protocols, and open-source implementation.
result Facilitates fair and rigorous evaluation of 3D deep learning methods.
Study on reproducibility in optimization with bounds on limits.
problem Limits of reproducibility in noisy or error-prone optimization procedures.
method Defined a quantitative measure of reproducibility and analyzed convex optimization settings.
result Revealed a fundamental trade-off between computation and reproducibility.
We propose a novel adaptive learning algorithm based on iterative orthogonal projections in the Cartesian product of multiple reproducing kernel Hilbert spaces (RKHSs). The task is estimating/tracking nonlinear functions which are supposed to contain multiple components such as (i) linear and nonlinear components, (ii)…
This work is a part of ICLR Reproducibility Challenge 2019, we try to reproduce the results in the conference submission PADAM: Closing The Generalization Gap of Adaptive Gradient Methods In Training Deep Neural Networks. Adaptive gradient methods proposed in past demonstrate a degraded generalization performance than …
Study of regularized least squares in RKKS with indefinite kernels.
problem Asymptotic properties of regularized least squares with indefinite kernels in RKKS.
method Introducing a bounded hyper-sphere constraint, theoretical demonstration of globally optimal solution, modified error decomposition techniques, matrix perturbation theory.
result Derivation of learning rates in RKKS, same as RKHS under certain conditions.
Survival analysis models research reproducibility, offering new insights.
problem Reproducibility crisis in machine learning research.
method Survival analysis to model reproducibility as a continuous process.
result Survival analysis provides deeper insights into research reproducibility.
The paper advocates for interpretable, accountable, reproducible machine learning in medicine.
problem Black box models in medicine lack transparency and regulatory approval.
method Intrinsically interpretable modeling approaches and collaborative learning paradigms.
result Interpretable machine learning models can support clinical decisions and gain regulatory approval.
Clarifies the scope of 'reproducibility' in AI and ML.
problem Unclear meaning of 'reproducibility' in AI and ML.
method Analyzes the community's works on reproducibility.
result Identifies eight general topic areas of reproducibility.
Despite the recent progress in deep reinforcement learning field (RL), and, arguably because of it, a large body of work remains to be done in reproducing and carefully comparing different RL algorithms. We present catalyst.RL, an open source framework for RL research with a focus on reproducibility and flexibility. Ma…
This chapter introduces reproducibility in machine learning for medical imaging.
problem Lack of reproducibility in machine learning for medical imaging.
method Distinguishes and defines types of reproducibility, outlines requirements, and discusses utility.
result Discussion on benefits and a plea for a non-dogmatic approach to reproducibility.
New quadrature method using randomly pivoted Cholesky outperforms existing techniques.
problem Efficiently approximating integrals of functions in reproducing kernel Hilbert spaces.
method Nodes drawn by randomly pivoted Cholesky algorithm.
result Randomly pivoted Cholesky quadrature is fast and achieves comparable accuracy to more computationally intensive methods.
HECT tests climate model outputs for reproducibility.
problem Ensuring climate models accurately reflect physical processes.
method Probabilistic classifiers for high-dimensional spatio-temporal data.
result A principled way to assess statistical reproducibility of climate models.
Paper tackles tensor decomposition for unaligned observations using RKHS and novel loss functions.
problem Tackles tensor decomposition for unaligned observations.
method Uses functions in RKHS to represent mode with unaligned observations, introduces versatile loss function, proposes optimization algorithm and stochastic gradient method.
result Demonstrates improved tensor decomposition efficiency and effectiveness with synthetic and real data.
The paper tackles domain generalization using functional regression.
problem Learning a model that generalizes well across different source distributions.
method Functional regression approach to learn a linear operator between marginal and conditional distributions.
result The proposed algorithm achieves finite sample error bounds for the idealized risk.
Bayesian optimization algorithm reduces regret with efficient region pruning.
problem Sequential optimization of unknown functions in high-dimensional spaces.
method Gaussian process-based, domain shrinking through tree-based region pruning.
result Order-optimal regret performance with reduced computational complexity.
Much of human knowledge sits in large databases of unstructured text. Leveraging this knowledge requires algorithms that extract and record metadata on unstructured text documents. Assigning topics to documents will enable intelligent search, statistical characterization, and meaningful classification. Latent Dirichlet…
New method stabilizes machine learning predictions across random seeds.
problem Machine learning predictions vary across random seeds, causing instability.
method Introduces adaptive cross-bagging to eliminate seed dependence.
result Adaptive cross-bagging achieves targeted stability in debiased machine learning.
Proposes a method to improve neural architectures reproducibly.
problem Lack of reproducibility in Neural Architecture Transformer (NAT).
method Differentiable Neural Architecture Transformation (DNAT).
result DNAT outperforms NAT and is applicable to various models and datasets.
The paper aims to improve reproducibility of machine learning experiments.
problem Lack of reproducibility in machine learning experiments.
method Investigates factors affecting reproducibility and proposes FAIR data practices.
result Preliminary results show the role of ProvBook in capturing and comparing provenance.
We estimate Radon-Nikodym derivatives using regularization in reproducing kernel Hilbert spaces.
problem Estimating Radon-Nikodym derivatives in various applications.
method General regularization scheme in reproducing kernel Hilbert spaces.
result High order accuracy in reconstructing Radon-Nikodym derivatives at any point.
Paper studies a robust online learning algorithm for regression.
problem Develops a robust online learning algorithm for regression problems.
method Introduces an online learning algorithm with a robust loss function over RKHS.
result The algorithm achieves optimal convergence rates in mean square and RKHS.
Study on how sampling works for complex data functions.
problem Analyzing convergence of sampling algorithms for RKHS functions.
method Minimalistic assumptions on kernel and data, error estimates in RKHS norm, uniform convergence on compact domains.
result New convergence rates for Lipschitz and Hölder continuous kernels.
New algorithms compute Koopman operators on RKHSs efficiently and accurately.
problem Data-driven spectral analysis of Koopman operators on RKHSs.
method General, provably convergent algorithms for RKHSs.
result Optimal algorithms with error control and spectral measures.
NeurIPS 2019 program improves reproducibility in machine learning.
problem Ensuring machine learning research results are reproducible and reliable.
method Code submission policy, reproducibility challenge, and checklist integration.
result Improved reproducibility standards across the machine learning community.
Algorithm generates realistic metaorders from public trade data.
problem Generating realistic metaorders from public data.
method Novel algorithm that recovers stylized facts of metaorders impact.
result Average realized short-term price impact has a mechanical origin.
Unified platform for optimal stopping problems in R.
problem Optimal stopping problems in machine learning.
method Unified implementation of Regression Monte Carlo algorithms.
result Unified and reproducible platform for RMC algorithms.