Proposes a Siamese NN for algorithm selection focusing on alike performing instances.
problem Lack of effective meta-features for algorithm selection via meta-learning.
method Siamese Neural Network architecture with 'Algorithm-Performance Personas' concept.
result Proposed metric outperforms standard performance metrics in training sample selection.
The book explores alternatives to worst-case analysis for algorithm performance.
problem Providing strong worst-case guarantees for many algorithms is impossible.
method Surveying and detailing various nuanced analysis approaches.
result More nuanced analysis approaches are needed for fundamental problems.
New bounds derived for KG algorithm's performance in finite time.
problem Best arm identification problem in multi-armed bandit.
method Theoretical analysis of finite-time performance, deriving bounds for sample allocation, error probability, and regret.
result Upper and lower bounds for the probability of error and simple regret of the KG algorithm.
The paper proposes a method to predict the performance of data-driven algorithms using surrogate models.
problem Improving the performance prediction of data-driven knowledge discovery algorithms.
method Surrogate-assisted performance prediction using evolutionary modeling of clinical pathways.
result The proposed approach provides interpretable prediction of algorithm performance and quality.
New method improves consistency of reinforcement learning performance evaluations.
problem Inconsistent performance results in reinforcement learning due to flawed evaluation metrics.
method Proposes a new comprehensive evaluation methodology for reinforcement learning algorithms.
result Demonstrates improved reliability of performance measurements for reinforcement learning algorithms.
Identifies the best-performing algorithm from a set of candidates.
problem Finding the best-performing machine learning algorithm from benchmark performance.
method Subset selection for multinomial distributions, providing asymptotic and finite-sample schemes.
result Significantly improved methods for identifying the best-performing algorithm.
Study limits of testing algorithms without assumptions, finding key performance bounds.
problem Understanding algorithm performance without making assumptions.
method Exploring fundamental limits of algorithm performance testing.
result Hardness results for evaluating algorithm performance and comparing multiple algorithms.
Dataset pruning is the process of removing sub-optimal tuples from a dataset to improve the learning of a machine learning model. In this paper, we compared the performance of different algorithms, first on an unpruned dataset and then on an iteratively pruned dataset. The goal was to understand whether an algorithm (s…
One significant challenge to scaling entity resolution algorithms to massive datasets is understanding how performance changes after moving beyond the realm of small, manually labeled reference datasets. Unlike traditional machine learning tasks, when an entity resolution algorithm performs well on small hold-out datas…
Researchers analyze how algorithmic and implementation choices affect RL performance.
problem Difficulty in separating algorithmic and implementation differences in RL performance.
method Unified derivations through a single control-as-inference objective, categorizing algorithms as EM or KL minimization.
result Implementation details are co-adapted with algorithmic choices, some transferable across algorithms.
A TS algorithm improves performance in multi-task bandits with transfer.
problem Improving performance across multiple related bandit tasks.
method A TS-type algorithm for online multi-task learning with a novel concentration inequality.
result The TS-type algorithm achieves nearly-optimal performance guarantees.
Algorithms often have tunable parameters that impact performance metrics such as runtime and solution quality. For many algorithms used in practice, no parameter settings admit meaningful worst-case bounds, so the parameters are made available for the user to tune. Alternatively, parameters may be tuned implicitly with…
One of the classical problems in machine learning and data mining is feature selection. A feature selection algorithm is expected to be quick, and at the same time it should show high performance. MeLiF algorithm effectively solves this problem using ensembles of ranking filters. This article describes two different wa…
Paper compiles ML algorithm performance benchmarks on OpenML datasets.
problem Finding optimal hyperparameters for ML algorithms efficiently.
method Generated benchmark data for 7 ML algorithms on 39 datasets, fixed hyperparameters before testing.
result Comprehensive dataset of ML algorithm performance sensitivity.
Algorithm for recognizing and performing Reidemeister moves in Gauss diagrams.
problem Recognizing and performing Reidemeister moves in Gauss diagrams.
method Simple algorithm for recognizing and performing Reidemeister moves in Gauss diagrams.
result Simple algorithm for recognizing and performing Reidemeister moves in Gauss diagrams.
New algorithms improve machine learning performance with explicit regret bounds.
problem Improving machine learning performance with explicit regret bounds.
method Projection-based linear regression algorithms with a focus on modern machine-learning models and their algorithmic performance.
result Established a priori regret bounds with explicit λ-dependence.
Daily streamflow forecasting through data-driven approaches is traditionally performed using a single machine learning algorithm. Existing applications are mostly restricted to examination of few case studies, not allowing accurate assessment of the predictive performance of the algorithms involved. Here we propose sup…
The study provides theoretical foundations for using smaller instances to predict algorithm performance on larger ones.
problem Scalability challenge in evaluating algorithms on large instances.
method Formalized size generalization, providing theoretical guarantees for predicting algorithm performance on large instances using smaller, representative instances.
result Characterized the subsample size sufficient to ensure performance on the subsample reflects performance on the full instance.
Improved batched SH algorithm maintains original performance.
problem Maintaining performance in batched multi-armed bandits.
method Simple batch version of Sequential Halving algorithm.
result Batching does not degrade SH algorithm's performance.
Algorithm ranks assets in fluctuating markets.
problem Ranking assets in nonstationary time series.
method Naive Bayes asset ranker that adjusts weights based on performance.
result Outperforms traditional methods and S&P 500 index.
Random forest (RF) missing data algorithms are an attractive approach for dealing with missing data. They have the desirable properties of being able to handle mixed types of missing data, they are adaptive to interactions and nonlinearity, and they have the potential to scale to big data settings. Currently there are …
Approximate algorithms for structured prediction problems---such as LP relaxations and the popular alpha-expansion algorithm (Boykov et al. 2001)---typically far exceed their theoretical performance guarantees on real-world instances. These algorithms often find solutions that are very close to optimal. The goal of thi…
As more data are produced each day, and faster, data stream mining is growing in importance, making clear the need for algorithms able to fast process these data. Data stream mining algorithms are meant to be solutions to extract knowledge online, specially tailored from continuous data problem. Many of the current alg…
New algorithm improves group fairness in social classification problems by exploiting performativity.
problem Inequities in social classification problems due to performativity.
method Develops algorithmic fairness practices that leverage performativity to achieve stronger group fairness guarantees.
result Achieves stronger group fairness guarantees compared to non-performative settings.
We propose accelerated randomized coordinate descent algorithms for stochastic optimization and online learning. Our algorithms have significantly less per-iteration complexity than the known accelerated gradient algorithms. The proposed algorithms for online learning have better regret performance than the known rando…
SMAC3 optimizes machine learning hyperparameters efficiently.
problem Optimizing hyperparameters for machine learning algorithms.
method Bayesian Optimization framework with facades for various use cases.
result Improves performance with minimal evaluations.
Stochastic Gradient Descent has been widely studied with classification accuracy as a performance measure. However, these stochastic algorithms cannot be directly used when non-decomposable pairwise performance measures are used such as Area under the ROC curve (AUC) which is a common performance metric when the classe…
The paper introduces metrics to evaluate NILM algorithms' performance on unseen buildings.
problem Assessing NILM algorithms' performance on new, unseen buildings.
method Developed several metrics to evaluate NILM algorithms' generalization ability.
result Demonstrated the utility of the proposed metrics through two case studies.
Online algorithms stabilize in feedback loops of performative prediction.
problem Feedback loops in algorithmic predictions influence data distributions.
method Martingale argument and randomization to avoid distributional assumptions.
result No-regret algorithms converge to performatively stable equilibria.
A benchmark for simulation-based inference methods.
problem Lack of a public benchmark for 'likelihood-free' algorithms.
method Provided a benchmark with tasks and performance metrics, including neural networks and ABC methods.
result State-of-the-art algorithms have room for improvement, and neural network-based approaches generally perform better.
Improved analysis of UCBVI algorithm with better empirical performance.
problem Improving the UCBVI algorithm's performance and understanding its bounds.
method Refined analysis of UCBVI algorithm with improved bonus terms and regret analysis.
result Improving multiplicative constants in UCBVI bounds enhances empirical performance.
The paper proposes calibration to improve algorithm performance using machine learning predictions.
problem Improving real-world performance of online algorithms with machine learning predictions.
method Calibration as a tool to bridge the gap between prediction uncertainty and algorithm design.
result Calibrated advice leads to more effective guidance in high-variance settings and significant performance improvements in real-world data.
This study validates BN structure learning algorithms under noisy data, revealing performance discrepancies.
problem Inconsistent performance claims across BN structure learning algorithms due to inconsistent evaluation methods.
method Applied 15 algorithms to noisy data in multiple studies, evaluating with various criteria.
result Traditional synthetic performance may overestimate real-world performance by 10-50%.
Equations track profits and losses in trading algorithms.
problem Evaluating the performance of trading algorithms.
method Formal equations incorporating spread.
result Evaluate trading model algorithms' performance.
Several recent deep neural networks experiments leverage the generalist-specialist paradigm for classification. However, no formal study compared the performance of different clustering algorithms for class assignment. In this paper we perform such a study, suggest slight modifications to the clustering procedures, and…
Iterative reweighted algorithms, as a class of algorithms for sparse signal recovery, have been found to have better performance than their non-reweighted counterparts. However, for solving the problem of multiple measurement vectors (MMVs), all the existing reweighted algorithms do not account for temporal correlation…
New algorithm finds causal parent nodes without graph learning.
problem Finding causal relationships without knowing the graph structure.
method Developed an efficient algorithm using atomic interventions.
result Algorithm optimally performs interventions with sublinear complexity.
Benchmarked over 70 graph clustering algorithms.
problem Lack of comprehensive performance comparison for graph clustering algorithms.
method Evaluated 70+ graph clustering programs for runtime and quality on weighted and unweighted graphs, analyzed ground truth characteristics.
result Supply a start point for engineers and viewpoint for researchers.
New NMF algorithm uses Toeplitz matrix for facial recognition.
problem Facial recognition performance improvement.
method Proposes TNMF algorithm with Toeplitz penalty for NMF.
result TNMF outperforms ZNMF and other constrained NMF algorithms.
This paper tackles ranking-based performance normalization for optimization algorithms.
problem Ranking optimization algorithms across diverse numerical scales disrupts performance comparisons.
method Introduces absolute ranking and a sampling-based computational method to address numerical scale variation.
result Provides a more robust framework for assessing performance across multiple algorithms and problems.
New algorithm outperforms existing ones in multi-player bandit problems without sensing.
problem Decentralized multi-player multi-armed bandit problem without collision or sensing info.
method Randomized Selfish KL-UCB, inspired by Selfish KL-UCB, with low complexity.
result Randomized Selfish KL-UCB outperforms state-of-the-art algorithms in almost all environments.
First private Bayesian optimization algorithm with provable performance.
problem Privacy-preserving Bayesian optimization in outsourced settings.
method Random projection-based transformation to preserve privacy and distances.
result Regret bound similar to non-private GP-UCB established for PO-GP-UCB.
New algorithm closes empirical gap in PFSGD performance.
problem Empirical performance gap between tuned SGD and PFSGD.
method Parameter-free algorithm based on Coin-Betting ODE updates.
result New algorithm outperforms tuned baselines and matches optimal performance.
We consider the correlated multiarmed bandit (MAB) problem in which the rewards associated with each arm are modeled by a multivariate Gaussian random variable, and we investigate the influence of the assumptions in the Bayesian prior on the performance of the upper credible limit (UCL) algorithm and a new correlated U…
Paper proposes TDQN, a DRL strategy for optimal stock trading.
problem Optimal trading position determination in stock markets.
method Deep reinforcement learning (DRL) with Trading Deep Q-Network (TDQN) algorithm.
result TDQN strategy significantly improves Sharpe ratio performance.
The goal of data-driven algorithm design is to obtain high-performing algorithms for specific application domains using machine learning and data. Across many fields in AI, science, and engineering, practitioners will often fix a family of parameterized algorithms and then optimize those parameters to obtain good perfo…
Regularized EM algorithm improves clustering performance with small sample sizes.
problem Performance reduction in EM algorithm due to small sample size and poorly conditioned covariance matrices.
method Regularized EM algorithm that uses prior knowledge to ensure positive definiteness of covariance matrices.
result The regularized EM algorithm outperforms standard EM in clustering tasks with small sample sizes.
Using a low-dimensional parametrization of signals is a generic and powerful way to enhance performance in signal processing and statistical inference. A very popular and widely explored type of dimensionality reduction is sparsity; another type is generative modelling of signal distributions. Generative models based o…