The paper quantifies uncertainty in aggregated machine learning metrics.
problem Uncertainty in summarizing model performance across multiple tasks.
method Statistical methodologies including bootstrapping and Bayesian modeling.
result Insights into model performance dominance for specific tasks.
We introduce a novel aggregation method to efficiently perform image denoising. Preliminary filters are aggregated in a non-linear fashion, using a new metric of pixel proximity based on how the pool of filters reaches a consensus. We provide a theoretical bound to support our aggregation scheme, its numerical performa…
We introduce a nonlinear aggregation type classifier for functional data defined on a separable and complete metric space. The new rule is built up from a collection of M arbitrary training classifiers. If the classifiers are consistent, then so is the aggregation rule. Moreover, asymptotically the aggregation rule b…
BOA improves financial forecasting by combining expert models.
problem Challenges in choosing between multiple machine learning models for financial forecasting.
method Online aggregation of expert models using Bernstein Online Aggregation (BOA) procedure.
result BOA leads to better portfolio performance, higher Sharpe Ratio, and lower shortfall.
We extend the recently introduced theory of Lovasz-Bregman (LB) divergences (Iyer & Bilmes 2012) in several ways. We show that they represent a distortion between a "score" and an "ordering", thus providing a new view of rank aggregation and order based clustering with interesting connections to web ranking. We show ho…
We extend the recently introduced theory of Lovasz-Bregman (LB) divergences (Iyer & Bilmes, 2012) in several ways. We show that they represent a distortion between a 'score' and an 'ordering', thus providing a new view of rank aggregation and order based clustering with interesting connections to web ranking. We show h…
Distance metric learning is successful in discovering intrinsic relations in data. However, most algorithms are computationally demanding when the problem size becomes large. In this paper, we propose a discriminative metric learning algorithm, and develop a distributed scheme learning metrics on moderate-sized subsets…
Study uses echo-sounder buoys to analyze tuna schools' association with dFADs globally.
problem Understanding temporal trends of tuna schools' association to drifting objects.
method Applied Machine Learning to examine binary and regression outputs of tuna schools' colonization and disaggregation times.
result Median colonization and disaggregation times varied by ocean, with Pacific having longest soak and colonization times.
We solve the multi-criteria benchmarking problem by formalizing it as a social choice problem and identifying conditions for meaningful rankings.
problem Aggregating multiple metrics into a single ranking for models in benchmarking problems.
method Formalizing multi-criteria benchmarking as a social choice problem and identifying sufficient conditions for meaningful rankings.
result We prove that meaningful multi-criteria benchmarking becomes possible under certain preference conditions (single-peaked, group-separable, distance-restricted).
DAGgr aggregates multiple DAGs to stabilize causal structure learning.
problem Stability in learning causal structure from data.
method Model averaging of candidate DAGs weighted by predictive likelihood, with acyclicity enforced.
result DAGgr consistently outperforms individual DAGs and bootstrap-aggregation baselines.
We propose a framework, named Aggregated Wasserstein, for computing a dissimilarity measure or distance between two Hidden Markov Models with state conditional distributions being Gaussian. For such HMMs, the marginal distribution at any time spot follows a Gaussian mixture distribution, a fact exploited to softly matc…
Probabilistic graphical models are graphical representations of probability distributions. Graphical models have applications in many fields including biology, social sciences, linguistic, neuroscience. In this paper, we propose directed acyclic graphs (DAGs) learning via bootstrap aggregating. The proposed procedure i…
We propose a framework, named Aggregated Wasserstein, for computing a dissimilarity measure or distance between two Hidden Markov Models with state conditional distributions being Gaussian. For such HMMs, the marginal distribution at any time position follows a Gaussian mixture distribution, a fact exploited to softly …
The problem of explaining deep learning models, and model predictions generally, has attracted intensive interest recently. Many successful approaches forgo global approximations in order to provide more faithful local interpretations of the model's behavior. LIME develops multiple interpretable models, each approximat…
In classification, the de facto method for aggregating individual losses is the average loss. When the actual metric of interest is 0-1 loss, it is common to minimize the average surrogate loss for some well-behaved (e.g. convex) surrogate. Recently, several other aggregate losses such as the maximal loss and average t…
The study introduces measures of collective mobility from aggregated OD data.
problem Understanding large-scale mobility patterns from aggregated data.
method Developed a framework using synthetic and real data to interpret network-level mobility.
result Aggregated mobility measures reveal network structure and flow constraints.
New tool for summarizing time-varying data shapes.
problem Understanding dynamic data shapes.
method Introducing crocker stacks for time-varying metric spaces.
result Demonstrated utility in parameter identification task.
We propose regression networks for the problem of few-shot classification, where a classifier must generalize to new classes not seen in the training set, given only a small number of examples of each class. In high dimensional embedding spaces the direction of data generally contains richer information than magnitude.…
Paper proposes a supervised similarity framework for corporate bonds using RF proximities.
problem Challenges in measuring similarity for corporate bonds due to noisy data and lack of ground truth.
method Proposes a supervised similarity framework using Random Forest for corporate bonds, introducing a novel metric to evaluate similarities.
result Random Forest outperforms other methods in evaluating similarities for corporate bonds.
Census data provide detailed information about population characteristics at a coarse resolution. Nevertheless, fine-grained, high-resolution mappings of population counts are increasingly needed to characterize population dynamics and to assess the consequences of climate shocks, natural disasters, investments in infr…
HierarchicalForecast provides a Python framework for coherent hierarchical forecasting.
problem Ensuring forecasts at disaggregate levels add up to aggregate forecasts.
method Preprocessed datasets, evaluation metrics, and statistical baseline models.
result Python-based reference framework for statistical and ML forecasting.
Study analyzes neural network models to understand generalization performance.
problem Understanding good generalization in neural networks.
method Analyzed a corpus of models from a public contest, breaking ALPHAHAT into scale and shape metrics.
result Identified a Simpson's paradox in metric performance across different model depths and regularization hyperparameters.
Performance metrics (error measures) are vital components of the evaluation frameworks in various fields. The intention of this study was to overview of a variety of performance metrics and approaches to their classification. The main goal of the study was to develop a typology that will help to improve our knowledge a…
The study improves sentiment analysis of 10-K filings, revealing aggregation effects on accuracy and correlation with market outcomes.
problem Lack of sentiment analysis for 10-K filings, particularly for risk disclosures.
method Supervised lexicon-learning approach applied to 10-K filings and Item 1A risk-factor sections, trained against return and volatility labels at different levels of aggregation.
result Sentiment analysis of Item 1A sections performs better at the individual-firm level, while full-filing text is more accurate at sector and portfolio levels.
New method accounts for uncertainty in medical AI evaluations.
problem Uncertainty in ground truth affects AI model performance estimates.
method Statistical aggregation approach to infer probabilities of medical conditions.
result Performance estimates are significantly lower when uncertainty is accounted for.
Proposes a new optimization-based method for aggregating sets in neural networks.
problem Limited representational power of existing aggregation methods.
method Equilibrium Aggregation: an optimization-based approach.
result Equilibrium Aggregation outperforms existing methods in various tasks.
The Wisdom of Crowds (WOC), as a theory in the social science, gets a new paradigm in computer science. The WOC theory explains that the aggregate decision made by a group is often better than those of its individual members if specific conditions are satisfied. This paper presents a novel framework for unsupervised an…
Turbo-Aggregate reduces secure aggregation time from quadratic to nearly linear.
problem Quadratic overhead in secure model aggregation for federated learning.
method Multi-group circular strategy, additive secret sharing, and coding techniques.
result Achieves O(NlogN) overhead, compared to O(N2), for up to 50% user dropout. New method improves CATE model selection with optimal regret rates.
problem Nontrivial task of selecting accurate CATE models.
method Causal Q-aggregation using doubly robust loss.
result Achieves optimal oracle model selection regret rates of log(M)/n.
This paper challenges the current metrics used for evaluating long-term forecasting models.
problem Current metrics focus on pointwise error reduction, ignoring structural properties.
method Proposes a multi-dimensional evaluation approach that includes statistical fidelity, structural coherence, and decision-level relevance.
result Current progress in forecasting may reflect specialization in benchmark configurations rather than deeper understanding of temporal dynamics.
Study aggregation of statistical evidence under unknown dependence using group-invariance.
problem Aggregating statistical evidence under unknown and complex dependence structures.
method Develops a framework using group-invariance and permutation-based constructions to aggregate evidence across transformed datasets.
result Shows uniform improvement in critical values for single-batch aggregation over deterministic calibrations, adapting to unknown dependence structures.
New LCM aggregator improves GNN performance and efficiency.
problem Graph neural networks' sensitivity to aggregation function choice.
method Learnable commutative monoid for graph aggregation.
result LCM aggregator achieves performance competitive with recurrent aggregators.
Unified approach to aggregating models and preferences.
problem Consistent aggregation of models and preferences.
method Formal definition and weighted averaging of models and preferences.
result All rational aggregation rules are weighted averages of highest-ranked models/experts.
In order to scale standard Gaussian process (GP) regression to large-scale datasets, aggregation models employ factorized training process and then combine predictions from distributed experts. The state-of-the-art aggregation models, however, either provide inconsistent predictions or require time-consuming aggregatio…
Aggregation challenges causal interpretation of IV estimators.
problem Aggregation of fine-grained components into an aggregate treatment variable.
method Characterization of conditions for identifying aggregate causal effects.
result Standard IV estimators cannot identify aggregate causal effects due to ambiguous dependencies.
SACP aggregates nonconformity scores from multiple predictors to create more efficient uncertainty sets.
problem Combining predictive uncertainties from multiple models for efficient and reliable uncertainty quantification.
method SACP (Symmetric Aggregated Conformal Prediction) aggregates nonconformity scores using a flexible symmetric aggregation function.
result SACP consistently improves efficiency and often outperforms state-of-the-art model aggregation baselines.
We consider the forecast aggregation problem in repeated settings, where the forecasts are done on a binary event. At each period multiple experts provide forecasts about an event. The goal of the aggregator is to aggregate those forecasts into a subjective accurate forecast. We assume that experts are Bayesian; namely…
DUA-D2C refines D2C for better deep learning generalization.
problem Overfitting in deep learning models due to data outliers and noise.
method Dynamic uncertainty-aware aggregation of D2C subsets.
result Significant improvement in generalization performance.
Paper analyzes sparse aggregation in GLMs with Kullback-Leibler risk bounds.
problem Sparse aggregation in GLMs for parameter approximation.
method Exponential weighted aggregation scheme with Kullback-Leibler risk bounds.
result Sharp oracle inequality for Kullback-Leibler risk with leading constant 1 and minimax-optimal rate of aggregation.
Aggregates predictions from multiple regression models using random projections and kernel methods.
problem Combining predictions from multiple regression models to improve accuracy.
method Random projection of high-dimensional feature space, followed by kernel-based consensual aggregation.
result The aggregation scheme performs similarly to using the original high-dimensional features, with high probability.
The paper introduces a method to control false splits in tree-based data aggregation.
problem Identifying the correct subgroups to treat as a single entity in tree-based data.
method Introduces the 'false split rate' and proposes a multiple hypothesis testing algorithm for tree-based aggregation.
result The proposed algorithm controls the false split rate, demonstrating its effectiveness on stock volatility and taxi fare data.
Estimates variance function using aggregation methods in regression models.
problem Estimating variance function in regression models.
method Two-step procedure involving model selection or convex aggregation, using two independent samples.
result Consistency of the proposed method in L2 error for MS and C aggregations.
This work analyzes two methods for combining multiple binary labels in bipartite ranking.
problem Combining multiple binary labels for optimal bipartite ranking.
method Loss aggregation vs. label aggregation approaches.
result Label aggregation is preferable to loss aggregation due to label dictatorship issues.
Boosts kernel two-sample test power with multiple kernels.
problem Detecting differences between two distributions over metric spaces.
method Combining MMD estimates over multiple kernels using Mahalanobis distance.
result More powerful in detecting a wide range of alternatives in finite samples.
We present a method that "meta" classifies whether seg-ments predicted by a semantic segmentation neural networkintersect with the ground truth. For this purpose, we employ measures of dispersion for predicted pixel-wise class probability distributions, like classification entropy, that yield heat maps of the input sce…
PREMA recovers detailed data from aggregated views.
problem Reconstructing detailed data from aggregated views.
method Low-rank tensor factorization.
result Recovery guarantees under certain conditions.
Paper presents a novel approach for global feature aggregation in Graph Neural Networks.
problem Graphs lack a straightforward way to perform non-local feature aggregation like images and texts.
method Utilizes Latent Fixed Data Structure (LFDS) to aggregate feature vectors from local extraction.
result Proposed methods achieve competitive or better results with linear computational complexity.
New algorithm for aggregate inference in HMMs with continuous observations.
problem Inference in large populations with indistinguishable individuals and continuous measurements.
method Continuous observation collective forward-backward algorithm extending existing discrete case algorithm.
result Efficacy demonstrated through numerical experiments.