The paper tackles attributing forecast gaps in complex model suites.
problem Attributing forecast gaps to individual component models in complex model suites.
method Formalized walk analysis, adapted LMDI and Shapley value approaches.
result Developed efficient formulas for gap attribution in practical portfolio-scale examples.
Attributing forecast gaps to component models in complex model suites
problem Attributing forecast gaps between model-suite forecasts and realized outcomes
method Formalizing walk analysis and adapting order-independent attribution frameworks
result Deriving efficient formulas for elementwise and vectorized gap attribution
Hyperparameter optimization and neural architecture search can become prohibitively expensive for regular black-box Bayesian optimization because the training and evaluation of a single model can easily take several hours. To overcome this, we introduce a comprehensive tool suite for effective multi-fidelity Bayesian o…
Paper introduces OARF benchmark suite for federated learning systems.
problem Limited diversity in federated learning benchmarks.
method Characterizes OARF benchmark suite with diverse data and applications.
result Federated learning can effectively increase end-to-end throughput.
CURL uses contrastive learning to improve reinforcement learning performance.
problem Improving reinforcement learning performance on complex tasks.
method Contrastive learning to extract high-level features from raw pixels, followed by off-policy control.
result CURL outperforms prior methods on DeepMind Control Suite and Atari Games.
New method QMLE performs well in complex action spaces without policy gradients.
problem Why policy gradients outperform action-value methods in complex action spaces.
method QMLE framework for action-value methods based on three principles.
result QMLE performs comparably to policy gradient methods in complex action spaces.
Study introduces a benchmark suite for evaluating neural MI estimators on real-world unstructured datasets.
problem Lack of comprehensive evaluation methods for neural MI estimators on real-world unstructured datasets.
method Developed a benchmark suite using same-class sampling and a binary symmetric channel trick.
result Showed accurate manipulation of true MI values of real-world datasets.
LLM Pro Finance Suite enhances financial NLP with instruction-tuned models.
problem Limited NLP capabilities for financial tasks in generalist models.
method Instruction-tuned large language models fine-tuned on financial data.
result Consistent improvement over state-of-the-art baselines in finance tasks.
This paper evaluates how well neural models can solve complex tasks by breaking them into simpler ones.
problem Measuring neural models' ability to solve complex tasks by breaking them into simpler subtasks.
method Characterized axes of compositional generalization, introduced a benchmark suite of tasks, and improved Transformer models' attention mechanisms.
result Modified Transformer models generally perform better than natural baselines in solving complex tasks, but challenges remain.
Study shows LLC correlates with neural network compressibility.
problem Evaluating limits of neural network compression.
method Extended minimum description length principle using singular learning theory.
result Complexity estimates based on LLC are linearly correlated with compressibility.
Machine learning research depends on objectively interpretable, comparable, and reproducible algorithm benchmarks. We advocate the use of curated, comprehensive suites of machine learning tasks to standardize the setup, execution, and reporting of benchmarks. We enable this through software tools that help to create an…
We study learning problems in which the underlying class is a bounded subset of Lp and the target Y belongs to Lp. Previously, minimax sample complexity estimates were known under such boundedness assumptions only when p=∞. We present a sharp sample complexity estimate that holds for any p>4. It is b…
New probabilistic complexity measures for linear and kernel methods.
problem Limitations of linear and kernel methods in machine learning.
method Introducing approximate notions of dimensional and margin complexity.
result Approximate complexity measures are both sufficient and necessary for learning.
Many domains of science have developed complex simulations to describe phenomena of interest. While these simulations provide high-fidelity models, they are poorly suited for inference and lead to challenging inverse problems. We review the rapidly developing field of simulation-based inference and identify the forces …
DUNs improve active learning by dynamically adjusting model complexity.
problem Active learning datasets grow, leading to model bias or overfitting.
method Depth Uncertainty Networks (DUNs) infer model depth for flexibility and complexity.
result DUNs outperform other BNN variants on active learning tasks, reducing overfitting.
MinShap improves feature selection accuracy and stability in complex models.
problem Challenging feature selection in unknown non-linear relationships.
method Combines Shapley values with a modified approach to feature selection.
result MinShap outperforms state-of-the-art algorithms in accuracy and stability.
PINNs solve differential geometry problems in complex shapes.
problem Solving differential geometry problems in complex shapes.
method Training neural networks with loss functions inspired by differential conditions.
result PINNs are effective for differential geometry problems.
New framework minimizes model complexity for improved few-shot learning.
problem Empirical benefits of pre-training scale with data size but lack theoretical explanation.
method Complexity Minimization framework for meta-representation learning.
result Theoretical analysis shows error rate improves with more meta-training data.
Given an n-manifold M and an n-category C, we define a chain complex (the "blob complex") B_*(M;C). The blob complex can be thought of as a derived category analogue of the Hilbert space of a TQFT, and as a generalization of Hochschild homology to n-categories and n-manifolds. It enjoys a number of nice formal properti…
New AMP algorithms for rotationally invariant models with reduced complexity.
problem Signal estimation in generalized linear models with arbitrary spectral design matrices.
method Rotationally invariant approximate message passing (AMP) algorithms.
result Performance close to Vector AMP with significantly lower complexity.
FLBench automates federated learning benchmarking.
problem Manual dataset partitioning fails to simulate real-world isolated data islands.
method Develops a federated learning benchmark suite with three domains.
result Automates evaluation of federated learning systems and algorithms.
New models optimize quotes for automated market makers considering various price dynamics and demand variability.
problem Optimizing quotes for automated market makers in volatile price environments.
method Advanced models incorporating stochastic volatility, jumps, Hawkes processes, and Markov-modulated Poisson processes.
result Optimal quotes can be computed using numerical methods tailored to each model.
Torelli space (in genus g) is the moduli space of compact Riemann surfaces of genus g together with a symplectic basis of their first homology group. It is the quotient of the genus g Teichmuller space by the Torelli group T_g and is a model of the classifying space of T_g. It is known that almost all T_g are not finit…
Unified framework identifies nonlinear systems using characteristic curves and neural networks.
problem Balancing interpretability and flexibility in nonlinear system identification.
method Combines differential equation structure with neural networks, using characteristic curves as modular components.
result NN-CC approach outperforms other methods in complex nonlinear systems.
Bayesian Optimisation (BO) refers to a suite of techniques for global optimisation of expensive black box functions, which use introspective Bayesian models of the function to efficiently search for the optimum. While BO has been applied successfully in many applications, modern optimisation tasks usher in new challeng…
MLPerf, an emerging machine learning benchmark suite strives to cover a broad range of applications of machine learning. We present a study on its characteristics and how the MLPerf benchmarks differ from some of the previous deep learning benchmarks like DAWNBench and DeepBench. We find that application benchmarks suc…
Study accelerates NAS research with a large dataset of ZC proxies.
problem Speeding up neural architecture search with ZC proxies.
method Created NAS-Bench-Suite, evaluated 13 ZC proxies across 28 tasks, and provided a unified codebase.
result ZC proxies capture substantial complementary information and can improve NAS algorithm performance.
New neural networks model complex phenomena with fewer parameters.
problem Challenges in studying higher-order interactions in neural networks.
method Introducing curved neural networks using the maximum entropy principle.
result Curved neural networks accelerate memory retrieval and exhibit explosive phase transitions.
Bayesian nonparametric models improve OOD detection, especially with complex covariance structures.
problem Improving out-of-distribution detection methods, especially in complex scenarios.
method Proposes Bayesian nonparametric mixture models with hierarchical priors that generalize the Mahalanobis distance score.
result Bayesian nonparametric methods outperform existing OOD methods, especially in complex scenarios.
The past decade has seen substantial work on the use of non-negative matrix factorization and its probabilistic counterparts for audio source separation. Although able to capture audio spectral structure well, these models neglect the non-stationarity and temporal dynamics that are important properties of audio. The re…
Feature attribution methods, or saliency maps, are one of the most popular approaches for explaining the decisions of complex machine learning models such as deep neural networks. In this study, we propose a stochastic optimization approach for the perturbation-based feature attribution method. While the original optim…
This paper describes HyperStream, a large-scale, flexible and robust software package, written in the Python language, for processing streaming data with workflow creation capabilities. HyperStream overcomes the limitations of other computational engines and provides high-level interfaces to execute complex nesting, fu…
FinSurvival provides a large-scale financial survival modeling benchmark.
problem Lack of large-scale, realistic, and freely available datasets for benchmarking AI survival models.
method Derived 16 survival modeling tasks from cryptocurrency lending data using an automated pipeline.
result Demonstrated that existing AI survival models are not well-suited for these challenging tasks.
Generative model uses ODEs and RKHSs for measure matching.
problem Minimum divergence generative modeling and sampling.
method Diffeomorphic matching and image registration principles applied to ODEs and RKHSs.
result Theoretical error bounds and extensive numerical experiments demonstrate the method's properties and applicability.
This paper solves the multiple reference model problem in RLHF with exact solutions and sample complexity guarantees.
problem Limitations of single reference models in aligning LLMs with human feedback.
method Integrates multiple reference models into RLHF frameworks, addressing theoretical challenges with exact solutions and sample complexity guarantees.
result First exact solution to the multiple reference model problem in reverse KL-regularized RLHF.
Study of Betti numbers in prodsimplicial complexes for directed graphs, focusing on DNA recombination.
problem Analyzing Betti numbers in directed graphs for DNA recombination.
method Custom prodsimplicial complexes for acyclic directed graphs, investigating Betti numbers.
result Investigated Betti numbers and cycles in prodsimplicial complexes for DNA recombination.
We introduce two numerical conjugacy invariants for dynamical systems -- the complexity and weak complexity indices -- which are well-suited for the study of "completely integrable" Hamiltonian systems. These invariants can be seen as "slow entropies", they describe the polynomial growth rate of the number of balls (fo…
This paper introduces the Partition Tree Weighting technique, an efficient meta-algorithm for piecewise stationary sources. The technique works by performing Bayesian model averaging over a large class of possible partitions of the data into locally stationary segments. It uses a prior, closely related to the Context T…
Conformal prediction offers distribution-free inference for complex models.
problem Traditional predictive inference methods are limited by assumptions about data distributions and model details.
method Conformal prediction uses symmetry assumptions and treats learning algorithms as black boxes.
result Conformal prediction provides exact finite-sample guarantees, even under limited assumptions.
Tensor2Tensor is a library for deep learning models that is well-suited for neural machine translation and includes the reference implementation of the state-of-the-art Transformer model.
A normalizing flow models a complex probability density as an invertible transformation of a simple density. The invertibility means that we can evaluate densities and generate samples from a flow. In practice, autoregressive flow-based models are slow to invert, making either density estimation or sample generation sl…
FinTSBridge evaluates financial time series models for asset pricing.
problem Lack of effective evaluation methods for financial time series models.
method Developed FinTSBridge suite with new metrics and tasks.
result Showcased new metrics for financial time series models.
New methods make Bayesian inference feasible for complex cognitive models.
problem Bayesian inference is hard for complex, unknown models.
method Amortized neural network estimators for simulation-based inference.
result Effort in training networks amortizes over multiple uses.
Frugal Flows learn complex data and infer marginal causal effects.
problem Challenges in estimating marginal causal effects from complex data.
method Frugal Flows use normalizing flows to flexibly learn data and infer causal quantities.
result Frugal Flows can generate synthetic data that closely matches real-world data and exactly parameterize causal quantities.
Develops a transparent surrogate model for complex data.
problem Balancing accuracy and transparency in complex decision-making models.
method Partial dependence effects for feature engineering, smart segmentation, and GLM fitting.
result The maidrr GLM closely approximates a black box model and outperforms benchmarks.
The learnability of different neural architectures can be characterized directly by computable measures of data complexity. In this paper, we reframe the problem of architecture selection as understanding how data determines the most expressive and generalizable architectures suited to that data, beyond inductive bias.…
Paper establishes baselines for offline RL from visual observations.
problem Challenges in offline reinforcement learning from visual observations with continuous action spaces.
method Simple baselines and benchmarking tasks for offline RL from visual observations.
result Simple modifications to existing online RL algorithms outperform existing offline RL methods.
Stabilizing black-box algorithms through task-oriented randomization
problem Ensuring stability of black-box models
method Task-oriented randomization
result Established rigorous theoretical foundations and demonstrated effectiveness through simulations and real-world applications