Unified perspective unites Bayesian optimization and active learning for efficient goal-oriented optimization.
problem Efficiently optimize expensive engineering and scientific problems with limited data.
method Unified framework linking Bayesian infill criteria and active learning criteria.
result Unified approach formalizes Bayesian infill criteria and active learning criteria.
This work tackles online memory selection in continual learning using information theory.
problem Online selection of a representative replay memory from data streams.
method Information-theoretic criteria (surprise, learnability) and Bayesian model for efficient computation.
result InfoRS improves robustness against data imbalance compared to reservoir sampling.
A new criterion selects models in overparameterized settings.
problem Model selection for overparameterized models with more parameters than data.
method Establishes Bayesian duality and introduces the Interpolating Information Criterion.
result The Interpolating Information Criterion selects models in overparameterized settings.
The sBIC outperforms other model selection criteria in LDA topic modeling.
problem Selecting the optimal number of topics in Latent Dirichlet Allocation (LDA) models.
method Monte Carlo simulations comparing sBIC to other criteria.
result sBIC is superior for choosing the number of topics in LDA models.
New algorithms improve experimental design efficiency and approximation quality.
problem Finding optimal subset of vectors for expensive measurements.
method Bayesian experimental design using determinantal point processes.
result Developed efficient algorithms for optimal design under multiple criteria.
The paper derives an equation linking WAIC and WBIC for singular models.
problem In singular models, conventional criteria fail due to likelihood and posterior breakdown.
method Theoretical derivation linking WAIC and WBIC.
result An asymptotic equation linking WAIC and WBIC for singular models.
Bayesian active learning improves holistic educational assessments.
problem Gap between holistic CJ and criterion-based rubrics in education.
method Extends Bayesian CJ to handle multiple LO components, using entropy-based active learning.
result Enhanced predictive rankings with uncertainty estimates and quantified assessor agreement.
Proposes SNML for selecting word2vec Skip-gram dimensionality.
problem Selecting optimal dimensionality for word2vec Skip-gram models.
method Information criteria (AIC, BIC, SNML) applied to SG and SG Negative Sampling models.
result SNML outperforms AIC and BIC, selecting closer optimal dimensionality.
vsOED optimizes experiment design with reinforcement learning for Bayesian models.
problem Optimizing the sequence of experiments in Bayesian models for efficient data collection.
method Reinforcement learning with variational posterior approximations to optimize design policy.
result vsOED achieves superior sample efficiency compared to existing methods.
We test three common information criteria (IC) for selecting the order of a Hawkes process with an intensity kernel that can be expressed as a mixture of exponential terms. These processes find application in high-frequency financial data modelling. The information criteria are Akaike's information criterion (AIC), the…
Study evaluates various regularization methods for electricity price forecasting.
problem Improving accuracy of electricity price predictions.
method Applied ten different penalty functions to two model structures in two electricity markets.
result LQ and elastic net consistently produce more accurate forecasts than other regularization types.
Proposes a model selection method for t-SNE perplexity.
problem Manual tuning of t-SNE perplexity is difficult and requires expertise.
method Develops a model selection objective for t-SNE perplexity that minimizes extra computation.
result Perplexity settings found by the approach are consistent with human expert preferences.
New method speeds up model selection for complex scientific tasks.
problem Exhaustive model selection is computationally infeasible for large model spaces.
method Branch-and-bound algorithm with non-monotonic criteria.
result Guaranteed identification of optimal models with significant computational speedups.
LS improves model selection for singular statistical models.
problem Challenges in model selection for singular statistical models.
method Integrates empirical loss from WAIC and sBIC penalty term.
result Enhanced utility for model selection without regularity constraints.
Estimating the dependences between random variables, and ranking them accordingly, is a prevalent problem in machine learning. Pursuing frequentist and information-theoretic approaches, we first show that the p-value and the mutual information can fail even in simplistic situations. We then propose two conditions for r…
Expands Bayesian experiment design framework to account for model discrepancies.
problem Model misspecification in Bayesian optimal experiment design.
method Introduces Expected General Information Gain and Expected Discriminatory Information criteria.
result Demonstrates improved robustness and detection capabilities in experiment design.
Framework quantifies semantic similarity between groups of embeddings.
problem Quantifying semantic similarity between groups of embeddings.
method Formulates model comparison task, contrasts generative models, uses information criteria.
result Achieves competitive results in Semantic Textual Similarity tasks.
Bayesian symbolic regression automates model discovery from data.
problem Learning closed-form mathematical models from data using heuristic methods.
method Probabilistic approach to symbolic regression, connecting to information theory and statistical physics.
result Probabilistic approach provides model plausibility and performance guarantees.
Paper improves feature selection accuracy using transfer learning.
problem Improving feature selection accuracy in information criteria-based methods.
method Proposes TLCp, a transfer learning procedure based on Mallows' Cp.
result TLCp outperforms conventional Cp in accuracy and stability.
When performing regression or classification, we are interested in the conditional probability distribution for an outcome or class variable Y given a set of explanatoryor input variables X. We consider Bayesian models for this task. In particular, we examine a special class of models, which we call Bayesian regression…
Bayesian method discovers PDEs with variable coefficients robustly.
problem Discovering PDEs from noisy data is challenging.
method Bayesian sparse learning with tBGL-SS and Gibbs sampler.
result Method enhances robustness and model selection criteria.
Bayesian optimization has been proposed as a practical and efficient tool through which to tune parameters in many difficult settings. Recently, such techniques have been combined with real-time fMRI to propose a novel framework which turns on its head the conventional functional neuroimaging approach. This closed-loop…
Novel path planning improves UAV detection of extreme anomalies.
problem Improper criteria for UAV anomaly detection.
method Mathematical criteria guiding UAV towards strong anomalies.
result Proposed approach outperforms traditional methods in various applications.
AutoGMM automates Gaussian mixture modeling in Python.
problem Automatic clustering of complex data with uncertainty-aware grouping.
method Strategic initialization using an agglomerative Mahalanobis heuristic, parallelized model selection by information criteria.
result Strong out-of-the-box performance on classic benchmarks and real datasets.
Automates model selection for GLMs using optimization.
problem Automating model selection for generalized linear models.
method Mixed-integer conic optimization for feature subset selection and model constraints.
result Optimization of AIC and BIC criteria with multicollinearity constraints.
This paper optimizes stock portfolios considering ESG criteria using Bayesian optimization.
problem Optimizing financial investments while incorporating ESG criteria.
method Bayesian optimization to maximize stock portfolio performance under ESG constraints.
result A scalable approach to optimize stock portfolios that balance financial performance and ESG compliance.
We study tick-by-tick financial returns belonging to the FTSE MIB index of the Italian Stock Exchange (Borsa Italiana). We can confirm previously detected non-stationarities. However, scaling properties reported in the previous literature for other high-frequency financial data are only approximately valid. As a conseq…
Bayesian BIC for multi-trial data improves VAR model order selection.
problem Optimal VAR model order selection for multi-trial event-based data.
method Derive and apply Bayesian Information Criterion (BIC) for multi-trial ensemble data.
result Multi-trial BIC successfully recovers real model order and estimates small model order.
The principle of peer review is central to the evaluation of research, by ensuring that only high-quality items are funded or published. But peer review has also received criticism, as the selection of reviewers may introduce biases in the system. In 2014, the organizers of the ``Neural Information Processing Systems\r…
Paper proposes a new sparse VAR model for high-dimensional time series.
problem Non-identifiability, computational intractability, and difficulty of interpretation for high-dimensional time series.
method Sparse infinite-order VAR model with ℓ1-regularized estimation methods. result Greater statistical efficiency and interpretability achieved with little loss of temporal information.
A new criterion HBIC improves model selection for factor analysis with missing data.
problem Model selection for factor analysis with incomplete data.
method Proposes a novel criterion HBIC that uses actual observed information in the penalty term.
result HBIC is more accurate than BIC when missing data rates are high.
Detecting and recovering labels in binomial logistic mixtures is challenging due to an information gap.
problem Detecting and recovering labels in binomial logistic mixtures
method Propose two feasibility-aware inference procedures
result Avoid misleading component selections and improve label probability calibration
SIC detects elbows in error curves automatically.
problem Automatic elbow detection in error curves.
method Spectral information criterion (SIC) extracts geometric features of error curves.
result SIC provides a subset of models with smaller cardinality than total possible models.
A government has to finance a risk for its population. It shares the charges among the population with a fixed scale based on economic criteria. Various organisms have to collect and to redistribute fairly the subsidies. Under these conditions, when the size of the organisms is varied, the distribution's laws of the cr…
We introduce a new criterion to determine the order of an autoregressive model fitted to time series data. It has the benefits of the two well-known model selection techniques, the Akaike information criterion and the Bayesian information criterion. When the data is generated from a finite order autoregression, the Bay…
This paper treats prediction markets as Bayesian inverse problems to quantify uncertainty and identify event outcomes.
problem Uncertainty and identifiability in prediction market outcomes from price-volume histories.
method Formulates prediction markets as Bayesian inverse problems, introduces a log-odds observation model, and derives posterior uncertainty quantification and identifiability criteria.
result Explicit diagnostics for informative and stable inference regimes, and validation through synthetic data experiments.
The article explains how to use Mixture-of-Experts models for complex data.
problem Modeling complex data generating processes (DGPs).
method Constructing Mixture-of-Experts (MoE) models using maximum quasi-likelihood (MQL) estimators and blockwise-MM algorithms.
result MQL estimators are consistent and asymptotically normal under certain conditions.
The ultimate goal of optimization is to find the minimizer of a target function.However, typical criteria for active optimization often ignore the uncertainty about the minimizer. We propose a novel criterion for global optimization and an associated sequential active learning strategy using Gaussian processes.Our crit…
Introduces new metrics to measure performance of deep Bayesian neural networks.
problem Lack of specific criteria to measure performance of deep Bayesian neural networks.
method Proposes several metrics including model calibration, data rejection ability, and uncertainty divergence.
result Introduces more specific criteria for measuring deep Bayesian neural network performance.
Model selection based on classical information criteria, such as BIC, is generally computationally demanding, but its properties are well studied. On the other hand, model selection based on parameter shrinkage by ℓ1-type penalties is computationally efficient. In this paper we make an attempt to combine their st…
Developed criteria for selecting non-normalized models using NCE and score matching.
problem No information criteria for non-normalized models estimated by NCE or score matching.
method Developed information criteria based on discrepancy measures for non-normalized models estimated by NCE or score matching.
result The proposed criteria enable selection of the appropriate non-normalized model in a data-driven manner.
The paper discusses the impact of prior densities on Bayesian model selection.
problem The sensitivity of marginal likelihood to prior choice in Bayesian model selection.
method Analyzes the role of prior densities in model selection, discusses improper priors, and proposes solutions.
result Marginal likelihood can be sensitive to prior choice, but improper priors can still be used with caution.
Suggests stopping criteria for feature selection using mutual information.
problem Automatic determination of optimal feature subset size and stopping criterion.
method Monitoring conditional mutual information (CMI) among groups of variables using Renyi's α-entropy.
result Easy to implement stopping criteria for feature selection.
One of the most popular copulas for modeling dependence structures is t-copula. Recently the grouped t-copula was generalized to allow each group to have one member only, so that a priori grouping is not required and the dependence modeling is more flexible. This paper describes a Markov chain Monte Carlo (MCMC) method…
Bayesian method identifies causal sets across populations without graph knowledge.
problem Transporting causal information across populations without causal graph knowledge.
method Combines observational and experimental data to identify s-admissible backdoor sets.
result Proves asymptotic convergence and corrects transportability bias in simulations.
A new method improves robustness and efficiency of Bayesian LOO-CV.
problem Computational expense and unreliability of classical LOO-CV in high-dimensional Bayesian models.
method Proposes a mixture estimator to compute Bayesian LOO-CV criteria with finite asymptotic variance.
result Improved robustness and efficiency in high-dimensional problems.
The paper analyzes and proposes a new stopping criterion for recursive Bayesian classification.
problem Limitations of conventional stopping criteria in recursive Bayesian classification.
method Geometric interpretation of state posterior progression and analysis of conventional criteria.
result Proposes a new stopping criterion to overcome limitations of conventional methods.
We exhibit a strong link between frequentist PAC-Bayesian risk bounds and the Bayesian marginal likelihood. That is, for the negative log-likelihood loss function, we show that the minimization of PAC-Bayesian generalization risk bounds maximizes the Bayesian marginal likelihood. This provides an alternative explanatio…