We introduce a new criterion to determine the order of an autoregressive model fitted to time series data. It has the benefits of the two well-known model selection techniques, the Akaike information criterion and the Bayesian information criterion. When the data is generated from a finite order autoregression, the Bay…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,742 papers · 148 categories
Trend · papers per month
48 results for “generalized information criterion”
New criterion improves predictive evaluation in weighted inference scenarios.
problem Improving predictive evaluation in scenarios with different likelihoods for estimation and evaluation.
method Developed the posterior covariance information criterion (PCIC) to handle weighted likelihood inference.
result PCIC is asymptotically unbiased for quasi-Bayesian generalization error in weighted inference.
SIC detects elbows in error curves automatically.
problem Automatic elbow detection in error curves.
method Spectral information criterion (SIC) extracts geometric features of error curves.
result SIC provides a subset of models with smaller cardinality than total possible models.
A widely applicable Bayesian information criterion (Watanabe, 2013) is applicable for both regular and singular models in the model selection problem. This criterion tends to overestimate the log marginal likelihood. We identify an overestimating term of a widely applicable Bayesian information criterion. Adjustment of…
Proposes a new hyperprior and predictive criterion for weakly informative hyperprior in relevance vector machine.
problem Capturing non-homogeneous data structure with limited kernel functions.
method Uses inverse gamma hyperprior with a shape parameter close to zero and a scale parameter not close to zero. Applies multiple kernel method with different widths. Proposes extended predictive information criterion for scale parameter selection.
result Obtains a multiple kernel relevance vector regression model with good predictive accuracy.
Study shows the corrected Akaike criterion is inadmissible for estimating Kullback-Leibler discrepancy.
problem Inadmissibility of the corrected Akaike information criterion for estimating Kullback-Leibler discrepancy.
method Loss estimation framework to demonstrate inadmissibility and provide improved estimators.
result Improved estimators of Kullback-Leibler discrepancy are provided and perform well in reduced-rank situations.
Complete criterion for VoI in multi-decision influence diagrams established.
problem Analyzing safety and fairness properties of AI systems using influence diagrams.
method Introduced ID homomorphisms and Tree of Systems to prove properties of multi-decision influence diagrams.
result First complete graphical criterion for VoI in influence diagrams with multiple decisions.
In the information-based paradigm of inference, model selection is performed by selecting the candidate model with the best estimated predictive performance. The success of this approach depends on the accuracy of the estimate of the predictive complexity. In the large-sample-size limit of a regular model, the predicti…
LS improves model selection for singular statistical models.
problem Challenges in model selection for singular statistical models.
method Integrates empirical loss from WAIC and sBIC penalty term.
result Enhanced utility for model selection without regularity constraints.
SplitWise enhances stepwise regression by adaptively encoding numeric predictors into binary features.
problem Capturing nonlinear relationships in regression models without sacrificing interpretability.
method Adaptive encoding of numeric predictors into binary features using shallow decision trees, assessed by AIC or BIC.
result Consistently produces more parsimonious and generalizable models than traditional techniques.
An Analysis of the Value of Information when Exploring Stochastic, Discrete Multi-Armed Banditscs.AI
In this paper, we propose an information-theoretic exploration strategy for stochastic, discrete multi-armed bandits that achieves optimal regret. Our strategy is based on the value of information criterion. This criterion measures the trade-off between policy information and obtainable rewards. High amounts of policy …
We test three common information criteria (IC) for selecting the order of a Hawkes process with an intensity kernel that can be expressed as a mixture of exponential terms. These processes find application in high-frequency financial data modelling. The information criteria are Akaike's information criterion (AIC), the…
The paper derives an equation linking WAIC and WBIC for singular models.
problem In singular models, conventional criteria fail due to likelihood and posterior breakdown.
method Theoretical derivation linking WAIC and WBIC.
result An asymptotic equation linking WAIC and WBIC for singular models.
A new criterion selects models in overparameterized settings.
problem Model selection for overparameterized models with more parameters than data.
method Establishes Bayesian duality and introduces the Interpolating Information Criterion.
result The Interpolating Information Criterion selects models in overparameterized settings.
A new criterion HBIC improves model selection for factor analysis with missing data.
problem Model selection for factor analysis with incomplete data.
method Proposes a novel criterion HBIC that uses actual observed information in the penalty term.
result HBIC is more accurate than BIC when missing data rates are high.
IIC provides a PAC-Bayes bound for interpolating models, revealing factors affecting generalization.
problem Theoretical challenges in understanding overparameterized models and their performance.
method PAC-Bayesian perspective applied to the Interpolating Information Criterion (IIC).
result Test error for overparameterized models achieving zero training error depends on various factors.
A new method for automatic gradient tree boosting using information theory.
problem Automatic selection of tree complexity and number in gradient boosting.
method Optimism of greedy leaf splitting procedure modeled as a Cox-Ingersoll-Ross process, leading to an information criterion for model selection.
result The method achieves significant speedups (10-1400) compared to xgboost without sacrificing predictive power.
Statistical inference is considered for variables of interest, called primary variables, when auxiliary variables are observed along with the primary variables. We consider the setting of incomplete data analysis, where some primary variables are not observed. Utilizing a parametric model of joint distribution of prima…
The paper describes a method to infer the signal-to-noise ratio in portfolio optimization.
problem Estimating the signal-to-noise ratio in portfolio optimization problems.
method A statistic similar to the Sharpe Ratio Information Criterion is used for inference.
result The method works well for reasonable sample and asset universe sizes.
This paper introduces Kernel-based Information Criterion (KIC) for model selection in regression analysis. The novel kernel-based complexity measure in KIC efficiently computes the interdependency between parameters of the model using a variable-wise variance and yields selection of better, more robust regressors. Expe…
Paper designs a penalty for model order selection using information criteria.
problem Selecting the correct model order from a set of candidate models.
method Designs a penalty for the generalized information criterion (GIC) to minimize underestimation.
result Optimal penalty minimizes underestimation while keeping overestimation below a specified level.
Factorized information criterion (FIC) is a recently developed approximation technique for the marginal log-likelihood, which provides an automatic model selection framework for a few latent variable models (LVMs) with tractable inference algorithms. This paper reconsiders FIC and fills theoretical gaps of previous FIC…
A simple application of FIC to model selectionphysics.data-an
We have recently proposed a new information-based approach to model selection, the Frequentist Information Criterion (FIC), that reconciles information-based and frequentist inference. The purpose of this current paper is to provide a simple example of the application of this criterion and a demonstration of the natura…
Factorized Information Criterion (FIC) is a recently developed information criterion, based on which a novel model selection methodology, namely Factorized Asymptotic Bayesian (FAB) Inference, has been developed and successfully applied to various hierarchical Bayesian models. The Dirichlet Process (DP) prior, and one …
New objective function improves model robustness.
problem Machine Learning robustness issues.
method Conditional Entropy Bottleneck (CEB) objective function.
result CEB models improve robust generalization.
When the in-sample Sharpe ratio is obtained by optimizing over a k-dimensional parameter space, it is a biased estimator for what can be expected on unseen data (out-of-sample). We derive (1) an unbiased estimator adjusting for both sources of bias: noise fit and estimation error. We then show (2) how to use the adjust…
A statistical model or a learning machine is called regular if the map taking a parameter to a probability distribution is one-to-one and if its Fisher information matrix is always positive definite. If otherwise, it is called singular. In regular statistical models, the Bayes free energy, which is defined by the minus…
We consider a problem of data integration. Consider determining which genes affect a disease. The genes, which we call predictor objects, can be measured in different experiments on the same individual. We address the question of finding which genes are predictors of disease by any of the experiments. Our formulation i…
We study tick-by-tick financial returns belonging to the FTSE MIB index of the Italian Stock Exchange (Borsa Italiana). We can confirm previously detected non-stationarities. However, scaling properties reported in the previous literature for other high-frequency financial data are only approximately valid. As a conseq…
Paper improves feature selection accuracy using transfer learning.
problem Improving feature selection accuracy in information criteria-based methods.
method Proposes TLCp, a transfer learning procedure based on Mallows' Cp.
result TLCp outperforms conventional Cp in accuracy and stability.
Proposes SNML for selecting word2vec Skip-gram dimensionality.
problem Selecting optimal dimensionality for word2vec Skip-gram models.
method Information criteria (AIC, BIC, SNML) applied to SG and SG Negative Sampling models.
result SNML outperforms AIC and BIC, selecting closer optimal dimensionality.
New criterion for almost-complex 4-manifolds using polyhedral decompositions.
problem Bounding the genus of surfaces in almost-complex 4-manifolds.
method Polyhedral decompositions and adjunction criterion.
result Established adjunction inequality for almost-complex 4-manifolds.
Active feature selection uses mutual information to choose fewer labels for better feature selection.
problem Selecting features with limited labeled data.
method Uses active feature selection with mutual information criterion, optimizing label selection for better feature quality.
result Algorithm selects features with higher mutual information using fewer labels than the data set size.
Safe Bayesian optimization method using information theory.
problem Optimizing unknown functions while respecting safety constraints.
method Information-theoretic exploration criterion for continuous domains.
result The method learns the value of the safe optimum up to arbitrary precision.
Fast leave-one-cluster-out cross-validation using clustered Network Information Criterion (NICc)stat.ME
Paper introduces NICc for fast cluster-based validation of prediction models.
problem Validation of prediction models on clustered data.
method Derived NICc to approximate leave-one-cluster-out deviance for standard regression models.
result NICc provides more accurate model size and variable selection, especially with strong clustering.
Zero-Flow Encodersstat.ML
Paper introduces a flow-based framework for representation learning.
problem Capturing fine-grained structural details in complex data distributions.
method Zero-flow criterion for conditional independence and a tractable loss function.
result The zero-flow criterion enables learning of sufficient information from data.
Comparing with traditional learning criteria, such as mean square error (MSE), the minimum error entropy (MEE) criterion is superior in nonlinear and non-Gaussian signal processing and machine learning. The argument of the logarithm in Renyis entropy estimator, called information potential (IP), is a popular MEE cost i…
A new tradeoff between regularization and sharpness improves model performance in overparameterized settings.
problem Improving model performance in overparameterized settings with minimum-norm interpolators.
method Proposes a regularization-sharpness tradeoff for overparameterized linear regression with an ℓ^p penalty.
result Empirical validation shows the tradeoff terms can distinguish performant linear interpolators.
Framework for generating multiple clusterings from multi-view data.
problem Challenges in finding optimal clustering criteria and handling incomplete multi-view data.
method DiMVMC framework that optimizes multiple decoder deep networks to complete data views and generate shared representations.
result DiMVMC outperforms state-of-the-art competitors in generating multiple clusterings with high diversity and quality.
Information Forestscs.LG
We describe Information Forests, an approach to classification that generalizes Random Forests by replacing the splitting criterion of non-leaf nodes from a discriminative one -- based on the entropy of the label distribution -- to a generative one -- based on maximizing the information divergence between the class-con…
Multi-class Active Learning: A Hybrid Informative and Representative Criterion Inspired Approachcs.LG
Labeling each instance in a large dataset is extremely labor- and time- consuming . One way to alleviate this problem is active learning, which aims to which discover the most valuable instances for labeling to construct a powerful classifier. Considering both informativeness and representativeness provides a promising…
Accurate model selection is a fundamental requirement for statistical analysis. In many real-world applications of graphical modelling, correct model structure identification is the ultimate objective. Standard model validation procedures such as information theoretic scores and cross validation have demonstrated poor …
A new criterion for deep active learning selects minimal labeled data points.
problem Efficiently select minimal labeled data points for deep neural networks.
method Diffuses label information over a graph of data representations to switch between exploration and refinement.
result The diffusion-based criterion outperforms existing methods in deep active learning.
The use of variable selection methods is particularly appealing in statistical problems with functional data. The obvious general criterion for variable selection is to choose the `most representative' or `most relevant' variables. However, it is also clear that a purely relevance-oriented criterion could lead to selec…
Selection of tuning parameters in bridge regression models via Bayesian information criterionstat.ME
We consider the bridge linear regression modeling, which can produce a sparse or non-sparse model. A crucial point in the model building process is the selection of adjusted parameters including a regularization parameter and a tuning parameter in bridge regression models. The choice of the adjusted parameters can be v…
We inspect Vietoris-Rips complexes of certain metric spaces using a new generalization of Bestvina-Brady discrete Morse theory. Our main result is a pair of metric criteria on , called the Morse Criterion and Link Criterion, that allow us to deduce information about the homotopy types of certain $VR_t(…
Novel approach to learn CTBNs from data with minimal interventions.
problem Learning CTBNs from time-course data with limited resources.
method Variational approximation of expected information gain for experimental design.
result Semi-analytical expression for structure and parameter learning.
Pragmatic Information Rates, Generalizations of the Kelly Criterion, and Financial Market Efficiencycs.IT
This paper is part of an ongoing investigation of "pragmatic information", defined in Weinberger (2002) as "the amount of information actually used in making a decision". Because a study of information rates led to the Noiseless and Noisy Coding Theorems, two of the most important results of Shannon's theory, we begin …