Model selection based on classical information criteria, such as BIC, is generally computationally demanding, but its properties are well studied. On the other hand, model selection based on parameter shrinkage by -type penalties is computationally efficient. In this paper we make an attempt to combine their st…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
169,181 papers · 148 categories
Trend · papers per month
48 results for “Information Criteria”
Developed criteria for selecting non-normalized models using NCE and score matching.
problem No information criteria for non-normalized models estimated by NCE or score matching.
method Developed information criteria based on discrepancy measures for non-normalized models estimated by NCE or score matching.
result The proposed criteria enable selection of the appropriate non-normalized model in a data-driven manner.
Proposes SNML for selecting word2vec Skip-gram dimensionality.
problem Selecting optimal dimensionality for word2vec Skip-gram models.
method Information criteria (AIC, BIC, SNML) applied to SG and SG Negative Sampling models.
result SNML outperforms AIC and BIC, selecting closer optimal dimensionality.
Suggests stopping criteria for feature selection using mutual information.
problem Automatic determination of optimal feature subset size and stopping criterion.
method Monitoring conditional mutual information (CMI) among groups of variables using Renyi's α-entropy.
result Easy to implement stopping criteria for feature selection.
Three LF training criteria improve neural network acoustic models without cross-entropy pre-training.
problem Improving purely sequence-trained neural network acoustic models.
method Comparison of three lattice-free discriminative training criteria (MMI, bMMI, sMBR) on LVCSR tasks.
result LF-bMMI models outperform plain LF-MMI models by 5% WER on Switchboard datasets.
This work tackles online memory selection in continual learning using information theory.
problem Online selection of a representative replay memory from data streams.
method Information-theoretic criteria (surprise, learnability) and Bayesian model for efficient computation.
result InfoRS improves robustness against data imbalance compared to reservoir sampling.
This paper introduces efficient approximations for fairness criteria in regression models.
problem Measuring fairness in real-valued outcomes (regression settings) is computationally challenging.
method Fast approximations of mutual information for independence, separation, and sufficiency fairness criteria.
result The method achieves state-of-the-art accuracy/fairness tradeoffs in real-world datasets.
New method speeds up model selection for complex scientific tasks.
problem Exhaustive model selection is computationally infeasible for large model spaces.
method Branch-and-bound algorithm with non-monotonic criteria.
result Guaranteed identification of optimal models with significant computational speedups.
Cost-effective framework for eliciting and aggregating preferences.
problem Eliciting preferences efficiently under budget constraints.
method Iterative computation of cost-effective questions using Plackett-Luce model and various information criteria.
result Carefully designed information criteria lead to more accurate predictions with fewer questions.
A new criterion selects models in overparameterized settings.
problem Model selection for overparameterized models with more parameters than data.
method Establishes Bayesian duality and introduces the Interpolating Information Criterion.
result The Interpolating Information Criterion selects models in overparameterized settings.
We test three common information criteria (IC) for selecting the order of a Hawkes process with an intensity kernel that can be expressed as a mixture of exponential terms. These processes find application in high-frequency financial data modelling. The information criteria are Akaike's information criterion (AIC), the…
Automated model assesses online health info quality using machine learning.
problem Low quality health information on the internet poses risks to patients.
method Used machine learning models, specifically hierarchical encoder attention-based neural networks (HEA) with BERT and BioBERT embeddings.
result HEA models outperform traditional models in evaluating health info quality.
The sBIC outperforms other model selection criteria in LDA topic modeling.
problem Selecting the optimal number of topics in Latent Dirichlet Allocation (LDA) models.
method Monte Carlo simulations comparing sBIC to other criteria.
result sBIC is superior for choosing the number of topics in LDA models.
Selective regression allows abstention to improve fairness criteria.
problem Selective regression can exacerbate disparities between subgroups.
method Proposes new fairness criteria and two approaches to mitigate performance disparity.
result Proposed fairness criteria ensures performance improvement for every subgroup with reduced coverage.
High-dimensional predictive models, those with more measurements than observations, require regularization to be well defined, perform well empirically, and possess theoretical guarantees. The amount of regularization, often determined by tuning parameters, is integral to achieving good performance. One can choose the …
Paper designs a penalty for model order selection using information criteria.
problem Selecting the correct model order from a set of candidate models.
method Designs a penalty for the generalized information criterion (GIC) to minimize underestimation.
result Optimal penalty minimizes underestimation while keeping overestimation below a specified level.
Framework benchmarks optimizers on multiple criteria.
problem Benchmarking optimizers across diverse test functions.
method Union-free generic depth function for partial orders/rankings.
result Identifies central and outlying rankings of optimizers.
The paper derives an equation linking WAIC and WBIC for singular models.
problem In singular models, conventional criteria fail due to likelihood and posterior breakdown.
method Theoretical derivation linking WAIC and WBIC.
result An asymptotic equation linking WAIC and WBIC for singular models.
Unified perspective unites Bayesian optimization and active learning for efficient goal-oriented optimization.
problem Efficiently optimize expensive engineering and scientific problems with limited data.
method Unified framework linking Bayesian infill criteria and active learning criteria.
result Unified approach formalizes Bayesian infill criteria and active learning criteria.
Paper discusses prediction errors for penalized regressions using GAMP and LOOCV.
problem Prediction accuracy of penalized regression models.
method Derives prediction error estimators using GAMP and LOOCV.
result Information criteria and LOOCV error estimators differ in large parameter regions.
IndiSeek learns disentangled representations by balancing independence and completeness.
problem Learning disentangled representations with mutual information in multi-modal data.
method Combines independence-enforcing objective with a reconstruction loss that bounds conditional mutual information.
result Demonstrates effectiveness on synthetic data, CITE-seq, and real-world multi-modal benchmarks.
Optimal reinsurance and investment strategies are derived under mean-variance criteria with partial information.
problem Optimal reinsurance and investment strategies for an insurance firm under mean-variance criteria with partially observable market dynamics.
method Formulated as a stochastic LQ control problem, solved using separation principle and stochastic filtering theory for partial information, and viscosity solution for full information.
result Efficient strategies and efficient frontier presented in closed forms via solutions to extended stochastic Riccati equations.
This paper proposes new methods for ALR that consider informativeness, representativeness, and diversity.
problem Efficiently label samples for regression models with limited labeled data.
method Integrates informativeness, representativeness, and diversity in pool-based sequential active learning.
result Demonstrates effectiveness of new ALR approaches on 12 datasets.
Automatically assesses the quality of online health articles.
problem Lack of automated tools to evaluate the quality of online health information.
method Data mining approach using 10 quality criteria and feature selection.
result Classifier achieved 84%-90% accuracy on 10 criteria.
A novel online feature selection method using DPP for diversity.
problem Online feature selection for diverse feature sets.
method DPP-based framework with three stages: sampling, local criteria, and global criteria.
result Demonstrated better compactness and comparable/outsuperior performance.
Estimating the dependences between random variables, and ranking them accordingly, is a prevalent problem in machine learning. Pursuing frequentist and information-theoretic approaches, we first show that the p-value and the mutual information can fail even in simplistic situations. We then propose two conditions for r…
Framework quantifies semantic similarity between groups of embeddings.
problem Quantifying semantic similarity between groups of embeddings.
method Formulates model comparison task, contrasts generative models, uses information criteria.
result Achieves competitive results in Semantic Textual Similarity tasks.
New methods ensure fairness in noisy protected groups.
problem Noisy or biased protected group information complicates fairness audits.
method Robust optimization techniques to enforce fairness on true groups.
result Robust approaches achieve better true group fairness guarantees.
We study tick-by-tick financial returns belonging to the FTSE MIB index of the Italian Stock Exchange (Borsa Italiana). We can confirm previously detected non-stationarities. However, scaling properties reported in the previous literature for other high-frequency financial data are only approximately valid. As a conseq…
LS improves model selection for singular statistical models.
problem Challenges in model selection for singular statistical models.
method Integrates empirical loss from WAIC and sBIC penalty term.
result Enhanced utility for model selection without regularity constraints.
Graph embedding provides an efficient solution for graph analysis by converting the graph into a low-dimensional space which preserves the structure information. In contrast to the graph structure data, the i.i.d. node embedding can be processed efficiently in terms of both time and space. Current semi-supervised graph…
We use variational Gaussian approximations to analyze parametric models with unknown data-generating distributions.
problem Analyzing inference and learning in parametric models with unknown or intractable data-generating distributions.
method Replica method with variational Gaussian approximation in grand canonical formalism.
result Stationarity conditions adaptively determine parameters of the trial Hamiltonian for each dataset.
SIC detects elbows in error curves automatically.
problem Automatic elbow detection in error curves.
method Spectral information criterion (SIC) extracts geometric features of error curves.
result SIC provides a subset of models with smaller cardinality than total possible models.
Study evaluates various regularization methods for electricity price forecasting.
problem Improving accuracy of electricity price predictions.
method Applied ten different penalty functions to two model structures in two electricity markets.
result LQ and elastic net consistently produce more accurate forecasts than other regularization types.
Proposes a model selection method for t-SNE perplexity.
problem Manual tuning of t-SNE perplexity is difficult and requires expertise.
method Develops a model selection objective for t-SNE perplexity that minimizes extra computation.
result Perplexity settings found by the approach are consistent with human expert preferences.
Detecting and recovering labels in binomial logistic mixtures is challenging due to an information gap.
problem Detecting and recovering labels in binomial logistic mixtures
method Propose two feasibility-aware inference procedures
result Avoid misleading component selections and improve label probability calibration
New algorithms improve experimental design efficiency and approximation quality.
problem Finding optimal subset of vectors for expensive measurements.
method Bayesian experimental design using determinantal point processes.
result Developed efficient algorithms for optimal design under multiple criteria.
Enhances Federated Learning by prioritizing device contributions and adjusting aggregation parameters.
problem Data privacy and security in distributed learning systems.
method Integrates multiple criteria for device contribution, uses prioritized aggregation, and adapts aggregation parameters online.
result The proposed approach outperforms standard Federated Learning methods in distributed learning tasks.
New GIC improves model selection for structured sparse models.
problem Model selection and regularization in high-dimensional scenarios.
method Proposes a new Generalized Information Criteria (GIC) for structured sparse models.
result Obtains non-asymptotic model selection bounds and sufficient conditions for model selection consistency.
MOSAIC selects few informative exemplars from high-dimensional data with non-linear structures.
problem Representative selection from high-dimensional data with non-linear structures.
method MOSAIC uses a multi-criteria approach with a quadratic formulation to maximize global representation power, diversity, and outlier detection.
result MOSAIC maximizes data coverage in a transformed space and achieves robustness to various outlier types.
Meta-analysis improves personalized treatment rules across multiple sites.
problem Lack of generalizability in learning individualized treatment rules across different medical sites.
method Developed a method for individual-level meta-analysis of ITRs, borrowing sign-coherency information between sites.
result Jointly learned site-specific ITRs with improved generalizability.
vsOED optimizes experiment design with reinforcement learning for Bayesian models.
problem Optimizing the sequence of experiments in Bayesian models for efficient data collection.
method Reinforcement learning with variational posterior approximations to optimize design policy.
result vsOED achieves superior sample efficiency compared to existing methods.
Paper tackles item cold-start problem using active learning and attributes.
problem Cold-start issue in recommender systems for new items.
method Proposes a novel recommendation scheme combining active learning and item attributes.
result Experimental results show superior performance compared to traditional methods.
Batch Active Learning uses derivative information for Gaussian Process regression.
problem Efficiently selecting data batches in Gaussian Process regression models.
method Proposes using the predictive covariance matrix to select data batches, exploiting full correlation.
result Demonstrates the effectiveness of incorporating derivative information across diverse applications.
In this work we present a review of the state of the art of information theoretic feature selection methods. The concepts of feature relevance, redundance and complementarity (synergy) are clearly defined, as well as Markov blanket. The problem of optimal feature selection is defined. A unifying theoretical framework i…
New method improves cross-validation for sparse reduced rank regression models.
problem Inconsistent parameter selection in cross-validation for high-dimensional data.
method Proposes cross-validation of projection-selection patterns to avoid inconsistency issues.
result Develops new scale-free information criteria for minimax optimal error rate.
The article explains how to use Mixture-of-Experts models for complex data.
problem Modeling complex data generating processes (DGPs).
method Constructing Mixture-of-Experts (MoE) models using maximum quasi-likelihood (MQL) estimators and blockwise-MM algorithms.
result MQL estimators are consistent and asymptotically normal under certain conditions.
Bayesian active learning improves holistic educational assessments.
problem Gap between holistic CJ and criterion-based rubrics in education.
method Extends Bayesian CJ to handle multiple LO components, using entropy-based active learning.
result Enhanced predictive rankings with uncertainty estimates and quantified assessor agreement.