Introduces relative information gain for improving Gaussian process regression rates.
problem Improving the sample complexity of estimating or maximizing unknown functions.
method Introduces relative information gain, interpolates between effective dimension and information gain, and proves PAC-Bayesian bounds.
result Obtains minimax-optimal rates of convergence through the relative information gain.
Active inference minimizes expected free energy for optimal behavior.
problem Understanding and optimizing behavior in complex systems.
method Combines Bayesian decision theory, optimal Bayesian design, and the free energy principle.
result Active inference emerges as a unified framework for information-seeking, utility maximization, and goal-directed behavior.
Study calculates arbitrage gains between two markets with limited liquidity.
problem Arbitrage gains between markets with limited liquidity.
method Modeling arbitrage gains using relative liquidity and trading volume, assuming quadratic trading costs.
result Arbitrage gains depend on relative liquidity and trading volume between markets.
Relative to the large literature on upper bounds on complexity of convex optimization, lesser attention has been paid to the fundamental hardness of these problems. Given the extensive use of convex optimization in machine learning and statistics, gaining an understanding of these complexity-theoretic issues is importa…
Eluder dimension and information gain are equivalent for reproducing kernel Hilbert spaces.
problem Complexity measures in bandit and reinforcement learning.
method Equivalence of eluder dimension and information gain for reproducing kernel Hilbert spaces.
result Eluder dimension and information gain are equivalent for reproducing kernel Hilbert spaces.
New measures for causal entropy and information gain studied.
problem Quantifying causal relationships in machine learning.
method Formal study of causal entropy and information gain.
result Established fundamental properties and relationships.
AV-ASR system improves speech recognition with visual context.
problem Improving speech recognition accuracy with visual information.
method Transformer-based architecture with multiresolution and multimodal training.
result Multiresolution training speeds up convergence and improves WER by 18%.
Hypermodels improve exploration efficiency and accuracy.
problem Efficiently approximating Thompson sampling with large ensembles.
method Introducing hypermodels as a generalization of ensembles, including linear and neural network hypermodels.
result Hypermodels enable more accurate exploration and performance gains over Thompson sampling.
Proposes a new method to enhance neural learning by maximizing information gain.
problem Improving neural learning by selecting key variables to maximize information gain.
method Adaptive Ensemble Kalman Filter to quantify uncertainty and maximize information gain.
result The proposed method enables the neural network to learn more effectively from stochastic systems.
Decision trees algorithms use a gain function to select the best split during the tree's induction. This function is crucial to obtain trees with high predictive accuracy. Some gain functions can suffer from a bias when it compares splits of different arities. Quinlan proposed a gain ratio in C4.5's information gain fu…
PC-GAIN improves GAIN's imputation by incorporating category information.
problem Missing data in incomplete datasets.
method Pre-training with pseudo-labels and incorporating an auxiliary classifier into GAIN.
result Significantly improved imputation quality compared to GAIN.
This paper extends SLS controllers to two stocks, proving the RPE property with cross-coupling.
problem Extending SLS controllers to two stocks without exploiting correlations.
method Developed a novel architecture for cross-coupling two SLS controllers, derived a closed-form expected value, and proved the RPE property.
result Guaranteed RPE property with cross-coupling for a large class of stock dynamics.
ICYM2I corrects missingness bias in multimodal learning.
problem Missingness patterns between source and target environments affect multimodal learning performance.
method ICYM2I uses inverse probability weighting to correct missingness bias in predictive performance and information gain.
result ICYM2I improves multimodal learning performance by accounting for missingness.
New framework analyzes regret in guided diffusion for optimizing structured inputs.
problem Understanding regret behavior in guided-diffusion black-box optimization for structured design problems.
method Developed a certificate-based expected simple-regret framework that avoids assumptions breaking down in modern diffusion BO pipelines.
result Explains how exponential and polynomial convergence can arise from mass lift in near-optimal designs.
A new method models financial returns by separating sign and magnitude, improving forecasting accuracy.
problem Capturing nonlinear predictability in financial return dynamics.
method Decomposes returns into sign and magnitude components, using a joint distribution model.
result Significantly outperforms traditional linear models in forecasting U.S. stock market returns.
Model-based clustering is widely-used in a variety of application areas. However, fundamental concerns remain about robustness. In particular, results can be sensitive to the choice of kernel representing the within-cluster data density. Leveraging on properties of pairwise differences between data points, we propose a…
Active inference selects actions to maximize information gain, aiding structure learning.
problem Learning the structure of underlying world models.
method Active inference selects actions based on expected free energy, which includes information gain and value.
result Actions that maximize information gain help disambiguate among alternative models.
Proposes a framework to handle missing data in traffic forecasting with sensor blackouts.
problem Missing data in traffic forecasting due to sensor blackouts, especially when correlated with traffic conditions.
method Latent state-space framework that models traffic dynamics and sensor dropout probabilities.
result Improves traffic forecasting by reducing blackout imputation RMSE from 7.02 to 4.23, with MNAR modeling providing additional gains.
We present and study a Minority Game based model of a financial market where adaptive agents -- the speculators -- interact with deterministic agents -- called producers. Speculators trade only if they detect predictable patterns which grant them a positive gain. Indeed the average number of active speculators grows wi…
The article calculates asymptotic expansions for quantum invariants from surgeries on Whitehead link components.
problem Calculating quantum invariants for 3-manifolds resulting from surgeries on Whitehead link components.
method Asymptotic expansion of relative Reshetikhin-Turaev and Turaev-Viro invariants.
result Asymptotic formulas for both invariants are derived.
Transformers for binary decisions are sensitive to evidence order, leading to unreliable outcomes.
problem Order sensitivity in Transformers for binary decisions leads to unreliable outcomes.
method Formalized an expectation-realization gap and developed QMV and EDFL bounds.
result Uniform permutation mixtures reduce dispersion and improve reliability.
Proposes EPIG for active learning to improve predictive performance.
problem Suboptimal predictive performance of traditional active learning methods.
method Introduces EPIG, a new acquisition function measuring information gain in the space of predictions.
result EPIG leads to stronger predictive performance compared to BALD across various datasets and models.
Heuristic optimisers which search for an optimal configuration of variables relative to an objective function often get stuck in local optima where the algorithm is unable to find further improvement. The standard approach to circumvent this problem involves periodically restarting the algorithm from random initial con…
We study the problem of online path learning with non-additive gains, which is a central problem appearing in several applications, including ensemble structured prediction. We present new online algorithms for path learning with non-additive count-based gains for the three settings of full information, semi-bandit and…
Expands Bayesian experiment design framework to account for model discrepancies.
problem Model misspecification in Bayesian optimal experiment design.
method Introduces Expected General Information Gain and Expected Discriminatory Information criteria.
result Demonstrates improved robustness and detection capabilities in experiment design.
Algorithm refines matrix ratings using hierarchical graph clustering.
problem Matrix completion with side information from social graphs.
method Hierarchical graph clustering followed by iterative refinement of matrix ratings.
result Achieves optimal sample complexity for matrix completion.
Active sampling algorithm improves accuracy of inferred scores from pairwise comparisons.
problem Inference of accurate scores from time-consuming pairwise comparisons.
method Approximate message passing and expected information gain maximization.
result ASAP offers the highest accuracy of inferred scores compared to existing methods.
New method improves robustness of Bayesian experimental design.
problem Bayesian experimental design's sensitivity to prior distribution changes.
method Introduces robust expected information gain (REIG) and uses KL-divergence ambiguity sets.
result REIG stabilizes sampling-based EIG estimation and compensates for prior variability.
FisherSFT selects informative examples to fine-tune LLMs efficiently.
problem Adapting large language models to new domains efficiently.
method Selects examples maximizing information gain using Hessian of log-likelihood.
result Empirically demonstrates improved performance with reduced computational cost.
Maximizes robustness in Bayesian experimental design under model uncertainty.
problem Brittleness of Bayesian experimental design under model misspecification.
method Formulates as a max--min game, uses Sibson's α-MI, and adopts PAC-Bayes framework.
result Establishes robust belief update and conditional information gain measure.
New method corrects active learning for distribution shifts and outliers.
problem Conventional active learning methods fail to account for test-time distribution.
method JEPIG, a hybrid of BALD and EPIG, maximizes expected predictive information gain.
result JEPIG outperforms conventional methods in active learning with distribution shifts.
This paper describes a general framework for learning Higher-Order Network Embeddings (HONE) from graph data based on network motifs. The HONE framework is highly expressive and flexible with many interchangeable components. The experimental results demonstrate the effectiveness of learning higher-order network represe…
Upper bound derived for informed traders' gains in a model, akin to thermodynamics.
problem Informed traders' gains in a financial model with finite horizon.
method Bayesian inference and entropic inequality.
result Upper bound for expected gain, analogous to thermodynamics.
In many professons employees are rewarded according to their relative performance. Corresponding economy can be modeled by taking N independent agents who gain from the market with a rate which depends on their current gain. We argue that this simple realistic rate generates a scale free distribution even though intr…
Study adds investment gains and losses to recursive utility model, proving existence and uniqueness of utility process.
problem Existence and uniqueness of utility process in a recursive utility model with investment gains and losses.
method Generalized recursive utility model with constant elasticity of intertemporal substitution and relative risk aversion degree. Proved existence and uniqueness in a specific, finite-state Markovian setting.
result Utility process exists and is unique when agent derives nonnegative gain-loss utility, and non-existent or non-unique otherwise.
Estimates expected information gain using density approximations and dimension reduction.
problem Estimating expected information gain in nonlinear and non-Gaussian settings.
method Flexible transport-based schemes for EIG estimation, optimal sample allocation, and gradient-based upper bounds on mutual information.
result Optimal sample allocation and dimension reduction schemes improve EIG estimation accuracy and convergence rate.
New bandit algorithm maximizes information gain.
problem Optimizing decision-making in uncertain environments.
method Approximates information maximization using entropy and free energy principles.
result Asymptotic optimality proven for two-armed bandit problem.
Training deep recurrent neural network (RNN) architectures is complicated due to the increased network complexity. This disrupts the learning of higher order abstracts using deep RNN. In case of feed-forward networks training deep structures is simple and faster while learning long-term temporal information is not poss…
We approximate differential entropy for efficient Bayesian experimental design.
problem Efficiently estimating expected information gain in large-scale inference problems.
method Approximate differential entropy using Monte Carlo or quasi-Monte Carlo surrogates.
result Our approach achieves comparable or better convergence rates than state-of-the-art methods.
Momentum based stochastic gradient methods such as heavy ball (HB) and Nesterov's accelerated gradient descent (NAG) method are widely used in practice for training deep networks and other supervised learning models, as they often provide significant improvements over stochastic gradient descent (SGD). Rigorously speak…
A new algorithm STE for model-based RL improves learning rates.
problem Sparse rewards and computational intractability of estimating information gain.
method Developed a novel algorithm based on Stein Information Directed Exploration (STE)E.
result Achieves sublinear Bayesian regret, outperforming prior approaches.
A new algorithm for conversational recommendation systems using dueling bandits in GLMs.
problem Limited user feedback in existing conversational bandit methods.
method Integrates dueling bandits with relative feedback in generalized linear models.
result Theoretical and empirical validation of ConDuel's efficacy.
Improved asset pricing using uncertainty-adjusted sorting in machine learning models.
problem Ignoring asset-specific estimation uncertainty in portfolio construction.
method Uncertainty-adjusted prediction bounds for sorting assets.
result Improves portfolio performance across various ML models and equity panels.
As Computer Vision moves from a passive analysis of pixels to active analysis of semantics, the breadth of information algorithms need to reason over has expanded significantly. One of the key challenges in this vein is the ability to identify the information required to make a decision, and select an action that will …
Meta-active learning optimizes control of safety-critical systems by efficiently learning dynamics and configurations.
problem Efficiently learning system dynamics and optimal configurations for safety-critical systems like deep brain stimulation.
method Meta-learning an acquisition function using LSTM, cast as meta-learning, with a mixed-integer linear program policy.
result Achieved a 46% increase in information gain and a 20% speedup in computation time over baselines.
Inference for normal and Monte Carlo distributions using minimum relative entropy.
problem Inference from partial information on expectations and covariances.
method Minimum relative entropy sub-manifolds, analytical formulas, Monte Carlo simulations.
result Improved numerical implementation for inference from partial information.
Enhances robustness in experimental design through Generalised Bayesian inference.
problem Poor inference and estimates of information gain when statistical model is incorrectly specified.
method Generalised Bayesian (Gibbs) inference framework applied to experimental design.
result GBOED enhances robustness to outliers and incorrect assumptions about noise distribution.
This paper extends financial theory to measure learnable market structure under computational constraints.
problem Understanding learnable market structure under bounded computational capacity.
method Introduces financial epiplexity as a measure of learnable market structure, extending classical information theory.
result Proves that equal entropy does not imply equal epiplexity and derives thresholds for useful regimes.