In the information-based paradigm of inference, model selection is performed by selecting the candidate model with the best estimated predictive performance. The success of this approach depends on the accuracy of the estimate of the predictive complexity. In the large-sample-size limit of a regular model, the predicti…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,657 papers · 148 categories
Trend · papers per month
48 results for “information-based complexity”
A simple application of FIC to model selectionphysics.data-an
We have recently proposed a new information-based approach to model selection, the Frequentist Information Criterion (FIC), that reconciles information-based and frequentist inference. The purpose of this current paper is to provide a simple example of the application of this criterion and a demonstration of the natura…
There are three principle paradigms of statistical inference: (i) Bayesian, (ii) information-based and (iii) frequentist inference. We describe an objective prior (the weighting or -prior) which unifies objective Bayes and information-based inference. The -prior is chosen to make the marginal probability an unbia…
We apply information-based complexity analysis to support vector machine (SVM) algorithms, with the goal of a comprehensive continuous algorithmic analysis of such algorithms. This involves complexity measures in which some higher order operations (e.g., certain optimizations) are considered primitive for the purposes …
Method quantifies uncertainties in complex MRF models.
problem Uncertainties in MRF predictions due to data, modeling, and approximations.
method Information-based uncertainty quantification using MRF graphical structure.
result Tight bounds on predictions for quantities of interest in MRFs.
New algorithms improve convergence rates for non-log-concave sampling and log-partition estimation.
problem Efficiently sampling from non-log-concave distributions and estimating their log-partition function.
method Analysis of information-based complexity, study of polynomial-time sampling algorithms.
result Optimal rates for sampling and log-partition estimation sometimes exceed those for optimization.
Change-point analysis is a flexible and computationally tractable tool for the analysis of times series data from systems that transition between discrete states and whose observables are corrupted by noise. The change-point algorithm is used to identify the time indices (change points) at which the system transitions …
We consider a class of generalized capital asset pricing models in continuous time with a finite number of agents and tradable securities. The securities may not be sufficient to span all sources of uncertainty. If the agents have exponential utility functions and the individual endowments are spanned by the securities…
Estimation of Rényi Entropy and Mutual Information Based on Generalized Nearest-Neighbor Graphsstat.ML
We present simple and computationally efficient nonparametric estimators of Rényi entropy and mutual information based on an i.i.d. sample drawn from an unknown, absolutely continuous distribution over . The estimators are calculated as the sum of -th powers of the Euclidean lengths of the edges of the `genera…
Bayesian Neural Networks improve high-dimensional level set estimation.
problem Scalability issue in existing LSE methods for high-dimensional inputs.
method Bayesian Neural Networks with information-based acquisition functions.
result Proposed method achieves better results than state-of-the-art approaches.
New study shows exponential sample growth for ReQU neural networks.
problem Computing neural network approximations from samples is challenging.
method Information-based complexity tools.
result Functions can be approximated by ReQU neural networks at arbitrary rates but require exponentially growing samples.
Jensen-Shannon Information Based Characterization of the Generalization Error of Learning Algorithmscs.IT
New bound on machine learning model performance using Jensen-Shannon information.
problem Understanding the performance of machine learning models.
method Proposes a new information-theoretic bound on generalization error.
result Shows that the new bound can be tighter than mutual information-based bounds under certain conditions.
We present arguments for the formulation of unified approach to different standard continuous inference methods from partial information. It is claimed that an explicit partition of information into a priori (prior knowledge) and a posteriori information (data) is an important way of standardizing inference approaches …
This article attempts to place the emergence of probabilistic numerics as a mathematical-statistical research field within its historical context and to explore how its gradual development can be related both to applications and to a modern formal treatment. We highlight in particular the parallel contributions of Sul'…
We analyze random feature and two-layer neural networks using duality framework.
problem Understanding and analyzing functions within and Barron spaces.
method Duality framework based on information-based complexity (IBC).
result Sharp bounds for learning using RFMs without curse of dimensionality for .
Trustworthy AI is a critical issue in machine learning where, in addition to training a model that is accurate, one must consider both fair and robust training in the presence of data bias and poisoning. However, the existing model fairness techniques mistakenly view poisoned data as an additional bias to be fixed, res…
We analyze the adversarial examples problem in terms of a model's fault tolerance with respect to its input. Whereas previous work focuses on arbitrarily strict threat models, i.e., -perturbations, we consider arbitrary valid inputs and propose an information-based characteristic for evaluating tolerance to diverse …
Complex dynamical systems driven by the unravelling of information can be modelled effectively by treating the underlying flow of information as the model input. Complicated dynamical behaviour of the system is then derived as an output. Such an information-based approach is in sharp contrast to the conventional mathem…
Unknown constraints arise in many types of expensive black-box optimization problems. Several methods have been proposed recently for performing Bayesian optimization with constraints, based on the expected improvement (EI) heuristic. However, EI can lead to pathologies when used with constraints. For example, in the c…
In the information-based approach to asset pricing the market filtration is modelled explicitly as a superposition of signals concerning relevant market factors and independent noise. The rate at which the signal is revealed to the market then determines the overall magnitude of asset volatility. By letting this inform…
This paper provides sufficient conditions for the time of bankruptcy (of a company or a state) for being a totally inaccessible stopping time and provides the explicit computation of its compensator in a framework where the flow of market information on the default is modelled explicitly with a Brownian bridge between …
Proposes mutual information for regression without prior knowledge.
problem Regression tasks without prior model knowledge.
method Mutual information learning formulation, SGD convergence analysis.
result High dimensionality can be beneficial with a threshold.
A fast feature selection method using OLS and SOCC for classification.
problem Feature selection for linear classification.
method Orthogonal Least Squares (OLS) with Squared Orthogonal Correlation Coefficient (SOCC).
result The proposed method outperforms other feature selection methods in speed and accuracy.
We are working to develop automated intelligent agents, which can act and react as learning machines with minimal human intervention. To accomplish this, an intelligent agent is viewed as a question-asking machine, which is designed by coupling the processes of inference and inquiry to form a model-based learning unit.…
In this paper we introduce a class of information-based models for the pricing of fixed-income securities. We consider a set of continuous- time information processes that describe the flow of information about market factors in a monetary economy. The nominal pricing kernel is at any given time assumed to be given by …
Information of Interestq-fin.PR
A pricing formula for discount bonds, based on the consideration of the market perception of future liquidity risk, is established. An information-based model for liquidity is then introduced, which is used to obtain an expression for the bond price. Analysis of the bond price dynamics shows that the bond volatility is…
AES uses α-divergence to select informative points for BO, improving optimization performance.
problem Optimizing complex functions with limited evaluations.
method AES uses α-divergence to select points based on dependency with global maximum.
result AES outperforms other information-based acquisition functions in various experiments.
A new method improves uncertainty estimation in deep learning, especially for hard-to-label samples.
problem Improving uncertainty estimation for hard-to-label samples in deep learning.
method Introduces Fisher Information Matrix (FIM) to dynamically reweight objective loss terms.
result Consistently outperforms traditional evidential neural networks in uncertainty estimation tasks.
In a standard setting of Bayesian optimization (BO), the objective function evaluation is assumed to be highly expensive. Multi-fidelity Bayesian optimization (MFBO) accelerates BO by incorporating lower fidelity observations available with a lower sampling cost. In this paper, we focus on the information-based approac…
Feature selection methods have an important role on the readability of data and the reduction of complexity of learning algorithms. In recent years, a variety of efforts are investigated on feature selection problems based on unsupervised viewpoint due to the laborious labeling task on large datasets. In this paper, we…
Outsider Tradingq-fin.TR
In this paper we examine inefficiencies and information disparity in the Japanese stock market. By carefully analysing information publicly available on the internet, an `outsider' to conventional statistical arbitrage strategies--which are based on market microstructure, company releases, or analyst reports--can never…
TES optimizes black-box functions efficiently with minimal approximations.
problem Efficient Bayesian optimization with minimal approximations and generalization to batch BO.
method TES acquisition function measures information gain on trusted maximizers.
result TES achieves state-of-the-art performance with minimal approximations.
Multivariate pattern analyses approaches in neuroimaging are fundamentally concerned with investigating the quantity and type of information processed by various regions of the human brain; typically, estimates of classification accuracy are used to quantify information. While a extensive and powerful library of method…
This paper presents a novel data-driven technique based on the spatiotemporal pattern network (STPN) for energy/power prediction for complex dynamical systems. Built on symbolic dynamic filtering, the STPN framework is used to capture not only the individual system characteristics but also the pair-wise causal dependen…
Information-Based Asset Pricingq-fin.PR
A new framework for asset price dynamics is introduced in which the concept of noisy information about future cash flows is used to derive the price processes. In this framework an asset is defined by its cash-flow structure. Each cash flow is modelled by a random variable that can be expressed as a function of a colle…
We propose a betting strategy based on Bayesian logistic regression modeling for the probability forecasting game in the framework of game-theoretic probability by Shafer and Vovk (2001). We prove some results concerning the strong law of large numbers in the probability forecasting game with side information based on …
GIBBON unifies Bayesian optimization for various problem types.
problem Bayesian optimization across noisy, multi-fidelity, and batch optimisations.
method Novel approximation of information gain for a single framework.
result Significantly lower computational overhead compared to existing approaches.
Study identifies a Strategic Gap in market efficiency due to AI-driven timing and complexity in disclosure.
problem Market inefficiency due to structural influence of disclosure timing and complexity.
method Introduces Autonomous Disclosure Regulator, a multi-node AI framework to audit disclosure complexity and unpredictability.
result Companies use confusing language and unpredictable timing to slow down market learning, creating a 60% Structural Gap.
This paper optimizes sampling for least-squares approximation.
problem Optimizing sampling for least-squares approximation in arbitrary linear spaces.
method Introducing the Christoffel function to construct near-optimal random sampling strategies.
result The number of samples scales log-linearly in the dimension of the approximation space.
Improves Bayesian optimization for multi-fidelity functions.
problem Inefficient estimation of black-box functions due to ignored or oversimplified correlations between fidelities.
method Proposes DNN-MFBO using deep neural networks to capture complex relationships between fidelities.
result Shows significant improvement in optimization performance on synthetic and real-world datasets.
New method controls bias in training data for fair outcomes.
problem Ensuring equal treatment between different groups in machine learning.
method Contrastive information estimation to control mutual information between representations and protected attributes.
result Our method provides strong theoretical guarantees on the parity of any downstream algorithm.
A new framework for asset pricing based on modelling the information available to market participants is presented. Each asset is characterised by the cash flows it generates. Each cash flow is expressed as a function of one or more independent random variables called market factors or "X-factors". Each X-factor is ass…
In this paper we present a novel quasi-Newton algorithm for use in stochastic optimisation. Quasi-Newton methods have had an enormous impact on deterministic optimisation problems because they afford rapid convergence and computationally attractive algorithms. In essence, this is achieved by learning the second-order (…
Regularization is a big issue for training deep neural networks. In this paper, we propose a new information-theory-based regularization scheme named SHADE for SHAnnon DEcay. The originality of the approach is to define a prior based on conditional entropy, which explicitly decouples the learning of invariant represent…
Regularization is a big issue for training deep neural networks. In this paper, we propose a new information-theory-based regularization scheme named SHADE for SHAnnon DEcay. The originality of the approach is to define a prior based on conditional entropy, which explicitly decouples the learning of invariant represent…
Unified bounds linking compressibility, fractal dimensions, and mutual information.
problem Understanding generalization in stochastic learning algorithms.
method Rate-distortion theory applied to machine learning generalization.
result Unified bounds linking compressibility, fractal dimensions, and mutual information.
Linear and nonlinear market correlations: characterizing financial crises and portfolio optimizationq-fin.ST
Pearson correlation and mutual information based complex networks of the day-to-day returns of US S&P500 stocks between 1985 and 2015 have been constructed in order to investigate the mutual dependencies of the stocks and their nature. We show that both networks detect qualitative differences especially during (recent)…
Two new undersampling methods improve classification accuracy for imbalanced datasets.
problem Class imbalance and distributional differences in large datasets lead to biased models and poor predictive performance.
method Mutual information-based stratified simple random sampling and support points optimization.
result Empirical results show higher balanced classification accuracy compared to traditional techniques.