MIC consistently estimates dependence in large datasets.
problem Estimating dependence between variable pairs in large datasets.
method Proving consistency of MIC as an estimator.
result MIC is a consistent estimator of population statistic MIC*.
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,657 papers · 148 categories
Trend · papers per month
48 results for “Maximal Information Coefficient”
Distance Correlation Methods for Discovering Associations in Large Astrophysical Databasesastro-ph.CO
High-dimensional, large-sample astrophysical databases of galaxy clusters, such as the Chandra Deep Field South COMBO-17 database, provide measurements on many variables for thousands of galaxies and a range of redshifts. Current understanding of galaxy formation and evolution rests sensitively on relationships between…
A measure of dependence is said to be equitable if it gives similar scores to equally noisy relationships of different types. Equitability is important in data exploration when the goal is to identify a relatively small set of strongest associations within a dataset as opposed to finding as many non-zero associations a…
Reshef et al. recently proposed a new statistical measure, the "maximal information coefficient" (MIC), for quantifying arbitrary dependencies between pairs of stochastic quantities. MIC is based on mutual information, a fundamental quantity in information theory that is widely understood to serve this need. MIC, howev…
Proposes an adversarial algorithm to learn unbiased representations via HGR coefficient.
problem Learning fair representations without sensitive attribute information.
method Adversarial algorithm using Hirschfeld-Gebelein-Renyi (HGR) maximal correlation coefficient.
result Significant improvements in bias mitigation compared to existing methods.
Reshef & Reshef recently published a paper in which they present a method called the Maximal Information Coefficient (MIC) that can detect all forms of statistical dependence between pairs of variables as sample size goes to infinity. While this method has been praised by some, it has also been criticized for its lack …
A new method for multi-objective Bayesian optimization using entropy search and variational lower bound maximization.
problem Efficiently optimizing multiple objectives in continuous domains.
method Approximates the Pareto-frontier using a mixture distribution and optimizes the balance through variational lower bound maximization.
result Demonstrated effectiveness especially with many objective functions.
The principle of absence of arbitrage opportunities allows obtaining the distribution of stock price fluctuations by maximizing its information entropy. This leads to a physical description of the underlying dynamics as a random walk characterized by a stochastic diffusion coefficient and constrained to a given value o…
Study S-shaped utility maximization with VaR constraint and unobservable drift.
problem Maximizing utility with a Value at Risk (VaR) constraint and unknown drift.
method Bayesian filter, concavification principle, change of measure, semi-closed integral representation, algorithms (Lagrange, simulation, deep neural network).
result Critical wealth level determining solution feasibility and optimal solution existence.
Sharp bounds for eigenfunctions on product spaces restricted to submanifolds.
problem Establishing bounds for eigenfunctions on product spaces.
method Combining asymptotics of Jacobi polynomials and positivity of Fourier coefficients of spherical functions.
result Sharp bounds for eigenfunctions on products of rank-one symmetric spaces.
The maximal information coefficient (MIC) is a tool for finding the strongest pairwise relationships in a data set with many variables (Reshef et al., 2011). MIC is useful because it gives similar scores to equally noisy relationships of different types. This property, called {\em equitability}, is important for analyz…
New examples show high twisting doesn't guarantee open book maximality.
problem Maximality of open book pages under high twisting.
method Double branched covers of closed braids.
result Arbitrarily high twisting does not guarantee maximal Euler characteristic.
This paper studies the question of filtering and maximizing terminal wealth from expected utility in a partially information stochastic volatility models. The special features is that the only information available to the investor is the one generated by the asset prices, and the unobservable processes will be modeled …
A new model detects complex network communities using node attributes.
problem Lack of methods integrating node attributes for community detection in attributed networks.
method BCSBM model that integrates betweenness centrality and clustering coefficient of nodes.
result BCSBM model outperforms other methods in detecting various network structures.
The paper studies the robust maximization of utility of terminal wealth in the diffusion financial market model. The underlying model consists with risky tradable asset, whose price is described by diffusion process with misspecified trend and volatility coefficients, and non-tradable asset with a known parameter. The …
We investigate the ergodic problem of growth-rate maximization under a class of risk constraints in the context of incomplete, Itô-process models of financial markets with random ergodic coefficients. Including {\em value-at-risk} (VaR), {\em tail-value-at-risk} (TVaR), and {\em limited expected loss} (LEL), these cons…
Sparse linear (or generalized linear) models combine a standard likelihood function with a sparse prior on the unknown coefficients. These priors can conveniently be expressed as a maximization over zero-mean Gaussians with different variance hyperparameters. Standard MAP estimation (Type I) involves maximizing over bo…
We consider the problem of recovering block-sparse signals whose structures are unknown \emph{a priori}. Block-sparse signals with nonzero coefficients occurring in clusters arise naturally in many practical scenarios. However, the knowledge of the block structure is usually unavailable in practice. In this paper, we d…
Framework uses deep learning and statistical models to solve PDEs with discontinuous coefficients.
problem Solving PDEs with discontinuous coefficients.
method Two-stage physics-informed deep learning and statistical mixture models.
result Framework achieves adaptability and accurate parameter identification.
Abstract: Determines thermoelastic coefficients from boundary data.
problem Determining coefficients of thermoelastic system from boundary information.
method Explicit expression for thermoelastic Dirichlet-to-Neumann map with variable coefficients.
result Thermoelastic Dirichlet-to-Neumann map uniquely determines coefficients on the manifold.
Modelling highly multi-modal data is a challenging problem in machine learning. Most algorithms are based on maximizing the likelihood, which corresponds to the M(oment)-projection of the data distribution to the model distribution. The M-projection forces the model to average over modes it cannot represent. In contras…
An SMP-Based Algorithm for Solving the Constrained Utility Maximization Problem via Deep Learningq-fin.CP
New algorithm solves utility maximization with deep learning for constrained problems.
problem Maximizing utility under convex constraints with random coefficients.
method Developed a new algorithm using stochastic maximum principle and deep learning.
result The new algorithm outperforms existing methods in accuracy and applicability.
Influence maximization (IM) is one of the most important problems in social network analysis. Its objective is to find a given number of seed nodes that maximize the spread of information through a social network. Since it is an NP-hard problem, many approximate/heuristic methods have been developed, and a number of th…
Investigates the use of Information Coefficient as a stock selection model performance measure.
problem The adequacy and effectiveness of Information Coefficient (IC) for evaluating stock selection models is unclear.
method Simulation and simple statistical modeling to examine IC behavior statically and dynamically.
result Proposes two practical procedures for IC-based ongoing performance monitoring of stock selection models.
The paper extends physics-based information maximization to complex bandit problems.
problem Designing efficient decision-making policies for complex bandit problems.
method Information and free-energy maximization principles adapted to three distinct bandit types.
result Information maximization leads to strong performance in complex bandit problems.
New quantum states capture more information, enabling advanced processing tasks.
problem Quantum information processing challenges with limited statistical information.
method Introducing Random-Coefficient Pure States (RCPS) and exploiting their higher-order statistics.
result RCPS provide richer information than density operators, enabling new quantum tasks.
New bandit algorithm maximizes information gain.
problem Optimizing decision-making in uncertain environments.
method Approximates information maximization using entropy and free energy principles.
result Asymptotic optimality proven for two-armed bandit problem.
New index formula for hypoelliptic operators on manifolds.
problem Index computation for hypoelliptic differential operators.
method Generalized index formula for *-maximally hypoelliptic operators.
result Explicit index computations for Hormander's sum of squares operators.
In this work, we consider a manufactory process which can be described by a multiple-instance logistic regression model. In order to compute the maximum likelihood estimation of the unknown coefficient, an expectation-maximization algorithm is proposed, and the proposed modeling approach can be extended to identify the…
This paper concerns the recursive utility maximization problem. We assume that the coefficients of the wealth equation and the recursive utility are concave. Then some interesting and important cases with nonlinear and nonsmooth coefficients satisfy our assumption. After given an equivalent backward formulation of our …
Proposes a new method to enhance neural learning by maximizing information gain.
problem Improving neural learning by selecting key variables to maximize information gain.
method Adaptive Ensemble Kalman Filter to quantify uncertainty and maximize information gain.
result The proposed method enables the neural network to learn more effectively from stochastic systems.
A new method estimates the learning coefficient using empirical loss.
problem Estimating the learning coefficient in complex models.
method Numerical estimation using Empirical Loss.
result Our method outperforms existing techniques in terms of bias and variance.
The speed of convergence of the Expectation Maximization (EM) algorithm for Gaussian mixture model fitting is known to be dependent on the amount of overlap among the mixture components. In this paper, we study the impact of mixing coefficients on the convergence of EM. We show that when the mixture components exhibit …
On low-sampling-rate Kramers-Moyal coefficientscond-mat.stat-mech
We analyze the impact of the sampling interval on the estimation of Kramers-Moyal coefficients. We obtain the finite-time expressions of these coefficients for several standard processes. We also analyze extreme situations such as the independence and no-fluctuation limits that constitute useful references. Our results…
The effectiveness of utility-maximization techniques for portfolio management relies on our ability to estimate correctly the parameters of the dynamics of the underlying financial assets. In the setting of complete or incomplete financial markets, we investigate whether small perturbations of the market coefficient pr…
Global inequality in energy consumption from 1980 to 2010physics.data-an
We study the global probability distribution of energy consumption per capita around the world using data from the U.S. Energy Information Administration (EIA) for 1980-2010. We find that the Lorenz curves have moved up during this time period, and the Gini coefficient G has decreased from 0.66 in 1980 to 0.55 in 2010,…
The study uses DCC for financial market analysis, revealing hidden correlations.
problem Identifying hidden nonlinear correlations in financial markets.
method Agglomerative hierarchical clustering with distance correlation coefficient.
result DCC reveals more information than Pearson correlation for financial data.
Improves functional linear regression with shape transfer learning.
problem Data scarcity in functional linear models.
method Shape-based transfer learning from auxiliary to target domains.
result Enhances robustness and generalizability of functional linear models.
On a real analytic 5-dimensional CR-generic submanifold M^5 in C^4 of codimension 3, hence of CR dimension 1, which enjoys the generically satisfied nondegeneracy condition that Lie brackets up to length 3 of T^{1,0}M generate CTM, a canonical Cartan connection is constructed after reduction to a certain partially expl…
The paper analyzes generalization of noisy, iterative algorithms using maximal leakage.
problem Analyzing the generalization behavior of noisy, iterative learning algorithms.
method Information-theoretic framework with maximal leakage metric.
result Explicit upper bounds on maximal leakage for various scenarios.
Proposes a framework to maximize mutual information in VAE models for better latent code representation.
problem Lack of explicit measurement of the quality of learned representations in VAE models.
method Variational Mutual Information Maximization Framework for VAE.
result Maximizes mutual information between latent codes and observations, improving latent code representation.
Study finds Stäckel equivalence for superintegrable systems via invariant quadrics.
problem Understanding Stäckel equivalence in superintegrable systems.
method Using invariant quadrics to determine Stäckel classes of superintegrable systems.
result Stäckel classes of superintegrable systems can be derived from associated invariant quadrics.
Generalizes underlap coefficient for multivariate group separation.
problem Quantifying distributional separation across groups in statistical learning.
method Generalizes underlap coefficient (UNL) to multivariate settings, studies its relationship with Bayes risk and mutual information, proposes an efficient importance sampling estimator.
result UNL as a measure of dependence between group labels and variables of interest, interpretable measure of partition-covariate dependence in clustering.
Kernel methods linked to feature subspaces and maximal correlation kernels.
problem Understanding kernel methods and their relationship to feature extraction.
method Established a correspondence between feature subspaces and kernels, introduced maximal correlation kernels, and demonstrated their optimality.
result Kernel SVM on maximal correlation kernel achieves minimum prediction error.
Recently, crowdsourcing has emerged as an effective paradigm for human-powered large scale problem solving in various domains. However, task requester usually has a limited amount of budget, thus it is desirable to have a policy to wisely allocate the budget to achieve better quality. In this paper, we study the princi…
Information-maximization clustering learns a probabilistic classifier in an unsupervised manner so that mutual information between feature vectors and cluster assignments is maximized. A notable advantage of this approach is that it only involves continuous optimization of model parameters, which is substantially easie…
The paper provides formulas for Hadamard coefficients using Green's operators.
problem Calculating Hadamard coefficients from Green's operators.
method Various methods including resolvents, powers of Green's operators, and product with the real line.
result Formulas for Hadamard coefficients in terms of Green's operators.
The Price of Informationq-fin.MF
Investors pay for additional asset information based on utility maximization.
problem Determining the optimal price for additional asset information.
method Solving a stochastic control problem with partial information and utility maximization.
result Investors choose to purchase information at a deterministic time.