Study complexity in financial market using Shannon entropy.
problem Measuring complexity in financial market information traffic.
method Reconstructing financial dynamics from share prices, calculating Shannon entropy.
result Shannon entropy quantifies complexity in financial market information.
We discuss the systemic risk implied by the interbank exposures reconstructed with the maximum entropy method. The maximum entropy method severely underestimates the risk of interbank contagion by assuming a fully connected network, while in reality the structure of the interbank network is sparsely connected. Here, we…
Predicts trainability of deep neural networks using reconstruction entropy.
problem Predicting the initial conditions for trainability of deep neural networks.
method Cascade of auxiliary networks to reconstruct input from activation layers, computing relative entropy.
result Predicts trainability of deep feedforward networks on various datasets with a single epoch.
EVODiff optimizes DM inference by reducing conditional entropy, improving image generation.
problem Slow and inaccurate inference in diffusion models.
method Entropy-aware variance optimization for efficient inference.
result Significant improvement in image generation quality and efficiency.
We propose a permutation-invariant loss function designed for the neural networks reconstructing a set of elements without considering the order within its vector representation. Unlike popular approaches for encoding and decoding a set, our work does not rely on a carefully engineered network topology nor by any addit…
Entropy data replaces classical charts for smooth manifolds.
problem Establishing smooth structures on topological manifolds.
method Using entropy data to define admissible coordinate functions and reconstruct smooth atlases.
result Entropy-smooth structures are equivalent to classical smooth structures and stable under perturbations.
This work enhances collaborative inference privacy by minimizing conditional entropy and boosting robustness against model inversion attacks.
problem Privacy leakage in collaborative inference systems via model inversion attacks.
method Theoretical proof and derivation of a differentiable measure for bounding conditional entropy, followed by a CEM algorithm to maximize it.
result Theoretical proof and experimental validation show that CEM consistently boosts inversion robustness without compromising feature utility or efficiency.
PACE-GGM uses Gaussian mechanism for private covariance estimation.
problem Private estimation of covariance matrices in high dimensions.
method Data-adaptive selection of entries, Gaussian mechanism, maximum-entropy reconstruction.
result Consistent improvements in estimation error compared to Gaussian mechanism and baselines.
Study uses holography to analyze entanglement entropy in deformed CFTs.
problem Analyzing entanglement entropy in $Tar{T}$-deformed CFTs.
method Holographic methods and direct bulk gravitational action evaluation.
result Agreement with known results for entanglement entropy.
This paper presents a novel method for the reconstruction of a neural network connectivity using calcium fluorescence data. We introduce a fast unsupervised method to integrate different networks that reconstructs structural connectivity from neuron activity. Our method improves the state-of-the-art reconstruction meth…
New method recovers curvature from heat diffusion data.
problem Recovering Riemannian curvature from heat diffusion.
method Information-theoretic approach using relative entropy.
result Local curvature determined by heat diffusion.
The interdependent nature of the global economy has become stronger with increases in international trade and investment. We propose a new model to reconstruct the international trade network and associated cost network by maximizing entropy based on local information about inward and outward trade. We show that the tr…
A new method for feature fusion in U-Net decoders using difference-based gating.
problem Precise fusion of high-level semantics and low-level details in U-Net decoder reconstruction.
method Proposes two difference-based gating approaches: Feature-difference gating (FDG) and Entropy-difference gating (EDG).
result Both FDG and EDG methods outperform existing attention-based fusion methods, with EDG showing superior performance.
We address the two fundamental problems of spatial field reconstruction and sensor selection in heterogeneous sensor networks: (i) how to efficiently perform spatial field reconstruction based on measurements obtained simultaneously from networks with both high and low quality sensors; and (ii) how to perform query bas…
New results on inferring hidden states in trackable weak models.
problem Inferring hidden states in trackable weak models.
method Analyzing strongly-connected trackable weak models and reconstructing branch choices.
result The number of hypotheses in strongly-connected trackable models is bounded by a constant.
Tree-AMP simplifies inference in complex tree-structured models.
problem Inference in high-dimensional tree-structured models.
method Approximate Message Passing algorithms for various machine learning tasks.
result Theoretical performance predictions and automated entropy estimation.
A good representation for arbitrarily complicated data should have the capability of semantic generation, clustering and reconstruction. Previous research has already achieved impressive performance on either one. This paper aims at learning a disentangled representation effective for all of them in an unsupervised way…
We present a novel synthesis of Fisher information and asset pricing theory that yields a practical method for reconstructing the probability density implicit in security prices. The Fisher information approach to these inverse problems transforms the search for a probability density into the solution of a differential…
We discuss how maximum entropy methods may be applied to the reconstruction of Markov processes underlying empirical time series and compare this approach to usual frequency sampling. It is shown that, at least in low dimension, there exists a subset of the space of stochastic matrices for which the MaxEnt method is mo…
New methods optimize sums of bivariate functions on finite domains.
problem Optimizing functions with multiple arguments that are sums of bivariate functions.
method Measure-valued extensions, ℓ2-approximation, entropy-regularization, linear programming, coordinate ascent. result Tractable problem formulations solvable with various methods.
Analyzes how BPE tokenisation affects corpus statistics and model entropy in transformer models.
problem Understanding how natural language properties relate to tokenisation schemes in transformer models.
method Analyzes Shannon entropy of corpora under Zipfian distribution, investigates BPE transformations, trains language models, and uses attention diagnostics.
result Transformer models trained on BPE-tokenised corpora increasingly agree with Zipfian predictions as BPE depth increases, indicating reduced local token dependencies.
Diagnostic stroke imaging with C-arm cone-beam computed tomography (CBCT) enables reduction of time-to-therapy for endovascular procedures. However, the prolonged acquisition time compared to helical CT increases the likelihood of rigid patient motion. Rigid motion corrupts the geometry alignment assumed during reconst…
Deep learning speeds up protein mapping entropy calculation.
problem Efficiently calculating the mapping entropy of protein structures.
method Deep graph networks for accelerating mapping entropy computation.
result Deep graph networks achieve a speedup factor of up to 10^5.
Reconstructing patterns of interconnections from partial information is one of the most important issues in the statistical physics of complex networks. A paramount example is provided by financial networks. In fact, the spreading and amplification of financial distress in capital markets is strongly affected by the in…
Framework for analyzing dynamic topological changes in point clouds using persistent homology and dynamic optimal transport.
problem Analyzing transient structural reorganizations during dynamic phase transitions in time-evolutionary point clouds.
method Hierarchical dynamic evaluation framework driven by topological and hypergraph reconstruction strategy.
result Combining transport-based alignment with multi-scale entropy diagnostics for dynamic topological analysis.
A new method for VAEs improves latent space disentanglement without violating probability laws.
problem Improving latent space disentanglement in VAEs without violating probability laws.
method Developed a Renyi VAE with a conditional distribution not learned, using Singular Value Decomposition for evaluation.
result Improved latent space disentanglement without violating probability laws.
A new deep learning framework for efficient IoT data compression and inference.
problem Limited bandwidth and power resources in IoT sensors.
method Co-designed deep learning framework that maximizes sensing goal accuracy.
result Superior performance compared to benchmark models.
Develops statistical framework for resolving reward function ambiguity in inverse reinforcement learning.
problem Non-uniqueness of reward functions in inverse reinforcement learning.
method Entropy regularization combined with least-squares reconstruction of the reward from the soft Bellman residual.
result Least-squares reward function is unique and consistent with the expert policy.
This work extends entropic optimal transport to non-product reference couplings, focusing on Gaussian cases.
problem Finding a diffuse coupling between two measures with non-product reference couplings.
method Reduction of the entropic optimal transport problem to a matrix optimization problem.
result Complete description of the solution for non-product reference couplings, including primal and dual variables.
In this manuscript we propose two objective terms for neural image compression: a compression objective and a cycle loss. These terms are applied on the encoder output of an autoencoder and are used in combination with reconstruction losses. The compression objective encourages sparsity and low entropy in the activatio…
In this paper we estimate the propagation of liquidity shocks through interbank markets when the information about the underlying credit network is incomplete. We show that techniques such as Maximum Entropy currently used to reconstruct credit networks severely underestimate the risk of contagion by assuming a trivial…
Assessing systemic risk in financial markets is of great importance but it often requires data that are unavailable or available at a very low frequency. For this reason, systemic risk assessment with partial information is potentially very useful for regulators and other stakeholders. In this paper we consider systemi…
EPSTE: A geometric token and deep learning approach to estimating transfer entropy in neuroimaging time series
problem Inferring directed interactions between neural systems from EEG and MEG
method Reframing TE estimation as a learnable problem operating on structured symbolic representations
result EPSTE achieves near-perfect recovery of ground-truth directed structure and significantly lower absolute error than the baseline
New method improves sparse signal reconstruction using 1RSB-AMP.
problem Sparse signal reconstruction with improved accuracy.
method Developed 1RSB-AMP and 1RSB-SE for SCAD penalty minimization.
result 1RSB-AMP achieves improved reconstruction compared to RS-AMP.
Proposes a link between randomness and compression in deep learning.
problem Improving efficiency in deep learning training.
method Introduces a novel tomographic compression framework called Dual Tomographic Compression (DTC).
result Demonstrates high correlation between learning performance and Gibbs entropy over compression ratios.
This paper proposes a new method for efficient data compression using Bayesian neural networks.
problem Efficient compression of data represented as functions mapping coordinates to signal values.
method Overfitting variational Bayesian neural networks to the data and compressing an approximate posterior weight sample using relative entropy coding.
result Our method achieves strong performance on image and audio compression while retaining simplicity.
A neural network derived from first principles using MaxEnt.
problem Developing a neural network from first principles.
method Derived a neural network using the principle of Maximum Entropy, with linear dimension-reducing transformations and conditional mean estimators.
result Unified theoretical justification for activation functions like sigmoid, softplus, and relu.
Unified framework for estimating reward functions in competitive games.
problem Estimating unknown reward functions in competitive games.
method Unified framework with entropy regularization for reward function recovery.
result Strong theoretical guarantees and practical effectiveness demonstrated.
New method learns cell trajectories and network interactions from single-cell data.
problem Network inference in systems biology from steady-state data.
method Min-entropy estimation for stochastic dynamics, leveraging both temporal and perturbational data.
result Jointly learns cellular trajectories and network interactions.
CNN accurately reconstructs lattice topology with strong thermal fluctuations.
problem Reconstructing lattice topology with strong thermal fluctuations and unbalanced data.
method Deep convolutional neural network (CNN) mapping local magnetic moments to coupling probabilities.
result CNN accurately reconstructs lattice topology where thermal fluctuations dominate.
Here we present an application of two maxentropic procedures to determine the probability density distribution of compound sums of random variables, using only a finite number of empirically determined fractional moments. The two methods are the Standard method of Maximum Entropy (SME), and the method of Maximum Entrop…
Diffusion models improve image compression at low bit-rates.
problem Efficiently compressing images at very low bit-rates.
method Encoding into an embedding, using diffusion models to refine the embedding iteratively.
result Realistic reconstructions can be generated at extremely low bit-rates.
In information theory, Fisher information and Shannon information (entropy) are respectively used to quantify the uncertainty associated with the distribution modeling and the uncertainty in specifying the outcome of given variables. These two quantities are complementary and are jointly applied to information behavior…
A new method for generative modeling of discrete data using geometric latent subspaces.
problem Learning generative models for discrete data with statistical dependencies.
method Geometric latent-subspace framework in exponential parameter space of product manifolds of categorical distributions.
result Low-dimensional latent space encodes statistical dependencies and accurately models high-dimensional discrete data.
We show that classical thermodynamics has a formulation in terms of Hamilton-Jacobi theory, analogous to mechanics. Even though the thermodynamic variables come in conjugate pairs such as pressure/volume or temperature/entropy, the phase space is odd-dimensional. For a system with n thermodynamic degrees of freedom it …
Bipartite networks provide an insightful representation of many systems, ranging from mutualistic networks of species interactions to investment networks in finance. The analysis of their topological structures has revealed the ubiquitous presence of properties which seem to characterize many - apparently different - s…
DeepCMC compresses CSI for massive MIMO systems, reducing overhead and improving performance.
problem High CSI overhead in massive MIMO systems limits spectral efficiency.
method Deep learning-based fully convolutional neural network with residual layers and entropy coding.
result DeepCMC outperforms state-of-the-art schemes in CSI reconstruction quality for the same compression rate.
Improved GAN performance using higher-order Wasserstein moments.
problem Stabilizing and enhancing GANs for better mode coverage and stability.
method Deriving and training a GAN with a modified Wasserstein distance using higher-order moments.
result Training a GAN with higher-order Wasserstein moments improves performance, even with increased computational cost.