Study shows how information loss and operation loss are related in feature representations.
problem Understanding the relationship between information loss and operation loss in feature representations.
method Analyzes the interplay between weak information loss and operation loss in continuous representations.
result Specific forms of vanishing information loss imply vanishing MPE loss in classification.
Informer model with GMADL loss outperforms benchmarks in high frequency Bitcoin trading.
problem Developing automated trading strategies for high frequency Bitcoin data.
method Informer architecture with RMSE, GMADL, and Quantile loss functions.
result Informer model with GMADL loss function outperforms benchmarks in trading outcomes.
The paper decomposes probabilistic scores into reliability, uncertainty, and information loss.
problem Understanding the reliability and uncertainty of probabilistic predictions.
method Developed decomposition identities for proper losses, quantifying reliability, residual uncertainty, and information gain.
result A three-term identity for classification scores, revealing miscalibration, grouping term, and feature-level uncertainty.
Topology-enhanced loss improves 3D object reconstruction from 2D images.
problem Challenges in reconstructing 3D objects from 2D images, especially capturing shape information.
method Integrates multi-scale topological features into the reconstruction loss using cubical complexes and optimal transport distance.
result Topology-aware loss substantially improves 3D reconstruction quality.
New tools analyze data selection methods, reducing information losses.
problem Analyzing and reducing information losses in data selection.
method Information theoretic losses framework applied to Facility Location Selection and Transductive Experimental Design.
result Proves Facility Location Selection and Transductive Experimental Design reduce information losses.
We present and study a partial-information model of online learning, where a decision maker repeatedly chooses from a finite set of actions, and observes some subset of the associated losses. This naturally models several situations where the losses of different actions are related, and knowing the loss of one action p…
Forecastability measures predictive information across horizons.
problem How much predictive information is available at each prediction horizon?
method Develops the consequences of mutual information between future observations and information set.
result Forecastability is a profile reflecting process dependence structure, with properties like compression and truncation error.
The paper bounds information losses in neural classifiers from sampling.
problem Information losses in neural classifiers from finite datasets.
method Proves a relationship between information losses and expected total variation of the estimated neural model, bounds this expected total variation as a function of dataset size.
result Obtains bounds on information losses that are less sensitive to input compression and much smaller than existing bounds.
The abstract discusses how diversification and securitization lead to information losses in financial risk optimization.
problem Information loss in financial risk optimization practices.
method Information theoretic concepts to quantify information losses in financial transformations and portfolios.
result Diversification and securitization increase information sensitivity, leading to maximal information losses when assets are uncorrelated.
New bounds on neural network test loss derived from conditional information measures.
problem Estimating test loss of neural networks trained on limited data.
method Framework based on conditional information density between hypothesis and training set.
result Tail bounds on test loss decay as 1/n, improving over previous 1/sqrt{n} bounds.
New PAC-Bayes method updates priors without losing confidence information.
problem Lack of sequential prior updates in PAC-Bayes without losing confidence information.
method Recursive PAC-Bayes decomposition of expected loss.
result Sequential prior updates with no information loss.
Cross-entropy loss linked to metric learning, outperforming complex pairwise losses.
problem Improving metric learning performance without complex optimization schemes.
method Theoretical analysis linking cross-entropy to pairwise losses, showing cross-entropy as an upper bound and equivalent to mutual information maximization.
result Minimizing cross-entropy is equivalent to maximizing mutual information, leading to state-of-the-art performance.
Anonymization reduces economic signal extraction from financial texts.
problem Reducing meaningful economic signals from financial texts due to anonymization.
method Analyzed the impact of anonymization on textual understanding and economic signal extraction.
result Information loss due to anonymization is severe and pervasive, outweighing its benefits in certain financial applications.
Automates loss function selection for faster, more robust model training.
problem Manual selection of loss functions is inefficient and model-specific.
method Meta-learning to learn parametric loss functions that generalize across tasks.
result Meta-trained loss functions improve model performance in supervised and reinforcement learning.
DE-QT detects optimal Q-learning stopping points.
problem Information loss in Q-learning during prolonged training.
method Introducing DE-QT to detect entropy changes in Q-tables.
result DE-QT identifies the best stopping point for Q-learning.
Loss to followup is a significant issue in healthcare and has serious consequences for a study's validity and cost. Methods available at present for recovering loss to followup information are restricted by their expressive capabilities and struggle to model highly non-linear relations and complex interactions. In this…
We stabilize MI-based losses by adding a regularization term, improving their performance and stability.
problem Instability of MI-based losses in machine learning.
method Added a novel regularization term to stabilize MI-based losses.
result Regularization stabilizes training and improves the performance of MI-based losses.
Calibrating classifiers reduces grouping loss using sufficiency criteria.
problem Grouping loss in probabilistic classifier calibration is often overlooked.
method Revisited Langford & Zadrozny's probing reduction approach and introduced Brier curves.
result The probing reduction approach reduces grouping loss and supports sufficient calibration.
Unified framework for faster neural network training with less information loss.
problem Time-consuming backpropagation and loss of unpropagated gradient information.
method Unified sparse backpropagation framework and memorized sparse backpropagation algorithm.
result Convergence in probability with certain conditions and effective information loss mitigation.
Feature noise causes loss discrepancies across groups even with equal data.
problem Loss discrepancies observed in learning procedures across different groups.
method Characterized the effect of feature noise on loss discrepancy in linear regression.
result Feature noise leads to loss discrepancy even when groups have equal data.
Boosting can efficiently optimize any loss function without requiring first-order information.
problem Boosting's efficiency in optimizing loss functions without first-order information.
method Extending gradient-based optimization to use only zeroth-order information.
result Boosting can optimize any loss function efficiently, including non-convex, non-differentiable, and non-continuous ones.
Geodesics connect model modes in neural network loss landscapes.
problem Connecting modes in neural network loss landscapes.
method Reframed mode connectivity in Information Geometry, hypothesized geodesics as mode-connecting paths, proposed algorithm to approximate geodesics.
result Geodesics achieve mode connectivity in neural networks.
Paper proposes fitting loss functions to data using source functions from information geometry.
problem Choosing appropriate loss functions for machine learning models.
method Introduces source functions from information geometry to fit loss functions to the domain at hand.
result Significant improvements over state-of-the-art methods in model training.
A new method for survival analysis models that ensures fairness without using sensitive demographic data.
problem Ensuring fairness in survival analysis models without relying on sensitive demographic information.
method A worst-case error minimization approach using a training loss function that does not know sensitive demographic information.
result The proposed method often scores better on fairness metrics without a significant drop in prediction accuracy compared to baselines.
This paper improves traffic flow modeling by using multi-gradient descent algorithms for physics-informed machine learning.
problem Combining physics-based and data-driven approaches in traffic flow modeling.
method Introducing multi-gradient descent algorithms to explore the Pareto front in a multi-objective setting.
result Multi-gradient descent algorithms significantly outperform scalarization-based methods in complex PIML scenarios.
A new method for automatic gradient tree boosting using information theory.
problem Automatic selection of tree complexity and number in gradient boosting.
method Optimism of greedy leaf splitting procedure modeled as a Cox-Ingersoll-Ross process, leading to an information criterion for model selection.
result The method achieves significant speedups (10-1400) compared to xgboost without sacrificing predictive power.
Proposes a cross entropy loss for better ranking algorithms.
problem Improving the theoretical understanding and performance of ranking algorithms.
method Introduces a cross entropy-based loss function that is a convex bound on NDCG and consistent with NDCG.
result Empirically, the proposed method outperforms existing algorithms in quality and robustness.
New bounds using samplewise evaluated CMI for deep neural networks.
problem Improving generalization bounds for deep neural networks.
method Introduced a new family of information-theoretic generalization bounds using samplewise evaluated conditional mutual information (CMI).
result The new bounds can be tighter than previous ones for deep neural networks.
Sign information is the key to overcoming the inevitable saturation error in compressive sensing systems, which causes information loss and results in bias. For sparse signal recovery from saturation, we propose to use a linear loss to improve the effectiveness from existing methods that utilize hard constraints/hinge …
Sampling a fraction of pairs can match full evaluation in machine learning losses.
problem High computational cost of full pairwise loss evaluation.
method Survey sampling techniques targeting informative pairs.
result Performance close to full pairwise evaluation achieved with frugal sampling.
New methods evaluate data representations by complexity of low-loss predictor learning.
problem Evaluating quality of data representations for downstream tasks.
method Surplus Description Length (SDL) and ε Sample Complexity (εSC) methods.
result Methods measure the information needed to approximate optimal predictor up to specified tolerance.
Investors suffer welfare loss despite having better information.
problem Welfare loss among investors with absolute information advantages.
method Examined financial markets with heterogenous investors and objective measures of welfare.
result Investors incur welfare loss even with better information, revealing a double loss phenomenon.
Improved malware detection by adding auxiliary loss terms to a neural network.
problem Malware detection accuracy with a single label.
method Fit deep neural networks to multiple auxiliary prediction targets derived from metadata.
result Significant improvement in detection performance, reducing false negatives by 42.6% at a low false positive rate.
The study examines how formal index insurance compares to informal risk sharing in managing natural disasters.
problem The challenges of natural disasters and the effectiveness of index insurance in risk management.
method A three-strategy evolutionary game model to analyze the competitive relationship between formal index insurance, informal risk sharing, and non-insurance.
result Basis risk and loss ratio significantly impact the adoption rate of index insurance, with different strategies preferred under varying conditions.
Using established principles from Statistics and Information Theory, we show that invariance to nuisance factors in a deep neural network is equivalent to information minimality of the learned representation, and that stacking layers and injecting noise during training naturally bias the network towards learning invari…
This study examines fees in AMMs to reduce losses from informed orderflow.
problem Minimizing losses from informed orderflow in AMMs.
method Modeling arbitrage dynamics and sensitivity to fee choices.
result Identified fees that mimic price directionality to reduce losses.
Proposes a new loss function for robust training of deep neural networks against noisy labels.
problem Training deep neural networks with noisy labels, especially instance-independent noise.
method Introduces a novel information-theoretic loss function, L_DMI, based on Determinant based Mutual Information (DMI).
result L_DMI is the first provably robust loss function to instance-independent label noise, without requiring auxiliary information.
Paper improves phase identification in power systems using information theory.
problem Improving supervised learning accuracy in phase identification.
method Developed two new techniques based on information theory.
result Significant improvement in phase identification accuracy (e.g., from 51.7% to 97.3%).
New bounds derived for machine learning algorithms using convex functions.
problem Bounding generalization error in machine learning.
method Using strongly convex functions and subgaussian loss tails, derived new generalization bounds.
result Generalization bounds can be derived using any strongly convex function of the joint input-output distribution.
Improves GANs by incorporating class information with a multi-hinge loss.
problem Improving GANs to better respect class information.
method Proposes a multi-class generalization of the Hinge loss for GANs.
result Improves Inception Scores and Frechet Inception Distance on Imagenet.
Develops LSH schemes for f-divergences and mutual information loss.
problem Approximating nearest neighbors in high-dimensional probability distributions.
method General framework and specific LSH schemes for f-divergences and mutual information loss.
result Generalized Jensen-Shannon divergence can be approximated by Hellinger distance.
Optimal insurance contracts are designed to screen risk preferences and risk types under asymmetric information.
problem Designing optimal insurance contracts under asymmetric information and risk types.
method Constructing a menu of contracts that maximizes mean-variance utilities, subject to truth-telling constraints.
result Equilibrium contracts exhibit nonlinear pricing with decreasing risk loadings, inducing self-selection.
Poisson learning doesn't solve graph semi-supervised learning issues.
problem Global information loss in graph-based semi-supervised learning.
method Poisson learning is Laplace regularization with thresholding.
result Poisson learning cannot overcome the global information loss problem.
Ranking is a key aspect of many applications, such as information retrieval, question answering, ad placement and recommender systems. Learning to rank has the goal of estimating a ranking model automatically from training data. In practical settings, the task often reduces to estimating a rank functional of an object …
This paper proposes an alternative to E2E training for deep networks, reducing memory footprint.
problem High GPUs memory footprint in end-to-end training of deep networks.
method Locally supervised learning with information propagation loss to avoid information collapse.
result The proposed method achieves competitive performance with less than 40% memory footprint compared to E2E training.
Proposes a method to improve financial time series forecasting using compact representations and contrastive loss.
problem Financial time series forecasting with small datasets and overfitting issues.
method Class-conditioned latent variable model, mutual information maximization, contrastive loss, deep autoregressive models.
result Empirical experiments show improved performance compared to state-of-the-art methods.
Unified framework for expert selection with bandit and lower-bound feedback.
problem Selecting the best expert in scenarios with bandit feedback and lower-bound information.
method Introduces a new feedback model combining bandit and lower-bound information, proving optimal regret bounds for modified Exp3 algorithms.
result Optimal regret bounds for modified Exp3 algorithms, generalizing both bandit and full-information settings.
Study excess risk in statistical inference with transformations.
problem Excess risk in estimating random variables from feature vectors and transformations.
method Characterize lossless transformations, develop test statistics, and information-theoretic bounds.
result Strongly consistent partitioning test statistic for lossless transformations.