Deep learning shows ETF imbalances are more informative than market imbalances.
problem Determining causality between ETF and market imbalances.
method Deep learning econometric methodology applied to stock and ETF transactions.
result ETF imbalance messages are more informative than market imbalance messages.
Improved stock price prediction model using generalized order flow imbalance.
problem Improving stock price prediction models using new order flow imbalance indicators.
method Proposed a generalized order flow imbalance construction method and applied it to CSI 500 stocks.
result Generalized Stationarized Order Flow Imbalance (log-GOFI) shows significant improvement in explaining stock price changes.
This paper examines biases in foundation models under long-tailed data and proposes a method to mitigate parameter imbalance.
problem The bias introduced by imbalanced training data in foundation models affects long-tailed downstream tasks.
method The paper examines parameter imbalance and data imbalance, proposing a backdoor adjustment method to mitigate parameter imbalance.
result An average performance increase of about 1.67% on each dataset.
Recent studies have shown that imbalance ratio is not the only cause of the performance loss of a classifier in imbalanced data classification. In fact, other data factors, such as small disjuncts, noises and overlapping, also play the roles in tandem with imbalance ratio, which makes the problem difficult. Thus far, t…
We investigate the probability distribution of order imbalance calculated from the order flow data of 43 Chinese stocks traded on the Shenzhen Stock Exchange. Two definitions of order imbalance are considered based on the order number and the order size. We find that the order imbalance distributions of individual stoc…
This chapter tackles class imbalance in datasets to promote data democracy.
problem Class imbalance in datasets leading to biased decisions and policies.
method Statistical measures and data-level methods (oversampling, undersampling, etc.) applied to a real dataset.
result Popular data-level methods improve performance in handling class imbalance.
We analyze anomaly detection class imbalance using a solvable model.
problem Class imbalance hampers anomaly detection performance.
method We use an exact solution of the teacher-student perceptron model through replica theory.
result Optimal train imbalance is often different from 50%, influenced by intrinsic imbalance and data abundance.
Theoretical and empirical taxonomy of imbalance in binary classification.
problem Class imbalance degrades binary classification performance.
method Proposed a principled framework based on three scales: imbalance coefficient, sample-dimension ratio, and intrinsic separability. Derived closed-form Bayes errors and analyzed degradation across models.
result The triplet (η, κ, Δ) provides a model-agnostic explanation of imbalance-induced deterioration.
Proposes methods to improve multi-label learning by addressing local label imbalance.
problem Local label imbalance within minority class examples degrades multi-label learning performance.
method Introduces a measure to assess local label imbalance and two sampling approaches (MLSOL, MLUL) to address it.
result Experimental results show MLSOL and MLUL improve performance on multi-label datasets.
Online class imbalance learning constitutes a new problem and an emerging research topic that focusses on the challenges of online learning under class imbalance and concept drift. Class imbalance deals with data streams that have very skewed distributions while concept drift deals with changes in the class imbalance s…
Modeling price dynamics in response to order flow imbalance in Chinese futures markets.
problem Understanding price dynamics in markets with order flow imbalance.
method Modeling order flow imbalance as an Ornstein-Uhlenbeck process with memory and mean-reverting characteristics.
result Horizon-dependent heterogeneity in conventional metrics' interaction with order flow imbalance.
Without any specific way for imbalance data classification, artificial intelligence algorithm cannot recognize data from minority classes easily. In general, modifying the existing algorithm by assuming that the training data is imbalanced, is the only way to handle imbalance data. However, for a normal data handling, …
The study examines how class imbalance impacts logistic regression models in low-default credit portfolios.
problem The impact of class imbalance on logistic regression models in low-default credit portfolios.
method Simulation study with controlled data-generating mechanisms to vary class imbalance and predictor-response association strength.
result Classification accuracy decreases significantly as event rate decreases, and optimal cut-off shifts with imbalance.
New method addresses class imbalance in federated learning.
problem Class imbalance in federated learning training data.
method Proposes a monitoring scheme to infer training data composition and a new loss function, Ratio Loss, to mitigate imbalance.
result Demonstrates effectiveness in mitigating class imbalance in federated learning, outperforming previous methods.
The study analyzes trading imbalances from SEC Form 13F-HR filings to identify profitable trading opportunities.
problem Identifying profitable trading opportunities based on SEC Form 13F-HR filings.
method Statistical analysis of holdings variations and imbalances between consecutive reporting periods.
result A profitable trading strategy can be implemented by betting against the sign of imbalances with a 1-2 month time horizon.
Analysis shows data imbalance slows learning curves for minority and majority classes.
problem Data imbalance impacts learning dynamics and model performance.
method Theoretical analysis of GD and SGD, including per-class normalization.
result Data imbalance causes sub-optimal learning trajectories for minority and majority classes.
Study characterizes and mitigates imbalances in neurosymbolic learning.
problem Characterizing and mitigating class-specific risks in neural classifiers.
method Theoretical analysis and practical techniques including estimating marginal gold labels and mitigating imbalances at training and testing time.
result Learning imbalances can be greatly impacted by the symbolic component σ, unlike in supervised and weakly supervised learning.
Many real-world classification problems are significantly class-imbalanced to detriment of the class of interest. The standard set of proper evaluation metrics is well-known but the usual assumption is that the test dataset imbalance equals the real-world imbalance. In practice, this assumption is often broken for vari…
A novel approach reduces class imbalance in network traffic classification.
problem Severe class imbalance in network traffic leads to poor classification performance.
method Group & Reweight strategy: clusters classes, updates weights, optimizes model.
result Improves comprehensive performance in prediction and reduces class imbalance.
In this study, we systematically investigate the impact of class imbalance on classification performance of convolutional neural networks (CNNs) and compare frequently used methods to address the issue. Class imbalance is a common problem that has been comprehensively studied in classical machine learning, yet very lim…
Framework tackles class imbalance and noisy labels in active learning.
problem Class imbalance and noisy labels in real-world datasets.
method Uses foundation model priors to select informative samples for active learning.
result Substantial annotation savings (over 50%) with preserved performance and robustness.
Study reveals class disparities in balanced datasets through spectral imbalance.
problem Class disparities in balanced datasets are overlooked despite model performance gaps.
method Developed a theoretical framework and studied 11 encoders to diagnose spectral imbalance.
result Identified spectral imbalance as a source of class disparities in balanced datasets.
DIGRAC clusters directed graphs using flow imbalance, outperforming existing methods.
problem Clustering directed networks without label supervision.
method DIGRAC uses a graph neural network with a novel imbalance loss for directed flow imbalance.
result DIGRAC outperforms 10 state-of-the-art methods on directed graph clustering.
Analyzes how class imbalance and heterogeneity affect diffusion model learning dynamics.
problem Understanding how class imbalance and heterogeneity impact the learning dynamics of diffusion models.
method Developed a high-dimensional analytical framework to study class-dependent learning in score-based diffusion models.
result Class variance is the primary determinant of learning order, favoring higher-variance classes; centroid geometry plays a secondary role.
The paper introduces a new price model based on entropy that better fits high-frequency market data.
problem Understanding fair prices in high-frequency markets with bid-ask imbalance.
method A parametrized family of prices derived from the Maximum Entropy Principle, minimizing bias given volume imbalance.
result The model can generate higher kurtosis and heavy-tailed distributions compared to standard models.
Paper optimizes internal balancing of wind and hydropower to reduce intraday market volatility.
problem Reduction of intraday market volatility for power producers with wind and hydropower assets.
method Internal balancing within the same river system and sales/purchase in a pay-as-bid intraday market.
result Reduction in short-term marginal cost and risk through internal balancing.
A novel probabilistic approach forecasts imbalance prices in Belgium.
problem Forecasting imbalance prices in short-term energy markets.
method Two-step approach: compute net regulation volume state transition probabilities, then infer imbalance prices.
result The probabilistic approach outperforms deterministic and Gaussian Process models.
APC overcomes missing data and class imbalance in time series data.
problem Missing data and class imbalance in time series data.
method Self-supervised learning with Autoregressive Predictive Coding (APC).
result APC improves classification performance on real-world medical datasets.
Paper introduces a new performance metric for class imbalance datasets.
problem Challenges in selecting and comparing models for imbalanced datasets.
method Proposes a new performance measure based on the harmonic mean of Recall and Selectivity normalized in class labels.
result The proposed measure is less sensitive to changes in the majority class and more sensitive to changes in the minority class.
Study shows neural collapse is invariant to class imbalances under certain conditions.
problem Neural collapse properties are only valid for balanced data.
method Adopted UFM and introduced SELI for invariant characterization.
result Embeddings and classifiers always interpolate a simplex-encoded label matrix regardless of class imbalances.
Proposes a multimodal deep generative model for semi-supervised learning with class imbalance.
problem Class imbalance in semi-supervised learning with partial supervision.
method Separate encoders for each modality, sharing latent variables, and using Student's t-distributions for prior, encoder, and decoder.
result Outperforms baseline methods in generalization and classification performance for partially labeled multimodal data.
Study evaluates three class imbalance techniques across diverse datasets.
problem Class imbalance in binary classification tasks.
method Synthetic Minority Over-sampling Technique (SMOTE), Class Weights tuning, Decision Threshold Calibration.
result Decision Threshold Calibration is the most consistently effective technique.
Improves Active Learning by considering class imbalance and difficulty.
problem Active Learning's focus on individual samples ignores class distribution and difficulty.
method Proposes a method based on Bayes' rule to incorporate class imbalance, using a Variational Auto Encoder (VAE).
result Significantly outperforms state-of-the-art methods on datasets with heavy data imbalance.
Solves dual imbalance in detecting sparse anomalies in MIL.
problem Detecting scarce and sparse anomalous samples in MIL.
method Reformulates MIL as a fine-grained PU learning problem, addressing imbalance at both macro and micro levels.
result Demonstrates effectiveness of BFGPU framework on synthetic and real-world datasets.
Unified framework suppresses model bias in semi-supervised learning with decoupled sampling control.
problem Class imbalance in semi-supervised learning, especially with distributional mismatches.
method Unified framework SC-SSL with decoupled sampling control, explicit expansion capability, and adaptive sampling probabilities.
result Consistent and state-of-the-art performance across various benchmark datasets and distribution settings.
Adam outperforms gradient descent on language models due to handling heavy-tailed class imbalance.
problem Heavy-tailed class imbalance in language tasks.
method Comparing Adam and gradient descent on various architectures and data types, focusing on the impact of class imbalance.
result Class imbalance causes slow convergence for gradient descent, while Adam and sign-based methods are less affected.
The paper tackles imbalance in production data by proposing sampling methods to improve model performance on underrepresented observations.
problem Imbalance in production data negatively impacts model predictive performance on underrepresented observations.
method Three sampling approaches are investigated to adjust for imbalance in training data and improve model performance.
result Fitting a model using sampled data yields a small reduction in overall predictive performance but a better performance on underrepresented observations.
Paper addresses class imbalance in disk SMART dataset using GANs and genetic algorithms.
problem Class imbalance in disk SMART dataset.
method Data synthesised by multivariate GANs mixed with genetic algorithms.
result Higher disk fault classification prediction accuracy.
Study reveals optimal price prediction through volume imbalance analysis.
problem Understanding the relationship between prices and volume imbalance in high-frequency trading.
method Developed a market-making model to analyze price-imbalance connection and solve optimization problems.
result Optimal quoting of predictive imbalance is confirmed, useful for financial regulation.
Study finds option volume imbalance predicts equity market returns.
problem Predicting equity market returns using option volume imbalance.
method Nonlinear analysis of option volumes decomposed into five market participant classes.
result Strong signals of predictability of excess market returns from Market-Maker volumes.
Short-term probabilistic forecasting of German electricity imbalance prices.
problem Uncertainty in renewable energy capacity and electricity prices.
method Combining lasso with bootstrap, gamlss, and probabilistic neural networks for forecasting imbalance prices.
result Sophisticated methods improve empirical coverage of imbalance prices but do not substantially outperform the intraday continuous price index.
In this comment we discuss the problem of reconciling the linear efficiency of price returns with the long-memory of supply and demand. We present new evidence that shows that efficiency is maintained by a liquidity imbalance that co-moves with the imbalance of buyer vs. seller initiated transactions. For example, duri…
The paper investigates how data imbalance affects fairness and accuracy in differentially private deep learning.
problem Impact of data imbalance on fairness and accuracy in differentially private deep learning.
method Study the effects of different levels of imbalance in the data on the accuracy and fairness of decisions made by a model trained with differential privacy.
result Small imbalances and loose privacy guarantees can cause disparate impacts on model accuracy and fairness.
The study examines the generalization of Macro-AUC in multi-label learning, identifying label imbalance as a critical factor.
problem Theoretical understanding of Macro-AUC in multi-label learning is lacking.
method Characterization of generalization properties of learning algorithms based on surrogate losses w.r.t. Macro-AUC, identification of label imbalance as a critical factor.
result The widely-used univariate loss-based algorithm is more sensitive to label imbalance than pairwise and reweighted loss-based ones, implying worse performance.
New metric shows how different regularization methods affect deep linear networks.
problem Understanding the training dynamics of deep linear networks.
method Introduced a new metric called layer imbalance to analyze training dynamics. Demonstrated behavior of different regularization methods and stochastic gradient descent.
result Different regularization methods behave similarly, leading to a flat minima.
Study of linear classifiers in infinite imbalance scenarios.
problem Behavior of linear discriminant functions in extreme imbalance conditions.
method Analysis of linear classifiers under infinite imbalance, focusing on weight function properties and limit behavior.
result Limiting coefficient vectors reflect robustness or conservatism, optimizing against worst-case alternatives.
This paper improves robot traders' market impact sensitivity.
problem Market impact in automated trading systems.
method Critiqued existing methods, introduced MLOFI, and demonstrated new algorithms.
result New imbalance-sensitive trader-agents exhibit market impact effects.
Class imbalance is a pervasive issue among classification models including deep learning, whose capacity to extract task-specific features is affected in imbalanced settings. However, the challenges of handling imbalance among a large number of classes, commonly addressed by deep learning, have not received a significa…