ADAPT method controls balance between sparse generator and discriminator in GANs.
problem High computational complexity in GANs training.
method Balanced Dynamic Sparse Training (ADAPT) method.
result ADAPT method achieves a good trade-off between performance and computational cost.
MTAdam optimizes multiple loss terms in neural models, balancing gradients dynamically.
problem Balancing multiple loss terms in neural model training is challenging and computationally demanding.
method Generalized Adam algorithm that computes separate derivatives and balances gradients across layers dynamically.
result Training with MTAdam leads to faster recovery from suboptimal initial loss weighting and matches conventional training outcomes.
This work improves policy-based training by proposing an evaluation balance objective for GFlowNets.
problem Reliable estimation of policy divergence under directed acyclic graphs remains challenging.
method Proposes an evaluation balance objective over partial episodes to measure policy divergence and improve policy-based training reliability.
result Evaluation balance strengthens policy-based training reliability and broadens its flexibility.
Unified formula for training dynamics of linear networks combining lazy and balanced regimes.
problem Training dynamics of linear networks in two distinct setups: lazy and balanced/active.
method Unified formula for the evolution of the learned matrix, combining lazy and balanced regimes.
result Unified formula allows for rapid convergence and low rank bias, proving a complete phase diagram.
Balanced Activation improves object detection performance on long-tailed datasets.
problem Mismatch between training and testing label distributions in object detection.
method Introduces Balanced Activation (Balanced Softmax and Balanced Sigmoid) to address label distribution shift.
result Balanced Activation provides ~3% gain in mAP on LVIS-1.0 compared to state-of-the-art methods.
Compared to in-clinic balance training, in-home training is not as effective. This is, in part, due to the lack of feedback from physical therapists (PTs). Here, we analyze the feasibility of using trunk sway data and machine learning (ML) techniques to automatically evaluate balance, providing accurate assessments out…
Proposes unbiased estimators for training mixture of experts models.
problem Efficiently training large-scale mixture of experts models on modern hardware.
method Two unbiased estimators based on principled stochastic assignment procedures.
result Both estimators are more effective and robust than biased alternatives.
Proposes φ-balancing for more balanced expert utilization in MoE models.
problem Balanced expert utilization in MoE models to avoid bias.
method Directly targets population-level balance by minimizing a convex potential function.
result Consistently outperforms prior methods in stability and effectiveness.
Proposes a simple framework to balance task difficulty in multi-task learning.
problem Varying difficulty levels among different tasks in multi-task learning.
method Introduces a Balanced Multi-Task Learning (BMTL) framework that transforms training losses to balance task difficulty.
result Empirical studies show state-of-the-art performance of the proposed BMTL framework.
TAET tackles long-tailed distributions in adversarial robustness.
problem Long-tailed distributions complicate adversarial robustness in real-world applications.
method TAET integrates an initial stabilization phase followed by a stratified equalization adversarial training phase.
result TAET achieves significant improvements in robustness and efficiency.
In many important machine learning applications, the training distribution used to learn a probabilistic classifier differs from the testing distribution on which the classifier will be used to make predictions. Traditional methods correct the distribution shift by reweighting the training data with the ratio of the de…
Paper introduces Balanced Meta-Softmax for better long-tailed visual recognition.
problem Long-tailed distribution mismatch between training and testing data.
method Balanced Meta-Softmax, an unbiased extension of Softmax, using a Meta Sampler.
result Balanced Meta-Softmax outperforms state-of-the-art solutions on visual recognition and instance segmentation.
Variational inference improves training of generative flow networks.
problem Training generative flow networks efficiently and accurately.
method Define variational objectives in terms of KL divergences and optimize convex combinations.
result Variational inference methods can reduce the variance of gradients in training generative flow networks.
TempBalance improves neural network training by balancing learning rates across layers.
problem Improving neural network training performance.
method Layer-wise learning rate scheduling based on HT-SR Theory.
result TempBalance significantly outperforms ordinary SGD and state-of-the-art optimizers.
Paper proposes ARB-Loss to improve classification precision in imbalanced datasets.
problem Improving classification precision on minor classes in imbalanced datasets.
method Introduces Attraction-Repulsion-Balanced Loss (ARB-Loss) to balance gradients across different classes.
result ARB-Loss achieves state-of-the-art performance with one-stage training.
Proposes a method to balance tasks in multitask learning with a single gradient step update.
problem Balancing tasks in multitask learning to avoid imbalance.
method Gradient-based meta-learning to balance tasks at the gradient level, training shared and task-specific layers separately.
result Achieves state-of-the-art performance on various multitask computer vision problems.
TempBalance boosts model performance with low data.
problem Low-data training and fine-tuning in model alignment.
method Inspired by HT-SR theory, TempBalance balances training quality across layers.
result TempBalance improves model performance as data decreases.
We study optimal covariate balance for causal inferences from observational data when rich covariates and complex relationships necessitate flexible modeling with neural networks. Standard approaches such as propensity weighting and matching/balancing fail in such settings due to miscalibrated propensity nets and inapp…
Study examines how balancing methods affect model behavior in imbalanced classification problems.
problem Impact of balancing methods on model behavior in imbalanced classification problems.
method Used Explainable Artificial Intelligence tools (variable importance method, partial dependence profile, accumulated local effects) to compare model behavior before and after balancing.
result Significant changes in model behavior due to balancing methods, leading to biased models.
FairACE improves fairness in GNNs by balancing node performance across degree groups.
problem Degree biases in GNNs lead to unequal prediction performance among nodes with varying degrees.
method Integrates asymmetric contrastive learning with adversarial training to balance performance between high-degree and low-degree nodes.
result Significantly improves degree fairness metrics while maintaining competitive accuracy.
New method trains GFlowNets from partial episodes to improve convergence and stability.
problem Improving convergence and stability of GFlowNets training.
method Introducing SubTB(λ) for GFlowNet training from partial action subsequences. result SubTB(λ) accelerates GFlowNet convergence and enables training in longer action sequences. CUBE explains models by balanced experiments and contrasts.
problem Post-hoc explanation of trained predictive models.
method Design-based framework using balanced low-high probes.
result Reveals dominant learned effect structure and clarifies query efficiency.
Distributed machine learning training is one of the most common and important workloads running on data centers today, but it is rarely executed alone. Instead, to reduce costs, computing resources are consolidated and shared by different applications. In this scenario, elasticity and proper load balancing are vital to…
Image classification datasets are often imbalanced, characteristic that negatively affects the accuracy of deep-learning classifiers. In this work we propose balancing GAN (BAGAN) as an augmentation tool to restore balance in imbalanced datasets. This is challenging because the few minority-class images may not be enou…
Stable GFlowNets prevent loss spikes and mode collapse in training.
problem Unstable training of GFlowNets leading to loss spikes and mode collapse.
method Assessed sensitivity of GFlowNet objectives, derived loss-to-TV bounds, and proposed Stable GFlowNets.
result Stable GFlowNets improve training behavior and distributional fidelity.
Regularization leads to balancedness in deep linear networks.
problem Balancedness in deep linear networks.
method Geometric invariant theory and Riemannian geometry of fibers.
result Balancing flows converge to the balanced manifold at a uniform exponential rate.
New approach tackles class imbalance in long-tailed datasets using domain adaptation techniques.
problem Class imbalance in long-tailed datasets leading to poor model performance.
method Proposes a meta-learning approach to estimate differences between class-conditioned distributions.
result Validated approach on six benchmark datasets and three loss functions.
Proposes a balanced multi-component and multi-layer neural network for efficient function approximation.
problem Accurately and efficiently approximating complex functions with high degrees of freedom and computational cost.
method Inspired by a multi-component approach, MMNN combines single-layer networks with a multi-layer decomposition strategy.
result Significant reduction in training parameters, more efficient training process, and improved accuracy compared to FCNNs or MLPs.
This paper proposes BAT to balance accuracy and robustness in adversarial training.
problem Balancing accuracy and robustness in adversarial training models.
method Blind adversarial training (BAT) uses a cutoff-scale strategy to adaptively estimate a nonuniform budget for AEs.
result BAT improves the overall robustness of adversarial training models.
Balance corrects biased survey data for more accurate insights.
problem Bias in survey data leads to inaccurate insights and underperforming models.
method Three steps: bias understanding, weight adjustment, and evaluation.
result Corrected data leads to more accurate ML model training and insights.
BOSS learns from one labeled sample per class to match fully supervised performance.
problem Achieving fully supervised performance with minimal labeled data.
method Combines class prototype refining, class balancing, and self-training.
result BOSS achieves comparable test accuracies to fully supervised learning.
DiPriMe forests use private medians to create balanced tree splits for privacy-protected data.
problem Privacy concerns in training random forests due to multiple data queries.
method Proposes DiPriMe forests, which use a private median to generate balanced splits, ensuring differential privacy.
result DiPriMe forests achieve high utility while maintaining differential privacy, as shown both theoretically and empirically.
New meta-learning method improves domain generalization by balancing parameters closer to domain centroids.
problem Improving domain generalization by reducing overfitting to specific domains.
method Arithmetic meta-learning with arithmetic-weighted gradients to balance parameters closer to domain centroids.
result Experimental validation of improved domain generalization performance.
New insights into how large learning rates affect transformer training dynamics.
problem Understanding how large learning rates impact the training of transformer models.
method Analyzing a simplified linear transformer model with a two-factor product map.
result Large learning rates can lead to various training outcomes including cycles, chaos, or divergence.
Contextual bandit algorithms are sensitive to the estimation method of the outcome model as well as the exploration method used, particularly in the presence of rich heterogeneity or complex outcome models, which can lead to difficult estimation problems along the path of learning. We develop algorithms for contextual …
Doubly robust self-training improves semi-supervised learning by balancing labeled and pseudo-labeled data.
problem Improving semi-supervised learning performance with limited labeled data.
method Introduces doubly robust self-training, a method that combines labeled and pseudo-labeled data to balance between labeled-only and pseudo-labeled-only training.
result Demonstrates superior performance of doubly robust self-training on ImageNet and nuScenes datasets.
Paper proposes D3M to improve anti-spoofing detection by balancing loss function and using complementary features.
problem Improving automatic speaker verification systems against high-quality playback attacks.
method D3M uses a balanced focal loss function to dynamically scale loss based on sample traits, and combines three feature types for robust detection.
result D3M systems outperform conventional methods significantly, achieving min-tDCF of 0.0124 and EER of 0.55%.
CBDA improves active learning for semantic segmentation, especially with imbalanced classes.
problem Class imbalance degrades performance in domain adaptive active learning.
method Class Balanced Dynamic Acquisition (CBDA) selects more balanced labels for active learning.
result CBDA increases minority class performance and outperforms baselines by 0.6-2.4 mIoU.
Adaptive weights improve physics-informed neural networks and deep operator networks.
problem Training physics-informed neural networks and deep operator networks can be challenging, leading to unsatisfactory accuracy and efficiency.
method Proposes a pointwise adaptive weighting method that balances the residual decay rate across different training points.
result Our proposed approach of balanced residual decay rates offers advantages including bounded weights, high prediction accuracy, fast convergence rate, low training uncertainty, low computational cost, and ease of hyperparameter tuning.
A new GCN model detects cryptocurrency fraud by considering network evolution and balance theory.
problem Detecting fraud in evolving signed cryptocurrency trust networks.
method Motif-aware temporal GCN using balance theory and learnable weights.
result The model outperforms existing methods on bitcoin datasets.
This study improves electricity price forecasting in the Irish balancing market.
problem Limited and inconsistent research on short-term price forecasting in volatile balancing markets.
method Compared statistical, machine learning, and deep learning models using a public dataset and framework.
result LEAR, a statistical approach, outperforms complex models in the balancing market.
Flashback Learning balances model stability and plasticity in continual learning.
problem Balancing model stability and plasticity in continual learning.
method Flashback Learning (FL) uses a bidirectional regularization approach to balance stability and plasticity.
result FL improves model accuracy by up to 4.91% in Class-Incremental and 3.51% in Task-Incremental settings.
Study shows pre-trained models can handle long-tailed relations well, improving classifier performance.
problem Challenges in long-tailed relation classification due to class imbalance.
method Used instance-balanced sampling to pre-train models and then improved classifier performance through attentive relation routing.
result Robust classifier with attentive relation routing achieves better long-tailed classification ability.
The paper addresses classification imbalance by framing it as a transfer learning problem.
problem Classification imbalance where one class is much rarer than the other.
method The paper studies oversampling procedures to balance classes, focusing on SMOTE and bootstrapping.
result The excess risk decomposes into balanced training rate and transfer cost, with SMOTE having a higher transfer cost.
In real-world classification problems, the class balance in the training dataset does not necessarily reflect that of the test dataset, which can cause significant estimation bias. If the class ratio of the test dataset is known, instance re-weighting or resampling allows systematical bias correction. However, learning…
Gradient descent in deep networks tends to find flat minima, which are nearly balanced.
problem Understanding the effect of gradient descent on the structure of minima in deep neural networks.
method Characterized flat minima in linear neural networks trained with a quadratic loss.
result Flat minima correspond to nearly balanced networks where the gain from input to intermediate representations is nearly constant.
A new Federated Learning approach balances personalization and global training.
problem Breaking the curse of data heterogeneity in Federated Learning.
method Splitting variables into global and local parameters, using a simple algorithm.
result The approach allows each client to fit their data perfectly, breaking the curse of data heterogeneity.
The ability to rank candidate architectures is the key to the performance of neural architecture search~(NAS). One-shot NAS is proposed to reduce the expense but shows inferior performance against conventional NAS and is not adequately stable. We investigate into this and find that the ranking correlation between archi…