Dynamic weights improve multimodal emotion and gender recognition.
problem Improving performance in multiple classification tasks with a single model.
method Dynamic joint loss weights for multimodal emotion and gender recognition.
result Lower joint loss and better generalizability than static weights.
Square loss performs comparably or better than cross-entropy in neural architectures for various tasks.
problem The superiority of cross-entropy loss over square loss in classification tasks is debated.
method Comparison of several neural architectures on NLP, ASR, and computer vision datasets using both loss functions.
result Square loss often produces better results in the majority of tasks, especially in NLP and ASR.
A new method optimizes neural sequence models for better task performance.
problem Training neural sequence models with maximum likelihood estimation ignores task losses.
method Maximum likelihood guided parameter search (MGS) in the parameter space.
result MGS optimizes sequence-level losses, reducing repetition and non-termination.
Generalizes robust loss functions for improved performance in various tasks.
problem Improving performance on tasks like registration and clustering.
method Introduces a continuous robustness parameter into loss functions, allowing them to be generalized.
result Improves performance on learning-based tasks like generative image synthesis and unsupervised depth estimation.
Adapts auxiliary losses using gradient similarity to improve neural network performance.
problem Statistical inefficiency in neural networks and difficulty in selecting helpful auxiliary tasks.
method Uses cosine similarity between gradients of tasks to adaptively weight auxiliary losses.
result Guaranteed convergence to critical points of the main task and practical usefulness across domains.
HydaLearn dynamically adjusts task weights for better MTL performance.
problem Constant loss weights in MTL lead to poor results due to drifting relevance and varying mini-batch composition.
method HydaLearn uses mini-batch gradients to dynamically adjust task weights.
result HydaLearn improves performance on synthetic and real-world data.
Task loss matching misrepresents similarity between neural network layers.
problem Measuring similarity between neural network layers using task loss matching.
method Task loss matching vs. direct matching; comparison with CCA and CKA.
result Direct matching provides a better similarity index than task loss matching.
We consider the problem of training probabilistic conditional random fields (CRFs) in the context of a task where performance is measured using a specific loss function. While maximum likelihood is the most common approach to training CRFs, it ignores the inherent structure of the task's loss function. We describe alte…
Paper introduces Fenchel-Young losses for supervised learning tasks.
problem Choosing the right loss function for supervised learning tasks.
method Introduces Fenchel-Young losses as a generic way to construct convex loss functions.
result Fenchel-Young losses unify and create new loss functions.
TOPNet integrates task-based evaluation into machine learning models.
problem Non-differentiable task-based evaluation criteria in real-world applications.
method Task-Oriented Prediction Network (TOPNet) with learnable surrogate loss function.
result TOPNet significantly outperforms traditional and heuristic models in financial prediction tasks.
Tamed Cross Entropy (TCE) loss outperforms standard CE loss in noisy classification tasks.
problem Improving classification performance in noisy data scenarios.
method Introducing Tamed Cross Entropy (TCE) loss, a derivative of Cross Entropy (CE) loss.
result TCE loss outperforms CE loss in all tested noisy classification scenarios.
Paper analyzes proper losses and their performance in machine learning tasks.
problem Understanding the performance of estimators and forecasters in machine learning tasks.
method Analyzes surrogate regret and convergence rates for strictly proper losses.
result Strongly proper losses achieve the optimal convergence rate.
Proposes a simple framework to balance task difficulty in multi-task learning.
problem Varying difficulty levels among different tasks in multi-task learning.
method Introduces a Balanced Multi-Task Learning (BMTL) framework that transforms training losses to balance task difficulty.
result Empirical studies show state-of-the-art performance of the proposed BMTL framework.
Novel ramp loss method improves weakly supervised machine translation and parsing.
problem Training neural models without gold labels in weak supervision scenarios.
method Adapted ramp loss objectives to promote positive outputs and discourage negative ones.
result Bipolar ramp loss objectives outperform other methods on weakly supervised tasks.
Automates loss function selection for faster, more robust model training.
problem Manual selection of loss functions is inefficient and model-specific.
method Meta-learning to learn parametric loss functions that generalize across tasks.
result Meta-trained loss functions improve model performance in supervised and reinforcement learning.
Unified framework for fair regression under demographic parity.
problem Ensuring fairness in regression tasks subject to demographic parity constraints.
method Proposes a unified framework applicable to various regression tasks with a broad spectrum of loss functions, derived a novel characterization of the fair risk minimizer, and established theoretical consistency and convergence rates.
result Effective minimization of risk while satisfying fairness constraints across various regression settings.
TPG-DNN predicts user intent using GRU loss and multi-task learning.
problem Improving user experience and shopping efficiency in e-commerce.
method Adaptive GRU loss function with multi-task learning.
result TPG-DNN outperforms existing CTR models on Taobao datasets.
TA-VAAL improves active learning by better utilizing task structures and overall data distribution.
problem High labeling cost limits deep learning applications; active learning selects informative samples.
method Task-aware variational adversarial active learning (TA-VAAL) modifies VAAL by relaxing task loss prediction and using ranking loss information.
result TA-VAAL outperforms state-of-the-arts on various datasets, including balanced and imbalanced labels.
TaskMet learns a metric to improve model performance on unseen tasks.
problem Deep models trained on one task may struggle on another task due to conflicting objectives.
method TaskMet learns a metric in the prediction space to balance task and prediction losses.
result TaskMet achieves better performance on downstream tasks without altering the prediction model.
AuxiLearn combines auxiliary tasks into a single loss function.
problem Improving neural network performance on a main task using auxiliary tasks.
method Implicit differentiation to learn a network that combines auxiliary tasks into a single coherent objective function.
result AuxiLearn consistently outperforms competing methods in various tasks and domains.
Improves continual learning with theoretical guarantees and a new algorithm.
problem Learning incremental tasks with dynamic data distributions.
method Contrastive and distillation losses with theoretical performance guarantees.
result Theoretical performance bounds and improved continual learning performance.
New method finds better loss functions for neural nets.
problem Finding effective loss functions for deep neural networks.
method Optimizes multivariate Taylor polynomial parameterizations using CMA-ES.
result TaylorGLO finds loss functions that outperform existing methods.
Improves cross-modal retrieval by integrating unlabeled data.
problem Lack of semantic similarity constraints and unlabeled data in cross-modal retrieval.
method Integrates quadruplet ranking loss and semi-supervised contrastive loss in a multi-task learning architecture.
result Boosts cross-modal retrieval accuracy by exploiting unlabeled data.
PACMAN provides bounds for classification tasks considering accuracy vs. negative log-loss mismatch.
problem Mismatch between accuracy and negative log-loss in classification tasks.
method Point-wise PAC approach over generalization gap, using likelihood ratio and concentration inequalities.
result PACMAN provides point-wise PAC bounds for the generalization problem.
Paper proposes a principled method to learn loss functions for supervised learning tasks.
problem Choosing an appropriate loss function for supervised learning tasks.
method The paper revisits and generalizes the SLIsotron algorithm using Bregman divergences.
result The BregmanTron algorithm learns both the loss and classifier, with convergence guarantees.
TripletGAN uses triplet loss to improve generative models, preventing mode collapse.
problem Mode collapse in generative models.
method Substituting discriminator's classification loss with triplet loss.
result TripletGAN helps prevent mode collapse and converges to the given distribution.
The paper predicts loss scaling across different datasets and compute scales.
problem Predicting loss scaling across different datasets and compute scales.
method Derive shifted power law relationships between train and test losses.
result Shifted power law relationships hold for various datasets and tasks, improving prediction accuracy.
Paper tackles Byzantine resilience in distributed multi-task learning.
problem Resilience of distributed algorithms in the presence of Byzantine agents.
method Online weight assignment rule based on accumulated loss and filtering.
result Aggregation with proposed weight assignment rule improves expected regret.
Adaptive loss function improves performance by aligning training and evaluation metrics.
problem Loss-metric mismatch in machine learning training.
method Adaptive loss alignment through meta-learning of a dynamic loss function.
result Significant performance improvements across various tasks and data.
The Z-loss improves efficiency and flexibility in neural network training.
problem Efficiency and flexibility in training large neural networks.
method Introducing the Z-loss, a spherical loss function.
result The Z-loss significantly outperforms log-softmax in efficiency and flexibility.
Novel model improves clinical risk prediction by transferring knowledge between tasks over time.
problem Negative transfer in multi-task learning for clinical risk prediction.
method Temporal Probabilistic Asymmetric Multi-Task Learning (TPAMTL).
result Significantly outperforms various deep learning models for time-series prediction.
Enhances ENet's prediction accuracy while maintaining uncertainty estimation.
problem Gradient shrinkage problem in ENet's loss function.
method Proposes a multi-task learning framework with a modified MSE loss function.
result Improves ENet's prediction accuracy without losing uncertainty estimation.
Proposes measures for uncertainty quantification using proper scoring rules.
problem Uncertainty quantification for prediction tasks.
method Decomposes proper scoring rules into divergence and entropy components, tailoring uncertainty quantification to specific tasks.
result Flexibility in uncertainty quantification improves performance in selective prediction and active learning.
VIABLE learns a loss function for better few-shot learning.
problem Few-shot learning underfits with standard loss functions.
method Meta-learning to learn a differentiable loss function.
result Learning a relational loss function improves performance and sample efficiency.
This research improves PAC-Bayesian bounds for classification tasks using convexified loss.
problem Deriving generalization bounds for classification tasks with non-convex loss functions.
method Shift focus to misclassification excess risk bounds for PAC-Bayesian classification using convex surrogate loss and leveraging PAC-Bayesian relative bounds in expectation.
result Improved PAC-Bayesian bounds for classification tasks with convex surrogate loss.
The study offers new theoretical insights into structured prediction with convex loss minimization.
problem The challenge of structured prediction with efficient convex surrogate loss minimization.
method Constructing a convex surrogate loss and proving tight bounds on the calibration function.
result Formalizes the intuition that some task losses make learning harder than others, and that 0-1 loss is ill-suited for general structured prediction.
New methods evaluate data representations by complexity of low-loss predictor learning.
problem Evaluating quality of data representations for downstream tasks.
method Surplus Description Length (SDL) and ε Sample Complexity (εSC) methods.
result Methods measure the information needed to approximate optimal predictor up to specified tolerance.
Paper proposes a CNN for speech emotion recognition using center loss and reconstruction.
problem Speech emotion recognition (SER) in audio signals.
method Convolutional Neural Network (CNN) with center loss and reconstruction as regularizers.
result Proposed method achieves highly discriminative features for SER.
FAN improves attention weights for better relation emphasis.
problem Learning attention weights for better relation emphasis.
method Introduced a novel center-mass cross entropy loss and a focused attention backbone.
result Focused supervision leads to improved attention distribution and enhanced representation.
A new method approximates loss functions asymmetrically to prevent catastrophic forgetting.
problem Catastrophic forgetting in deep neural networks.
method Approximating a true loss function using an asymmetric quadratic function with one side overestimated.
result Achieves state-of-the-art accuracy close to upper-bound performance on benchmark datasets.
Extends quadratic loss for SVM and deep learning to improve pattern correlation.
problem Improving generalization in supervised binary classification and regression tasks.
method Extends quadratic loss, restarts from problem (8) in [3], proposes new algorithms, uses multiple kernel learning.
result Comparable results with standard losses and parameterized quadratic loss.
Boosted CVaR Classification improves tail performance in classification tasks.
problem Maximizing tail performance in classification tasks.
method Proposed Boosted CVaR Classification framework using randomized classifiers and LPBoost algorithm.
result Minimizing CVaR loss over randomized classifiers leads to better tail performance.
AdaRound improves post-training quantization of neural networks.
problem Improving the accuracy of quantized weights in neural networks.
method Adaptive rounding mechanism that adapts to data and task loss.
result AdaRound outperforms rounding-to-nearest and achieves state-of-the-art performance.
Paper corrects Max-Margin loss for multi-label tasks.
problem Max-Margin loss inconsistency in multi-label classification.
method Introduced Restricted-Max-Margin loss.
result Consistent loss for multi-label tasks under milder conditions.
Classification and regression tasks in overparameterized models show different generalization properties.
problem Comparing classification and regression in overparameterized models.
method Comparison of least-squares minimum-norm interpolation and hard-margin SVM using different loss functions.
result Interpolating solutions generalize well with 0-1 loss but not with square loss.
Abstract: Bounds for multi-task and multi-category learning.
problem Risk bounds for vector-valued functions.
method Framework for deriving risk bounds with feature maps and loss functions.
result Conditions for shared representations beneficial in multi-task and multi-category learning.
We propose a principle and loss functions for efficient single-class classification.
problem Efficient binary classification for specific classes in high-dimensional data.
method Define Single Logit Classification (SLC) task, propose Principle of Logit Separation, and design loss functions.
result Loss functions aligned with the Principle of Logit Separation improve SLC accuracy by 20%.
Improved speaker recognition with deep metric learning.
problem Performance gap between training and unseen speakers.
method Optimized speaker embedding model with prototypical network loss (PNL).
result Outperforms state-of-the-art models in speaker verification and identification.