Paper introduces new loss functions for Siamese networks using FDA.
problem Training Siamese networks with improved loss functions.
method Proposes Fisher Discriminant Triplet (FDT) and Fisher Discriminant Contrastive (FDC) loss functions based on FDA.
result Shows effectiveness of FDT and FDC on MNIST and histopathology datasets.
Generalized dual discriminator GANs improve upon traditional GANs by using two discriminators and a flexible loss function.
problem Mode collapse in GANs.
method Introducing dual discriminator α-GANs and extending the approach to arbitrary functions. result The approach reduces the optimization problem to a linear combination of an f-divergence and a reverse f-divergence. New approach to GANs using convex loss functions and kernel-based discriminators.
problem Minimizing the f-divergence between true and fake data distributions.
method Introducing a minimizing general loss viewpoint and using kernel-based discriminators.
result Maximizing the general loss is equivalent to the min-max problem in GAN.
Proposes L-Softmax loss for CNNs to improve feature discriminativeness.
problem Lack of explicit feature discriminativeness in cross-entropy loss.
method Introduces L-Softmax loss that encourages intra-class compactness and inter-class separability.
result Deeply learned features with L-Softmax loss are more discriminative, boosting performance.
Paper introduces negative margin loss for better few-shot classification accuracy.
problem Improving few-shot classification accuracy with metric learning.
method Introduces negative margin loss and analyzes its impact on feature discriminability.
result Negative margin loss outperforms regular softmax loss on few-shot classification benchmarks.
Proposes a new loss function for deep neural networks.
problem Deep neural networks lack a direct method to discriminate between correct and competing classes.
method Introduces a discriminative loss function based on negative log likelihood ratio.
result Significantly outperforms cross-entropy loss on image classification tasks.
This work improves GAN training by optimizing multiple discriminator losses.
problem Training GANs with multiple discriminators using single-objective methods.
method Formulates multi-objective optimization of multiple discriminator losses.
result Hypervolume maximization outperforms previous methods in sample quality and computational cost.
CcGAN tackles conditional image generation for continuous labels.
problem Mathematical challenges in conditioning on continuous, scalar labels.
method Proposes novel empirical losses and label input methods for continuous conditional GANs.
result CcGAN generates diverse, high-quality images from continuous labels.
This paper analyzes implicit bias in Deep Linear Discriminant Analysis.
problem The implicit bias of Deep Linear Discriminant Analysis.
method Analyzing gradient flow on a L-layer diagonal linear network.
result Under balanced initialization, the network transforms additive updates into multiplicative updates, conserving the (2/L) quasi-norm.
Fisher loss improves deep domain adaptation by learning discriminative within-class compact and between-class separable representations.
problem Improving deep domain adaptation performance by learning discriminative representations.
method Proposes a Fisher loss to learn discriminative representations that are within-class compact and between-class separable.
result Noticeable improvements in deep domain adaptation performance, e.g., 6.67% absolute improvement in mean accuracy on the Office-Home dataset.
DNLL loss improves deep LDA accuracy and consistency.
problem Pathological solutions in unconstrained Deep LDA.
method Introducing Discriminative Negative Log-Likelihood (DNLL) loss.
result Deep LDA trained with DNLL produces clean latent spaces and better calibrated probabilities.
Improved acoustic word embeddings using shared decoder in multi-view encoders.
problem Learning discriminative acoustic word embeddings from text labels.
method Combining Siamese multi-view encoders with a shared decoder network to maximize the relationship between acoustic and text embeddings.
result 11.1% relative improvement in average precision on acoustic word discrimination task with WSJ dataset.
Solves GAN mode collapse by assigning minibatches to multiple discriminators.
problem Mode collapse in GANs where models generate similar samples.
method Multiple discriminators, microbatching, and changing tasks.
result Promotes sample diversity in generated sets.
New method improves GANs by estimating density ratios in feature space with SP loss.
problem Filtering out unrealistic images from GANs trained with suboptimal discriminators.
method Develops DRE-F-SP method based on Softplus loss for density ratio estimation in feature space, and proposes three subsampling methods.
result Empirically shows substantial improvement over existing methods on synthetic and CIFAR-10 datasets.
Lower-dimensional video discriminators improve GAN performance.
problem High curvature in unconstrained video discriminator loss surfaces.
method Proposed Lower-Dimensional Video Discriminators (LDVD GANs).
result LDVD GANs double Temporal-GAN performance and achieve state-of-the-art performance.
Paper proposes angular loss for better face recognition and object classification.
problem Improving intra-class compactness and preventing overfitting in face recognition and object classification.
method Angular loss function to maximize angular gradient, reducing overfitting and requiring only one adjustable constant.
result Our method outperforms other methods in accuracy, discriminative information, and time-efficiency.
Parametric divergences are effective for generative modeling despite being non-optimal.
problem Training high-dimensional distributions with GANs.
method Generalization of GAN losses to parametric divergences, focusing on sensitivity to specific distribution moments.
result Parametric divergences are more suitable for learning high-dimensional distributions due to their sensitivity to specific aspects of the distribution.
Linking GANs to binary classification through divergences.
problem Training GANs and understanding their divergence properties.
method Revisiting the discriminator's role in computing f-divergences.
result Alternative training perspective for f-GANs by designing discriminator loss.
Paper analyzes and improves GANs' generalization and stability.
problem Poor generalization of GANs' discriminators in practical settings.
method Proposes a zero-centered gradient penalty to improve discriminator's generalization.
result Improves GANs' generalization and convergence through the proposed penalty.
Improved GAN sampling by collaborating discriminator and generator.
problem Loss of discriminator information during standard GAN sampling.
method Collaborative sampling between generator and discriminator, gradient-based updates.
result Generated samples are closer to real data distribution.
Paper proposes SDRL to improve continual learning with less computational cost.
problem Catastrophic forgetting in continual learning.
method SDRL method that refines gradients from memorized samples to reduce gradient diversity.
result SDRL shows better performance than state-of-the-art methods on multiple benchmark tasks.
Optimal domain adaptation model using Fisher's Linear Discriminant.
problem Improving classification accuracy across different domains.
method Convex combination of source and target hypotheses, derived under 0-1 loss.
result Effective classifier can be computed without direct source task information.
Stacked GANs improve image generation quality.
problem Generating high-quality images from hierarchical representations.
method Top-down stack of GANs, conditional and entropy losses.
result SGAN generates higher quality images than single GAN.
We unify f-divergences, Bregman divergences, surrogate loss bounds (regret bounds), proper scoring rules, matching losses, cost curves, ROC-curves and information. We do this by systematically studying integral and variational representations of these objects and in so doing identify their primitives which all are rela…
The paper revisits discriminative vs. generative classifiers, showing naive Bayes requires fewer samples.
problem Comparing discriminative and generative classifiers in multiclass settings.
method Theoretical analysis and simulations of naive Bayes vs. logistic regression.
result Multiclass naive Bayes requires fewer samples to approach asymptotic error compared to logistic regression.
Efficient algorithm for contextual bandits with first-order guarantees.
problem Adapting to low noise in contextual bandits.
method Reduction to online regression with logarithmic loss.
result Optimal and efficient first-order guarantees for contextual bandits.
New method trains generative models without discriminators, improving stability and accuracy.
problem Training implicit generative models with adversarial discriminators leads to instability and mode-dropping.
method Invariant statistical loss function, avoiding discriminators.
result Successfully trains generative models for various complex distributions without mode-dropping.
Enhances deep learning robustness to noise without sacrificing clean data accuracy.
problem Robustness of deep neural networks to input noise.
method Discriminative loss at penultimate layer and class-wise feature alignment with Gaussian noise.
result Improves robustness to various perturbations without degrading clean data accuracy.
Paper proposes a loss extension for neural networks to improve OSR performance.
problem Open set recognition problem, distinguishing known and unknown classes.
method Introduces a loss function extension to find more discriminative polar representations.
result Significantly improves performance on datasets from different domains.
Proposes a test to ensure predictive algorithms predict intended outcomes better than unintended ones.
problem Unintended model behavior leading to prediction of unintended outcomes.
method Falsification framework using nonparametric hypothesis testing to compare prediction losses across outcomes.
result Establishes discriminant validity with respect to gender but not race in an admissions setting.
Paper analyzes the tradeoff between a discriminator's ability to identify true distribution and its ability to generalize.
problem Discrimination-generalization tradeoff in GANs.
method Analyzes the linear span of discriminator sets and their relationship to generalization and discriminative ability.
result Discriminator sets with dense linear span are both discriminative and generalizable.
New method samples triplets from data distributions for training Triplet networks.
problem Training robust Triplet networks with discriminative triplets.
method Bayesian updating of multivariate normal distributions for dynamic class embedding sampling.
result Experimental validation on MNIST and histopathology CRC datasets shows effectiveness of the proposed method.
New approach stabilizes GANs by leveraging implicit competitive regularization.
problem Stability issues in GAN training due to discriminator's exploitation of generator errors.
method Competitive Gradient Descent (CGD) for opponent-aware modeling of generator and discriminator.
result Significant improvement in GAN training stability without explicit regularization.
Paper proposes a CNN for speech emotion recognition using center loss and reconstruction.
problem Speech emotion recognition (SER) in audio signals.
method Convolutional Neural Network (CNN) with center loss and reconstruction as regularizers.
result Proposed method achieves highly discriminative features for SER.
Adaptive density discrimination improves metric learning for better classification and feature extraction.
problem Challenges in distance metric learning, especially in performance and feature extraction compared to modern classification algorithms.
method Explicitly models class distributions, adapts similarity assessment, and penalizes class overlap.
result Achieves state-of-the-art classification results, surpassing softmax classifier and triplet loss.
CapsuleGAN uses capsule networks in GANs for better image data modeling.
problem Improving image data modeling in GANs.
method CapsuleGAN uses capsule networks as discriminators in GANs, with a new objective function incorporating margin loss.
result CapsuleGAN outperforms standard GANs on image data modeling and semi-supervised classification.
Fence GAN improves anomaly detection by modifying GAN loss.
problem Anomaly detection on complex high-dimensional data.
method Modified GAN loss to encourage generated samples at the boundary of real data distribution.
result Fence GAN yields the best anomaly classification accuracy.
Rob-GAN combines generator, discriminator, and adversarial attack for improved robustness and quality.
problem Improving robustness and quality of GAN-generated images under adversarial attacks.
method Rob-GAN framework that jointly optimizes generator and discriminator in the presence of adversarial attacks.
result Rob-GAN improves convergence speed, image quality, and robustness of discriminators under strong adversarial attacks.
EBGAN uses energy-based discriminator for stable GAN training.
problem Stability and high-resolution generation in GANs.
method EBGAN views discriminator as an energy function, generator as minimizing energy, and uses auto-encoder architecture with reconstruction error.
result EBGAN exhibits more stable behavior and generates high-resolution images.
Proposes a neural network approach to enforce direct supervision in hidden layers for better feature learning.
problem Gradient vanishing and lack of discriminative features in hidden layers of CNNs.
method Introduces a label consistency regularization loss for late hidden layers combined with classification error loss.
result Achieves state-of-the-art performance on public benchmarks for action and object recognition.
QFDA combines machine learning and information theory for image classification.
problem Lack of literature on combining machine learning and information theory.
method Quantized Fisher Discriminant Analysis (QFDA) using a cost function for rate-distortion optimization.
result QFDA achieves at least as good classification accuracy as FDA on quantized images.
BlackOut speeds up RNNLMs with large vocabularies.
problem Training RNNLMs with large vocabularies is computationally expensive.
method BlackOut uses a discriminative loss and weighted sampling to efficiently train RNNLMs with million-word vocabularies.
result BlackOut achieves the lowest perplexity scores on the one billion word benchmark.
A new method quantifies feature-map discriminativeness for efficient pruning of deep neural networks.
problem Efficiently pruning deep neural networks to reduce computation while maintaining accuracy.
method Presented a novel mathematical formulation (Discriminant Information, DI) to quantify feature-map discriminativeness, enabling efficient pruning and intra-layer mixed precision quantization.
result Our pruned ResNet50 achieves 44% FLOPs reduction without any Top-1 accuracy loss.
We analyze GANs using neural tangent kernels, revealing flaws and advancing understanding.
problem Flaws in previous GAN analysis models.
method Neural Tangent Kernel framework for infinite-width discriminator.
result New insights into GAN convergence and generated distribution.
ASA improves ASR by adapting SD models to SI model's deep feature distribution.
problem Improving ASR performance on new speakers with limited data.
method Adversarial learning to regularize SD model's deep features to match SI model's.
result ASA achieves significant word error rate improvements over SI models.
Study minimax rates for nonparametric density estimation with adversarial losses.
problem Estimating densities under various adversarial loss functions.
method General framework for analyzing minimax rates with different loss functions.
result Determines the minimax rate based on loss choice and density smoothness.
End-to-end training improves segmental models for speech recognition.
problem Training segmental models is challenging and lacks understanding under different settings and losses.
method Investigated a model class based on recent successful approaches, trained end-to-end with various losses.
result Marginal log loss provides the most consistent strong performance without requiring ground-truth alignments.
A new GAN loss function based on cumulant generating functions improves stability and robustness.
problem Improving the stability and performance of GANs.
method Cumulant GAN loss function based on variational R{é}nyi divergence.
result Cumulant GAN achieves linear convergence to Nash equilibrium and superior performance in image generation.