Unbounded output networks improve classification performance.
problem Improving classification accuracy in neural networks.
method Introducing UnBounded output network (UBnet) with unbounded output units and a modified mean-squared error objective.
result UBnets achieve high classification performance on MNIST, CIFAR-10, and CIFAR-100 datasets.
Prototype networks on hyperspheres improve classification and regression.
problem Improving classification and regression performance.
method Using hyperspherical prototypes for classification and regression, optimizing prototypes through data-independent margin separation.
result Hyperspherical prototype networks outperform other methods in classification, regression, and their combination.
EPFGNN models graph connections for better node classification.
problem Graph node classification issues due to feature aggregation.
method EPFGNN models graph as a Markov Random Field with explicit pairwise factors and a GNN backbone.
result EPFGNN improves semi-supervised node classification performance.
Solves regression problems with CP by converting to classification.
problem Challenges in CP for heteroscedastic, multimodal, or skewed regression outputs.
method Converts regression to classification, uses CP for classification to obtain CP sets for regression.
result Simple approach yields good results on practical problems.
Proposes a deep tree-ensemble model for multi-output prediction.
problem Lack of efficient solutions for multi-output prediction.
method Integrates tree-embeddings into deep tree-ensembles for structured output prediction.
result Superior performance in multi-label classification and multi-target regression tasks.
Gradient boosting adapted for multi-label and multi-output tasks.
problem Joint prediction of multiple classification or regression outputs.
method Gradient tree boosting with random output projections.
result Random projection improves adaptation to different output correlation patterns.
Study shows MSE with sigmoid can match SCE in classification tasks, especially with noisy data.
problem Inconsistent errors in neural network classification tasks.
method Introduced Output Reset algorithm to use MSE with sigmoid activation.
result MSE with sigmoid activation achieves comparable accuracy and convergence rates to Softmax Cross-Entropy, especially in noisy data scenarios.
Efficiently approximates uncertainty in classification models using Dirichlet distributions.
problem Inefficient computation of uncertainty estimates in Bayesian deep learning.
method Revised Laplace Bridge method to construct a Dirichlet approximation of softmax output distributions.
result The Dirichlet approximation leads to more efficient computation and better uncertainty estimates.
Extreme classification problems are multiclass and multilabel classification problems where the number of outputs is so large that straightforward strategies are neither statistically nor computationally viable. One strategy for dealing with the computational burden is via a tree decomposition of the output space. Whil…
Proposes fair classification method using Wasserstein-1 distances.
problem Ensuring fairness in classification models by protecting sensitive information.
method Minimizes Wasserstein-1 distances to enforce independence between classifier outputs and sensitive information.
result Empirical performance superior to fairness baselines on benchmark datasets.
The paper proves neural networks with ReLU and softmax can approximate any function.
problem Approximating functions and class labels in neural networks.
method Extended universal approximator theory to neural networks with ReLU and softmax.
result Neural networks with ReLU and softmax can approximate any function and class labels.
This paper analyzes output activation functions for adversarial losses.
problem Understanding which output activation functions form a well-behaved adversarial loss.
method Variational divergence minimization and a comparative framework for adversarial losses.
result There is no single winning combination of output activation functions and regularization approaches across all settings.
A new method uses RBMs to handle incomplete multi-output data.
problem Handling incomplete multi-output data with dependencies between features and labels.
method Adapted RBM algorithm based on mean-field equations for joint imputation and classification.
result Efficiently solves problems with missing features and labels.
Visual integration helps understand ensemble model performance.
problem Lack of comprehensibility in ensemble models.
method Visual integration of data and model space for effective exploration and manipulation of ensemble models.
result Improved understanding of how each model contributes to ensemble performance.
Multi-output Gaussian processes (MOGP) are probability distributions over vector-valued functions, and have been previously used for multi-output regression and for multi-class classification. A less explored facet of the multi-output Gaussian process is that it can be used as a generative model for vector-valued rando…
We adapt the idea of random projections applied to the output space, so as to enhance tree-based ensemble methods in the context of multi-label classification. We show how learning time complexity can be reduced without affecting computational complexity and accuracy of predictions. We also show that random output spac…
Unified framework for generating set-valued outputs.
problem Handling unordered set outputs with varying sizes.
method Sequential Set Generation (SSG) framework.
result SSG outperforms baseline methods in experiments.
Conditional Restricted Boltzmann Machines (CRBMs) are rich probabilistic models that have recently been applied to a wide range of problems, including collaborative filtering, classification, and modeling motion capture data. While much progress has been made in training non-conditional RBMs, these algorithms are not a…
A CAE improves DNN's outlier and adversary defense.
problem Improving DNN's robustness against outliers and adversaries.
method Proposes a classification-autoencoder (CAE) that compresses samples into disjoint spaces and uses a decoder to classify and defend against adversaries.
result The CAE achieves state-of-the-art outlier recognition and near-lossless classification of adversaries.
Paper trains SNNs for classification using first-to-spike decoding.
problem Training SNNs for classification under GLM model.
method Proposes first-to-spike decoding method for SNNs.
result Improves accuracy and efficiency of SNN classification.
Optimistic bounds for multi-output learning using self-bounding Lipschitz condition.
problem Learning vector-valued functions from supervised data.
method Introducing self-bounding Lipschitz condition and proving optimistic bounds using local Rademacher complexity and Srebro's inequality.
result Minimax optimal generalization bounds for multi-output learning, up to logarithmic factors.
The paper analyzes deep neural networks using information theory to improve classification accuracy.
problem Improving classification accuracy in deep neural networks.
method Modeling the output of convolutional filters as a random variable conditioned on class and network structure, computing conditional entropy as a compact code.
result The conditional entropy feature analysis leads to higher classification accuracy than the original CNN.
Enhances fairness in multi-output models using optimal transport.
problem Improving fairness in multi-output models like multi-task/multi-class classification and representation learning.
method Post-processing method using optimal transport mappings to move model outputs towards empirical Wasserstein barycenter.
result Demonstrates effectiveness of the proposed approach on multi-task/multi-class classification and representation learning tasks.
LLMs produce volatile sentence-level sentiment classifications that affect financial decision-making.
problem Volatile outputs from LLMs impact financial text understanding tasks.
method Case study on US equity market investing via news sentiment analysis.
result Volatile LLM outputs lead to significant variations in portfolio construction and returns.
New bounds enable training of probabilistic models for deep networks.
problem Training scalable latent variable models for deep networks.
method Introducing new variational bounds for specific output layers of neural networks.
result Analytical bounds for certain output layers allow training without re-parameterization or Monte Carlo approximations.
Multi-Output Dependence (MOD) learning is a generalization of standard classification problems that allows for multiple outputs that are dependent on each other. A primary issue that arises in the context of MOD learning is that for any given input pattern there can be multiple correct output patterns. This changes the…
Aggregates diverse zero-shot LLM outputs for better corporate disclosure classification.
problem Combining varied zero-shot LLM predictions for improved stock return prediction.
method Multi-prompt framework with three fixed zero-shot LLM classifiers, logistic meta-classifier aggregation.
result Aggregated model outperforms single classifiers and baseline models, increasing balanced accuracy from 0.566 to 0.606.
RTM extends TM for continuous output problems using conjunctive clauses.
problem Continuous output problems in machine learning.
method Modified inner inference mechanism to produce a single continuous output.
result RTM achieves better regression accuracy with fewer clauses.
Paper proposes a SIMO DNN for indoor localization using Wi-Fi fingerprints.
problem Indoor localization with high accuracy and efficiency.
method Single-input and multi-output deep neural network architecture.
result SIMO-DNN scheme outperforms existing methods in floor detection and location accuracy.
The study shows removing fully connected output layers improves efficiency without sacrificing performance.
problem Large number of parameters in fully connected layers for high-category datasets.
method Examined architectures replacing fully connected output layers with fixed layers and compared performance.
result Fixed classifiers offer no additional benefit over removing the output layer and its parameters.
Paper classifies lung nodules in CT scans, improving diagnostic accuracy.
problem Early detection of lung nodules for better treatment planning.
method Proposes four 3D neural networks for direct mapping from 3D images to class labels.
result 3D multi-output DenseNet achieves state-of-the-art classification accuracy.
Optimal transport improves multivariate prediction uncertainty quantification.
problem Uncertainty quantification in multivariate learning tasks, especially in regression and classification.
method Introducing a novel Conformal Prediction procedure using optimal transport to handle multivariate score functions and construct flexible prediction regions.
result Ensures finite-sample, distribution-free coverage guarantees for multivariate prediction sets.
Paper introduces conformal prediction for reliable uncertainty quantification in landmark localization.
problem Systematic underestimation of total predictive uncertainty in landmark localization.
method Conformal prediction framework for multi-output regression, generating flexible prediction regions.
result Methods outperform existing approaches in validity and efficiency across 2D and 3D datasets.
Reduced-rank method improves least-squares regression under output regularity.
problem Least-squares regression with infinite dimensional outputs.
method Reduced-rank method for solving least-squares problems with output regularity assumptions.
result Learning bounds and improved statistical performance compared to full-rank method.
Meta-learner reduces few-shot learning errors by nulling out error signals.
problem Few-shot learning accuracy issues in neural networks.
method Linear transformer for null-space projection of neural network outputs.
result Meta-learner achieves best or near-best image classification accuracies.
Expands weak supervision by allowing partial labels from multiple noisy sources.
problem Creating models without labeled data using heuristic labelers.
method Probabilistic generative model estimating partial label accuracies.
result Improved model accuracy on various tasks (8.6% on text, comparable to zero-shot methods on images).
Proposes a new method to approximate Gaussian inference in classification tasks.
problem Uncertainty quantification in classification tasks using softmax functions.
method Develops a new formalism to approximate Gaussian distributions over logit space and proposes using element-wise normCDF or sigmoid instead of softmax.
result Improves uncertainty quantification compared to softmax Monte Carlo sampling.
Classification Ensemble, which uses the weighed polling of outputs, is the art of combining a set of basic classifiers for generating high-performance, robust and more stable results. This study aims to improve the results of identifying the Persian handwritten letters using Error Correcting Output Coding (ECOC) ensemb…
Efficient graph-based decoding improves extreme classification accuracy.
problem Learning algorithms for extreme classification with large label sets.
method ECOC with loss-based decoding on graph-induced output codes.
result Efficient loss-based decoding on graph output codes improves classification accuracy.
Improved sample efficiency with normalized RBF kernels in neural networks.
problem Learning more with less data in deep learning models.
method Two-phase method to train neural networks with normalized RBF kernels as output layer.
result Normalized RBF kernel networks achieve higher sample efficiency, compactness, and separability.
This chapter introduces supervised classification methods and their applications.
problem Automatically assigning objects to classes based on numerical measurements.
method Learning from examples to map features to class labels.
result Understanding and applying various classification techniques.
This paper generalizes an important result from the PAC-Bayesian literature for binary classification to the case of ensemble methods for structured outputs. We prove a generic version of the \Cbound, an upper bound over the risk of models expressed as a weighted majority vote that is based on the first and second stat…
Simplifies neural network constraints with computationally efficient method.
problem Implementing hard output constraints in neural networks.
method Additional neural network layer for output constraints.
result Computational simplicity with complexity O(n*m) for linear constraints.
A method for learning discontinuous functions using clustering, classification, and regression.
problem Supervised learning with highly nonlinear and discontinuous outputs.
method Three stages: clustering, classification, and separate regression for each class.
result Combining clustering, classification, and regression provides a robust and powerful approach.
Factorization machines and polynomial networks are supervised polynomial models based on an efficient low-rank decomposition. We extend these models to the multi-output setting, i.e., for learning vector-valued functions, with application to multi-class or multi-task problems. We cast this as the problem of learning a …
A tensor-based method improves classification accuracy on spatiotemporal data in echo state networks.
problem Loss of spatial and temporal correlations when using standard linear algebra techniques on multidimensional hidden layer states.
method Orthogonal Tucker decompositions of tensors to preserve and exploit the multidimensional nature of hidden layer states.
result The tensor-based approach outperforms the standard linear output weight approach in classification accuracy.
Paper tackles novelty detection in text classification.
problem Traditional text classification assumes known classes in testing, but often encounters unexpected instances.
method Converts problem to pair-wise matching, uses CNN with embedding matrices.
result Proposed method outperforms state-of-the-art baselines.
The paper analyzes and corrects disparate impact in machine learning models using information theory.
problem Systematic discrimination in machine learning models based on sensitive attributes.
method Information-theoretic framework to quantify and correct disparate impact.
result Closed-form expressions for efficient correction of input distributions to achieve statistically indistinguishable output distributions.