This work addresses encoding biases in neural networks by tailoring models with unsupervised losses.
problem Improving neural network representations and reducing the generalization gap.
method Inspired by transductive learning, the authors propose tailoring and meta-tailoring to optimize unsupervised losses during prediction time.
result Models trained with tailoring and meta-tailoring perform better on the task objective after adapting to unsupervised losses.
A robust loss for anomaly mitigation and unsupervised contamination classification
problem Detecting and mitigating contamination in supervised and unsupervised settings
method Neural Bayesian Anomaly Mitigation (NBAM)
result Recovering the structure of contamination and identifying label-flip pairs
Algorithm learns optimal arm selection in unsupervised sequential selection with contextual information.
problem Learning optimal arm selection in unsupervised sequential selection with contextual information.
method Proposes an algorithm for the contextual USS problem under the CWD property, demonstrating sub-linear regret.
result Demonstrates sub-linear regret for the proposed algorithm.
Novel approach trains ASR models with less supervision using bilevel optimization.
problem Training acoustic models for ASR with minimal supervision.
method Bilevel optimization with unsupervised and supervised losses.
result Achieves superior performance compared to existing methods.
We investigate the use of a non-parametric independence measure, the Hilbert-Schmidt Independence Criterion (HSIC), as a loss-function for learning robust regression and classification models. This loss-function encourages learning models where the distribution of the residuals between the label and the model predictio…
We present a new flavor of Variational Autoencoder (VAE) that interpolates seamlessly between unsupervised, semi-supervised and fully supervised learning domains. We show that unlabeled datapoints not only boost unsupervised tasks, but also the classification performance. Vice versa, every label not only improves class…
Unsupervised segmentation learns features without labels, improving accuracy.
problem Discover and localize semantically meaningful categories in images without annotations.
method Separates feature learning from cluster compactification; distills unsupervised features into discrete semantic labels using a contrastive loss function.
result Significant improvement over prior state of the art on semantic segmentation challenges.
A new method detects concept drift without true labels.
problem Detecting concept drift in unsupervised settings.
method Student-teacher learning paradigm for drift detection.
result The method outperforms state-of-the-art approaches in experiments.
We study the emergence of sparse representations in neural networks. We show that in unsupervised models with regularization, the emergence of sparsity is the result of the input data samples being distributed along highly non-linear or discontinuous manifold. We also derive a similar argument for discriminatively trai…
Improved unsupervised probing for ranking tasks using Contrast-Consistent Ranking.
problem Improving self-consistency in language model rankings.
method Adapting Contrast-Consistent Search (CCS) to Contrast-Consistent Ranking (CCR) for ranking tasks.
result CCR probing outperforms prompting techniques across different models and datasets.
In this paper we take a problem of unsupervised nodes clustering on graphs and show how recent advances in attention models can be applied successfully in a "hard" regime of the problem. We propose an unsupervised algorithm that encodes Bethe Hessian embeddings by optimizing soft modularity loss and argue that our mode…
Hutter (2007) recently introduced the loss rank principle (LoRP) as a generalpurpose principle for model selection. The LoRP enjoys many attractive properties and deserves further investigations. The LoRP has been well-studied for regression framework in Hutter and Tran (2010). In this paper, we study the LoRP for clas…
A new unsupervised learning method calibrates rough volatility models efficiently.
problem Efficient calibration of rough volatility models with minimal data.
method Unsupervised learning using BSDE representation and neural networks.
result The proposed scheme minimizes loss and approximates BSDE solution.
Thompson Sampling tackles USS, a sequential selection problem without feedback.
problem Unsupervised Sequential Selection (USS) problem with fixed costs and ordered arms.
method Thompson Sampling algorithm for USS problem.
result Thompson Sampling achieves near optimal regret and better performance than existing algorithms.
Unsupervised method removes satellite noise without paired data.
problem Image artifacts from satellite sensor noises affect quality and applications.
method Wavelet subband cycleGAN using adversarial and cycle-consistency losses.
result Effectively removes satellite noise while preserving high frequency features.
Improved speech enhancement using diffusion models with MSE loss.
problem Efficient incorporation of noisy speech in generative speech enhancement.
method Augmented diffusion-based generative model with a MSE loss for enhanced speech.
result Proposed method improves speech enhancement performance compared to original diffusion model.
Unsupervised space partitioning improves ANNS performance without pre-processing.
problem Efficient nearest neighbor search in high-dimensional spaces.
method Custom unsupervised learning framework for space partitioning and learning-to-search.
result Our method outperforms state-of-the-art approaches on ANNS benchmarks.
Paper addresses unsupervised learning from incomplete measurements in inverse problems.
problem Learning from incomplete measurements is challenging in inverse problems.
method Use multiple measurement operators to overcome nullspace issues; propose a novel unsupervised learning loss.
result Presented necessary and sufficient conditions for successful unsupervised learning.
A new framework for deep matrix factorizations improves model consistency and flexibility.
problem Inconsistent loss functions in deep matrix factorizations.
method Introduces two new loss functions and a generic optimization framework.
result Demonstrates improved model performance on synthetic and real data.
DEQGAN uses GANs to solve differential equations without supervision.
problem Solving differential equations with neural networks.
method Generative Adversarial Networks (GANs) to learn the loss function.
result DEQGAN achieves lower mean squared errors and competitive solution accuracy compared to traditional methods.
RKUM is an R package for robust kernel-based unsupervised methods.
problem Robust analysis under contaminated or noisy data conditions.
method Robust kernel covariance and cross-covariance operators using generalized loss functions.
result RKUM reduces sensitivity to contamination and effectively identifies outliers.
End-to-end training solves deep unsupervised contrastive learning problems.
problem Theoretical analysis of unsupervised contrastive learning.
method End-to-end training of deep neural networks.
result Approximate stationary solutions found for non-convex contrastive loss.
A new unsupervised contrastive learning framework improves time series representation learning.
problem Lack of labeled data in time series data.
method Proposes an unsupervised contrastive learning framework using a novel contrastive loss and data augmentation.
result Framework outperforms other approaches on univariate and multivariate time series, and benefits transfer learning.
Super-AND improves unsupervised embedding learning with 89.2% accuracy on CIFAR-10.
problem Extracting good representations from data without labels.
method Super-AND uses unique losses to gather similar samples and maintain features.
result Super-AND achieves 89.2% accuracy on CIFAR-10 image classification.
This paper proposes a new method for automatically selecting the optimal kernel bandwidth in density estimation.
problem The challenge of selecting the optimal kernel bandwidth in unsupervised density estimation.
method The approach uses a topology-based loss function for automated bandwidth selection.
result Demonstrates the potential of the topology-based approach across different dimensions.
Ensembling word embeddings to improve distributed word representations has shown good success for natural language processing tasks in recent years. These approaches either carry out straightforward mathematical operations over a set of vectors or use unsupervised learning to find a lower-dimensional representation. Th…
We show how to estimate a model's test error from unlabeled data, on distributions very different from the training distribution, while assuming only that certain conditional independencies are preserved between train and test. We do not need to assume that the optimal predictor is the same between train and test, or t…
For better classification generative models are used to initialize the model and model features before training a classifier. Typically it is needed to solve separate unsupervised and supervised learning problems. Generative restricted Boltzmann machines and deep belief networks are widely used for unsupervised learnin…
SFAVEL distills features from language models for fact verification without annotations.
problem Fact verification with semantically meaningful and compact features.
method Self-supervised pretraining using contrastive loss with language models.
result Achieved state-of-the-art results on FB15k-237 and FEVER.
This paper rethinks confidence calibration under covariate shifts.
problem Calibration methods struggle with covariate shifts and unstable importance weighting.
method Derives Expectation consistency condition and proposes Expectation consistency loss (ECL).
result ECL loss is compatible with various types of calibration and has the same sample complexity as ECE.
Proposes a new method for disentangling data representations using topological analysis.
problem Learning disentangled representations for better model explainability and robustness.
method Integrates a multi-scale topological loss term into the training of deep learning models.
result Improves disentanglement scores compared to state-of-the-art methods.
This paper proposes RAS, a novel unsupervised loss function for speech separation.
problem Unsupervised speech separation without teacher models or synthesized mixtures.
method Train a neural network to separate sources given one channel, using Wiener filtering and SI-SDR.
result Achieves significant SDR improvement with minimal labeled data.
Deep Reinforcement Learning (DRL) algorithms are known to be data inefficient. One reason is that a DRL agent learns both the feature and the policy tabula rasa. Integrating prior knowledge into DRL algorithms is one way to improve learning efficiency since it helps to build helpful representations. In this work, we co…
Convolutional neural networks provide visual features that perform remarkably well in many computer vision applications. However, training these networks requires significant amounts of supervision. This paper introduces a generic framework to train deep networks, end-to-end, with no supervision. We propose to fix a se…
Unsupervised model detects healthcare fraud from patient visit data.
problem Detecting fraudulent healthcare bills from patient visit data.
method Uses LSTM and seq2seq models for anomaly detection, normalizes scores with EDF.
result Improves anomaly detection for high class imbalance problems.
A scalable system detects price anomalies in online marketplaces to improve customer experience.
problem Inaccurate prices on online marketplaces lead to poor customer experience and revenue loss.
method MoatPlus uses unsupervised statistical features and an ensemble of models to generate upper price bounds.
result Our approach improves precise anchor coverage by up to 46.6% in high-vulnerability item subsets.
We present a generalization of the Cauchy/Lorentzian, Geman-McClure, Welsch/Leclerc, generalized Charbonnier, Charbonnier/pseudo-Huber/L1-L2, and L2 loss functions. By introducing robustness as a continuous parameter, our loss function allows algorithms built around robust loss minimization to be generalized, which imp…
AutoEmbedder clusters unlabeled data using semi-supervised DNN embedding.
problem Clustering unlabeled data efficiently and effectively.
method Semi-supervised DNN embedding system using Siamese network architecture.
result AutoEmbedder outperforms existing DNN-based semi-supervised methods.
Unsupervised framework captures acquisition variability in structural connectomes.
problem Acquisition differences across sites, scanners, and protocols complicate structural connectome analysis.
method An unsupervised framework using architectural annealing to balance discrete and continuous latent variables.
result Architectural annealing produces stronger site learning than baseline models.
Study detects boundaries in unlabeled noisy images without labels.
problem Detecting boundaries in unlabeled noisy images without labels.
method Proposed a continuous hinge-type surrogate loss for boundary detection, combined with deep neural networks.
result Deep neural network achieves minimax-optimal boundary recovery rate under piecewise smooth boundary model.
The paper introduces a new loss function to prevent overfitting in semi-supervised graph networks.
problem Overfitting in semi-supervised graph networks trained with cross-entropy loss.
method Proposes an unsupervised manifold smoothness loss to regularize the graph convolutional networks.
result Adding the proposed loss consistently improves performance of graph networks.
Optimizes PnL using linear signals in quantitative finance.
problem Maximizing profit and loss in financial trading.
method Unsupervised machine learning approach that maximizes Sharpe Ratio through linear relationships and parameter optimization.
result Empirical validation and effectiveness of the model on U.S. Treasury ETF.
In this paper, we propose a simple model referred as Contradistinguisher (CTDR) for unsupervised domain adaptation whose objective is to jointly learn to contradistinguish on unlabeled target domain in a fully unsupervised manner along with prior knowledge acquired by supervised learning on an entirely different domain…
Paper proposes a new speech representation benchmark and model.
problem Lack of benchmarks for comparing speech representations.
method Unsupervised triplet-loss objective for training a universal non-semantic speech representation.
result Proposed representation outperforms other models on benchmark and transfer learning tasks.
A good representation for arbitrarily complicated data should have the capability of semantic generation, clustering and reconstruction. Previous research has already achieved impressive performance on either one. This paper aims at learning a disentangled representation effective for all of them in an unsupervised way…
Data representations that contain all the information about target variables but are invariant to nuisance factors benefit supervised learning algorithms by preventing them from learning associations between these factors and the targets, thus reducing overfitting. We present a novel unsupervised invariance induction f…
This work proposes an unsupervised neural network framework for solving combinatorial optimization problems on graphs.
problem Challenges in neural networks solving combinatorial optimization problems without labeled instances.
method Inspired by Erdos' probabilistic method, a neural network parametrizes a probability distribution over sets, optimizing it to find low-cost integral solutions.
result The method provides valid solutions to the maximum clique problem and local graph clustering, achieving competitive results.
In the last decade, supervised deep learning approaches have been extensively employed in visual odometry (VO) applications, which is not feasible in environments where labelled data is not abundant. On the other hand, unsupervised deep learning approaches for localization and mapping in unknown environments from unlab…