We describe a nonparametric topic model for labeled data. The model uses a mixture of random measures (MRM) as a base distribution of the Dirichlet process (DP) of the HDP framework, so we call it the DP-MRM. To model labeled data, we define a DP distributed random measure for each label, and the resulting model genera…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,657 papers · 148 categories
Trend · papers per month
48 results for “randomized label”
Proposes MGPLL for PL learning with non-random noise.
problem Partial label learning with non-random label noise.
method Bi-directional mapping framework, conditional noise label generation, multi-class predictor, adversarial learning.
result Demonstrates state-of-the-art performance in partial label learning.
Neural networks learn patterns in random data, improving downstream performance.
problem Understanding what deep networks learn with random labels.
method Analytical and empirical study of convolutional and fully connected networks pre-trained on random labels.
result Pre-trained networks on random labels transfer faster to real datasets, despite specialization effects.
We propose using five data-driven community detection approaches from social networks to partition the label space for the task of multi-label classification as an alternative to random partitioning into equal subsets as performed by RAkELd: modularity-maximizing fastgreedy and leading eigenvector, infomap, walktrap an…
Label ranking aims to learn a mapping from instances to rankings over a finite number of predefined labels. Random forest is a powerful and one of the most successful general-purpose machine learning algorithms of modern times. In this paper, we present a powerful random forest label ranking method which uses random de…
Study compares memorization of SimCLR to supervised and random labels training.
problem Understanding memorization in contrastive learning.
method Investigated SimCLR's memorization properties compared to supervised and random labels training.
result SimCLR's memorization is similar to random labels training in terms of training object complexity distribution.
The paper cleans label noise in supervised classification using Bernoulli sampling.
problem Label noise degrades supervised classifier performance.
method Proposes a label noise cleaning method based on Bernoulli random sampling.
result The method separates clean and noisy observations without prior label information.
Gradient descent amplifies random features in neural networks to useful ones.
problem Generalization in neural networks trained on corrupted data.
method Characterization of feature-learning process in two-layer ReLU networks trained by gradient descent.
result Gradient descent amplifies random features to useful ones, achieving near optimal generalization error.
Unsupervised Domain Adaptation (DA) is used to automatize the task of labeling data: an unlabeled dataset (target) is annotated using a labeled dataset (source) from a related domain. We cast domain adaptation as the problem of finding stable labels for target examples. A new definition of label stability is proposed, …
Proposes a constrained labeling method for weakly supervised learning.
problem Combining weak supervision signals while navigating misleading correlations.
method Randomized constrained labeling within a defined space.
result Randomized constrained labeling converges after few iterations and outperforms other methods.
Deep neural networks (DNNs) can easily fit a random labeling of the training data with zero training error. What is the difference between DNNs trained with random labels and the ones trained with true labels? Our paper answers this question with two contributions. First, we study the memorization properties of DNNs. O…
Dirichlet Active Learningstat.ML
DiAL uses Bayesian Dirichlet random fields for active learning with sparse labels.
problem Active learning with limited labeled data.
method Bayesian Dirichlet random field for feature-conditional class probabilities, calibrating with graph Laplacian.
result Competitive performance in low-label rate graph learning tasks.
We present a novel approach to leverage large unlabeled datasets by pre-training state-of-the-art deep neural networks on randomly-labeled datasets. Specifically, we train the neural networks to memorize arbitrary labels for all the samples in a dataset and use these pre-trained networks as a starting point for regular…
This paper proposes a method to select relevant features for multi-label learning.
problem Feature selection in multi-label learning to retain important information with minimal features.
method Random manifold sampling and joint sparse regularization to solve multicollinearity and obtain sparse feature sets.
result The proposed method outperforms other methods in selecting relevant features for multi-label learning.
Study shows label noise impacts neural representations' information content, revealing double descent behavior.
problem Impact of label noise on neural network hidden representations.
method Information Imbalance proxy of conditional mutual information to compare hidden representations.
result Representations learned with noisy labels are more informative than those with clean labels in the underparameterized regime, and equally informative in the overparameterized regime.
Unravelling hidden patterns in datasets is a classical problem with many potential applications. In this paper, we present a challenge whose objective is to discover nonlinear relationships in noisy cloud of points. If a set of point satisfies a nonlinear relationship that is unlikely to be due to randomness, we will l…
Statistical Query Hardness of Multiclass Linear Classification with Random Classification Noisecs.LG
Study shows SQ hardness for multiclass linear classification with random noise.
problem Complexity of multiclass linear classification with random noise.
method Proves super-polynomial SQ lower bounds for MLC with RCN.
result Super-polynomial SQ lower bounds for MLC with RCN.
Study shows perceptrons with random labels perform similarly to Gaussian data.
problem The assumption of Gaussian input data is often seen as a limitation in machine learning.
method Analyzed generalized linear classification (perceptron model) with random labels.
result Minimum training loss is independent of data covariance for high-dimensional input data.
In this paper we propose strategies for estimating performance of a classifier when labels cannot be obtained for the whole test set. The number of test instances which can be labeled is very small compared to the whole test data size. The goal then is to obtain a precise estimate of classifier performance using as lit…
NoiseRank reduces label noise without supervision, improving classification accuracy.
problem Label noise in datasets from noisy channels.
method NoiseRank uses Markov Random Fields to estimate and rank instances based on their noise probability.
result NoiseRank improves classification accuracy on noisy datasets.
We adapt the idea of random projections applied to the output space, so as to enhance tree-based ensemble methods in the context of multi-label classification. We show how learning time complexity can be reduced without affecting computational complexity and accuracy of predictions. We also show that random output spac…
Ensemble methods have been shown to be an effective tool for solving multi-label classification tasks. In the RAndom k-labELsets (RAKEL) algorithm, each member of the ensemble is associated with a small randomly-selected subset of k labels. Then, a single label classifier is trained according to each combination of ele…
Proposes a method to make statistical inferences robust in spatially dependent settings with missing at random labels.
problem Statistical inference challenges with missing at random labels and spatial dependence.
method Doubly robust estimator with cross-fit nuisances and jackknife spatial HAC variance correction.
result Asymptotically valid confidence intervals with improved finite-sample calibration.
We investigate the problem of sequentially predicting the binary labels on the nodes of an arbitrary weighted graph. We show that, under a suitable parametrization of the problem, the optimal number of prediction mistakes can be characterized (up to logarithmic factors) by the cutsize of a random spanning tree of the g…
Study shows how Laplacian semi-supervised learning behaves at low labeling rates.
problem Understanding behavior of Laplacian semi-supervised learning at very low label rates.
method Analysis of random geometric graphs and -convergence tools.
result For certain conditions, Laplacian learning becomes degenerate and spikes form; for others, it remains well-posed and consistent.
Paper introduces RPWithPrior for efficient label differential privacy in regression.
problem Protecting user privacy in regression tasks with minimal accuracy loss.
method Modeling responses as continuous random variables, avoiding discretization; estimating optimal intervals for randomized responses.
result RPWithPrior algorithm guarantees ε-label differential privacy and outperforms existing methods.
The paper compares aggregated data labels in curated and random bags for machine learning models.
problem Protecting user privacy in machine learning systems with aggregated data.
method Examined curated and random bags for training machine learning models and compared their performance.
result Gradient-based learning can be performed on aggregated data without performance degradation.
The labeled stochastic block model is a random graph model representing networks with community structure and interactions of multiple types. In its simplest form, it consists of two communities of approximately equal size, and the edges are drawn and labeled at random with probability depending on whether their two en…
This paper explores memorization in adversarial training and proposes a mitigation algorithm.
problem Understanding and mitigating robust overfitting in adversarial training.
method Demonstrated the capacity of deep networks to memorize adversarial examples, analyzed convergence and generalization issues, and proposed a new mitigation algorithm.
result Identified robust overfitting as a significant drawback of adversarial training and proposed a mitigation algorithm.
NURD improves model performance by distilling representations independent of nuisance variables.
problem Models trained under spurious correlations may fail on data with different nuisance-label relationships.
method Developed Nuisance-Randomized Distillation (NURD) to find representations independent of nuisance variables.
result NURD finds representations that perform better regardless of nuisance-label relationships.
Semi-supervised learning methods are motivated by the availability of large datasets with unlabeled features in addition to labeled data. Unlabeled data is, however, not guaranteed to improve classification performance and has in fact been reported to impair the performance in certain cases. A fundamental source of err…
The paper tackles noisy labels in high-dimensional data, showing low-dimensional intuitions fail and proposing an optimized method.
problem Noisy labels in high-dimensional data classification.
method Linear classifier with a label noisiness aware loss function, using random matrix theory and Gaussian mixture data model.
result The performance of the linear classifier in high-dimension converges to a limit involving scalar statistics of the data, and the optimal classifier in low-dimension fails.
Understanding the underlying mechanisms that enable the empirical successes of deep neural networks is essential for further improving their performance and explaining such networks. Towards this goal, a specific question is how to explain the "surprising" behavior of the same over-parametrized deep neural networks tha…
The field of few-shot learning has been laboriously explored in the supervised setting, where per-class labels are available. On the other hand, the unsupervised few-shot learning setting, where no labels of any kind are required, has seen little investigation. We propose a method, named Assume, Augment and Learn or AA…
The multilabel learning problem with large number of labels, features, and data-points has generated a tremendous interest recently. A recurring theme of these problems is that only a few labels are active in any given datapoint as compared to the total number of labels. However, only a small number of existing work ta…
Study online learning with set-valued feedback, showing differences between deterministic and randomized approaches.
problem Online learning with set-valued feedback, where labels are sets rather than single labels.
method Introduced new combinatorial dimensions (Set Littlestone and Measure Shattering) to characterize learnability.
result Characterized deterministic and randomized online learnability, and established bounds for various learning settings.
Study efficient learning of robust halfspaces with noise.
problem Learning robust halfspaces in the presence of adversarial perturbations and random label noise.
method Provides conditions for robust learnability and a simple algorithm for any ℓ_p perturbation.
result Simple computationally efficient algorithm for robust learning with random label noise.
Semi-supervised K-means++stat.ML
Traditionally, practitioners initialize the {\tt k-means} algorithm with centers chosen uniformly at random. Randomized initialization with uneven weights ({\tt k-means++}) has recently been used to improve the performance over this strategy in cost and run-time. We consider the k-means problem with semi-supervised inf…
New methods protect privacy while providing accurate prediction sets.
problem Privacy-preserving conformal prediction for untrusted aggregators.
method Two LDP approaches: k-ary randomized response and binary search response.
result Finite-sample coverage guarantees and robust coverage under randomization.
Conditional Random Fields (CRF) are frequently applied for labeling and segmenting sequence data. Morency et al. (2007) introduced hidden state variables in a labeled CRF structure in order to model the latent dynamics within class labels, thus improving the labeling performance. Such a model is known as Latent-Dynamic…
We study the problem of recovering the latent ground truth labeling of a structured instance with categorical random variables in the presence of noisy observations. We present a new approximate algorithm for graphs with categorical variables that achieves low Hamming error in the presence of noisy vertex and edge obse…
Paper tackles causal inference with partially labeled data, introducing robust methods.
problem Challenges in causal inference due to partially labeled datasets and potential bias.
method Decaying missing-at-random framework and BRSS estimator for doubly robust causal inference.
result Established asymptotic normality of BRSS estimator under decaying labeling propensity scores.
We study the problem of attacking a machine learning model in the hard-label black-box setting, where no model information is revealed except that the attacker can make queries to probe the corresponding hard-label decisions. This is a very challenging problem since the direct extension of state-of-the-art white-box at…
The goal of eXtreme Multi-label Learning (XML) is to automatically annotate a given data point with the most relevant subset of labels from an extremely large vocabulary of labels (e.g., a million labels). Lately, many attempts have been made to address this problem that achieve reasonable performance on benchmark data…
Estimates binary labels from dependent data using Markov Random Fields.
problem Statistical estimation from dependent data across spatial, temporal, and social domains.
method Modeling dependencies as Markov Random Fields and providing efficient estimation algorithms.
result Statistically efficient estimation rates for Ising models from a single sample.
Lower bound proves ridgeless regression performs poorly near interpolation threshold.
problem Proving performance of ridgeless regression near interpolation threshold.
method Distribution-independent lower bound for mean squared error in noisy ridgeless linear regression.
result Lower bound implies ridgeless regression performs poorly near interpolation threshold.
We study the performance of data-driven, a priori and random approaches to label space partitioning for multi-label classification with a Gaussian Naive Bayes classifier. Experiments were performed on 12 benchmark data sets and evaluated on 5 established measures of classification quality: micro and macro averaged F1 s…
Recent progress has shown that few-shot learning can be improved with access to unlabelled data, known as semi-supervised few-shot learning(SS-FSL). We introduce an SS-FSL approach, dubbed as Prototypical Random Walk Networks(PRWN), built on top of Prototypical Networks (PN). We develop a random walk semi-supervised lo…