DSI measures dataset separability for neural networks.
problem Difficulty in separating different classes of data in neural networks.
method Created the Distance-based Separability Index (DSI) to quantify dataset separability.
result DSI effectively measures dataset separability and indicates similar distributions of different classes.
Guiding the design of neural networks is of great importance to save enormous resources consumed on empirical decisions of architectural parameters. This paper constructs shallow sigmoid-type neural networks that achieve 100% accuracy in classification for datasets following a linear separability condition. The separab…
SepVAE separates patient-specific patterns from healthy ones using contrastive VAE.
problem Separating patient-specific patterns from healthy ones in medical datasets.
method SepVAE uses a contrastive VAE with disentangling and classification losses to differentiate between common and salient features.
result SepVAE outperforms previous methods in three medical applications and a CelebA dataset.
Monoaural audio source separation is a challenging research area in machine learning. In this area, a mixture containing multiple audio sources is given, and a model is expected to disentangle the mixture into isolated atomic sources. In this paper, we first introduce a challenging new dataset for monoaural source sepa…
Singing voice separation attempts to separate the vocal and instrumental parts of a music recording, which is a fundamental problem in music information retrieval. Recent work on singing voice separation has shown that the low-rank representation and informed separation approaches are both able to improve separation qu…
New approach for classification using trigonometric polynomial kernels from signal processing.
problem Classifying objects in arbitrary compact metric spaces.
method Localized trigonometric polynomial kernels for separating different probability measures.
result Minimal number of queried labels for perfect classification.
Paper proves mathematically that poisoning datasets can be detected.
problem Detecting data poisoning attacks in datasets.
method Mathematical definition and Conformal Separability Test.
result Dataset poisoning can be effectively detected.
Scattering networks maximize separation on low-dimensional data.
problem Maximizing separation capacity on low-dimensional datasets.
method Characterize and bound separation capacity for feature extractors, then apply to scattering networks with specific criteria.
result Design criteria for scattering networks to maximize separation on low-dimensional data.
The paper solves optimal bounds for separating data points in high dimensions.
problem Correcting AI errors and analyzing vulnerabilities in high-dimensional data.
method General stochastic separation theorems with optimal probability estimates.
result Explicit and optimal estimates of separation probabilities for important classes of distributions.
In this paper, we presented a novel semi-supervised one-class classification algorithm which assumes that class is linearly separable from other elements. We proved theoretically that class is linearly separable if and only if it is maximal by probability within the sets with the same mean. Furthermore, we presented an…
LLM safety alignment explained as divergence estimation.
problem Aligning large language models to avoid harmful outputs.
method Presented a theoretical framework showing alignment methods as divergence estimators.
result KLDO method improves safety alignment using compliance-refusal datasets.
Despite substantial progress in signal source separation, results for richly structured data continue to contain perceptible artifacts. In contrast, recent deep generative models can produce authentic samples in a variety of domains that are indistinguishable from samples of the data distribution. This paper introduces…
Efficient neural network for audio source separation.
problem End-to-end general purpose audio source separation.
method SuDoRMRF structure with simple one-dimensional convolutions for feature aggregation.
result SuDoRMRF achieves high quality audio source separation with minimal computational resources.
Recent progress in separating the speech signals from multiple overlapping speakers using a single audio channel has brought us closer to solving the cocktail party problem. However, most studies in this area use a constrained problem setup, comparing performance when speakers overlap almost completely, at artificially…
New method separates objects from images using deep neural networks trained to inpaint.
problem Fully self-supervised instance separation of occluded objects in images.
method Maximizes independence of two image regions given a fully self-supervised inpainting network.
result Method achieves similar segmentation performance to fully supervised methods on microscopy image datasets.
Gradient methods avoid overfitting on separable data.
problem Overfitting on separable data with gradient methods.
method Gradient methods (flow, descent, SGD) on separable data.
result Gradient methods asymptotically do not overfit on separable data.
SAVeD detects dataset versions without metadata, improving accuracy and separation.
problem Difficulty in identifying similar versions of structured datasets.
method Contrastive learning with modified SimCLR pipeline, generating and contrasting augmented table views.
result SAVeD achieves higher accuracy and separation scores on unseen tables.
We introduce Wavesplit, an end-to-end source separation system. From a single mixture, the model infers a representation for each source and then estimates each source signal given the inferred representations. The model is trained to jointly perform both tasks from the raw waveform. Wavesplit infers a set of source re…
A new NMF model for co-clustering and data approximation.
problem Finding a low rank approximation for nonnegative data.
method Generalizes separability assumption for NMF, proposing Co-Separable NMF (CoS-NMF).
result CoS-NMF outperforms state-of-the-art methods in co-clustering and data approximation.
Deep neural networks and decision trees operate on largely separate paradigms; typically, the former performs representation learning with pre-specified architectures, while the latter is characterised by learning hierarchies over pre-specified features with data-driven architectures. We unite the two via adaptive neur…
Inverse Drum Machine separates drum mixes using transcription and synthesis.
problem Separating individual drum tracks from mixed recordings.
method Analysis-by-synthesis framework combining deep learning and automatic transcription.
result Separation quality comparable to supervised methods requiring isolated stems.
Adam's bias shifts from full-batch to max-margin of different norms for separable data.
problem Understanding Adam's implicit bias in the incremental batch setting.
method Analyzing incremental Adam on linearly separable data, constructing datasets, and using a proxy algorithm.
result Incremental Adam can converge to different max-margin classifiers depending on the dataset and batching scheme.
We introduce a new metric to evaluate corruption robustness of ML classifiers.
problem Evaluating corruption robustness of machine learning classifiers.
method We propose a test data augmentation method using minimal class separation distance to derive a robustness distance ε and a metric MSCR.
result The MSCR metric allows interpretable comparison of classifier robustness on different datasets.
Modern large-scale datasets are frequently said to be high-dimensional. However, their data point clouds frequently possess structures, significantly decreasing their intrinsic dimensionality (ID) due to the presence of clusters, points being located close to low-dimensional varieties or fine-grained lumping. We test a…
Develops a Bayesian non-parametric approach for signal separation with varying components.
problem Signal separation with varying components across different input locations.
method Augments Gaussian Process Latent Variable Models with weighted sums of pure component signals and incorporates priors for linear weights.
result Framework allows for non-linear variations in signals and incorporates useful priors for linear weights.
Deep convolutional neural networks are known to specialize in distilling compact and robust prior from a large amount of data. We are interested in applying deep networks in the absence of training dataset. In this paper, we introduce deep audio prior (DAP) which leverages the structure of a network and the temporal in…
Single-microphone, speaker-independent speech separation is normally performed through two steps: (i) separating the specific speech sources, and (ii) determining the best output-label assignment to find the separation error. The second step is the main obstacle in training neural networks for speech separation. Recent…
A new method for separating mixed signals in space and time.
problem Nonlinear and nonstationary spatio-temporal data challenges.
method Identifiable autoregressive variational autoencoder.
result The method outperforms existing techniques in blind source separation and spatio-temporal prediction.
A hybrid K-NN and SVM technique improves classification accuracy.
problem Improving classification accuracy in pattern recognition.
method Discriminative nearest neighbour classification combined with SVM.
result The hybrid technique outperforms state-of-the-art methods.
Law explains how deep networks separate data for classification.
problem Black-box nature of deep learning limits architecture design and interpretation.
method Studied how deep neural networks process data in intermediate layers.
result Law of geometric data separation emerges in various architectures and datasets.
Spatial blind source separation simplifies multivariate spatial prediction.
problem Predicting multivariate measurements at unobserved locations with spatial dependencies.
method Spatial blind source separation as a pre-processing tool compared to Cokriging and neural networks.
result Spatial blind source separation simplifies spatial prediction by avoiding cross-dependencies.
Fisher loss improves deep domain adaptation by learning discriminative within-class compact and between-class separable representations.
problem Improving deep domain adaptation performance by learning discriminative representations.
method Proposes a Fisher loss to learn discriminative representations that are within-class compact and between-class separable.
result Noticeable improvements in deep domain adaptation performance, e.g., 6.67% absolute improvement in mean accuracy on the Office-Home dataset.
Standard methods in deep learning for natural language processing fail to capture the compositional structure of human language that allows for systematic generalization outside of the training distribution. However, human learners readily generalize in this way, e.g. by applying known grammatical rules to novel words.…
MixFT re-partitions data into sub-domains for better TSFM fine-tuning.
problem Improving zero-shot forecasting for new time series domains.
method MixFT re-divides data using Bayesian mixtures into homogeneous sub-domains for separate fine-tuning.
result MixFT outperforms per-dataset fine-tuning methods.
Isolating individual instruments in a musical mixture has a myriad of potential applications, and seems imminently achievable given the levels of performance reached by recent deep learning methods. While most musical source separation techniques learn an independent model for each instrument, we propose using a common…
Study evaluates Tree-Ring Watermarking in rectified flow-based models, revealing detection and separability limitations.
problem Detecting and separating Tree-Ring Watermarks in rectified flow-based models.
method Extensive experimentation comparing SD 2.1 and FLUX.1-dev models with various text guidance configurations and augmentation attacks.
result Inversion limitations affect watermark recovery and statistical separation.
Recent deep learning approaches have achieved impressive performance on speech enhancement and separation tasks. However, these approaches have not been investigated for separating mixtures of arbitrary sounds of different types, a task we refer to as universal sound separation, and it is unknown how performance on spe…
This paper ranks pre-trained DNNs using a novel SI measure.
problem Optimizing pre-trained DNN selection for transfer learning.
method Automated ranking via Separation Index (SI) on target datasets.
result Ranked pre-trained DNNs improve classification performance.
Generalizes underlap coefficient for multivariate group separation.
problem Quantifying distributional separation across groups in statistical learning.
method Generalizes underlap coefficient (UNL) to multivariate variables, establishes key properties, interprets as dependence measure, proposes efficient estimator.
result Highlights the UNL's utility in clustering for evaluating group structure dependence on covariates.
New deep-unfolded network improves video background separation.
problem Video foreground-background separation.
method Deep unfolding of an iterative RPCA algorithm with adaptive learning.
result Proposed network outperforms state-of-the-art in video foreground-background separation.
This paper proposes RAS, a novel unsupervised loss function for speech separation.
problem Unsupervised speech separation without teacher models or synthesized mixtures.
method Train a neural network to separate sources given one channel, using Wiener filtering and SI-SDR.
result Achieves significant SDR improvement with minimal labeled data.
New method disentangles sources of different timescales in planetary seismic data.
problem Unsupervised source separation of multi-scale seismic data from planetary missions.
method Wavelet scattering spectra for multi-scale clustering and variational autoencoder for source separation.
result Disentangles sources with different timescales in InSight mission seismic data.
Generalizes underlap coefficient for multivariate group separation.
problem Quantifying distributional separation across groups in statistical learning.
method Generalizes underlap coefficient (UNL) to multivariate settings, studies its relationship with Bayes risk and mutual information, proposes an efficient importance sampling estimator.
result UNL as a measure of dependence between group labels and variables of interest, interpretable measure of partition-covariate dependence in clustering.
Paper proposes data quality measures for large-scale high-dimensional data.
problem Lack of practical data quality measures for large-scale high-dimensional data.
method Proposes two data quality measures: class separability and in-class variability. Efficient algorithms based on random projections and bootstrapping are provided.
result Efficient algorithms for computing data quality measures on large-scale high-dimensional data.
Paper develops robust methods for panel data with latent groups, improving inference under group separation violations.
problem Inference in latent group panel models under group separation violations.
method Selective conditional inference approach to derive conditional distribution of coefficients given estimated group structure.
result Valid inference under violations of group separation, superior to traditional asymptotic methods.
Hybrid model improves music source separation by 1.4 dB.
problem Improving music source separation accuracy.
method End-to-end hybrid spectrogram and waveform model, using model decision for domain choice.
result 1.4 dB improvement in Signal-to-Distortion (SDR) on MusDB HQ dataset.
New framework assesses graph-learning datasets for better evaluation.
problem Insufficient evaluation of graph-learning datasets and methods.
method Introduces Rings framework for dataset ablations and proposes performance separability and mode complementarity measures.
result Demonstrates utility of Rings framework for graph-learning dataset evaluation.
In recent years, deep learning has surpassed traditional approaches to the problem of singing voice separation. The Wave-U-Net is a recent deep network architecture that operates directly on the time domain. The standard Wave-U-Net is trained with data augmentation and early stopping to prevent overfitting. Minimum hyp…