This paper strengthens the computational separation between multimodal and unimodal learning, showing unimodal learning is hard on typical instances.
problem Theoretical justification for empirical success of multimodal machine learning.
method Introduced a stronger average-case computational separation between unimodal and multimodal learning.
result For typical instances, unimodal learning is computationally hard, while multimodal learning is easy.
We simplify information measure computation using learned features.
problem Computing information measures from raw data is computationally expensive.
method Developed a separable design for computing information measures from learned feature representations.
result A variety of information measures can be computed efficiently through learned feature representations.
Reservoir computing's success depends on mapping different input time series to separable states.
problem Quantifying the ability of random linear reservoirs to map different input time series.
method Mathematical framework using spectral properties of the connectivity matrix.
result Separation capacity is fully characterized by the spectral properties of the connectivity matrix.
This paper provides a mathematical framework for time-delay reservoir computing.
problem Lack of rigorous mathematical foundations for reservoir computing properties.
method Control-theoretic framework, formal definitions of separation and fading memory, explicit lower bound derivation.
result Established formal definitions and connections to stability notions for time-delay systems.
Study shortest non-separating curves on non-orientable surfaces, proving NP-hardness and tractability.
problem Computing shortest non-separating simple closed curves on non-orientable surfaces.
method Developed tools for computing shortest curves, proving NP-hardness and tractability.
result Proved NP-hardness and fixed-parameter tractability for computing shortest orienting curves, and polynomial-time algorithm for non-orienting curves.
This paper proposes an alternative algorithm for multichannel variational autoencoder (MVAE), a recently proposed multichannel source separation approach. While MVAE is notable in its impressive source separation performance, the convergence-guaranteed optimization algorithm and that it allows us to estimate source-cla…
New approach models computer network activity as mixtures of sources.
problem Malicious activity detection in computer networks using standard algorithms is ineffective.
method Source separation approach to model short-term dynamics of computer network activity.
result Qualitative and quantitative experiments validate the approach.
The paper establishes a nearly-sharp statistical threshold for efficient learning in Latent MDPs with separated components.
problem Learning Latent Markov Decision Processes (LMDPs) with separated components.
method The paper considers various notions of separation and establishes a nearly-sharp statistical threshold for efficient learning. It also presents a quasi-polynomial algorithm with time complexity scaling in terms of the statistical threshold under a weaker assumption of separability under the optimal policy, and a near-matching time complexity lower bound under the exponential time hypothesis.
result Establishes a nearly-sharp statistical threshold for efficient learning in Latent MDPs with separated components.
Two commonly arising computational tasks in Bayesian learning are Optimization (Maximum A Posteriori estimation) and Sampling (from the posterior distribution). In the convex case these two problems are efficiently reducible to each other. Recent work (Ma et al. 2019) shows that in the non-convex case, sampling can som…
This work addresses the problem of learning sparse representations of tensor data using structured dictionary learning. It proposes learning a mixture of separable dictionaries to better capture the structure of tensor data by generalizing the separable dictionary learning model. Two different approaches for learning m…
Efficient neural network for audio source separation.
problem End-to-end general purpose audio source separation.
method SuDoRMRF structure with simple one-dimensional convolutions for feature aggregation.
result SuDoRMRF achieves high quality audio source separation with minimal computational resources.
The complement of a non-separating planar graph contains a K_n minor.
problem Characterizing the structure of complements of planar graphs.
method Analyzing the structure of complements of non-separating planar graphs and using examples to illustrate hypotheses.
result The order 2n-3 is the lowest possible for a non-separating planar graph whose complement contains a K_n minor.
A new method speeds up overlapping group lasso computations.
problem Time-consuming optimization of overlapping group lasso on large-scale problems.
method Non-overlapping statistical approximation to overlapping group lasso.
result The proposed penalty is statistically equivalent to overlapping group lasso.
Quantum computers outperform classical methods in density modeling.
problem Density modeling with quantum computers.
method Quantum-classical separation for density modeling.
result Quantum computers offer a super-polynomial advantage over classical algorithms for density modeling.
The paper introduces toric separable geometries and finds new extremal metrics.
problem Finding explicit extremal Kähler metrics on toric manifolds.
method Introducing toric separable geometries and analyzing their moduli space.
result Explicit computation of scalar curvature and derivation of necessary conditions for extremality.
Algorithm finds frequencies, amplitudes, and phases of sinusoids in noisy data.
problem Finding frequencies, amplitudes, and phases of sinusoids in noisy data.
method Maximum likelihood approach to estimate tone parameters from contaminated observations. Successively estimates frequencies and jointly optimizes amplitudes and phases.
result Near-linear computational complexity (O(N)) for estimating M number of sinusoidal sources. Separating an audio scene into isolated sources is a fundamental problem in computer audition, analogous to image segmentation in visual scene analysis. Source separation systems based on deep learning are currently the most successful approaches for solving the underdetermined separation problem, where there are more …
A parallel algorithm learns efficient Kronecker product dictionaries.
problem Sparse representation of 2D signals like images and hyperspectral data.
method Highly parallelizable algorithm for learning separable dictionaries.
result Competitive sparse representations at lower computational cost.
We compute the automorphism groups of the Torelli complex and the complex of separating curves for all but finitely many compact orientable surfaces. As an application, we show that the abstract commensurators of the Torelli group and the Johnson kernel for such surfaces are naturally isomorphic to the extended mapping…
Mirror flow optimizes separable data problems, converging to a maximum margin classifier.
problem Optimizing classification problems with separable data using mirror flow.
method Examine mirror flow on linearly separable classification problems, focusing on the horizon function of the mirror potential.
result Mirror flow converges to a maximum margin classifier for separable data under certain conditions.
Safe screening rule improves Group SLOPE efficiency.
problem Efficiently selecting groups of predictors in high-dimensional sparse learning.
method Safe screening rule for Group SLOPE, addressing block non-separable group effects.
result Significant computational efficiency gains without sacrificing accuracy.
Separating an audio scene such as a cocktail party into constituent, meaningful components is a core task in computer audition. Deep networks are the state-of-the-art approach. They are trained on synthetic mixtures of audio made from isolated sound source recordings so that ground truth for the separation is known. Ho…
Study on size and depth of neural networks for approximating benign functions, showing barriers and explicit results.
problem Understanding how size and depth of neural networks affect their ability to approximate benign functions.
method Analyzing ReLU networks for benign functions, proving barriers and explicit results.
result Explicit benign functions that cannot be approximated by networks of certain sizes or depths, showing barriers to size and depth separation.
Gaussian process (GP) audio source separation is a time-domain approach that circumvents the inherent phase approximation issue of spectrogram based methods. Furthermore, through its kernel, GPs elegantly incorporate prior knowledge about the sources into the separation model. Despite these compelling advantages, the c…
We compute the Floer homology of mapping classes which do not have any pseudo-Anosov components in the sense of Thurston's theory of surface diffeomorphisms. The formula for the Floer homology is obtained from a topological separation of fixed points and a separation mechanism for Floer connecting orbits. As examples, …
New separation concepts for Anosov representations help bound Thurston asymmetric metric.
problem Understanding diverging families of Anosov representations.
method Introducing separation concepts and analyzing combinatorial invariants.
result Critical exponent asymptotic to a graph invariant.
This work introduces sequential neural beamforming, which alternates between neural network based spectral separation and beamforming based spatial separation. Our neural networks for separation use an advanced convolutional architecture trained with a novel stabilized signal-to-noise ratio loss function. For beamformi…
New kernels allow learning from non-separable data.
problem Learning from non-separable data.
method Introducing entangled kernels and a two-step algorithm.
result Efficient algorithm for learning entangled kernels.
New criterion assesses cluster separability for validation.
problem Validating cluster analysis results and determining the number of clusters.
method Distinguishability criterion, combined loss function-based framework.
result Validated cluster configurations and determined the number of clusters.
AUC-spec optimizes graph-based SSL for complex label distributions.
problem Training accurate models with scarce labeled data and abundant unlabeled data.
method Computes a low-dimensional representation that maximizes class separation via AUC optimization.
result AUC-spec achieves competitive results on synthetic and real-world datasets.
Convolution Neural Network (CNN) has gained tremendous success in computer vision tasks with its outstanding ability to capture the local latent features. Recently, there has been an increasing interest in extending convolution operations to the non-Euclidean geometry. Although various types of convolution operations h…
A new method speeds up spectral normalization for neural nets.
problem Efficiently controlling the spectral norm of convolutional layers.
method Depthwise separable convolutions with spectral normalization.
result Significant reduction in computational and memory costs.
slimTrain simplifies DNN training by separating features and adapting hyperparameters.
problem Challenges in training deep neural networks, including non-convexity, non-smoothness, and hyperparameter sensitivity.
method slimTrain exploits separability in DNN architectures to reduce hyperparameter sensitivity and improve convergence.
result slimTrain outperforms existing methods with recommended hyperparameters and reduces sensitivity to remaining hyperparameters.
We define and discuss the first sparse coding algorithm based on closed-form EM updates and continuous latent variables. The underlying generative model consists of a standard `spike-and-slab' prior and a Gaussian noise model. Closed-form solutions for E- and M-step equations are derived by generalizing probabilistic P…
Randomly initialized neural networks can linearly separate arbitrary sets.
problem Mapping two arbitrary sets to linearly separable sets.
method Randomly initialized one-layer neural networks with sufficient width.
result With high probability, these networks can transform two sets into linearly separable sets.
Numerous algorithms are used for nonnegative matrix factorization under the assumption that the matrix is nearly separable. In this paper, we show how to make these algorithms efficient for data matrices that have many more rows than columns, so-called "tall-and-skinny matrices". One key component to these improved met…
Recently, a family of tractable NMF algorithms have been proposed under the assumption that the data matrix satisfies a separability condition Donoho & Stodden (2003); Arora et al. (2012). Geometrically, this condition reformulates the NMF problem as that of finding the extreme rays of the conical hull of a finite set …
Logistic regression is one of the most popular methods in binary classification, wherein estimation of model parameters is carried out by solving the maximum likelihood (ML) optimization problem, and the ML estimator is defined to be the optimal solution of this problem. It is well known that the ML estimator exists wh…
Higher granularity in MoE models boosts expressivity exponentially.
problem Expressivity of Mixture-of-Experts models with varying granularity.
method Comparing models with different numbers of active experts (granularity).
result Exponential separation in network expressivity based on granularity.
This work simplifies SVM parameter selection using S&S ratio.
problem SVM parameter tuning for optimal performance.
method S&S ratio to model SVM performance; automatic RP, kernel, and parameter selection.
result Optimized SVM parameters with reduced computational complexity.
Models for audio source separation usually operate on the magnitude spectrum, which ignores phase information and makes separation performance dependant on hyper-parameters for the spectral front-end. Therefore, we investigate end-to-end source separation in the time-domain, which allows modelling phase information and…
Tropical SVM tackles phylogenomics by classifying multi-locus data.
problem Classifying multi-locus data sets for phylogenetic analysis.
method Proposes tropical support vector machines (SVMs) for phylogenomics, formulated as linear programming problems.
result Developed methods for hard and soft margin tropical SVMs, proving necessary and sufficient conditions for separation.
In this work we show that randomized (block) coordinate descent methods can be accelerated by parallelization when applied to the problem of minimizing the sum of a partially separable smooth convex function and a simple separable convex function. The theoretical speedup, as compared to the serial method, and referring…
Improved tensor GLM estimation for complex data.
problem Complex tensor data in GLMs leads to high-dimensional, ill-posed estimation.
method Proposed LSRTR-M algorithm using Muon updates for faster convergence and lower errors.
result LSRTR-M converges faster and achieves lower errors than LSRTR.
Nonnegative matrix factorization (NMF) is a linear dimensionality technique for nonnegative data with applications such as image analysis, text mining, audio source separation and hyperspectral unmixing. Given a data matrix M and a factorization rank r, NMF looks for a nonnegative matrix W with r columns and a …
Invariant obstructs separating coassociative 4-folds.
problem Obstructing the separation of coassociative 4-folds.
method Defining a Z2-valued invariant for coassociative 4-folds with spin structures. result Invariant provides an obstruction to separating coassociative 4-folds.
New non-separable covariance kernels for spatiotemporal data derived from harmonic oscillator physics.
problem Capturing complex spatiotemporal dependencies in Gaussian processes.
method Hybrid spectral method based on the harmonic oscillator, deriving explicit covariance kernels.
result Explicit non-separable covariance kernels with space-time interactions.
Quantum speedup for Monte Carlo integration reduces integrand calls.
problem Reducing the number of calls to the integrand subroutine in high-dimensional Monte Carlo integration.
method Combining nested quantum amplitude estimation with pseudorandom numbers for separable integrands.
result Significant reduction in the number of integrand calls for high-dimensional integration.