Generative model synthesizes complex data structures with composite and nested types.
problem Synthesizing complex data structures with composite and nested types.
method Generic framework using causal transformers for struct and list generation.
result Consistently outperforms state-of-the-art models on standard and complex hierarchical datasets.
Reinterprets DNF as a deep generative LDA model for complex data.
problem Limited applicability of LDA in complex data scenarios.
method Proposes a discriminative normalization flow (DNF) model and interprets it as a deep generative LDA.
result DNF and its subspace version outperform conventional LDA in modeling complex data.
Improves stability in hyperbolic neural networks for complex data generation.
problem Numerical instability in hyperbolic neural networks hinders complex architecture development.
method Proposes a novel hyperbolic AE-GAN architecture with stable layers.
result Demonstrates state-of-the-art performance in generating complex data.
We study the sample complexity of private synthetic data generation over an unbounded sized class of statistical queries, and show that any class that is privately proper PAC learnable admits a private synthetic data generator (perhaps non-efficient). Previous work on synthetic data generators focused on the case that …
We present a study of generalization for data-dependent hypothesis sets. We give a general learning guarantee for data-dependent hypothesis sets based on a notion of transductive Rademacher complexity. Our main result is a generalization bound for data-dependent hypothesis sets expressed in terms of a notion of hypothe…
A new deep learning framework for topological data.
problem Developing models for data on complex topological domains.
method Introducing combinatorial complexes and developing attention-based CCNNs.
result CCNNs outperform existing models in tasks involving mesh shape analysis and graph learning.
Proposes a new method for causal inference in high-dimensional complex data.
problem Challenges in making causal inference with high-dimensional, nonlinear data.
method Combines deep learning techniques like sparse deep learning and stochastic neural networks.
result Outperforms existing methods in numerical studies.
Mixes higher-order simplicial complexes for data augmentation.
problem Lack of labeled data for complex systems with multiway interactions.
method Proposes mixup mechanisms for simplicial complexes, including linear and nonlinear mixup, and a convex clustering mixup.
result Synthetic simplicial complexes interpolate between existing data based on homomorphism densities.
Statistical learning theory has largely focused on learning and generalization given independent and identically distributed (i.i.d.) samples. Motivated by applications involving time-series data, there has been a growing literature on learning and generalization in settings where data is sampled from an ergodic proces…
Generative models can approximate high-dimensional data from lower dimensions without needing a latent dimension equal to or greater than the data's intrinsic dimension.
problem Theoretical limitations on the latent dimension required for generative models to approximate high-dimensional data distributions.
method Inspired by space-filling curves, the work demonstrates that generative networks can approximate distributions on d-dimensional manifolds from inputs of any arbitrary dimension, even lower than d. result Generative models can approximate high-dimensional data distributions from lower-dimensional inputs without needing a latent dimension equal to or greater than the data's intrinsic dimension.
This paper proves a generalization bound for complex-valued neural networks scaling with spectral complexity.
problem Ensuring the performance of complex-valued neural networks on unseen data.
method Theoretical derivation using Maurey Sparsification Lemma and Dudley Entropy Integral, empirical validation on various datasets.
result The spectral complexity of weight matrices is a significant factor in the generalization ability of complex-valued neural networks.
Study improves generalization bounds for equivariant networks on Markov data.
problem Challenges in integrating equivariance with Markov dependencies in neural networks.
method Applied McDiarmid's inequality and computed covering number using group theory.
result Derived upper bound on Rademacher complexity for equivariant neural networks on Markov datasets.
We present a novel notion of complexity that interpolates between and generalizes some classic existing complexity notions in learning theory: for estimators like empirical risk minimization (ERM) with arbitrary bounded losses, it is upper bounded in terms of data-independent Rademacher complexity; for generalized Baye…
Diffusion models generalize better with hierarchical data structure and regularization.
problem Understanding generalization in diffusion models with finite data.
method Analyzing diffusion models through data covariance spectra and developing a theoretical framework based on linear neural networks.
result Generalization in diffusion models improves with hierarchical data structure and regularization.
Rectified flows achieve optimal sample complexity for generating data.
problem Generating high-quality data samples efficiently.
method Rectified flows constrain transport trajectories to be linear, enabling efficient sampling.
result Achieve sample complexity of ildeO(ε−2), matching optimal rate for mean estimation. Complex embeddings handle non-metric proximity data better than traditional methods.
problem Proximities not always metric or inner product-based, causing convergence issues.
method Proposes complex-valued embeddings for non-vectorial data.
result Complex embeddings outperform traditional techniques on benchmarks.
Novel framework uses synthetic data to quantify uncertainty in complex data.
problem Uncertainty quantification in complex, unstructured data.
method Perturbation-Assisted Sample Synthesis (PASS) and Perturbation-Assisted Inference (PAI) framework.
result Statistically guaranteed validity in inference, enhancing reliability of synthetic data.
Quantum learning complexity reviewed using information theory.
problem Learning properties of quantum systems or processing data via quantum computing.
method Information-theoretic techniques focusing on data, copy, and model complexity.
result Copy complexity due to irreversible quantum measurements limits information extraction.
New bounds for quantum circuits depend on how data is encoded.
problem Lack of explicit dependence on data encoding in generalization bounds for PQCs.
method Derived generalization bounds that depend on data encoding strategies using Rademacher complexity and metric entropy.
result Optimal data-encoding strategies can be selected via structural risk minimization.
Transformers fine-tuned on synthetic data boost tabular data classification performance.
problem Improving tabular data classification accuracy.
method Fine-tuning ICL-transformers on synthetic datasets with complex decision boundaries.
result Fine-tuned ICL-transformers outperform regular neural networks on real-world datasets.
New method measures generalizability of deep neural networks based on decision boundary complexity.
problem Lack of generalization methods for deep neural networks.
method Created Decision Boundary Complexity (DBC) score to measure DNN complexity.
result Simpler decision boundaries lead to better generalizability, supporting Occam's Razor.
SCARY dataset generates complex causal scenarios for causality research.
problem Lack of complexity in existing causal datasets.
method Synthetic dataset with 40 scenarios, three seeds, and two data generation mechanisms.
result Provides a valuable resource for realistic causal discovery.
Graph Neural Networks struggle with generalization, especially OOD data; GRATIN solves this with Gaussian Mixture Model-based augmentation.
problem Graph Neural Networks struggle with generalization, particularly to unseen or out-of-distribution data.
method Theoretical framework using Rademacher complexity to compute a regret bound on generalization error. GRATIN algorithm leveraging Gaussian Mixture Models for efficient data augmentation.
result GRATIN outperforms existing augmentation techniques in terms of generalization and offers improved time complexity.
The paper presents a new framework for complex Support Vector Regression as well as Support Vector Machines for quaternary classification. The method exploits the notion of widely linear estimation to model the input-out relation for complex-valued data and considers two cases: a) the complex data are split into their …
Paper proposes an efficient causal discovery method with linear computational complexity.
problem Identifying causal relationships efficiently in large datasets.
method Approximate kernel-based generalized score function with low-rank technique and sampling algorithms.
result Significantly reduces computational costs while maintaining comparable accuracy.
RML improves generative modeling of complex distributions.
problem Learning complex distributions in applications.
method RML defines a forward process to a known distribution, then learns a reverse Markov process.
result RML efficiently captures complex distributions in simulations and climate data.
We study generalized complex manifolds from the point of view of symplectic and Poisson geometry. We start by showing that every generalized complex manifold admits a canonical Poisson structure. We use this fact, together with Weinstein's classical result on the local normal form of Poisson manifolds, to prove a local…
Generative adversarial networks (GANs) have been shown to provide an effective way to model complex distributions and have obtained impressive results on various challenging tasks. However, typical GANs require fully-observed data during training. In this paper, we present a GAN-based framework for learning from comple…
The paper provides results regarding the computational complexity of hybrid system identification. More precisely, we focus on the estimation of piecewise affine (PWA) maps from input-output data and analyze the complexity of computing a global minimizer of the error. Previous work showed that a global solution could b…
New framework improves worst-case generalization bounds for stochastic optimization.
problem Challenges in providing generalization guarantees for stochastic optimization algorithms.
method Introduces random set stability and empirically relevant complexity measures to avoid intractable mutual information terms.
result Bounded worst-case generalization error in terms of random set stability and empirically relevant complexity measures.
Proposes Neural Complexity (NC) for predicting and explaining generalization in deep neural networks.
problem Challenges in specifying a suitable complexity measure for deep neural networks to predict and explain generalization.
method A meta-learning framework that learns a scalar complexity measure through interactions with many heterogeneous tasks.
result Trained NC model can be added to standard training loss to regularize any task learner.
Study analyzes landscape complexity of empirical loss functions with correlated data.
problem Understanding the complexity of loss landscapes in machine learning with structured data.
method Kac-Rice formula and random matrix theory applied to high-dimensional empirical loss functions.
result Characterizes the average number of critical points in loss functions with structured data.
A complex-valued convolutional network (convnet) implements the repeated application of the following composition of three operations, recursively applying the composition to an input vector of nonnegative real numbers: (1) convolution with complex-valued vectors followed by (2) taking the absolute value of every entry…
Improves NF for complex data distributions with multiple modes.
problem Difficulty in handling data distributions with multiple isolated modes.
method Proposes a new framework using variational latent representation to improve NF.
result Significantly more powerful for generating data distributions with multiple modes.
Develops a measure-theoretic framework for complex co-occurrence data.
problem Modeling and interpreting complex co-occurrences in high-dimensional data.
method Introduces measure-theoretic probability and conditional probability, investigates E-integrals.
result Establishes a rigorous measure-theoretic foundation for co-occurrence modeling.
Paper introduces a neural network-based non-stationary influence kernel for complex event data.
problem Modeling complex, non-stationary, and dependent discrete event data.
method Neural Spectral Marked Point Processes (NSMPP) with a versatile non-stationary influence kernel.
result NSMPP outperforms state-of-the-art models on synthetic and real data.
Develops a private synthetic graph generator using Gromov-Wasserstein distance.
problem Creating private synthetic networks for complex data.
method Random connection model, fused Gromov-Wasserstein distance, differential privacy.
result Effective algorithm for generating private synthetic graphs with theoretical guarantees.
New bounds explain modern machine learning algorithms' generalization.
problem Explaining generalization behavior of modern machine learning algorithms.
method Proposes a new complexity measure based on empirical Rademacher complexity of an algorithm- and data-dependent hypothesis class.
result Obtains novel bounds with finite fractal dimension, simplifies proofs, and recovers known results.
Quantum models generalize well with little data, challenging traditional generalization theories.
problem Quantum machine learning models generalize well with few data, contradicting traditional theories.
method Systematic randomization experiments and theoretical constructions.
result Quantum neural networks can fit random states and labels, defying current generalization measures.
In this preregistration submission, we propose an empirical study of how networks handle changes in complexity of the data. We investigate the effect of network capacity on generalization performance in the face of increasing data complexity. For this, we measure the generalization error for an image classification tas…
We present a general theoretical analysis of structured prediction with a series of new results. We give new data-dependent margin guarantees for structured prediction for a very wide family of loss functions and a general family of hypotheses, with an arbitrary factor graph decomposition. These are the tightest margin…
We construct Hodge filtered cohomology groups for complex manifolds that combine the topological information of generalized cohomology theories with geometric data of Hodge filtered holomorphic forms. This theory provides a natural generalization of Deligne cohomology. For smooth complex algebraic varieties, we show th…
Study shows sample complexity for logistic regression with normal covariates.
problem Estimating parameters of logistic regression with normal design.
method Analyzes sample complexity in terms of dimension and inverse temperature.
result Shows two change-points in sample complexity curve based on inverse temperature.
No free lunch theorems suggest inductive biases are needed, but we show neural networks prefer low-complexity data.
problem The need for inductive biases in machine learning.
method Analysis of Kolmogorov complexity and neural network behavior on various datasets.
result Neural networks prefer low-complexity data, suggesting inductive biases are not always necessary.
Study shows low-complexity models can perform as well as state-of-the-art on small datasets.
problem Performance of deep learning models on small datasets.
method Wide variety of experiments with different deep learning architectures on small datasets.
result Low-complexity models can perform comparably well or better than state-of-the-art models on small datasets.
Adversarial training leads to clean data generalization with significant robust overfitting gap.
problem Significant robust generalization gap in adversarial training.
method Two theoretical views: representation complexity and training dynamics.
result ReLU nets with O(ND) extra parameters can achieve CGRO. Paper establishes a generalization bound for gradient flow using a data-dependent kernel.
problem Understanding the generalization properties of gradient-based optimization methods.
method Establishes a generalization bound for gradient flow through a data-dependent kernel called the loss path kernel (LPK).
result The LPK captures the entire training trajectory and leads to tighter generalization guarantees.
CT improves neural network performance on cell complex data.
problem Improving predictive performance of neural networks on complex data.
method Introducing the Cellular Transformer (CT) that generalizes graph-based transformers to cell complexes.
result CT achieves state-of-the-art performance on cell complex datasets without complex enhancements.