Introduces foundation priors for using model-generated data in empirical research.
problem Using model-generated data as real observations in empirical research.
method Introduces foundation priors as an exponential-tilted, generalized Bayesian update of the user's primitive prior.
result Synthetic data reflects both model patterns and user's priors, enabling principled use in empirical work.
Proposes a method to use generators as EBM foundations without latent inference.
problem Training EBMs from generator outputs without latent variables.
method Formulates a Hat EBM using generator outputs and residual variables.
result Strong performance on various generator tasks.
Grading function evaluates Bach-style chorales, outperforming human experts.
problem Difficulty in automatically evaluating musical style correctness.
method Introduces a grading function for evaluating four-part chorales in the style of J.S. Bach.
result Transformer model output is outperformed by the grading function at discriminating Bach chorales.
In this paper, we study the novel problem of not only predicting ingredients from a food image, but also predicting the relative amounts of the detected ingredients. We propose two prediction-based models using deep learning that output sparse and dense predictions, coupled with important semi-automatic multi-database …
Conformal Alignment ensures trustworthy outputs from foundation models.
problem Ensuring outputs from foundation models align with human values in high-stakes tasks.
method A framework that trains an alignment predictor using reference data to select trustworthy outputs.
result Conformal Alignment accurately identifies trustworthy outputs via lightweight training over moderate reference data.
We introduce a new neural architecture to learn the conditional probability of an output sequence with elements that are discrete tokens corresponding to positions in an input sequence. Such problems cannot be trivially addressed by existent approaches such as sequence-to-sequence and Neural Turing Machines, because th…
Efficient SGPRN model for imputation and visualization of missing data.
problem Imputation and visualization of missing data in time-varying correlation.
method Stochastic collapsed variational inference with structured Gaussian process regression network.
result Our model provides better imputation results on missing data than state-of-the-art methods.
New method uses machine learning to estimate sensitivity without binning.
problem Estimating sensitivity of high-dimensional data sets without binning.
method Combines machine-learning classification with likelihood-based inference tests using Kernel Density Estimators.
result Significance estimation is not sensitive to non-smooth probability distributions.
Method guarantees coherent factuality for language model outputs in reasoning tasks.
problem Ensuring correctness of language model outputs in reasoning tasks.
method Developed a conformal-prediction-based method applied to subgraphs within a deducibility graph.
result Achieved coherent factuality across target coverage levels, 90% on stricter definition.
We are given a video of a person performing a certain activity, from which we extract a controllable model. The model generates novel image sequences of that person, according to arbitrary user-defined control signals, typically marking the displacement of the moving body. The generated video can have an arbitrary back…
Proposes MEED framework for model interpretation.
problem Improving model interpretability and avoiding undesired characteristics.
method Adversarial Infidelity Learning (AIL) for effective feature selection.
result AIL mechanism helps learn desired conditional distribution.
Diffusion models generate private synthetic images with high quality.
problem Generating private synthetic images from sensitive datasets.
method Differentially private fine-tuning of pre-trained diffusion models.
result Improved synthetic data quality and accuracy compared to non-private models.
While it has not yet been proven, empirical evidence suggests that model generalization is related to local properties of the optima which can be described via the Hessian. We connect model generalization with the local property of a solution under the PAC-Bayes paradigm. In particular, we prove that model generalizati…
A new method for generating synthetic data using posterior distribution learning accelerates inference.
problem Generating high-quality synthetic data requires many discretization steps, which is computationally expensive.
method Learning the posterior distribution of clean data samples given noisy versions, using a scoring rule instead of regression loss.
result Consistently outperforms standard diffusion models at few discretization steps.
Artificial Intelligence (AI) can now automate the algorithm selection, feature engineering, and hyperparameter tuning steps in a machine learning workflow. Commonly known as AutoML or AutoAI, these technologies aim to relieve data scientists from the tedious manual work. However, today's AutoAI systems often present on…
This paper proposes a novel model for the rating prediction task in recommender systems which significantly outperforms previous state-of-the art models on a time-split Netflix data set. Our model is based on deep autoencoder with 6 layers and is trained end-to-end without any layer-wise pre-training. We empirically de…
Designing effective model-based reinforcement learning algorithms is difficult because the ease of data generation must be weighed against the bias of model-generated data. In this paper, we study the role of model usage in policy optimization both theoretically and empirically. We first formulate and analyze a model-b…
FCNv2 robustness tested under noise and random initial conditions.
problem Assessing AI weather forecasting model robustness to input noise.
method Two experiments with varying noise levels and random initial conditions.
result FCNv2 preserves hurricane features under low to moderate noise, but underestimates intensity and persistence.
In the domain of machine learning, Neural Memory Networks (NMNs) have recently achieved impressive results in a variety of application areas including visual question answering, trajectory prediction, object tracking, and language modelling. However, we observe that the attention based knowledge retrieval mechanisms us…
Improved neural transducer model outperforms attention model on longer sequences.
problem Improving performance of neural transducer models.
method Comparison of training criteria (marginalization vs. maximum approximation), model generalization, and output label topology.
result Final transducer model outperforms attention model by over 6% relative WER on Switchboard 300h.
Influence functions help study large language model generalization, revealing surprising decay patterns.
problem Understanding and mitigating risks in large language models (LLMs).
method Eigenvalue-corrected Kronecker-Factored Approximation (EK-FAC) to scale influence functions to LLMs.
result Influences decay to near-zero when key phrases order is flipped, revealing a surprising limitation.
Improves model generalization by minimizing loss sharpness.
problem Overparameterized models often fail to generalize well despite low training loss.
method Sharpness-Aware Minimization (SAM) minimizes both loss value and sharpness.
result SAM improves model generalization across various datasets and models.
This paper investigates and compares currency substitution between the currencies of Central and Eastern European (CEE) countries and the euro. In addition, we develop a model with microeconomic foundations, which identifies difference between currency substitution and money demand sensitivity to exchange rate variatio…
This paper improves neural network generalization by dynamically learning kernel parameters.
problem Improving neural network generalization and adaptability.
method Diagonal adaptive kernel model that learns kernel eigenvalues and output coefficients during training.
result The diagonal adaptive kernel model significantly improves generalization over fixed-kernel methods.
Deep models generate geometric objects with global properties.
problem Comparing neural models' global properties from generated samples.
method Training on datasets of reflexive polytopes, comparing different representations.
result Models learn non-trivial global properties of geometric objects.
This paper addresses the problem of scalable optimization for L1-regularized conditional Gaussian graphical models. Conditional Gaussian graphical models generalize the well-known Gaussian graphical models to conditional distributions to model the output network influenced by conditioning input variables. While highly …
Theory explains creativity in diffusion models generating novel images.
problem Diffusion models generate highly original images far from training data.
method Identified locality and equivariance as inductive biases to prevent optimal score-matching.
result Analytic models predict diffusion model outputs with high accuracy.
WaveFit uses fixed-point iteration to create high-quality neural vocoders.
problem Creating high-quality neural vocoders with fast inference.
method Integrates GANs' adversarial training into a DDPM-like iterative framework based on fixed-point iteration.
result WaveFit synthesizes speech with naturalness comparable to human speech, and is significantly faster than existing methods.
Loss estimator improves model calibration and generalization.
problem Ensuring models generalize to unseen data.
method Train a loss estimator alongside the model using contrastive training.
result Improves model generalization and calibration.
A model learns causal graphs from summary statistics of synthetic data.
problem Causal discovery algorithms are brittle with large sets of variables and limited data.
method A supervised model trained on synthetic data predicts causal graphs from summary statistics.
result The model generalizes well beyond its training set and runs on large graphs.
New measures quantify how data augmentation improves model performance.
problem Understanding the effectiveness of data augmentation in deep learning.
method Introduced Affinity and Diversity measures to quantify augmentation performance.
result Augmentation performance is best achieved by optimizing both Affinity and Diversity.
SGMs fail to generate samples from complex distributions even when the score function is learned well.
problem Score-based Generative Models fail to produce high-quality samples from complex distributions.
method Score-based Generative Models (SGMs) are evaluated under conditions where the score function is learned well.
result SGMs can only generate Gaussian blurring of training data points, not complex distributions.
Diffusion models generate new samples by adding and removing noise.
problem Generating new samples from data.
method Apply noise to data, reverse the process to generate new samples.
result Diffusion models can improve classifier performance on imbalanced data.
Dependent MMD coresets help compare multiple related datasets.
problem Comparing multiple related datasets for insights into model generalization.
method Dependent MMD coresets for collections of datasets.
result Dependent MMD coresets facilitate comparison and understanding of multiple related datasets.
The study characterizes diffusion model generalization using data-dependent ridge manifolds.
problem Understanding where diffusion model-generated samples lie when not memorizing the training set.
method Introduced a time-dependent family of log-density ridge manifolds to characterize reverse-time inference.
result Generated samples evolve by a reach-align-slide mechanism, controlled by normal and tangential components of training error.
New approach improves model generalization through distributionally robust learning.
problem Improving model generalization in machine learning.
method Stochastic gradient descent applied to the outer minimization problem, with gradient estimation through multi-level Monte Carlo randomization.
result Our approach yields significant benefits over previous work in numerical experiments.
Neural waveform models such as the WaveNet are used in many recent text-to-speech systems, but the original WaveNet is quite slow in waveform generation because of its autoregressive (AR) structure. Although faster non-AR models were recently reported, they may be prohibitively complicated due to the use of a distillin…
Unified model for sequence labeling and classification.
problem Efficiently perform multiple sequence labeling tasks.
method Generative framework with shared natural language output space.
result Significant improvements in few-shot and low-resource slot labeling.
It has long been suggested that the biological brain operates at some critical point between two different phases, possibly order and chaos. Despite many indirect empirical evidence from the brain and analytical indication on simple neural networks, the foundation of this hypothesis on generic non-linear systems remain…
Paper introduces a new cost function to improve deep learning model generalization.
problem Overfitting and poor extrapolation of deep learning models.
method Introduces a 'whitening' cost function based on the Ljung-Box statistic.
result Significant improvement in generalization for RNNs and image autoencoders.
Unified CLIP space manipulations improve GAN adaptation with a single target image.
problem Overfitting or underfitting in fine-tuning a pre-trained generator with a single target image.
method Two-step training strategy: latent optimization in CLIP space followed by generator fine-tuning with CLIP space consistency loss.
result Our model generates diverse outputs with the target texture and outperforms baseline models.
The paper analyzes how clustering sensitive data can improve model generalization without revealing individual information.
problem Ensuring user data privacy in personalized recommendation systems.
method Look-alike clustering to replace sensitive features with cluster averages, analyzed using Convex Gaussian Minimax Theorem.
result Training models using anonymous cluster centers can improve generalization error, especially in high-dimensional settings.
New model simplifies symmetry handling in generative AI.
problem Symmetry handling in generative models for scientific tasks.
method Quotient-space diffusion models, viewing symmetry as quotient space.
result Improves performance over existing methods for molecular structure generation.
This work explores the generalization properties of diffusion models, providing theoretical and empirical insights.
problem Theoretical understanding of diffusion models' generalization capabilities remains underdeveloped.
method Theoretical exploration and quantitative analysis of generalization gaps in diffusion models.
result Established polynomially small generalization error (O(n−2/5+m−4/5)) for diffusion models, avoiding the curse of dimensionality. Diffusion models generalize well until a threshold is reached, preventing memorization.
problem Understanding why diffusion models don't memorize training data.
method Investigation of training dynamics and two timescales: τgen and τmem. result The threshold τmem increases linearly with training set size n, preventing memorization. SYNC learns time-aware causal representations to improve model generalization in evolving domains.
problem Spurious correlations and shortcut learning in existing EDG methods hinder model generalization.
method SYNC integrates dynamic causal factors and causal mechanism drifts into a sequential VAE framework.
result SYNC achieves superior temporal generalization performance on synthetic and real-world datasets.
Sig-DEG speeds up diffusion models by distilling them into faster approximations.
problem Computational intensity of diffusion models at inference time.
method Signature-based differential equation generation to summarize Brownian motion.
result Sig-DEG reduces inference steps by an order of magnitude while maintaining generation quality.
Pairwise Label Smoothing improves deep model generalization by reducing overconfidence.
problem Improving deep model generalization through regularization.
method PLS smooths labels for pairs of samples, learning distribution mass during training.
result PLS significantly outperforms LS and baseline models, reducing up to 30% classification error.