MO-PaDGAN generates diverse, high-performance designs with multiple metrics.
problem Challenges in generating diverse, high-performance designs with multiple metrics.
method MO-PaDGAN uses a new Determinantal Point Processes based loss function for probabilistic modeling of diversity and performances.
result MO-PaDGAN expands the design space towards high-performance regions and generates new designs with high diversity and performances.
Stochastic variance-reduced gradient (SVRG) is a classical optimization method. Although it is theoretically proved to have better convergence performance than stochastic gradient descent (SGD), the generalization performance of SVRG remains open. In this paper we investigate the effects of some training techniques, mi…
MO-PaDGAN improves multi-objective optimization by generating diverse and high-performing designs.
problem Challenges in parameterizing engineering designs for multi-objective optimization.
method MO-PaDGAN uses a generative adversarial network with a Determinantal Point Processes loss function to address these challenges.
result MO-PaDGAN generates designs with improved performance and coverage, even surpassing training data.
Using a low-dimensional parametrization of signals is a generic and powerful way to enhance performance in signal processing and statistical inference. A very popular and widely explored type of dimensionality reduction is sparsity; another type is generative modelling of signal distributions. Generative models based o…
This study evaluates zero-shot LLMs in finance, finding ChatGPT performs well but fine-tuned models are better.
problem Evaluating zero-shot LLMs in financial tasks.
method Comparison of ChatGPT and fine-tuned models on annotated data.
result Fine-tuned models generally outperform zero-shot LLMs.
Optimized nonlinearities enhance generalization in random feature models.
problem Improving generalization performance in random feature models.
method Analyzed and defined optimal nonlinearities from Gaussian model parameters.
result Optimized nonlinearities achieve better generalization performance than ReLU.
Invariant polynomials improve machine learning performance.
problem Improving machine learning algorithms using invariant polynomials.
method Developed and implemented Lorentz- and permutation-invariant polynomial generators in neural networks.
result Reduction in loss on training and validation data with Hironaka decompositions.
We investigate nearest neighbor and generative models for transferring pose between persons. We take in a video of one person performing a sequence of actions and attempt to generate a video of another person performing the same actions. Our generative model (pix2pix) outperforms k-NN at both generating corresponding f…
This paper proposes a new algorithm for controlling classification results by generating a small additive perturbation without changing the classifier network. Our work is inspired by existing works generating adversarial perturbation that worsens classification performance. In contrast to the existing methods, our wor…
UnmaskingTrees improves tabular data imputation and generation using gradient-boosted decision trees.
problem Traditional methods outperform advanced deep learning techniques on tabular data imputation benchmarks.
method UnmaskingTrees employs gradient-boosted decision trees to incrementally unmask features for imputation and generation.
result UnmaskingTrees outperforms state-of-the-art methods on tabular imputation and generation benchmarks.
Study on distributed linear regression performance, focusing on generalization error.
problem Performance of distributed learning in large-scale linear regression.
method Statistical learning approach, focusing on generalization error.
result Generalization error of distributed solution can be higher than centralized solution.
TD-GEN generates graphs using tree decomposition, improving efficiency and performance.
problem Efficiently generating graphs with statistical properties.
method Tree decomposition, permutation invariant tree generation, incremental graph generation.
result Improved graph generation efficiency and performance.
A sequential learning framework improves domain generalization performance.
problem Training models robust to domain shift across multiple domains.
method Inspired by lifelong learning, a sequential training approach optimizing for all following domains.
result Improves performance on DG benchmarks with a simple, fast algorithm.
Study how predictions affect the data they're based on, improving generalization guarantees.
problem How well do models generalize when predictions influence the data they're trained on?
method Embed performative predictions into statistical learning theory and prove generalization bounds.
result There's a fundamental trade-off between affecting data and learning from it.
Adaptive optimization algorithms, such as Adam and RMSprop, have shown better optimization performance than stochastic gradient descent (SGD) in some scenarios. However, recent studies show that they often lead to worse generalization performance than SGD, especially for training deep neural networks (DNNs). In this wo…
Transformers fine-tuned on synthetic data boost tabular data classification performance.
problem Improving tabular data classification accuracy.
method Fine-tuning ICL-transformers on synthetic datasets with complex decision boundaries.
result Fine-tuned ICL-transformers outperform regular neural networks on real-world datasets.
Regurgitative training with synthetic data harms LLM performance.
problem The impact of training LLMs with synthetic data generated by other LLMs.
method Fine-tuning GPT-3.5 with synthetic and real data in a machine translation task.
result Regurgitative training significantly reduces LLM performance.
We consider the problem of compressed sensing and of (real-valued) phase retrieval with random measurement matrix. We derive sharp asymptotics for the information-theoretically optimal performance and for the best known polynomial algorithm for an ensemble of generative priors consisting of fully connected deep neural …
This study compares machine learning algorithms for predictive performance and interpretability.
problem Comparing supervised machine learning algorithms for predictive ability and model interpretation.
method Simulation studies of extreme gradient boosting machines, random forests, and feedforward neural networks on structured data.
result Feedforward neural networks generally outperform tree-based boosting algorithms in smooth models, while tree-based models perform better in non-smooth models.
The study examines how averaging data improves model performance.
problem Understanding the generalization gap in machine learning models.
method Data averaging, covariance analysis, and stochastic gradient descent (SGD) noise modeling.
result A modified generalization gap is always non-negative for a large class of model parameter distributions.
AI benchmarks evaluate football team performance using generative models.
problem Evaluating human performance in complex interactive tasks is error-prone and unreliable.
method Trained Conditional VRNN Model on player and ball tracking data to imitate and predict team interactions.
result Trained model as a useful benchmark for evaluating team performance in football.
Variational autoencoders (VAEs) are powerful generative models with the salient ability to perform inference. Here, we introduce a quantum variational autoencoder (QVAE): a VAE whose latent generative process is implemented as a quantum Boltzmann machine (QBM). We show that our model can be trained end-to-end by maximi…
Novel nonparametric method for GLMs improves prediction and inference performance.
problem Improving prediction and inference in GLMs with minimal assumptions.
method Combines binary regression and latent variable formulations, extends parametric versions, introduces new classification statistic.
result Uniformly better prediction and inference performance over parametric formulation, especially with asymmetric data.
Closed-form flow matching yields similar performance to stochastic version, improving model performance.
problem Understanding why flow matching models generalize well.
method Empirical analysis and comparison of stochastic and closed-form flow matching losses.
result Closed-form flow matching can improve model performance.
Notwithstanding the significant efforts to develop estimators of long-range correlations (LRC) and to compare their performance, no clear consensus exists on what is the best method and under which conditions. In addition, synthetic tests suggest that the performance of LRC estimators varies when using different genera…
PaDGAN generates diverse, high-quality designs with improved performance.
problem Lack of diversity and performance improvement in generated designs.
method Integrates Determinantal Point Processes for diversity and quality, using GAN framework.
result PaDGAN generates higher quality designs with better diversity and without mode collapse.
We introduce a generative adversarial network (GAN) model to simulate the 3-dimensional Lagrangian motion of particles trapped in the recirculation zone of a buoyancy-opposed flame. The GAN model comprises a stochastic recurrent neural network, serving as a generator, and a convoluted neural network, serving as a discr…
Monotonic relationship found between in-distribution and out-of-distribution performance.
problem Understanding performance of machine learning models under distribution shifts.
method Analyzing ridge-regularized models and linear inverse problems under covariate shift.
result Monotonic relationship between in-distribution and out-of-distribution performance for certain models.
Random variables of the generalized Pareto distribution, can be transformed to that of the Pareto distribution. Explicit expressions exist for the maximum likelihood estimators of the parameters of the Pareto distribution. The performance of the estimation of the shape parameter of generalized Pareto distributed using …
MRCs minimize worst-case expected 0-1 loss and provide performance guarantees.
problem Minimizing expected 0-1 loss in classification.
method Minimizes worst-case expected 0-1 loss over uncertainty sets defined by linear constraints.
result Achieves efficient learning and generalization with performance guarantees.
We propose a mathematical framework for the study of a family of random fields--called forward performances--which arise as numerical representation of certain rational preference relations in mathematical finance. Their spatial structure corresponds to that of utility functions, while the temporal one reflects a Nisio…
Deep generative models (DGMs) are effective on learning multilayered representations of complex data and performing inference of input data by exploring the generative ability. However, it is relatively insufficient to empower the discriminative ability of DGMs on making accurate predictions. This paper presents max-ma…
Confidence intervals improve evaluation of binary prediction rules in data mining.
problem Uncertainty in performance measures estimation from finite datasets.
method Asymptotic normal approximations for confidence intervals, with a blurring correction.
result Improved finite sample coverage probabilities and general performance measures inference.
The paper investigates how symmetry in models affects their performance and generalization.
problem Understanding how symmetry in models impacts their performance and generalization.
method Formal unified investigation of intuitions about symmetry in models and data.
result Quantitative bounds and comparisons between model and data equivariance lead to optimal model performance.
Empirical evidence shows that ensembles, such as bagging, boosting, random and rotation forests, generally perform better in terms of their generalization error than individual classifiers. To explain this performance, Schapire et al. (1998) developed an upper bound on the generalization error of an ensemble based on t…
Proposes DFDG for robust domain generalization without source domain labels.
problem Robustness of deep learning models in real-world applications where train and test distributions differ.
method Model-agnostic, class-aware alignment of class relationships through saliency maps.
result Competitive performance on time series sensor and image classification datasets.
Ansor generates high-performance tensor programs for deep learning.
problem Generating high-performance tensor programs for deep learning on various hardware platforms is challenging.
method Ansor uses a hierarchical representation of the search space, sampling programs, and evolutionary search with a learned cost model to find high-performance programs.
result Ansor improves deep neural network execution performance up to 3.8x on Intel CPU, 2.6x on ARM CPU, and 1.7x on NVIDIA GPU.
We train a network to generate mappings between training sets and classification policies (a 'classifier generator') by conditioning on the entire training set via an attentional mechanism. The network is directly optimized for test set performance on an training set of related tasks, which is then transferred to unsee…
Algorithms often have tunable parameters that impact performance metrics such as runtime and solution quality. For many algorithms used in practice, no parameter settings admit meaningful worst-case bounds, so the parameters are made available for the user to tune. Alternatively, parameters may be tuned implicitly with…
The paper analyzes and improves the learning rates of distributed kernel ridge regression.
problem Generalization performance and learning rates of distributed kernel ridge regression.
method The paper derives optimal learning rates for DKRR in expectation and probability, proposes a communication strategy to improve learning performance, and evaluates these through theory and experiments.
result The communication strategy significantly improves the learning performance of DKRR, as demonstrated by both theoretical assessments and numerical experiments.
This large scale study focuses on quantifying what X-rays diagnostic prediction tasks generalize well across multiple different datasets. We present evidence that the issue of generalization is not due to a shift in the images but instead a shift in the labels. We study the cross-domain performance, agreement between m…
Study analyzes neural network models to understand generalization performance.
problem Understanding good generalization in neural networks.
method Analyzed a corpus of models from a public contest, breaking ALPHAHAT into scale and shape metrics.
result Identified a Simpson's paradox in metric performance across different model depths and regularization hyperparameters.
Paper tackles model collapse in recursive generative models using a weighted training scheme.
problem Model collapse in recursive generative models trained on synthetic data.
method Iteratively trains models on real and synthetic data, evaluates weighted training schemes.
result Optimal weighting scheme for synthetic data follows a unified expression, revealing a trade-off with model performance.
Estimates model performance under distribution shift using domain-invariant predictors.
problem Poor performance of models on test distributions different from training distributions.
method Uses domain-invariant predictors as a proxy for unknown target labels.
result Shows that the complexity of latent representations influences target risk.
Partially performative prediction studies how predictive models influence future data.
problem Distribution shift in predictive models due to endogenous and exogenous factors.
method Generalizing performative prediction to capture both endogenous and exogenous sources of distribution shift.
result Developed online analogues of performative stability and optimality for partially performative environments.
The goal of this study is to determine which strategic model, either IO or RBV, allows firms to generate the highest performance on a competitive market. Contrasting with classical studies that mobilize analyses as VARCOMP, we deploy a multi-agent system simulating the behavior of firms adopting RBV or IO strategic mod…
Equivariant neural networks improve performance and generalization in complex scalar field theory tasks.
problem Improving performance and generalization in neural networks for complex scalar field theory tasks.
method Incorporating translational equivariance into neural network architectures.
result Equivariant neural networks significantly outperform non-equivariant networks in various tasks, including those beyond the training set and across different lattice sizes.
Is cognition a collection of loosely connected functions tuned to different tasks, or can there be a general learning algorithm? If such an hypothetical general algorithm did exist, tuned to our world, could it adapt seamlessly to a world with different laws of nature? We consider the theory that predictive coding is s…