Nonlinear MCMC improves Bayesian machine learning sampling.
problem Sampling problems in Bayesian machine learning.
method Nonlinear MCMC technique with convergence guarantees.
result Improves sampling in Bayesian neural networks.
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,742 papers · 148 categories
Trend · papers per month
48 results for “CIFAR10”
DCTN uses tensor networks for image classification, achieving state-of-the-art results.
problem Improving image classification accuracy with deep neural networks.
method Developed a novel deep convolutional tensor network (DCTN) based on Entangled plaquette states (EPS).
result DCTN achieves state-of-the-art results on MNIST and FashionMNIST but overfits on CIFAR10.
Headless attacks bypass classification heads to fool transfer learning models.
problem Adversarial attacks against transfer learning models without access to the classification head.
method Label-blind adversarial attacks that do not require class-label information.
result Transfer attack lowers ResNet18 accuracy on CIFAR10 by over 40%.
RATIO improves neural network robustness and explainability.
problem Neural networks' lack of robustness to adversarial changes and uncertainty on out-distribution samples.
method RATIO: Adversarial Training on In- and Out-distribution.
result RATIO leads to robust models with reliable confidence estimates on out-distribution samples.
We propose a novel autoencoding model called Pairwise Augmented GANs. We train a generator and an encoder jointly and in an adversarial manner. The generator network learns to sample realistic objects. In turn, the encoder network at the same time is trained to map the true data distribution to the prior in latent spac…
New method defends against both single-step and iterative adversarial examples.
problem Defending against adversarial examples, especially iterative ones, is computationally expensive.
method Single-Step Adversarial Training (SST) with modifications.
result Our method outperforms state-of-the-art methods in both accuracy and training time.
Distributed learning is central for large-scale training of deep-learning models. However, they are exposed to a security threat in which Byzantine participants can interrupt or control the learning process. Previous attack models and their corresponding defenses assume that the rogue participants are (a) omniscient (k…
Plug-and-play multimodal controller improves class-conditional image generation.
problem Generating class-conditional images from user-specified labels.
method Introduces a `multimodal controller` to generate multimodal data without additional learning parameters.
result Multimodal controlled generative models produce higher quality class-conditional images and novel modalities.
Improves sampling speed of ARM models without losing accuracy.
problem Slow sampling from ARMs.
method Predictive sampling algorithm using ARM fixed-point iteration and learned forecasting modules.
result Significant improvements in sampling speed and efficiency.
New method makes neural networks more resilient to location-optimized adversarial patches.
problem Neural networks' vulnerability to adversarial patches that are visible but still effective.
method Developed a practical approach to optimize patch locations and applied adversarial training.
result Significantly improved robustness against adversarial patches on CIFAR10 and GTSRB.
Single model predicts multiple subnetworks for robustness.
problem Efficiently ensemble neural networks for robustness and uncertainty.
method Multi-input multi-output (MIMO) configuration to train independent subnetworks.
result Significant improvement in negative log-likelihood, accuracy, and calibration error.
HMQ improves quantization for edge devices with mixed precision.
problem Efficient quantization for edge devices with uniform, power-of-two thresholds.
method Introduces HMQ, a mixed precision quantization block that repurposes Gumbel-Softmax for searching over quantization schemes.
result Achieves competitive and state-of-the-art results on ImageNet despite restrictions.
Unify gradients to improve deep networks' robustness against black-box attacks.
problem Defending against score-based query attacks (SQAs) in deep neural networks.
method Unifying Gradients (UniG) to provide universal attack perturbations.
result Significantly improves real-world robustness of deep networks without sacrificing clean accuracy.
This paper introduces methods to handle discrete data by dequantization.
problem Handling discrete data in deep learning models.
method Dequantization framework, including importance-weighted and Rényi dequantization objectives, and autoregressive dequantization.
result Improved performance on uniform dequantization distributions and state-of-the-art negative log-likelihood on CIFAR10.
DeformRS certifies deep networks against various input deformations.
problem Vulnerability of deep networks to input deformations.
method Randomized smoothing reformulation for general deformations.
result Certifies rich deformations including translations, rotations, scaling, and affine.
ResNet models overfit benignly on Cifar10 but not on ImageNet due to label noise.
problem Understanding why benign overfitting fails in real-world classification tasks with label noise.
method Theoretical analysis of benign overfitting under a mild overparameterization setup.
result Benign overfitting can fail in the presence of label noise, unlike in heavy overparameterization settings.
Study shows dataset properties impact adversarial machine learning robustness.
problem Vulnerability of DNNs to adversarial attacks.
method Examined five datasets, analyzed input size and contrast effects.
result Input size and contrast significantly influence adversarial success.
DP-RandP improves privacy-utility tradeoff in DP-SGD by learning priors from random processes.
problem Improving the performance of differentially private stochastic gradient descent (DP-SGD) on private data.
method A three-phase approach that learns priors from images generated by random processes and transfers these priors to private data.
result New state-of-the-art accuracy on CIFAR10, CIFAR100, MedMNIST, and ImageNet for various privacy budgets.
BN helps learn fragile features, which can improve adversarial robustness.
problem The role of batch normalization in adversarial training and its impact on robustness.
method Investigated the expressiveness of BN in learning robust features compared to random features.
result Adversarially fine-tuning BN layers can result in non-trivial adversarial robustness.
01 loss neural network improves adversarial robustness on CIFAR10.
problem Improving machine learning models' resistance to adversarial attacks.
method Proposes a single hidden layer 01 loss neural network trained with stochastic coordinate descent.
result 01 loss network has the largest adversarial distortion compared to other models.
A new method uses adversarial attacks to detect other adversarial attacks.
problem Detecting iterative adversarial attacks on deep neural networks.
method Using the Carlini-Wagner (CW) attack as a detector itself, under certain assumptions.
result The method provides asymptotically optimal separation of original and attacked images.
MSLG generates soft labels to improve DNN performance on noisy datasets.
problem Significant performance degradation of DNNs due to noisy labels.
method Meta-learning techniques to estimate optimal label distribution and iteratively update soft labels.
result MSLG outperforms state-of-the-art methods by a large margin on various datasets.
New method AnInfoNCE uncovers latent factors in contrastive learning with practical variability.
problem Theoretical assumptions of contrastive learning loss overlook practical variability in positive pairs.
method AnInfoNCE, a generalization of InfoNCE, models anisotropic variability to uncover latent factors.
result AnInfoNCE increases recovery of latent factors in CIFAR10 and ImageNet, albeit at the cost of accuracy.
Dynamic pruning during training reduces deep network complexity without significant accuracy loss.
problem High memory and computational requirements of deep networks during training and inference.
method Dynamic pruning of convolutional filters during training, using L1 normalization for optimization.
result L1 normalization-based pruning yields up to 50% reduction in filters with minimal accuracy loss.
Tricks adversarial attacks to target specific classes, improving classifier accuracy.
problem Recent adversarial defense approaches have failed to protect classifiers from untargeted attacks.
method Target Training defense tricks untargeted attacks into targeted attacks on designated classes, then derives the real class.
result 86.2% accuracy for CW-L2 (confidence=0) in CIFAR10, outperforming unsecured classifiers.
Neural networks learn patterns in random data, improving downstream performance.
problem Understanding what deep networks learn with random labels.
method Analytical and empirical study of convolutional and fully connected networks pre-trained on random labels.
result Pre-trained networks on random labels transfer faster to real datasets, despite specialization effects.
DKGM improves image quality by debiasing kernel-based models.
problem Improving image quality in generative models without losing detail.
method Two-stage framework: KDE for density estimation, debiasing algorithm inspired by stochastic approximation.
result DKGM achieves comparable image quality to state-of-the-art models on CIFAR10.
Input-dependent smoothing mitigates classical issues but suffers from the curse of dimensionality.
problem Certifiably robust classifiers with input-dependent smoothing suffer from the curse of dimensionality.
method Proposed a theoretical and practical framework for input-dependent smoothing under strict restrictions.
result Input-dependent smoothing mitigates some classical issues but is limited by the curse of dimensionality.
Random deep neural networks are robust to adversarial examples, scaling with input size and dimension.
problem Adversarial examples challenge the reliability of deep learning algorithms.
method Analysis of random deep neural networks with Gaussian process equivalence and experiments on MNIST and CIFAR10.
result The distance of adversarial examples scales as norm of the input.
Deep learning architectures have demonstrated state-of-the-art performance for object classification and have become ubiquitous in commercial products. These methods are often applied without understanding (a) the difficulty of a classification task given the input data, and (b) how a specific deep learning architectur…
Neural nets learn simple distributions first, then more complex ones.
problem Understanding how neural networks generalize from simple to complex functions.
method Stochastic gradient descent training, synthetic data, CIFAR10, ImageNet pre-training.
result Neural networks initially use lower-order statistics, then higher-order ones.
New linear flows using exponential of linear transformations improve generative models.
problem Improving generative models in machine learning.
method Developed convolution exponentials and generalized Sylvester Flows using the exponential of linear transformations.
result Convolution exponentials and Convolutional Sylvester Flows outperform other models in log-likelihood.
New method makes deep nets robust to large perturbations.
problem Making deep nets robust to large perturbations.
method Simple weight initialization and rigorous evaluation on MNIST and CIFAR10.
result Learning on higher levels of noise is feasible with new initialization.
A semi-supervised learning method using predefined class centroids for image classification.
problem Reducing the need for labeled data in deep learning.
method Use a small number of labeled samples and data augmentation on unlabeled samples. Constrain all samples to predefined evenly-distributed class centroids (PEDCC) using loss functions.
result Achieves state-of-the-art results with minimal labeled data.
Study on dynamics of non-linear autoencoders learning principal components.
problem Technical difficulty in studying non-linear autoencoders due to non-trivial correlations.
method Derive asymptotically exact equations for SGD training of shallow, non-linear autoencoders.
result Autoencoders learn principal components sequentially and tie weights are ineffective.
Improved stochastic gradient estimation for deep learning in high dimensions.
problem Inadmissibility of mini-batch gradients in high-dimensional settings.
method Stein-rule shrinkage applied to gradient computation.
result The proposed SR-Adam outperforms Adam in large-batch settings.
We investigate under and overfitting in Generative Adversarial Networks (GANs), using discriminators unseen by the generator to measure generalization. We find that the model capacity of the discriminator has a significant effect on the generator's model quality, and that the generator's poor performance coincides with…
New margin-based regularization and selective sampling improve deep neural network performance.
problem Improving deep neural network performance on various classification tasks.
method Multi-margin regularization (MMR) and minimal margin score (MMS) for selective sampling.
result Improved results on multiple classification tasks across domains.
We propose an approach to address two issues that commonly occur during training of unsupervised GANs. First, since GANs use only a continuous latent distribution to embed multiple classes or clusters of data, they often do not correctly handle the structural discontinuity between disparate classes in a latent space. S…
We propose the labeled Čech complex, the plain labeled Vietoris-Rips complex, and the locally scaled labeled Vietoris-Rips complex to perform persistent homology inference of decision boundaries in classification tasks. We provide theoretical conditions and analysis for recovering the homology of a decision boundary fr…
Triplet networks are widely used models that are characterized by good performance in classification and retrieval tasks. In this work we propose to train a triplet network by putting it as the discriminator in Generative Adversarial Nets (GANs). We make use of the good capability of representation learning of the disc…
The paper studies stability of generative models trained on mixed data.
problem Training generative models on mixed datasets (real and synthetic data).
method Developed a framework to rigorously study stability under specific conditions.
result Proved the stability of iterative training under certain conditions.
Tensorial Neural Networks: Generalization of Neural Networks and Application to Model Compressionstat.ML
We propose tensorial neural networks (TNNs), a generalization of existing neural networks by extending tensor operations on low order operands to those on high order ones. The problem of parameter learning is challenging, as it corresponds to hierarchical nonlinear tensor decomposition. We propose to solve the learning…
New method lures adversaries to choose false directions in black-box attacks.
problem Improving robustness against black-box transfer attacks.
method Incorporates a removable neural network to induce a luring effect.
result Demonstrates effectiveness in thwarting state-of-the-art attacks.
DSDE: Using Proportion Estimation to Improve Model Selection for Out-of-Distribution Detectionstat.ML
DSDE improves OoD detection by estimating model library proportions.
problem Uncertainty quantification and balanced error rates in model selection for OoD detection.
method Inverted sequential p-value strategies, change-point detection, automatic hyperparameter selection.
result DSDE reduces FPR from 11.07% to 3.31% on CIFAR10.
It is becoming increasingly clear that many machine learning classifiers are vulnerable to adversarial examples. In attempting to explain the origin of adversarial examples, previous studies have typically focused on the fact that neural networks operate on high dimensional data, they overfit, or they are too linear. H…
Data selection methods, such as active learning and core-set selection, are useful tools for machine learning on large datasets. However, they can be prohibitively expensive to apply in deep learning because they depend on feature representations that need to be learned. In this work, we show that we can greatly improv…
Theory explains creativity in diffusion models generating novel images.
problem Diffusion models generate highly original images far from training data.
method Identified locality and equivariance as inductive biases to prevent optimal score-matching.
result Analytic models predict diffusion model outputs with high accuracy.