arXiv research

ICLR 2021 challenge in computational geometry and topology attracted 16 teams.

New adaptive attacks bypass many defenses to adversarial examples.

This work is a part of ICLR Reproducibility Challenge 2019, we try to reproduce the results in the conference submission PADAM: Closing The Generalization Gap of Adaptive Gradient Methods In Training Deep Neural Networks. Adaptive gradient methods proposed in past demonstrate a degraded generalization performance than …

Improved DP KDE with better privacy and efficiency.

New research shows many recent defenses against adversarial examples are ineffective against black-box attacks.

Paper bridges score estimation to parameter and density estimation in DDPMs.

TabPFN learns to approximate functions on tabular data.

Hard to estimate $L^2$-accurate scores without strong assumptions.

Although variational autoencoders (VAEs) represent a widely influential deep generative model, many aspects of the underlying energy function remain poorly understood. In particular, it is commonly believed that Gaussian encoder/decoder assumptions reduce the effectiveness of VAEs in generating realistic samples. In th…

We develop a family of techniques to align word embeddings which are derived from different source datasets or created using different mechanisms (e.g., GloVe or word2vec). Our methods are simple and have a closed form to optimally rotate, translate, and scale to minimize root mean squared errors or maximize the averag…

The workshop focuses on AI principles for structured data.

Study reveals bias in machine learning conference reviews.

Minimalist softmax attention learns constrained Boolean functions with supervision.

SAM optimizer benefits from normalization, stabilizing and guiding optimization.

Training neural networks is hard in fixed dimensions.

Recent works have cast some light on the mystery of why deep nets fit any data and generalize despite being very overparametrized. This paper analyzes training and generalization for a simple 2-layer ReLU net with random initialization, and provides the following improvements over recent works: (i) Using a tighter char…

We consider peer review in a conference setting where there is typically an overlap between the set of reviewers and the set of authors. This overlap can incentivize strategic reviews to influence the final ranking of one's own papers. In this work, we address this problem through the lens of social choice, and present…

We prove that the binary classifiers of bit strings generated by random wide deep neural networks with ReLU activation function are biased towards simple functions. The simplicity is captured by the following two properties. For any given input bit string, the average Hamming distance of the closest input bit string wi…

Improved neural network training by coupled initialization reduces neuron count.

Understanding and characterizing the subspaces of adversarial examples aid in studying the robustness of deep neural networks (DNNs) to adversarial perturbations. Very recently, Ma et al. (ICLR 2018) proposed to use local intrinsic dimensionality (LID) in layer-wise hidden representations of DNNs to study adversarial s…

A modular framework for knowledge distillation simplifies experiments and reproducibility.

New algorithm finds optimal policy with polynomial trajectories in deterministic systems.

A new method estimates Bayes error for deep networks, suggesting they may have reached the limit.

Generative models converge to data distribution but not principal latent factors.

New metric $m$-coherence measures gradient alignment during training, revealing surprising memorization patterns.

New method improves Bayesian optimization by considering budget and prior quality.

New method for faster convergence in non-convex optimization with unbounded smoothness.

Improved sampling from non-log-concave distributions with polynomial query complexity.