ICLR 2021 challenge in computational geometry and topology attracted 16 teams.
problem Designing and evaluating computational methods in differential geometry and topology.
method Designing and hosting an open-source competition with repositories Geomstats and Giotto-TDA.
result 16 teams participated in the challenge, showcasing innovative contributions to computational geometry and topology.
Hard to estimate L2-accurate scores without strong assumptions.
problem Estimating the score of unknown data distributions accurately.
method Reduction to generating samples and leveraging lattice-based cryptography hardness.
result Score estimation is computationally hard even with polynomial sample complexity.
SAM optimizer benefits from normalization, stabilizing and guiding optimization.
problem Improving deep neural network performance with SAM optimizer.
method Theoretical and empirical study of normalization in SAM for convex and non-convex functions.
result Normalization helps SAM in stabilizing and guiding optimization along a continuum of minima.
Improved neural network training by coupled initialization reduces neuron count.
problem Training neural networks efficiently with fewer neurons.
method Coupled initialization of weights into pairs of identical Gaussian vectors.
result Significantly reduced number of neurons required for network convergence.
New adaptive attacks bypass many defenses to adversarial examples.
problem Adversarial example defenses are not adequately evaluated using adaptive attacks.
method Detailed analysis of thirteen defenses, demonstrating their vulnerabilities to adaptive attacks.
result Adversarial example defenses are more vulnerable to adaptive attacks than previously thought.
This work is a part of ICLR Reproducibility Challenge 2019, we try to reproduce the results in the conference submission PADAM: Closing The Generalization Gap of Adaptive Gradient Methods In Training Deep Neural Networks. Adaptive gradient methods proposed in past demonstrate a degraded generalization performance than …
Improved DP KDE with better privacy and efficiency.
problem Kernel density estimation with differential privacy.
method Refined search tree construction for DP KDE.
result Improved query time and approximation ratio.
New research shows many recent defenses against adversarial examples are ineffective against black-box attacks.
problem The robustness of recent defenses against adversarial examples is insufficient, especially against black-box attacks.
method Evaluation of nine defenses on two black-box adversarial models and six attacks on CIFAR-10 and Fashion-MNIST datasets.
result Most recent defenses provide only marginal improvements in security (<25%) compared to undefended networks. Biotech IPOs in Q1 2021: advanced degrees, clinical trials, and IP key.
problem Identifying traits of biotech startups that go public.
method Database of biotech IPOs, analysis of leadership, technology, clinical trials, and financing.
result Advanced degrees, clinical trials, and IP are important for biotech startups.
New analysis shows ROI's predictive power for stock returns weakens significantly.
problem The predictive power of retail order imbalance (ROI) for future stock returns.
method Replicated Boehmer et al. (2021) using a more recent period and analyzed the effect of using alternative quote midpoint (QMP) method.
result Past ROI can no longer predict weekly returns on large-cap stocks, and the long-short strategy based on past ROI is no longer profitable.
Paper bridges score estimation to parameter and density estimation in DDPMs.
problem Efficiently estimating scores for generative models.
method Introduces a framework linking score estimation to parameter and density estimation.
result Denoising score-matching in DDPMs is asymptotically efficient for parameter estimation.
TabPFN learns to approximate functions on tabular data.
problem TabPFN tackles function approximation on tabular data.
method Treated as a black-box function approximator generator, observed behavior on varied datasets.
result Observed behavior that is both brilliant and baffling.
Rebellion Research's AI strategy outperformed the S&P 500 for 14 years.
problem Outperforming the S&P 500 for 14 years with AI.
method AI Global Equity strategy focused on machine learning.
result AI strategy has outperformed the S&P 500 for 14 years.
DCASE 2021 ASD task tackles domain-shifted anomalous sound detection.
problem Detecting unknown anomalous sounds under domain-shifted conditions.
method Ensemble of outlier exposure and inlier modeling detectors, feature learning from machine identification.
result Two types of remarkable approaches were adopted by top teams.
Although variational autoencoders (VAEs) represent a widely influential deep generative model, many aspects of the underlying energy function remain poorly understood. In particular, it is commonly believed that Gaussian encoder/decoder assumptions reduce the effectiveness of VAEs in generating realistic samples. In th…
New model selects robustly in adversarial reinforcement learning with unknown corruption.
problem Adversarial corruption in reinforcement learning with unknown total corruption amount.
method Model selection approach for finite-horizon tabular and linear MDPs.
result First worst-case optimal bound without knowledge of total corruption.
We develop a family of techniques to align word embeddings which are derived from different source datasets or created using different mechanisms (e.g., GloVe or word2vec). Our methods are simple and have a closed form to optimally rotate, translate, and scale to minimize root mean squared errors or maximize the averag…
The workshop focuses on AI principles for structured data.
problem Using AI on structured data for decision-making.
method Addressing principles of privacy, accountability, interpretability, robustness, and reasoning.
result Designing approaches to use structured data for reliable decisions.
Improved MMD test for non-Euclidean data with spectral regularization.
problem Inefficient and impractical MMD goodness-of-fit tests for non-Euclidean data.
method Spectral regularization of MMD test, extending results to general cases.
result Minimax optimal test for non-Euclidean data with appropriate regularization.
Study reveals bias in machine learning conference reviews.
problem Bias in machine learning conference review process.
method Comprehensive analysis of ICLR papers from 2017-2020.
result Strong institutional bias in accept/reject decisions.
Paper proposes a hybrid loss function for graph self-supervised learning.
problem Improving self-supervised representation learning for graphs.
method Hybrid loss function combining VICReg and HSIC.
result Hybrid loss function outperformed other methods in 4 out of 7 cases.
The paper introduces explainable k-means with axis-parallel hyperplanes for d-dimensional data.
problem Creating explainable clustering with axis-parallel hyperplanes for complex data.
method An efficient algorithm that finds an explainable clustering with a near-optimal cost function.
result The algorithm achieves a near-optimal k-means cost of k1−2/dpolylog(k) for d-dimensional data. WSB's investment advice significantly outperformed the S&P500 over 3 years, but not consistently.
problem Reliability of investment advice from WSB community.
method Data analysis of WSB posts and stock performance from 2019-2021.
result A WSB portfolio grew 200% over 3 years and 480% over 1 year, outperforming S&P500.
We improve adversarial robustness calibration analysis for broader hypothesis sets.
problem Improving calibration for adversarial robustness in machine learning.
method A finer definition of calibration for adversarial robustness.
result Our results cover most common hypothesis sets in machine learning.
Minimalist softmax attention learns constrained Boolean functions with supervision.
problem Learning k-bit Boolean functions, especially AND and OR. method Minimalist single-head softmax-attention mechanism with teacher forcing.
result Minimalist attention can solve Boolean tasks with supervision, but not without.
Paper uses SSC for identifying layers with identical community structures in DIMPLE networks.
problem Identifying layers with identical community structures in DIMPLE networks.
method Sparse Subspace Clustering (SSC) for identifying groups of layers with identical community structures.
result SSC leads to strongly consistent between-layer clustering under mild conditions.
Study proves fluid limits of fragmented limit-order markets.
problem Modeling fragmented limit-order markets with small and frequent orders.
method Proved convergence of discrete system to fluid limit characterized by coupled nonlinear ODEs.
result Fluid system converges to stationary equilibrium state over time.
Research characterizes intersection cohomology groups of gauge theories and cotangent bundles.
problem Characterizing intersection cohomology groups of Coulomb branch gauge theories.
method Uses geometric Satake correspondence for Kac-Moody settings.
result Sketches proof of conjecture in affine type A.
Training neural networks is hard in fixed dimensions.
problem Training two-layer neural networks is computationally hard in fixed dimensions.
method Parameterized complexity analysis considering dimension and number of neurons.
result Training two-layer neural networks is NP-hard for two dimensions.
Recent works have cast some light on the mystery of why deep nets fit any data and generalize despite being very overparametrized. This paper analyzes training and generalization for a simple 2-layer ReLU net with random initialization, and provides the following improvements over recent works: (i) Using a tighter char…
Improved approximation for socially fair clustering with ℓp-objective.
problem Finding a set of centers minimizing the maximum distance to all points in each group.
method Introduced a strengthened LP relaxation with an integrality gap of Θ(loglogℓlogℓ). result Improved approximation algorithm with (eO(p)loglogℓlogℓ)-approximation. Paper revisits HVA to address model risk in banking.
problem Model risk and dynamic hedging frictions in banking.
method Reconciles global fair valuation with local bank models.
result Local models should be excluded rather than managed via reserves.
Open problems in billiards and optics from a workshop.
problem Open problems in billiards and geometric optics.
method Collection of open problems from a workshop.
result No specific key result mentioned; collection of problems.
The paper proves rigidity theorems on holomorphic isometries into homogeneous domains.
problem Characterizing and comparing holomorphic isometries into homogeneous bounded domains.
method Two rigidity theorems on holomorphic isometries into homogeneous bounded domains.
result The flat (definite or indefinite) complex Euclidean space is not a relative of a homogeneous bounded domain.
DAIS improves AIS by resampling, avoiding gradient issues.
problem Low effective sample size in DAIS.
method DAIS with resampling step to improve efficiency.
result Resampling step avoids gradient variance issues.
Depth uncertainty networks don't improve with bias correction, contrary to expectations.
problem Improving performance in active learning with overparameterised models like NNs.
method Depth uncertainty networks, compared to underparameterised models, show no improvement in performance with bias correction.
result Depth uncertainty networks do not improve with bias correction, unlike underparameterised models.
Improved UDA framework using f-divergence measures.
problem Addressing distribution shifts in machine learning.
method Refined f-divergence-based discrepancy and f-domain discrepancy. result Novel target error and sample complexity bounds.
We consider peer review in a conference setting where there is typically an overlap between the set of reviewers and the set of authors. This overlap can incentivize strategic reviews to influence the final ranking of one's own papers. In this work, we address this problem through the lens of social choice, and present…
Gauge theory for families helps compare 4-manifold groups.
problem Comparing diffeomorphism and homeomorphism groups of 4-manifolds.
method Gauge theory applied to families of 4-manifolds.
result Improved understanding of group comparisons in 4-manifolds.
New DEC variant improves sample complexity bounds in decision making.
problem Understanding sample-efficient learning guarantees in decision making.
method Introducing a new Constrained Decision-Estimation Coefficient (DEC) and using it to derive improved lower bounds.
result New lower bounds improve upon prior work in three aspects: expectation, global applicability, and improper reference models.
DAIS minimizes symmetrized KL divergence between initial and target distributions.
problem Optimizing over initial distributions in importance sampling.
method Differentiable annealed importance sampling (DAIS) minimizing symmetrized KL divergence.
result DAIS minimizes symmetrized KL divergence between initial and target distributions.
Survey of gauge theory for families of 4-manifolds.
problem Understanding gauge theory for families of 4-manifolds.
method Developed gauge theory for diffeomorphism groups of 4-manifolds.
result New applications of gauge theory to diffeomorphism groups.
AI agents beat previous best on NetHack, but symbolic bots still outperform.
problem Developing AI agents that can ascend in the NetHack game.
method Used a procedurally generated NetHack Learning Environment for reinforcement learning.
result Symbolic bots outperform neural approaches on NetHack.
Study reveals clusters of resilient and vulnerable Spanish agri-food firms post-Ukraine-Russia war.
problem Financial resilience of agri-food companies in Spain during the Ukraine-Russia conflict.
method Cluster analysis using centred log-ratios for compositional data of financial ratios.
result Increase in resilient firms by 2023, highlighting sectoral adaptation to economic challenges.
Paper introduces kernel deformed exponential families for sparse continuous attention.
problem Creating efficient attention mechanisms for sparse data.
method Developed kernel deformed exponential families, theoretically and experimentally.
result Kernel deformed exponential families can attend to multiple compact regions of data.
We prove that the binary classifiers of bit strings generated by random wide deep neural networks with ReLU activation function are biased towards simple functions. The simplicity is captured by the following two properties. For any given input bit string, the average Hamming distance of the closest input bit string wi…
Winterization of Texas power system profitable but risky, estimated at $11.74bn over 30 years.
problem Profitability and risk of winterizing Texas power system infrastructure.
method Combined temperature-dependent load and outage estimates over 71 years of climate data.
result Large-scale winterization of gas infrastructure and power plants is profitable, but risks are high due to low-frequency of cold spells.
Cohen et al. (2021) show GD trajectories align on a bifurcation diagram.
problem Understanding the Edge of Stability (EoS) phenomenon in gradient descent.
method Empirical studies and rigorous mathematical proofs for two-layer networks and single-neuron networks.
result GD trajectories align on a specific bifurcation diagram independent of initialization.