Study explores various methods for detecting offensive language.
problem Detecting offensive language in text.
method Examined custom and off-the-shelf architectures, feature vector generation approaches.
result These methods perform well on small, noisy datasets.
A new framework learns system design using neural features in function space.
problem Learning system design with neural feature extractors.
method Introduces feature geometry in function space, nesting technique for optimal feature approximation.
result Optimal features found from data samples using off-the-shelf architectures and optimizers.
Study compares LSTM and ANN architectures for forex prediction, finding ANN more efficient.
problem Forex time series prediction efficiency and resource usage.
method Comparison of Long Short-Term Memory (LSTM) and specialized ANN architectures.
result Specialized ANN architecture performs better with fewer resources and faster execution.
EASIER-net uses sparse networks to improve prediction accuracy for high-dimensional data.
problem Limited use of neural networks in high-dimensional data with small samples.
method Ensemble by Averaging Sparse-Input Hierarchical networks (EASIER-net) with small modifications to neural network architecture and training procedure.
result EASIER-net achieves higher prediction accuracy than off-the-shelf methods on average.
This study evaluates different normalizing flow architectures for MCMC.
problem Lack of systematic comparison of normalizing flow architectures in MCMC.
method Extensive evaluation of various normalizing flow architectures on different MCMC methods and target distributions.
result Contractive residual flows are the best general-purpose models for MCMC.
DAC combines actor-critic methods for learning options.
problem Learning hierarchical policies in reinforcement learning.
method Double Actor-Critic (DAC) architecture using two parallel augmented MDPs.
result DAC outperforms existing methods in robot simulation tasks.
New method trains any neural network as a generative model.
problem Constrained design of normalizing flows due to analytical invertibility.
method Efficient gradient estimator for non-analytically invertible networks.
result Any dimension-preserving neural network can be used as a generative model.
WideBNet learns inverse scattering from wide-band data efficiently and stably.
problem Learning the inverse scattering map from wide-band scattering data.
method Combines butterfly factorization, FFT, and deep learning.
result WideBNet requires fewer training points and has stable training dynamics.
This work uses image generation models to find vision model bugs.
problem Automatically discovering failures in vision models.
method Conditional text-to-image generation and captioning models.
result Demonstrated utility of large-scale generative models to find vision model bugs.
JD.com uses a new CNN model to improve ad click prediction.
problem Improving CTR prediction for ads with visual content.
method Proposes Category-specific CNN (CSCNN) to incorporate category knowledge early in the feature extraction process.
result CSCNN outperforms existing methods in CTR prediction.
Labels distilled from images improve model training efficiency and flexibility.
problem Creating synthetic labels for a small set of real images to train models effectively.
method Introduce a more robust and flexible meta-learning algorithm for distillation and an effective first-order strategy based on convex optimization layers.
result Label distillation leads to improved results and greater flexibility in neural architectures.
We present a generative framework for generalized zero-shot learning where the training and test classes are not necessarily disjoint. Built upon a variational autoencoder based architecture, consisting of a probabilistic encoder and a probabilistic conditional decoder, our model can generate novel exemplars from seen/…
Paper introduces MSP Network for harmonizing images from different scanners.
problem Improving predictive performance and learning efficiency in data harmonization.
method Multi-Stage Prediction (MSP) Network integrating neural networks of different architectures.
result MSP Network shows 20% improvement in patch-based mean-squared error over state-of-the-art methods.
New method uses off-the-shelf classifiers to improve diffusion generation without extra training.
problem Improving sample quality and controllability in conditional diffusion generation.
method Leveraging off-the-shelf classifiers in a training-free fashion with calibration and pre-conditioning techniques.
result Significant performance improvements (up to 20%) over existing guidance schemes.
Checkmate optimizes DNN training time and memory usage.
problem Optimizing DNN training time and memory usage.
method Formalizes tensor rematerialization as an optimization problem, uses MILP solvers or approximation algorithms to find optimal schedules, and accelerates training with hardware-aware cost models.
result Reduces training cost and enables training with larger input sizes.
Generative adversarial networks benefit from optimal input dimension and adaptive generator architecture.
problem Minimizing generalization error in GANs through optimal input dimension.
method Introducing generalized GANs (G-GANs) with group penalty and architecture penalty for adaptive dimensionality reduction and network architecture identification.
result G-GANs achieve superior performance with 40%+ improvements in maximum mean discrepancy or Frechet inception distance compared to off-the-shelf methods.
SymDiff uses stochastic symmetrisation for equivariant diffusion models.
problem Constructing equivariant diffusion models for data augmentation.
method Stochastic symmetrisation for lightweight, efficient, and easy-to-implement equivariance.
result SymDiff achieves significant empirical benefit for E(3)-equivariant molecular generation. The paper proposes a method to incorporate prior knowledge into generative models using a regularizer.
problem Learning generative models with prior domain knowledge is costly and often impractical.
method Formulate a regularizer that makes generative model marginals follow prescribed feature relations.
result The method effectively incorporates prior knowledge into generative models without requiring specific model architectures.
BREEDS benchmarks assess model robustness to subpopulation shifts.
problem Measuring model robustness to novel subpopulation shifts.
method Controlled synthesis of realistic distribution shifts using class structure.
result Validated model sensitivity and effectiveness of robustness interventions.
Machine learning improves communications design without deep expertise.
problem Design high-performance communication systems with limited domain knowledge.
method Apply machine learning techniques to two communication problems.
result Deep learning discovered a simple and effective strategy for system design.
State-of-the-art speaker diarization systems utilize knowledge from external data, in the form of a pre-trained distance metric, to effectively determine relative speaker identities to unseen data. However, much of recent focus has been on choosing the appropriate feature extractor, ranging from pre-trained i−vectors…
A new method recovers rewards from behavior policies using classification and regression.
problem Recovering meaningful rewards from observed behavior in reinforcement learning.
method GenPQR, a modular procedure that estimates behavior policy, evaluates soft Q-function, and recovers normalized reward using classification and regression.
result GenPQR matches or improves reward recovery compared to DeepPQR, while being simpler and more modular.
BERT-XML automates ICD coding from EHR notes using BERT pretraining.
problem Manual ICD coding is time-consuming and expensive in hospitals.
method Trained a BERT model on EHR notes, adapting it for multi-label attention.
result First large-scale ICD-10 classification model using millions of EHR notes.
Modern neural networks are highly overparameterized, with capacity to substantially overfit to training data. Nevertheless, these networks often generalize well in practice. It has also been observed that trained networks can often be "compressed" to much smaller representations. The purpose of this paper is to connect…
New insights into attention mechanisms reveal dramatic trade-offs between rank and heads.
problem Dramatic trade-offs between rank and number of heads in attention mechanisms.
method Presented a simple target function and proved theoretical limits.
result Full-rank attention is necessary for long contexts, while low-rank is sufficient for short ones.
SS-GEN simulates rare events in heavy and light-tailed data.
problem Estimating probabilities of extreme events in multivariate data.
method Self-Similar Generative Estimation (SS-GEN) decomposes tail distribution into radial and angular components.
result SS-GEN generates representative extreme scenarios and estimates rare-event probabilities beyond observed data.
We characterize a prevalent weakness of deep neural networks (DNNs)---overthinking---which occurs when a DNN can reach correct predictions before its final layer. Overthinking is computationally wasteful, and it can also be destructive when, by the final layer, a correct prediction changes into a misclassification. Und…
Dynamic residual adapters improve performance across multiple latent domains without domain labels.
problem Overfitting to large domains and ignoring smaller ones in multi-domain learning.
method Dynamic residual adapters and augmentation strategies inspired by style transfer.
result Dynamic residual adapters significantly outperform standard models on multiple latent domains.
Defines metrics to compare neural network representations.
problem Comparing neural network representations across different architectures and tasks.
method Developed a family of metric spaces and modified existing measures to quantify representational dissimilarity.
result Identified relationships between neural representations and anatomical features.
New L1-Coverage objective simplifies exploration in reinforcement learning.
problem Challenges in exploration for high-dimensional domains.
method Introduces L1-Coverage objective to enable efficient exploration and planning. result First computationally efficient algorithms for online reinforcement learning with low coverability.
Benchopt automates machine learning benchmarking across languages and hardware.
problem Limited transparency and tedious re-implementation work in machine learning validation.
method A collaborative framework for automating, reproducing, and publishing optimization benchmarks.
result Demonstrates practical findings that highlight the importance of details in machine learning validation.
Improves ASR accuracy for domain mismatch using machine translation.
problem Domain mismatch in ASR systems leads to suboptimal results.
method Machine translation to map out-of-domain ASR errors to in-domain terms.
result 7% absolute improvement in word error rate, 4 point BLEU score improvement.
CoCoPIE shows AI can run on regular devices without special hardware.
problem Real-time AI on mobile devices without special hardware.
method Compression-compiler co-design using CoCoPIE framework.
result Real-time AI applications on off-the-shelf mobile devices.
We are in the middle of a remarkable rise in the use and capability of artificial intelligence. Much of this growth has been fueled by the success of deep learning architectures: models that map from observables to outputs via multiple layers of latent representations. These deep learning algorithms are effective tools…
PAVI speeds up Bayesian inference for large datasets.
problem Challenges in Bayesian inference for large population studies.
method Plate-amortized Variational Inference (PAVI) that shares parameterization across i.i.d. variables.
result Significant speedup in training variational distributions, orders of magnitude faster.
This paper examines data transfer methods to improve off-the-shelf Transformer models for retrosynthesis.
problem Improving prediction models for retrosynthesis using small datasets.
method Systematic examination of data transfer approaches on end-to-end generative models.
result Pre-training plus fine-tuning boosts the accuracy of a Transformer baseline, achieving state-of-the-art results.
Novel approach using Lagrangian Decomposition for neural network verification.
problem Computing tight bounds on neural network outputs.
method Lagrangian Decomposition, efficient supergradient ascent, proximal algorithm.
result Proves tighter bounds than previous dual algorithms.
This study investigates how much knowledge from natural images can be transferred to pathology images.
problem Quantifying how much knowledge from natural images can be transferred to pathology images.
method Proposes a framework to quantify knowledge gain by a particular layer, conducts empirical investigation in pathology image centered transfer learning.
result Early layers of deep models can transfer knowledge to pathology image classification tasks.
Since its introduction, unsupervised representation learning has attracted a lot of attention from the research community, as it is demonstrated to be highly effective and easy-to-apply in tasks such as dimension reduction, clustering, visualization, information retrieval, and semi-supervised learning. In this work, we…
Expands causal clustering framework with hierarchical and density-based methods.
problem Identifying heterogeneous treatment effects in unknown subgroup structure.
method Integrates hierarchical and density-based clustering algorithms into causal k-means clustering.
result Plug-in estimators for causal clustering are simple and readily implementable.
AFS uses attention to select features efficiently.
problem Efficiently selecting features from high-dimensional data.
method AFS combines an attention module and a learning module to address feature selection challenges.
result AFS outperforms state-of-the-art feature selection algorithms in accuracy and stability.
Develops a fast algorithm for robust logistic regression problems.
problem Distributionally robust logistic regression with Wasserstein distance.
method First-order algorithmic framework using linearized proximal ADMM.
result Sublinear convergence rate and up to 800x faster performance.
This paper learns hierarchical compositional models for image synthesis.
problem Creating interpretable and hierarchical image representations.
method Sparsifying a generator network to induce an AND-OR hierarchy.
result The method learns meaningful and interpretable hierarchical representations.
Optimizes matching in weighted graphs with semi-bandit sampling.
problem Finding optimal pairings in weighted graphs with sequential sampling.
method Leverages rank-1 assumption on adjacency matrix to reduce sample complexity and regret.
result Achieves linear dependency in the number of vertices for sample complexity and regret.
Investigates numerical issues in GP interpolation parameter estimation.
problem Numerical issues in maximum likelihood parameter estimation for Gaussian process interpolation.
method Investigates and proposes strategies to improve open-source software implementations.
result Improves reliability and reproducibility of studies relying on GP implementations.
Estimates sparse Gaussian graphical models using discrete optimization.
problem Learning a sparse graph from Gaussian graphical models.
method Proposes GraphL0BnB, an ℓ0-penalized MIP solved with a custom BnB framework. result Significant runtime and statistical performance improvements over existing methods.
Deep neural networks trained using a softmax layer at the top and the cross-entropy loss are ubiquitous tools for image classification. Yet, this does not naturally enforce intra-class similarity nor inter-class margin of the learned deep representations. To simultaneously achieve these two goals, different solutions h…
Deep NLP models benefit from underlying structures in the data---e.g., parse trees---typically extracted using off-the-shelf parsers. Recent attempts to jointly learn the latent structure encounter a tradeoff: either make factorization assumptions that limit expressiveness, or sacrifice end-to-end differentiability. Us…