Simple baselines improve multimodal utterance learning.
problem Learning rich multimodal utterance representations.
method Conditional factorization of utterances into unimodal factors; extending to bimodal and trimodal factors.
result Optimal embeddings can be derived in closed form.
Paper presents a universal baseline for binary prediction models.
problem Need a robust baseline to evaluate model performance.
method Dutch Draw (DD) baseline method for binary classification models.
result Reduces to almost always predicting zero or one in most situations.
Dark Experience improves continual learning with a simple, strong baseline.
problem General Continual Learning in scenarios where tasks are not sequential and offline training is not possible.
method Mixing rehearsal with knowledge distillation and regularization.
result Dark Experience outperforms consolidated approaches and leverages limited resources.
Bayesian Scattering offers a simple baseline for image data uncertainty.
problem Lack of interpretable, mathematically grounded uncertainty quantification methods for image data.
method Coupling wavelet scattering transform with a simple probabilistic head.
result Bayesian Scattering provides sensible uncertainty estimates under distribution shifts.
Graphs are complex objects that do not lend themselves easily to typical learning tasks. Recently, a range of approaches based on graph kernels or graph neural networks have been developed for graph classification and for representation learning on graphs in general. As the developed methodologies become more sophistic…
Investigates Q value evolution in Stable Baselines for DQL in simple vs complex environments.
problem DQL in Stable Baselines struggles with simple non-game environments.
method Comparison of TrafficLight and FrozenLake environments; Q value decomposition analysis.
result Q values meander far from optimal in complex relationships between states.
Simple method improves deep learning with noisy labels.
problem Deep learning's overfitting to noisy labels.
method Adds a variance regularization term to penalize neural network's Jacobian norm.
result Achieves state-of-the-art performance with high noise tolerance.
We propose a simple but strong baseline for time series classification from scratch with deep neural networks. Our proposed baseline models are pure end-to-end without any heavy preprocessing on the raw data or feature crafting. The proposed Fully Convolutional Network (FCN) achieves premium performance to other state-…
Simple tabular event prediction model outperforms existing methods.
problem Predicting events from tabular data with historic events.
method Standard autoregressive LLM-style transformers with elementary positional embeddings and causal language modeling.
result Simple model outperforms existing approaches across various datasets and use-cases.
Simple fine-tuning of GANs with frozen discriminator outperforms existing methods.
problem Fine-tuning GANs requires numerous data and heavy computation.
method Freeze lower layers of the discriminator for fine-tuning.
result FreezeD significantly outperforms previous techniques on various datasets.
The study introduces backward baselines to distinguish past prediction from future prediction in machine learning models.
problem Differentiating between past and future prediction in machine learning models.
method Theoretical, empirical, and normative arguments support a family of simple and efficient statistical tests called backward baselines.
result The study provides a meaningful backward baseline for auditing black-box prediction systems.
An important problem in sequential decision-making under uncertainty is to use limited data to compute a safe policy, i.e., a policy that is guaranteed to perform at least as well as a given baseline strategy. In this paper, we develop and analyze a new model-based approach to compute a safe policy when we have access …
Graph classification models are sensitive to initialisation and structure, but simple models perform well.
problem Graph classification models' performance is sensitive to initialisation and structure.
method Examined recent graph coarsening architectures and their performance sensitivity.
result Simple models like MLP, single-layer GCN, and fixed-weight GCN achieve competitive performance.
Simple neural net outperforms complex uncertainty methods.
problem Reliable uncertainty estimation from deterministic models.
method A simple baseline using a single softmax neural net with residual connections and spectral normalization.
result Simple neural net outperforms DUQ and SNGP on uncertainty prediction.
Policy gradient methods are a widely used class of model-free reinforcement learning algorithms where a state-dependent baseline is used to reduce gradient estimator variance. Several recent papers extend the baseline to depend on both the state and action and suggest that this significantly reduces variance and improv…
Study identifies and analyzes spurious correlations in data-driven models.
problem Spurious correlations in data-driven models are unreliable and hard to detect.
method Collect and analyze synthetic datasets generated from causal graphs to investigate spurious correlations.
result Patterns connecting spurious correlation hypotheses and model design choices were observed.
Simple model outperforms neural networks on language understanding tasks.
problem Neural networks struggle with creating novel expressions from familiar ones.
method Attention-inspired modification of a baseline model, focusing on sequential thinking and acting.
result Simple model achieves good performance on gSCAN tasks, validating the benchmark.
Paper identifies problematic baselines in Shapley value explanations and proposes a reweighting mechanism.
problem Identifying and addressing the suboptimality of baselines in Shapley value feature importance analysis.
method Analyzed suboptimality of baselines, identified problematic baseline, generalized uninformativeness, and designed a reweighting mechanism.
result Proposed uncertainty-based reweighting mechanism effectively accelerates computation and improves explanation quality.
We show that several popular few-shot learning benchmarks can be solved with varying degrees of success without using support set Labels at Test-time (LT). To this end, we introduce a new baseline called Centroid Networks, a modification of Prototypical Networks in which the support set labels are hidden from the metho…
Word embeddings have been shown to be useful across state-of-the-art systems in many natural language processing tasks, ranging from question answering systems to dependency parsing. (Herbelot and Vecchi, 2015) explored word embeddings and their utility for modeling language semantics. In particular, they presented an …
Proposes baselines for joint NAS and HPO optimization.
problem Joint optimization of neural architecture and hyperparameters for multiple objectives.
method Extends existing methods to jointly optimize with multiple objectives.
result Serves as simple baselines for future multi-objective joint NAS + HPO research.
Crowd-sourcing is a cheap and popular means of creating training and evaluation datasets for machine learning, however it poses the problem of `truth inference', as individual workers cannot be wholly trusted to provide reliable annotations. Research into models of annotation aggregation attempts to infer a latent `tru…
Paper establishes baselines for offline RL from visual observations.
problem Challenges in offline reinforcement learning from visual observations with continuous action spaces.
method Simple baselines and benchmarking tasks for offline RL from visual observations.
result Simple modifications to existing online RL algorithms outperform existing offline RL methods.
Graph classification has recently received a lot of attention from various fields of machine learning e.g. kernel methods, sequential modeling or graph embedding. All these approaches offer promising results with different respective strengths and weaknesses. However, most of them rely on complex mathematics and requir…
In this paper, we explore ordinal classification (in the context of deep neural networks) through a simple modification of the squared error loss which not only allows it to not only be sensitive to class ordering, but also allows the possibility of having a discrete probability distribution over the classes. Our formu…
Models predict patients at risk of uncontrolled hypertension.
problem Identifying patients at risk of uncontrolled hypertension.
method Developed machine learning models (logistic regression and recurrent neural networks) using EHR data.
result Best model achieved AUROC of 0.719, outperforming baseline.
A simple method improves batch active learning without high compute.
problem Efficient batch active learning in machine learning.
method Adapting standard single-point acquisition strategies to batch.
result Simple strategy performs as well as advanced batch methods.
A simple baseline outperforms deep learning methods in transportation forecasting.
problem The importance of stationarity and recurrent patterns in transportation data.
method A naive baseline based on average weekly patterns and linear regression.
result The baseline method achieves comparable or better results than state-of-the-art deep learning approaches.
This paper shows that a simple baseline based on a Bag-of-Words (BoW) representation learns surprisingly good knowledge graph embeddings. By casting knowledge base completion and question answering as supervised classification problems, we observe that modeling co-occurences of entities and relations leads to state-of-…
Simple Deep LDA models achieve accuracy competitive with softmax baselines.
problem Training Deep LDA models by maximum likelihood estimation leads to overlapping or collapsed class clusters.
method Proposed a constrained Deep LDA formulation with geometric constraints to fix class means and covariance.
result MLE becomes stable under geometric constraints, yielding well-separated class clusters.
Simple Euclidean models outperform hyperbolic graph learning models.
problem The effectiveness of hyperbolic graph learning models is questioned.
method Careful analysis of hyperbolic graph representation learning, identifying and addressing issues with baselines, modeling assumptions, and metric usage.
result Simple Euclidean models often outperform hyperbolic graph learning models, even on hyperbolic datasets.
Sales data in a commodity market (supermarket sales to consumers) has been analysed by studying the fluctuation spectrum and noise correlations. Three related products (ketchup, mayonnaise and curry sauce) have been analysed. Most noise in sales is caused by promotions, but here we focus on the fluctuations in baseline…
Ensemble models improve prediction calibration for mismatched distributions.
problem Calibration issues in deep neural networks with mismatched train and test distributions.
method Simple data augmentation and mixing techniques for ensemble models.
result Improves calibration and accuracy on CIFAR10 and CIFAR100 benchmarks.
A simple method flags images as out-of-distribution based on their distance to nearest neighbors.
problem Detecting images not aligned with a trained model's in-distribution data.
method Flag images as OOD if their average distance to K nearest neighbors is large in the classifier's representation space.
result Simple methods can outperform more complex ones when considering learned representations.
A new Bayesian model improves forecasting for intermittent demand.
problem Sparse observations, cold-start items, and obsolescence in intermittent demand forecasting.
method Hierarchical Bayesian TSB model with partial pooling and calibrated probabilistic configuration.
result TSB-HB achieves the lowest RMSE and RMSSE on the UCI Online Retail dataset.
SVD-RND detects blurred images better than conventional methods.
problem Blurred images can fool conventional OOD detection schemes.
method Constructs a novel RND-based detector that uses blurred images during training.
result SVD-RND outperforms baseline detectors in various domains.
Simple policy search outperforms advanced learnable test-time augmentation techniques.
problem Improving predictive performance through test-time data augmentation.
method Greedy policy search (GPS) for learning test-time augmentation policies.
result Augmentation policies learned with GPS achieve superior predictive performance and robustness.
Simple noise training improves image recognition robustness.
problem Modern neural networks struggle with unseen image corruptions.
method Additive Gaussian and Speckle noise training, adversarial training against worst-case noise.
result Significant improvement in robustness on ImageNet-C and MNIST-C benchmarks.
New learned optimizers outperform baselines by incorporating known and novel mechanisms.
problem Understanding how learned optimizers outperform traditional ones.
method Careful analysis and visualization of learned optimizers trained on various tasks.
result Learned optimizers incorporate known techniques like momentum and gradient clipping, as well as new forms of learning rate adaptation.
New algorithm trains neural nets on simple skills to learn complex tasks faster.
problem Learning complex tasks through simple imitation.
method Train neural networks on simple, easy-to-learn skills to accelerate learning of complex, hard-to-learn tasks.
result Consistently outperforms state-of-the-art baseline in training speed and performance.
We consider the problem of detecting out-of-distribution images in neural networks. We propose ODIN, a simple and effective method that does not require any change to a pre-trained neural network. Our method is based on the observation that using temperature scaling and adding small perturbations to the input can separ…
One of the basic tasks for Bayesian networks (BNs) is that of learning a network structure from data. The BN-learning problem is NP-hard, so the standard solution is heuristic search. Many approaches have been proposed for this task, but only a very small number outperform the baseline of greedy hill-climbing with tabu…
A simple regularization technique speeds up training of Neural ODEs.
problem Training Neural ODEs is computationally expensive.
method Randomly sampling the end time of the ODE during training.
result Significantly decreases training time and improves performance.
A simple baseline for extreme multi-label classification using random projections.
problem Automatically annotating data points with relevant labels from a large label vocabulary.
method On-the-fly global embedding using random projections, with an ensemble of learners.
result Competitive accuracy compared to existing methods, with significant speed-up and model-size reduction.
FrequentNet uses frequency domain basis vectors for image classification, making models more interpretable and efficient.
problem Image classification models are often complex and hard to interpret.
method FrequentNet selects filter vectors from frequency domain basis vectors instead of training them with back propagation.
result The method improves interpretability and efficiency of image classification models.
Simple method for black-box adversarial attacks with low query efficiency.
problem Efficient construction of adversarial images in the black-box setting.
method Randomly sampling from an orthonormal basis and adjusting images.
result Demonstrated unprecedented query efficiency in both untargeted and targeted attacks.
Bitcoin price prediction models fail to outperform a simple 'today's price' baseline, especially at longer horizons.
problem Lack of robust models that consistently outperform a naive price predictor at various horizons.
method Surveyed peer-reviewed papers, categorized by evaluation methodology, contrasted with social media discourse, and proposed methodological standards.
result No peer-reviewed study has shown robust superiority over the naive baseline across multiple market regimes at short-to-medium horizons.
Majority of semantic segmentation algorithms operate on a single frame even in the case of videos. In this work, the goal is to exploit temporal information within the algorithm model for leveraging motion cues and temporal consistency. We propose two simple high-level architectures based on Recurrent FCN (RFCN) and Mu…