The paper improves polyphonic music models by extracting salient features.
problem Improving performance of polyphonic music models.
method Training a neural network to predict both chords and notes, including extra features.
result Training a neural network to predict extra features improves model performance.
We present a novel hierarchical distance-dependent Bayesian model for event coreference resolution. While existing generative models for event coreference resolution are completely unsupervised, our model allows for the incorporation of pairwise distances between event mentions -- information that is widely used in sup…
We introduce a new approach to unsupervised estimation of feature-rich semantic role labeling models. Our model consists of two components: (1) an encoding component: a semantic role labeling model which predicts roles given a rich set of syntactic and lexical features; (2) a reconstruction component: a tensor factoriz…
A feature-rich Bitcoin trading assistant using reinforcement learning.
problem Predicting and analyzing Bitcoin market trends for long-term profits.
method Reinforcement learning with technical indicators and Twitter sentiment scores.
result Average profit of 69% over 685 days, including the Covid-19 period.
Proposes HBGNN for better recommendation systems using graph neural networks.
problem Sparse structured data in recommendation systems lacking feature richness.
method Hierarchical BiGraph Neural Network (HBGNN) using bigraph framework.
result Competitive performance compared to current methods.
Adversarially-trained models transfer better in limited data scenarios.
problem Improving transfer learning performance with limited data.
method Adversarially-train deep nets, freeze early layers, fine-tune last layers, observe shape vs texture bias.
result Adversarially-trained models transfer better than non-adversarially-trained models, especially with limited data.
Bayesian Matrix Factorization (BMF) is a powerful technique for recommender systems because it produces good results and is relatively robust against overfitting. Yet BMF is more computationally intensive and thus more challenging to implement for large datasets. In this work we present SMURFF a high-performance featur…
State-of-the-art machine learning frameworks support a wide variety of design features to enable a flexible machine learning programming interface and to ease the programmability burden on machine learning developers. Identifying and using a performance-optimal setting in feature-rich frameworks, however, involves a no…
This paper focuses on the problem of explaining predictions of psychological attributes such as attractiveness, happiness, confidence and intelligence from face photographs using deep neural networks. Since psychological attribute datasets typically suffer from small sample sizes, we apply transfer learning with two ba…
PyHHMM is a Python library for HHMMs with advanced features.
problem Handling heterogeneous observation models and missing data in HMMs.
method Object-oriented Python implementation with advanced features.
result PyHHMM supports a heterogeneous observation model and missing data inference.
We introduce a new benchmark for coreference resolution and NLI, Knowref, that targets common-sense understanding and world knowledge. Previous coreference resolution tasks can largely be solved by exploiting the number and gender of the antecedents, or have been handcrafted and do not reflect the diversity of naturall…
Datamodels predict model outcomes from training data subsets.
problem Understanding model behavior from training data.
method Conceptual framework for analyzing model behavior using datamodels.
result Simple linear datamodels can accurately predict model outputs.
Stochastic encoders outperform deterministic ones in 'perfect perceptual quality'.
problem Understanding when stochastic encoders outperform deterministic ones.
method Provided a toy example to illustrate performance.
result Stochastic encoders can significantly outperform deterministic ones in 'perfect perceptual quality'.
BiPE blends intra-segment and inter-segment encodings for better length extrapolation.
problem Improving length extrapolation in language models.
method Bilevel Positional Encoding (BiPE) that separates intra-segment and inter-segment encodings.
result BiPE enhances length extrapolation across various text modalities.
Enhanced Bayesian target encoding uses sampling techniques to improve model performance.
problem Improving target encoding for better model performance in machine learning.
method Using sampling techniques in Bayesian target encoding to extract intra-category distribution information.
result Improves generalization and reduces target leakage in machine learning models.
Paper proposes LCP for structural encodings, outperforming existing methods.
problem Improving Graph Neural Networks performance through effective structural encodings.
method Geometric perspective, Local Curvature Profiles (LCP) for structural encodings, combining with global positional encodings, comparing with rewiring techniques.
result LCP significantly outperforms existing structural encodings and combining LCP with global positional encodings improves performance.
Improved online AED models with multi-stage training and multi-task learning.
problem Enhance performance of online attention-based encoder-decoder models.
method Three-stage training with character encoder, BPE encoder, and attention decoder; multi-task learning at character and BPE levels; transfer learning from bidirectional encoder.
result 35% and 10% relative improvement over baselines for smaller and bigger models, respectively.
New definition reveals encoding explanations that retain predictive power.
problem Challenges in evaluating and identifying encoding explanations.
method Developed a definition of encoding based on conditional dependence.
result Existing evaluation scores do not rank non-encoding explanations correctly, but STRIPE-X does.
Study on encoding neural architectures for NAS, showing impact on performance.
problem Impact of different encodings on NAS performance.
method Formal definition and characterization of encodings, empirical study of encodings' effectiveness.
result Different encodings can significantly affect NAS performance.
Improved OOD detection across various shifts using multi-encoder fusion of RDMs.
problem Out-of-distribution detection across multiple types of distribution shifts.
method Statistical identification of encoder sensitivity, EncMin2L fusion, and Tippett minimum combination.
result Achieves AUROC ≥ 0.94 across four shift types, outperforming state-of-the-art detectors.
Single auto-encoder learns cross-domain image translation.
problem Cross-domain image-to-image translation using a single encoder-decoder architecture.
method Single auto-encoder with independent domain and content encodings.
result Cross-domain mapping achieved without separate encoders.
Binary encoding enables neural networks to extrapolate periodic functions.
problem Extrapolating periodic functions without prior knowledge of their form.
method Normalized Base-2 Encoding (NB2E) for continuous numerical values.
result MLPs using NB2E can successfully extrapolate diverse periodic signals.
This research compares two encoding methods for categorical attributes in machine learning, affecting model fairness.
problem The impact of encoding protected categorical attributes on fairness in machine learning models.
method Comparison of one-hot encoding and target encoding methods.
result Target encoding can lead to more unfair models compared to one-hot encoding due to induced bias.
Real-world complex networks describe connections between objects; in reality, those objects are often endowed with some kind of features. How does the presence or absence of such features interplay with the network link structure? Although the situation here described is truly ubiquitous, there is a limited body of res…
Quasi-orthonormal encoding reduces high dimensionality for categorical data.
problem High dimensionality and low sample size issues in categorical data encoding.
method Quasi-orthonormal encoding (QOE) for categorical data.
result QOE reduces dimensionality and improves machine learning performance.
For statistical learning, categorical variables in a table are usually considered as discrete entities and encoded separately to feature vectors, e.g., with one-hot encoding. "Dirty" non-curated data gives rise to categorical variables with a very high cardinality but redundancy: several categories reflect the same ent…
The paper identifies saddlepoints in unsupervised auto-encoding neural nets.
problem The risk landscape of unsupervised least squares in auto-encoding neural nets.
method Established an equivalence between unsupervised least squares and principal manifolds, discussed regularization strategies for auto-encoders.
result All non-trivial critical points in auto-encoding are saddlepoints, which are degenerate in overcomplete auto-encoding.
Regularized target encoding beats traditional methods for high cardinality features in ML.
problem Efficiently encoding high cardinality categorical variables for ML algorithms.
method Regularized target encoding compared to traditional encodings like integer and one-hot encoding.
result Regularized target encoding consistently provided the best results in a large-scale benchmark experiment.
We study the role of latent space dimensionality in Wasserstein auto-encoders (WAEs). Through experimentation on synthetic and real datasets, we argue that random encoders should be preferred over deterministic encoders. We highlight the potential of WAEs for representation learning with promising results on a benchmar…
We present a framework to learn privacy-preserving encodings of images that inhibit inference of chosen private attributes, while allowing recovery of other desirable information. Rather than simply inhibiting a given fixed pre-trained estimator, our goal is that an estimator be unable to learn to accurately predict th…
Study efficient neural operator learning using variation spaces.
problem Operator learning using encoder-decoder neural networks.
method Introduce variation space for nonlinear operators, establish approximation bounds.
result Algebraic approximation and learning rates for polynomially decaying input and output encoding errors.
We propose the Wasserstein Auto-Encoder (WAE)---a new algorithm for building a generative model of the data distribution. WAE minimizes a penalized form of the Wasserstein distance between the model distribution and the target distribution, which leads to a different regularizer than the one used by the Variational Aut…
Encoding a sequence of observations is an essential task with many applications. The encoding can become highly efficient when the observations are generated by a dynamical system. A dynamical system imposes regularities on the observations that can be leveraged to achieve a more efficient code. We propose a method to …
MUTE improves neural network performance with efficient target encoding.
problem Improving neural network performance with limited resources.
method MUTE optimizes Hamming distances among target encoding by understanding class confusion.
result MUTE offers better generalization and robustness with minimal overhead.
Object detection in point clouds is an important aspect of many robotics applications such as autonomous driving. In this paper we consider the problem of encoding a point cloud into a format appropriate for a downstream detection pipeline. Recent literature suggests two types of encoders; fixed encoders tend to be fas…
A new model SEQ clusters and classifies encoded features for better interpretability.
problem Lack of interpretability in classical supervised classification tasks.
method Proposes a novel supervised learning model named Supervised-Encoding Quantizer (SEQ) that applies a quantizer to cluster and classify encoded features.
result The quantizer provides an interpretable graph where each cluster represents a class with a particular style.
New method enforces encoder sparsity in HPF for more interpretable feature selection.
problem Lack of encoder sparsity in HPF leads to lack of column-clustering property.
method Enforces encoder sparsity using a generalized additive model (GAM).
result Gains ability to perform feature selection and relates each representation to original features.
DGA and DVGA learn disentangled graph representations to improve graph analysis.
problem Holistic graph auto-encoders fail to capture latent factors effectively.
method Design disentangled graph convolutional network and component-wise flow, impose independence constraints.
result Improved disentangled graph representations enhance graph analysis tasks.
A simple encoder and complex decoder for secure image encryption and decryption.
problem Secure and efficient image encryption and decryption.
method Uses a shallow encoder neural network for encryption and a deep decoder for decryption, trained independently.
result Decrypted images are nearly identical to the original, demonstrating the effectiveness of the framework.
STRING improves 2D and 3D position encodings for better performance.
problem Efficient and accurate position encoding for 2D and 3D applications.
method STRING extends Rotary Position Encodings with a unifying theoretical framework, maintaining translation invariance and low computational cost.
result STRING shows substantial gains in open-vocabulary object detection and robotics.
Given a convolutional dictionary underlying a set of observed signals, can a carefully designed auto-encoder recover the dictionary in the presence of noise? We introduce an auto-encoder architecture, termed constrained recurrent sparse auto-encoder (CRsAE), that answers this question in the affirmative. Given an input…
Auto-encoders have emerged as a successful framework for unsupervised learning. However, conventional auto-encoders are incapable of utilizing explicit relations in structured data. To take advantage of relations in graph-structured data, several graph auto-encoders have recently been proposed, but they neglect to reco…
Conformer encoder reverses sequence in time dimension, affecting decoder training.
problem Reversal of sequence in Conformer encoder impacts decoder training.
method Analyzed initial behavior of decoder cross-attention and proposed methods to avoid flipping.
result Self-attention module of Conformer starts dominating, allowing only reversed information to pass.
Neural networks encode inputs deterministically and categorically, behaving like hash encoders.
problem Understanding the encoding properties of neural networks.
method Analyzed the input space partitioned by ReLU-like activations in neural networks.
result Neural networks can be represented by unique activation patterns, similar to hash encoders.
Two graph auto-encoders decouple feature propagation from graph convolution layers.
problem Designing efficient graph auto-encoders with fixed receptive fields.
method L-GAE and L-VGAE using linear matrix computation before auto-encoder input.
result Comparable performance to VGAEs with smaller, simpler networks.
Neural encoding and decoding, which aim to characterize the relationship between stimuli and brain activities, have emerged as an important area in cognitive neuroscience. Traditional encoding models, which focus on feature extraction and mapping, consider the brain as an input-output mapper without inner states. In th…
New insights into how encoder-decoder networks generate attention matrices.
problem Understanding how encoder-decoder networks use attention matrices.
method Decomposing hidden states into temporal and input-driven components.
result Attention matrices are formed based on task requirements, not architecture type.
Randomized positional encodings boost transformer performance on longer sequences.
problem Transformers struggle with generalizing to sequences of arbitrary length.
method Introduced randomized positional encodings that simulate longer sequences and randomly select positions.
result Randomized positional encodings increase test accuracy by 12.0% on average for sequences of unseen length.