This paper analyzes Barlow Twins' representation efficiency using information-geometric methods.
problem Understanding and comparing the efficiency of self-supervised learning methods.
method Introduces an information-geometric framework to quantify representation efficiency and applies it to Barlow Twins.
result Proves that Barlow Twins achieves optimal representation efficiency (η=1).
Improves sample efficiency in reinforcement learning with input representation.
problem Poor sample efficiency in reinforcement learning.
method Attention-based method to project inputs into an invariant representation space.
result Representation space is m! smaller for inputs of m objects, improving sample efficiency. We found a new series to calculate Black-Scholes efficiently.
problem Calculating the Black-Scholes formula efficiently.
method Proved and tested a series representation for the European Black-Scholes call.
result Works in every market configuration and refines previous approximations.
Improved sample efficiency in reinforcement learning with object exchangeability.
problem Sample inefficiency in reinforcement learning, especially with complex input structures.
method Attention-based method to project inputs into an efficient representation space invariant under input ordering.
result Our representation reduces the search space by a factor of m! for m objects, improving sample efficiency.
New findings show good representations alone are insufficient for efficient reinforcement learning.
problem Understanding when good representations are enough for efficient reinforcement learning.
method Statistical analysis of reinforcement learning methods, focusing on value-based, model-based, and policy-based learning.
result Hard thresholds for reinforcement learning methods show good representations alone are insufficient, unless they meet certain quality criteria.
This work improves sample efficiency in meta-learning for nonlinear tasks.
problem Learning complex tasks efficiently with limited data.
method Subspace-based representations for nonlinear tasks.
result Subspace-based representations can be learned efficiently and improve future task performance.
Deep networks struggle to learn efficient representations of simple functions.
problem Can deep learning methods find efficient representations of simple functions?
method Trained deep neural networks on the parity function and fast Fourier transform, using gradient-based optimization.
result Deep networks require initialization close to exact solutions to learn efficient representations of simple functions.
New method improves robot learning from vision with better sample efficiency.
problem Scaling reinforcement learning to real robots from vision.
method State representation learning to extract relevant features.
result Improved sample efficiency and robustness to hyper-parameters.
Novelty search in low-dimensional space improves sample efficiency in exploration tasks.
problem Efficient exploration in complex environments with sparse rewards.
method Combines model-based and model-free objectives to learn a low-dimensional representation. Uses intrinsic novelty rewards based on nearest neighbor distances in this space.
result Our approach achieves more sample-efficient exploration compared to strong baselines on various tasks.
ReLEX algorithm improves RL efficiency by selecting optimal representations.
problem Improving reinforcement learning efficiency through better representation selection.
method Proposes ReLEX algorithm for both online and offline RL, focusing on bilinear transition kernels.
result ReLEX algorithms achieve optimal or near-optimal performance in both online and offline RL settings.
New method tackles MDPs by learning normalized representations efficiently.
problem Curse of dimensionality in MDPs.
method Contrastive representation learning for linear MDPs.
result First practical method with strong theoretical guarantees and empirical performance.
This work proposes a method to learn sparse representations that are more efficient for large-scale data retrieval.
problem Efficient retrieval of high-dimensional representations from large databases is computationally challenging.
method The approach minimizes the number of floating-point operations (FLOPs) by learning sparse embeddings with uniform non-zero entries.
result The proposed method achieves a similar or better speed-vs-accuracy tradeoff compared to existing baselines.
This paper tackles noise in raw datasets to improve representation learning efficiency.
problem Noise in real-world datasets degrades representation learning quality.
method Proposes denoising Cosine-Similarity (dCS) loss to learn robust representations.
result Empirical results show the dCS loss outperforms baseline objective functions.
Efficiently updates vertex representations for dynamic graphs using random walks.
problem Updating vertex representations for dynamic graphs without re-generating them on each update.
method Proposes algorithms that extend random walk-based methods to dynamic graphs, considering the extent and rate of changes.
result Achieves competitive results to state-of-the-art methods while being computationally efficient.
New model-free algorithms learn representations for low-rank MDPs efficiently.
problem Learning representations in reinforcement learning for low-rank MDPs.
method Developed minimax representation learning objective and interleaved with reward-free exploration.
result Proven sample efficiency and scalability to complex environments.
Empower efficient representation of distributions through moment-preserving methods.
problem Representing high-dimensional probability measures efficiently and accurately.
method Empower efficient representation of distributions through moment-preserving methods.
result Empowers efficient and accurate representation of high-dimensional probability measures.
InstantEmbedding efficiently generates node representations with less computation and memory.
problem Efficiently generating local node representations for large graphs.
method Local PageRank computations in sublinear time.
result Significantly faster and less memory-intensive than traditional methods.
Efficiently learns disentangled representations using conditional probability differences.
problem Learning disentangled representations with causal mechanisms.
method Approximates difference of conditional probabilities with model's generalization ability.
result 1.9--11.0imes more sample efficient and 9.4--32.4 times quicker than previous method. The paper formalizes criteria for non-spurious and disentangled representations using causal methods.
problem Formalizing criteria for non-spurious and disentangled representations in representation learning.
method Causal perspective, counterfactual quantities, observable consequences of causal assertions.
result Computable metrics for assessing representation learning based on observed data.
Neural networks learn molecule and material representations.
problem Learning efficient representations for molecules and materials.
method Continuous-filter convolutional network SchNet.
result SchNet accurately predicts chemical properties across various datasets.
New algorithm REFUEL shows multitask representation learning is more sample-efficient in RL.
problem Understanding the benefit of representation learning in reinforcement learning.
method Developed REFUEL algorithm for multitask low-rank RL, analyzing both upstream and downstream tasks.
result Multitask representation learning is provably more sample-efficient than individual task learning.
DeepGL learns hierarchical graph representations from attributed graphs.
problem Learning deep node and edge representations from large attributed graphs.
method Derives base features, learns multi-layered hierarchical graph representation, leverages previous layer outputs, supports attributed graphs, learns interpretable features, and is space-efficient.
result DeepGL learns relational functions that generalize across-networks and is effective for across-network transfer learning tasks.
Paper improves sample efficiency of transfer learning in diffusion models.
problem Diffusion models need too much data to train from scratch.
method Assumes shared low-dimensional representation across tasks for improved sample efficiency.
result Sample complexity of target tasks can be reduced with a well-learned representation.
Improved AutoDML estimator for causal inference using outcome-adapted shared covariate representation.
problem Efficiency in estimating treatment or policy effects in causal inference.
method Outcome-adapted AutoDML estimator that uses a shared covariate representation that is predictive of the outcome but not the Riesz representer.
result Outcome-adapted AutoDML estimator is asymptotically more efficient than baseline AutoDML.
Custom narrow-precision representations boost DNN inference speed by 7.6x with minimal accuracy loss.
problem Improving computational efficiency of deep neural networks.
method Exploring and utilizing unconventional narrow-precision floating-point representations for DNN weights and activations.
result Average speedup of 7.6x with less than 1% accuracy loss.
METEOR learns efficient representations from multi-modal data streams.
problem Efficiently interpreting multi-modal information in complex environments.
method METEOR learns compact representations by sharing parameters within semantically meaningful groups and preserving domain-agnostic semantics.
result METEOR reduces memory usage by around 80% compared to conventional methods.
Generative Multisensory Network learns 3D scene representations from multiple modalities.
problem Learning robust 3D scene representations from multiple sensory modalities.
method Amortized Product-of-Experts for efficient inference and cross-modal generation.
result The model can infer modality-invariant 3D scene representations efficiently from various sensory modalities.
Contrastive UCB improves RL by learning feature representations efficiently.
problem Improving feature learning in RL for online decision making.
method Proposes UCB-based contrastive learning algorithms for RL in MDPs and MGs.
result Proves sample efficiency in learning optimal policies and Nash equilibria.
POLAR learns efficient data acquisition policies using pretrained belief representations.
problem Challenges in learning effective policies for adaptive data acquisition.
method POLAR decouples representation learning from policy learning by leveraging pretrained predictive foundation models as belief-state encoders.
result POLAR outperforms state-of-the-art methods across diverse tasks while requiring fewer training samples.
Survey categorizes methods for learning state representations in reinforcement learning.
problem Addressing challenges in complex observation spaces for sequential decision making.
method Categorizes six main classes of methods for learning state representations.
result Enhances understanding of state representation learning in reinforcement learning.
New algorithm improves learning efficiency in multi-task contextual bandits.
problem Improving learning efficiency in multi-task contextual bandits.
method Alternating projected gradient descent (GD) and minimization estimator for low-rank feature matrix recovery.
result Proved regret bound for multi-task learning algorithm.
Proposes using gradients as features for efficient deep learning adaptation.
problem Efficient deep representation learning for different tasks.
method Designs a linear model incorporating gradients and activations of a pre-trained network.
result Shows strong results across various tasks and datasets.
Efficient neural network invariant to symmetry subgroups.
problem Designing neural networks invariant to symmetry subgroups for computational efficiency.
method A new G-invariant transformation module and multi-layer perceptron. result The proposed architecture is computationally and memory efficient, and universal.
The paper proposes efficient dictionary learning algorithms that avoid multiplications for sparse representations.
problem Sparse representation with reduced computational complexity.
method Factorizations of the dictionary into binary orthonormal, scaling, and shear transformations with closed-form solutions.
result The proposed methods are effective and can be compared to well-known transforms like FFT and DCT.
Graph representation converts complex networks into vectors for easier analysis.
problem Efficient analysis of large network data.
method Introduces graph representation and network embedding models.
result Efficiently converts graph data into low-dimensional vectors.
Efficiently learns quantizable embeddings for fast search.
problem Learning binary hamming code representations for search efficiency.
method Directly learns a quantizable embedding representation and sparse binary hash code end-to-end.
result Achieves state-of-the-art search accuracy and significant speedup.
A new method for fair representation learning using PLS.
problem Fairness in representation learning for data reduction.
method Proposes Fair Partial Least Squares (PLS) components with fairness constraints.
result The new method outperforms standard fair PCA methods on various datasets.
In this article, we study connections between representation theory and efficient solutions to the conjugacy problem on finitely generated groups. The main focus is on the conjugacy problem in conjugacy separable groups, where we measure efficiency in terms of the size of the quotients required to distinguish a distinc…
Efficiently learns representations across domains and tasks with few labels.
problem Learning representations that generalize across different domains and tasks with limited labeled data.
method Combines domain adversarial loss and metric learning for representation transfer. Optimizes on both labeled and unlabeled data in the target domain.
result Significantly outperforms fine-tuning on novel classes in new domains with few labeled examples.
Paper improves RL efficiency by learning dynamic embeddings.
problem Improving sample efficiency in reinforcement learning.
method Proposes a forward prediction objective for state and action embeddings that capture dynamics.
result Action embeddings alone improve RL performance; combined state and action embeddings achieve efficient learning.
A new method, REC, compresses images by encoding their latent representations efficiently.
problem Efficiently compressing single images with latent representations.
method Relative Entropy Coding (REC) that directly encodes latent representations with codelength close to relative entropy.
result REC is more efficient for single image compression compared to previous methods and is competitive for lossy compression.
EMDE efficiently estimates manifold densities for diverse recommendation systems.
problem Efficiently estimating manifold densities for multi-modal recommendation systems.
method EMDE (Efficient Manifold Density Estimator) framework for arbitrary vector representations.
result Established new state-of-the-art results in top-k and session-based recommendation settings.
Efficient PAC learning for contrastive linear representations is achieved.
problem Efficient PAC learning for contrastive linear representations.
method Relaxing the problem to a semi-definite program and using Rademacher complexity.
result First efficient PAC learning algorithm for contrastive learning.
Sparse representations improve network robustness and stability.
problem The benefits of sparse representations in artificial networks.
method Analysis of sparse networks with sparse weights and activations, simulations on MNIST and Google Speech Command Dataset.
result Sparse networks show significantly improved robustness and stability compared to dense networks.
A new method uses a frozen language model to improve sample efficiency in reinforcement learning.
problem Improving sample efficiency in reinforcement learning with partially observable environments.
method FROZEN Hopfield network and HELM (History Embedding Language Model) method.
result HELM achieves new state-of-the-art results on Minigrid and Procgen environments.
NodeSig efficiently computes binary node embeddings for scalable graph analysis.
problem Scalability issues in graph representation learning models.
method NodeSig uses random walk diffusion probabilities and stable random projections to compute binary node embeddings efficiently.
result NodeSig achieves a good balance between accuracy and efficiency on node classification and link prediction tasks.
DyRep learns dynamic graph node embeddings efficiently.
problem Efficiently encoding evolving information over dynamic graphs into low-dimensional representations.
method Inductive deep representation learning framework using time-scale dependent multivariate point process model.
result Significantly outperforms baselines on real-world datasets for dynamic link and event time prediction.
Proposes a new method to learn representations directly optimized for a task.
problem Learning representations optimized for unrelated tasks.
method Jointly learns representation and prediction function for a specific task.
result Learned representations outperform pre-trained ones and are more sample-efficient.