In neural networks, it is often desirable to work with various representations of the same space. For example, 3D rotations can be represented with quaternions or Euler angles. In this paper, we advance a definition of a continuous representation, which can be helpful for training deep neural networks. We relate this t…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,695 papers · 148 categories
Trend · papers per month
48 results for “continuous representation”
Paper proposes SDRL to improve continual learning with less computational cost.
problem Catastrophic forgetting in continual learning.
method SDRL method that refines gradients from memorized samples to reduce gradient diversity.
result SDRL shows better performance than state-of-the-art methods on multiple benchmark tasks.
Paper proposes a new FRL algorithm for continuous sensitive attributes using EIPM.
problem Existing FRL algorithms cannot handle continuous sensitive attributes.
method Introduces EIPM to assess fairness in representation space for continuous attributes and proposes FREM algorithm.
result FREM outperforms other methods in fairness evaluation for continuous sensitive attributes.
Representations based on random walks can exploit discrete data distributions for clustering and classification. We extend such representations from discrete to continuous distributions. Transition probabilities are now calculated using a diffusion equation with a diffusion coefficient that inversely depends on the dat…
A continual learning agent should be able to build on top of existing knowledge to learn on new data quickly while minimizing forgetting. Current intelligent systems based on neural network function approximators arguably do the opposite---they are highly prone to forgetting and rarely trained to facilitate future lear…
Controlling Overestimation Bias with Truncated Mixture of Continuous Distributional Quantile Criticscs.LG
Paper tackles overestimation bias in continuous control, improving performance by 25%.
problem Overestimation bias in off-policy learning.
method Truncated Quantile Critics (TQC) combines distributional representation, truncation, and ensembling of critics.
result TQC outperforms state-of-the-art methods by 25% on the Humanoid environment.
We present a framework for learning disentangled and interpretable jointly continuous and discrete representations in an unsupervised manner. By augmenting the continuous latent distribution of variational autoencoders with a relaxed discrete distribution and controlling the amount of information encoded in each latent…
Temporal-difference (TD) networks are a class of predictive state representations that use well-established TD methods to learn models of partially observable dynamical systems. Previous research with TD networks has dealt only with dynamical systems with finite sets of observations and actions. We present an algorithm…
Earth observation embeddings can convert discrete biome maps into continuous representations that better capture ecological variation.
problem Biome maps impose categorical boundaries that compress continuous variation in biotic communities.
method Fit a linear classifier on Earth observation embeddings to predict biome labels.
result Continuous biome representation outperforms discrete biome labels for predicting species occurrence.
Maximal representations in symplectic lattices proven for most cases.
problem Understanding maximal representations in symplectic lattices.
method Analyzing mapping class group orbits and continuous deformations of maximal diagonal representations.
result Proof of maximal representations in most lattices of Sp(2n,R).
Dual representations for robust risk measures and uncertainty sets.
problem Characterizing continuity of robust risk measures and their uncertainty sets.
method Develop dual representations for robust risk measures and uncertainty sets based on distinct geometric assumptions.
result Two dual frameworks for consolidated uncertainty sets are complementary, not interchangeable.
We classify hyperbolic monopoles with continuous symmetries and construct new examples.
problem Classifying and constructing hyperbolic monopoles with continuous symmetries.
method Developed a Structure Theorem and used representation theory to simplify the problem.
result Found constraints on structure groups and constructed novel spherically symmetric hyperbolic monopoles.
Framework for continuous-time network data representation learning.
problem Learning reliable representations of dynamic network interactions.
method Three-stage process: intensity estimation, projection learning, evolving node representation construction.
result Trajectories satisfy structural and temporal coherence, providing robust inference.
This paper builds on the connection between graph neural networks and traditional dynamical systems. We propose continuous graph neural networks (CGNN), which generalise existing graph neural networks with discrete dynamics in that they can be viewed as a specific discretisation scheme. The key idea is how to character…
Develops correlation number for specific potentials and Hitchin representations.
problem Analyzing correlation numbers for potentials with entropy gaps and Hitchin representations.
method Defines a correlation number for pairs of cusped Hitchin representations and explores its connection to the Manhattan curve.
result Establishes a connection between the correlation number and the Manhattan curve, revealing rigidity properties.
Can simple algorithms with a good representation solve challenging reinforcement learning problems? In this work, we answer this question in the affirmative, where we take "simple learning algorithm" to be tabular Q-Learning, the "good representations" to be a learned state abstraction, and "challenging problems" to be…
DeepCCG adapts classifiers to representation shifts in one step.
problem Adapting classifiers to shifts in continuous representation.
method Empirical Bayesian approach using class conditional Gaussian classifier and KL divergence for selection.
result DeepCCG reduces performance change due to representation shifts.
Distributed representations of sentences have become ubiquitous in natural language processing tasks. In this paper, we consider a continual learning scenario for sentence representations: Given a sequence of corpora, we aim to optimize the sentence encoder with respect to the new corpus while maintaining its accuracy …
American options can be equivalent to European options under certain conditions.
problem Determining when American options can be simplified to European options.
method Using methods from Jourdain and Martini, Chrsitensen, and convex duality.
result A first step towards verifying representability of American options.
Many loss functions in representation learning are invariant under a continuous symmetry transformation. For example, the loss function of word embeddings (Mikolov et al., 2013) remains unchanged if we simultaneously rotate all word and context embedding vectors. We show that representation learning models for time ser…
A new framework for graph representation learning.
problem Acquiring continuous representations of discrete objects like graphs.
method Nested SubSpace (NSS) arrangement and Disk-ANChor ARrangement (DANCAR).
result Successfully embedded WordNet in 20-dimensional space with high F1 score.
A new method uncovers discrete and continuous factors in gene expression data.
problem Jointly identifying discrete and continuous factors of variability without supervision.
method cpl-mixVAE framework using multiple interacting networks.
result The method successfully uncovers discrete and continuous factors in gene expression data.
New algorithm for context bandits with continuous actions.
problem Efficient decision-making with unknown action structures.
method Reduction-style algorithm combining supervised learning.
result Proven to work in general and validated with experiments.
Method estimates causal effects from incremental data, overcoming missing data challenges.
problem Estimating causal effects from non-stationary, incrementally available observational data.
method Continual Causal Effect Representation Learning
result Method achieves continual causal effect estimation without compromising original data.
Theory of Θ-positive representations for real closed fields.
problem Generalizing positive representations to real closed fields.
method Developing theory for Fuchsian groups to linear groups over real closed fields.
result Theory encompasses many generalizations of positive or Anosov representations.
We consider the problem of building a state representation model for control, in a continual learning setting. As the environment changes, the aim is to efficiently compress the sensory state's information without losing past knowledge, and then use Reinforcement Learning on the resulting features for efficient policy …
Replicated and validated Rank-N-Contrast for robust regression.
problem Deep regression models struggle with continuous sample orders.
method Contrastive learning of continuous representations by ranking samples.
result Improved performance and robustness of RNC framework.
Hybrid framework prevents forgetting in continual learning.
problem Avoiding forgetting in learning new tasks without forgetting old ones.
method Hybrid continual learning framework combining architecture growth and experience replay.
result Hybrid approach effectively avoids forgetting across multiple tasks.
DINo forecasts PDEs with flexible extrapolation and adaptability.
problem Fixed discretizations limit real-world PDE forecasting.
method DINo uses implicit neural representations for continuous-time dynamics.
result DINo outperforms other neural PDE forecasters.
A graph VAE framework optimizes neural architectures in a continuous space.
problem Discovering efficient neural architectures in a discrete space.
method Graph VAE framework with VAE and GNN components, joint learning of predictors and decoders.
result The framework discovers powerful neural architectures with both excellent performance and high computational efficiency.
Adaptive tensor modeling preserves continuity in multidimensional data.
problem Discretization of continuous multidimensional data loses important information.
method Functional Tucker decomposition (FTD) with RKHS modeling.
result FTD enables adaptive and expressive tensor modeling.
Characterizes continuity of monotone functionals in mixed topology.
problem Continuity of monotone functionals in mixed topology.
method Characterization through lower semicontinuity and dual representations.
result Continuity in mixed topology is equivalent to dual representation in terms of countably additive measures.
CLPF models continuous time-series data with improved representational power and variational approximations.
problem Fitting continuous time-series data with existing models faces challenges in representational power and variational quality.
method CLPF uses a time-dependent normalizing flow driven by a stochastic differential equation to decode continuous latent processes into continuous observables. Maximum likelihood optimization is achieved through a novel variational posterior process.
result CLPF outperforms state-of-the-art baselines on synthetic and real-world time-series data.
We consider the problem of building a state representation model in a continual fashion. As the environment changes, the aim is to efficiently compress the sensory state's information without losing past knowledge. The learned features are then fed to a Reinforcement Learning algorithm to learn a policy. We propose to …
In this paper we present our scientific discovery that good representation can be learned via continuous attention during the interaction between Unsupervised Learning(UL) and Reinforcement Learning(RL) modules driven by intrinsic motivation. Specifically, we designed intrinsic rewards generated from UL modules for dri…
Compressing Green's function using intermediate representation between imaginary-time and real-frequency domainscond-mat.str-el
New model-independent compact representations of imaginary-time data are presented in terms of the intermediate representation (IR) of analytical continuation. This is motivated by a recent numerical finding by the authors [J. Otsuki et al., arXiv:1702.03056]. We demonstrate the efficiency of the IR through continuous-…
Motivated by the human way of memorizing images we introduce their functional representation, where an image is represented by a neural network. For this purpose, we construct a hypernetwork which takes an image and returns weights to the target network, which maps point from the plane (representing positions of the pi…
Introduces CHL, a new loss function for continuous similarity learning.
problem Binary similarity learning limitations.
method CHL is a novel loss function that generalizes histogram loss to continuous similarities.
result CHL solves a wider range of tasks including similarity learning, representation learning, and data visualization.
We continue our discussion from part I.
Neural networks can represent complex piecewise functions efficiently.
problem Representing continuous piecewise affine functions with neural networks.
method Two hidden layers with ReLU activation, neurons for pieces.
result CPA functions can be represented by a neural network with linear size.
In this paper we establish necessary and sufficient conditions for the limit set of a projective Anosov representation to be a differentiable submanifold of projective space with Holder continuous derivatives. We also calculate the optimal value of the Holder constant in terms of the eigenvalue data of the Anosov repre…
New representation of PSL2(R) on infinite hyperbolic space via convex bodies.
problem Continuous irreducible actions of PSL2(R) on infinite-dimensional hyperbolic space.
method Using hyperbolic model for convex bodies, produce a continuous and irreducible representation.
result Yields a convex cocompact PSL2(R)-action on infinite-dimensional hyperbolic space with specific quotient properties.
Loop soup representation of zeta-regularised determinants and equivariant Symanzik identitiesmath.PR
Stochastic representation for determinants derived from Brownian loop soups.
problem Determinants of Laplace-type operators on manifolds.
method Stochastic representation using Brownian loop soups.
result Derives continuity of determinants and conformal invariance on surfaces.
The Geometric Alignment Tax: Tokenization vs. Continuous Geometry in Scientific Foundation Modelscs.LG
Foundation models fail to preserve continuous geometry, identified as the Geometric Alignment Tax.
problem Continuous geometry is lost in foundation models due to discrete categorical bottlenecks.
method Controlled ablations on synthetic systems and evaluation of 14 biological models using rate-distortion theory and MINE.
result Replacing cross-entropy with a continuous head reduces geometric distortion by up to 8.5x.
Towards Better Laplacian Representation in Reinforcement Learning with Generalized Graph Drawingcs.LG
New method learns high-quality Laplacian representations for reinforcement learning.
problem Lack of accurate Laplacian representations in large or continuous state spaces.
method Reformulated spectral graph drawing objective to have eigenvectors as unique global minimizer.
result Learned Laplacian representations more faithfully approximate the ground truth.
Recently, pre-trained language representation flourishes as the mainstay of the natural language understanding community, e.g., BERT. These pre-trained language representations can create state-of-the-art results on a wide range of downstream tasks. Along with continuous significant performance improvement, the size an…
A scattering transform defines a signal representation which is invariant to translations and Lipschitz continuous relatively to deformations. It is implemented with a non-linear convolution network that iterates over wavelet and modulus operators. Lipschitz continuity locally linearizes deformations. Complex classes o…
Study shows how information loss and operation loss are related in feature representations.
problem Understanding the relationship between information loss and operation loss in feature representations.
method Analyzes the interplay between weak information loss and operation loss in continuous representations.
result Specific forms of vanishing information loss imply vanishing MPE loss in classification.