Dictionaries are collections of vectors used for representations of random vectors in Euclidean spaces. Recent research on optimal dictionaries is focused on constructing dictionaries that offer sparse representations, i.e., -optimal representations. Here we consider the problem of finding optimal dictionaries …
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,742 papers · 148 categories
Trend · papers per month
48 results for “optimal representations”
RedEx improves neural network optimization with convex optimization guarantees.
problem Difficult optimization of neural networks.
method RedEx architecture using convex optimization with semi-definite constraints.
result RedEx can efficiently learn functions fixed methods cannot.
New method uses bi-level optimization to learn useful representations for imitation learning.
problem Learning useful representations for multiple tasks in imitation learning settings.
method Formulates representation learning as a bi-level optimization problem.
result Bi-level optimization framework provides sample complexity benefits for imitation learning.
Dictionaries are collections of vectors used for representations of elements in Euclidean spaces. While recent research on optimal dictionaries is focussed on providing sparse (i.e., -optimal,) representations, here we consider the problem of finding optimal dictionaries such that representations of samples of …
Introduces MPR to measure and optimize representation across intersectional groups in retrieval.
problem Harmful stereotypes, cultural erasure, and social disparities in image search and retrieval.
method Develops MPR metric, practical estimation methods, theoretical guarantees, and optimization algorithms.
result Optimizing MPR yields more proportional representation across multiple intersectional groups, often with minimal retrieval accuracy compromise.
Proposes a new method to learn representations directly optimized for a task.
problem Learning representations optimized for unrelated tasks.
method Jointly learns representation and prediction function for a specific task.
result Learned representations outperform pre-trained ones and are more sample-efficient.
Optimal transport metric transfers deep network representations efficiently.
problem Efficiently transfer deep network representations for new tasks.
method Use optimal transport to quantify representation similarity and regularize student network.
result Optimal transport distance promotes similarity between teacher and student representations.
Bézier-GAN optimizes airfoil design by reducing shape complexity.
problem High computational cost in aerodynamic shape optimization.
method Generative adversarial networks (GANs) to learn compact shape representations.
result Empirically accelerates optimization convergence by at least two times.
RPS uses graph-based representation learning for better portfolio optimization.
problem Improving portfolio optimization with better returns and lower risks.
method RPS redefines the distance matrix of financial assets using Representation Learning and Clustering algorithms.
result RPS proposes a heuristic to select closer to the optimal subset of assets.
DenseHMM improves HMMs by learning dense representations that enable gradient-based optimization.
problem Learning dense representations for hidden states and observables in HMMs.
method DenseHMM uses kernelized transition probabilities and two optimization schemes.
result DenseHMM achieves superior performance and expressiveness compared to standard HMMs.
Proposes Decodable Information Bottleneck for optimal representation learning.
problem Finding optimal representations for supervised learning.
method Integrates information retention and compression with the desired predictive family.
result Optimal representations lead to better expected test performance and can be estimated with guarantees.
A method for fair representation learning through bi-level optimization and implicit differentiation.
problem Ensuring fair predictors invariant across sub-groups.
method Bi-level optimization with inner-loop for invariant predictors, implicit path alignment for efficiency.
result Consistently better trade-off in prediction performance and fairness measurement.
Optimizing Variational Representations of Divergences and Accelerating their Statistical Estimationcs.LG
New method tightens variational representations of divergences for faster learning.
problem Improving tightness of variational representations of divergences for faster statistical estimation.
method Improved objective functionals constructed via an auxiliary optimization problem, leveraging neural network approximation.
result Tighter variational representations can result in significantly faster learning and more accurate estimation of divergences.
This work explores representation complexity in RL paradigms, revealing model-based RL as the easiest task.
problem Investigating the representation complexity gap among model-based, policy-based, and value-based RL.
method Demonstrated through analysis of Markov decision processes (MDPs) and introduced new classes of MDPs.
result Representation complexity hierarchy: model-based RL > policy-based RL > value-based RL.
A mathematical theory for understanding when abstract representations emerge in neural networksq-bio.NC
Math theory explains how neural networks learn abstract representations.
problem Understanding how neural networks learn abstract representations.
method Mathematical theory reformulating network optimization into mean field optimization over neural preactivations.
result Abstract representations of latent variables are guaranteed to appear in neural networks trained on tasks that depend on these variables.
Optimal feature transfer identified through bias-variance analysis.
problem Optimizing feature transfer in transfer learning.
method Simple linear model with fine-grained bias-variance decomposition.
result Optimal pretrained feature transform is naturally sparse.
Study on how optimal representations emerge during deep learning training, focusing on the role of implicit regularization.
problem Understanding how optimal representations for tasks are learned during training.
method Investigates the role of implicit regularization in learning minimal sufficient representations, analyzing changes in representation content during training.
result Semantically meaningful but ultimately irrelevant information is encoded in early transient dynamics of training, which is later discarded.
Optimizes basis for density-based atomic representations to enhance compactness and accuracy.
problem Improving the efficiency and accuracy of machine learning models for atomic properties.
method An unsupervised approach to determine the optimal basis set for atom density representations using splines.
result Optimal basis sets that encode structural information more compactly and accurately.
Recently, kernelized locality sensitive hashcodes have been successfully employed as representations of natural language text, especially showing high relevance to biomedical relation extraction tasks. In this paper, we propose to optimize the hashcode representations in a nearly unsupervised manner, in which we only u…
SOC-ICNN expands neural network representational capacity by using conic optimization.
problem Restrictive representational capacity of ReLU-based ICNNs.
method Proposes SOC-ICNN architecture that uses Second-Order Cone Programming.
result SOC-ICNN strictly expands representational space without increasing complexity.
New method computes optimal fairness-performance trade-off without complex models.
problem Intrinsic trade-off between fairness and classifier performance.
method Computes optimal Pareto front without training complex models.
result Optimal fair representations have useful structural properties enabling efficient computation.
CausalCOMRL improves RL task representations by integrating causal relationships, enhancing generalizability.
problem Spurious correlations in context-based offline meta-reinforcement learning.
method CausalCOMRL integrates causal representation learning to uncover and incorporate causal relationships among task components.
result CausalCOMRL achieves better performance on meta-reinforcement learning benchmarks.
Recent results on optimization and generalization properties of neural networks showed that in a simple two-layer network, the alignment of the labels to the eigenvectors of the corresponding Gram matrix determines the convergence of the optimization during training. Such analyses also provide upper bounds on the gener…
New method uses random projections to estimate densities and modes efficiently.
problem Estimating densities and modes from sparse representations.
method Expand-and-sparsify representations followed by linear function and mode recovery algorithms.
result Optimal rates for density and mode estimation achieved.
ProGraML uses graph-based machine learning to improve program optimization and analysis.
problem Improving program optimization and analysis with machine learning.
method Low-level, language agnostic graph representation and message passing neural networks.
result ProGraML achieves an average 94.0 F1 score on a benchmark dataset, significantly outperforming state-of-the-art approaches.
Paper proposes a new method for learning compact representations of sequential data.
problem Learning compact representations of sequential data capturing spatio-temporal cues.
method Contrastive representation learning via adversarial optimal transport on the Grassmann manifold.
result Empirical results show competitive performance in human action recognition.
Factor Representation and Decision Making in Stock Markets Using Deep Reinforcement Learningq-fin.ST
Paper uses deep reinforcement learning for optimal stock portfolio management.
problem Optimizing stock portfolio choices in complex market environments.
method Direct deep reinforcement learning to learn factor representations and make optimal decisions.
result Deep learning outperforms average market performance in portfolio allocation.
We systematically investigate the problem of representing Markov chains by families of random maps, and which regularity of these maps can be achieved depending on the properties of the probability measures. Our key idea is to use techniques from optimal transport to select optimal such maps. Optimal transport theory a…
Develops NPSVC++ to improve NPSVC performance through representation learning.
problem Feature suboptimality and class dependency in NPSVC training.
method Multi-objective optimization and end-to-end learning of NPSVC and its features.
result NPSVC++ ensures feature optimality across classes, overcoming training issues.
Representation learning becomes especially important for complex systems with multimodal data sources such as cameras or sensors. Recent advances in reinforcement learning and optimal control make it possible to design control algorithms on these latent representations, but the field still lacks a large-scale standard …
Optimal hedging framework with variational preferences under convex risk measures.
problem Optimal hedging with variational preferences under convex risk measures.
method Theoretical hedging optimization framework with dual representation of risk measures and utilities.
result Derivation of optimality and indifference pricing conditions.
We consider a set of probabilistic functions of some input variables as a representation of the inputs. We present bounds on how informative a representation is about input data. We extend these bounds to hierarchical representations so that we can quantify the contribution of each layer towards capturing the informati…
Contrastive learning adapts to data intrinsic dimensions, learning low-dimensional representations.
problem Learning high-dimensional representations from multi-modal data.
method Multi-modal contrastive learning with temperature optimization.
result Contrastive learning adapts to intrinsic dimensions of data, not specified dimensions.
Convex learning for diverse invariances in semi-inner-product space.
problem Efficiently learning invariant representations for a wide range of invariances.
method Developed a convex representation learning algorithm for generalized invariances modeled as semi-norms, introducing Euclidean embeddings for kernel representers in a semi-inner-product space.
result Accurate invariant representations learned efficiently and effectively, validated by experiments.
The paper tackles learning from similar but not identical linear representations, improving performance over single-task learning.
problem Understanding how to learn from tasks with similar but not exactly the same linear representations, especially when dealing with outlier tasks.
method Proposes adaptive and robust penalized empirical risk minimization and spectral methods.
result Both methods outperform single-task learning when representations are similar and perform at least as well otherwise, with minimax optimality demonstrated.
In this work we consider optimal stopping problems with conditional convex risk measures called optimised certainty equivalents. Without assuming any kind of time-consistency for the underlying family of risk measures, we derive a novel representation for the solution of the optimal stopping problem. In particular, we …
We prove the existence of optimal strategies for agents with cumulative prospect theory preferences who trade in a continuous-time illiquid market, transcending known results which pertained only to risk-averse utility maximizers. The arguments exploit an extension of Skorohod's representation theorem for tight sequenc…
Graph representation learning, aiming to learn low-dimensional representations which capture the geometric dependencies between nodes in the original graph, has gained increasing popularity in a variety of graph analysis tasks, including node classification and link prediction. Existing representation learning methods …
FairNN learns fair representations and decisions by optimizing a multi-objective loss function.
problem Fairness in machine learning models for decision-making.
method Joint feature representation and classification with multi-objective loss function.
result Joint approach outperforms separate treatment of fairness in representation learning or supervised learning.
A popular approach for predicting the future of dynamical systems involves mapping them into a lower-dimensional "latent space" where prediction is easier. We show that the information-theoretically optimal approach uses different mappings for present and future, in contrast to state-of-the-art machine-learning approac…
ReLEX algorithm improves RL efficiency by selecting optimal representations.
problem Improving reinforcement learning efficiency through better representation selection.
method Proposes ReLEX algorithm for both online and offline RL, focusing on bilinear transition kernels.
result ReLEX algorithms achieve optimal or near-optimal performance in both online and offline RL settings.
Representations of data that are invariant to changes in specified factors are useful for a wide range of problems: removing potential biases in prediction problems, controlling the effects of covariates, and disentangling meaningful factors of variation. Unfortunately, learning representations that exhibit invariance …
A key challenge in complex visuomotor control is learning abstract representations that are effective for specifying goals, planning, and generalization. To this end, we introduce universal planning networks (UPN). UPNs embed differentiable planning within a goal-directed policy. This planning computation unrolls a for…
Study optimization landscapes for overcomplete representations, showing benign geometric structures.
problem Optimizing overcomplete representations in high-dimensional data analysis.
method Formulate as -norm optimization problems with spherical constraint, analyze geometric properties.
result Nonconvex objectives have benign geometric structures, ensuring local search algorithms find target solutions.
On the Optimal Representation Efficiency of Barlow Twins: An Information-Geometric Interpretationcs.LG
This paper analyzes Barlow Twins' representation efficiency using information-geometric methods.
problem Understanding and comparing the efficiency of self-supervised learning methods.
method Introduces an information-geometric framework to quantify representation efficiency and applies it to Barlow Twins.
result Proves that Barlow Twins achieves optimal representation efficiency (η=1).
New metric captures individual neuron tuning across neural networks.
problem Need a metric that respects individual neuron tuning across different neural networks.
method Derived a 'soft' permutation-based metric using optimal transport theory.
result Metric avoids counter-intuitive outcomes and captures geometric insights.
This work improves disentanglement in latent space models without sacrificing generation quality.
problem Trade-off between disentanglement and generation quality in latent space models.
method Manifold optimization with a sum of autoencoder and PCA reconstruction errors, on the Stiefel manifold.
result Improves disentanglement without sacrificing generation quality.
Graph Convolutional Networks (GCNs) have shown very powerful for graph data representation and learning tasks. Existing GCNs usually conduct feature aggregation on a fixed neighborhood graph in which each node computes its representation by aggregating the feature representations of all its neighbors which is biased by…