We study the large scale geometry of the upper triangular subgroup of PSL(2,Z[1/n]), which arises naturally in a geometric context. We prove a quasi-isometry classification theorem and show that these groups are quasi-isometrically rigid with infinite dimensional quasi-isometry group. We generalize our results to a lar…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,695 papers · 148 categories
Trend · papers per month
48 results for “large dimensional context”
Optimal model selection for forecasting large collections of short time series using latent space.
problem Challenges in choosing among multiple forecasting methods for large, high-dimensional time series with limited data.
method Combining low-rank temporal matrix factorization with optimal model selection using cross-validation.
result Forecasting latent factors leads to significant performance gains compared to direct uni-variate model application.
CADE learns dual node representations for better generalization.
problem Transductive graph embeddings cannot generalize to unseen nodes or across different graphs.
method CADE combines real-time neighborhoods with neighbor-attentioned representation, preserving known node memory.
result CADE outperforms state-of-the-art methods in generalization and context-awareness.
Parallel recordings of neural spike counts have revealed the existence of context-dependent noise correlations in neural populations. Theories of population coding have also shown that such correlations can impact the information encoded by neural populations about external stimuli. Although studies have shown that the…
Approximate Message Passing (AMP) has been shown to be a superior method for inference problems, such as the recovery of signals from sets of noisy, lower-dimensionality measurements, both in terms of reconstruction accuracy and in computational efficiency. However, AMP suffers from serious convergence issues in contex…
This paper is concerned with a lesser-studied problem in the context of model-based, uncertainty quantification (UQ), that of optimization/design/control under uncertainty. The solution of such problems is hindered not only by the usual difficulties encountered in UQ tasks (e.g. the high computational cost of each forw…
Direct contextual policy search methods learn to improve policy parameters and simultaneously generalize these parameters to different context or task variables. However, learning from high-dimensional context variables, such as camera images, is still a prominent problem in many real-world tasks. A naive application o…
LLMs learn probability density functions in-context, showing distinct learning trajectories.
problem Density estimation of time series data in LLMs.
method Intensive Principal Component Analysis (InPCA) to visualize and analyze LLMs' learning dynamics.
result LLMs follow similar learning trajectories in a low-dimensional InPCA space, distinct from traditional methods.
Transformers learn new tasks from few examples via optimal approximation.
problem Learning new tasks from limited examples using large language models.
method Developed approximation and generalization error bounds for transformers trained on nonparametric regression tasks.
result Transformers achieve minimax optimal estimation risk in context.
High-dimensional data models, often with low sample size, abound in many interdisciplinary studies, genomics and large biological systems being most noteworthy. The conventional assumption of multinormality or linearity of regression may not be plausible for such models which are likely to be statistically complex due …
This work introduces a method to decompose uncertainty in in-context learning for large language models.
problem Understanding the sources of uncertainty in in-context learning for large language models.
method Variational uncertainty decomposition framework without sampling from latent parameter posterior.
result Quantitative and qualitative validation of decomposed epistemic and aleatoric uncertainties.
Learning Linear Regression with Low-Rank Tasks in-Contextcond-mat.dis-nn
This work analyzes how transformers learn common linear regression tasks.
problem Understanding how in-context learning operates in real-world applications with common task structures.
method Analyzing a linear attention model trained on low-rank regression tasks.
result Statistical fluctuations in finite pre-training data induce an implicit regularization, leading to a sharp phase transition in generalization error.
Wiener-Granger causality is a widely used framework of causal analysis for temporally resolved events. We introduce a new measure of Wiener-Granger causality based on kernelization of partial canonical correlation analysis with specific advantages in the context of large high-dimensional data. The introduced measure is…
Efficiently selects top-m designs for various contexts using sequential sampling.
problem Optimizing selection of top-m designs across different contexts.
method Formulated as a stochastic dynamic programming problem, developed sequential sampling policy.
result Asymptotically optimal sampling ratios for efficient selection.
Study reveals how depth of reasoning affects generalization in models.
problem Understanding scaling behavior of generalization with CoT depth.
method Theoretical model of CoT in linear regression using random matrix theory.
result Sharp phase transition between exponential and polynomial improvement, saturation, and overthinking.
Paper analyzes Transformer learning dynamics, proving benign landscape for in-context learning.
problem Understanding how Transformers learn in context with nonlinear features.
method Mean-field and two-timescale analysis of Transformer dynamics, proving nonconvex but benign landscape.
result Proves mean-field dynamics avoid saddle points, leading to improved optimization.
Context-aware recommender systems (CARS) have gained increasing attention due to their ability to utilize contextual information. Compared to traditional recommender systems, CARS are, in general, able to generate more accurate recommendations. Latent factors approach accounts for a large proportion of CARS. Recently, …
Proposes semi-supervised feature ranking for handling high-dimensional, unlabeled data.
problem Handling high-dimensional, unlabeled data in machine learning.
method Tree ensembles and Relief family algorithms for semi-supervised feature ranking.
result Semi-supervised feature ranking outperforms supervised methods across most datasets.
For low-dimensional data sets with a large amount of data points, standard kernel methods are usually not feasible for regression anymore. Besides simple linear models or involved heuristic deep learning models, grid-based discretizations of larger (kernel) model classes lead to algorithms, which naturally scale linear…
When can reliable inference be drawn in the "Big Data" context? This paper presents a framework for answering this fundamental question in the context of correlation mining, with implications for general large scale inference. In large scale data applications like genomics, connectomics, and eco-informatics the dataset…
LLMs perform well in financial sentiment analysis without fine-tuning.
problem Challenges in financial terminology, emotions, and ambiguous expressions.
method In-context learning methods for financial document-sentiment pairs.
result LLMs can generalize in-context demonstrations to new financial documents.
Paper argues context equals environment, improving AI generalization.
problem AI models struggle to generalize in new environments.
method In-Context Risk Minimization (ICRM) algorithm.
result ICRM leads to significant out-of-distribution performance improvements.
New insights into how large learning rates affect transformer training dynamics.
problem Understanding how large learning rates impact the training of transformer models.
method Analyzing a simplified linear transformer model with a two-factor product map.
result Large learning rates can lead to various training outcomes including cycles, chaos, or divergence.
Context-aware recommender systems (CARSs) apply sensing and analysis of user context in order to provide personalized services. Adding context to a recommendation model is challenging, since the addition of context may increases both the dimensionality and sparsity of the model. Recent research has shown that modeling …
Paper proposes linear transformers for efficient in-context learning without context length limitations.
problem Quadratic complexity of softmax transformers limits data processing speed.
method Investigates linear transformers under domain generalization, showing they learn mappings from context distributions to response functions.
result Linear transformers achieve in-context learning with a linear complexity in context length, offering a dimension-independent convergence rate.
The paper explores how regularization can improve multi-objective learning with high-dimensional data.
problem Improving multi-objective learning with high-dimensional and costly data.
method A two-stage MOL framework that leverages low-dimensional structure.
result Vanilla regularization approaches often fail in multi-objective learning, and a two-stage framework can successfully exploit low-dimensional structure.
Semi-supervised Generative Adversarial Networks (GANs) are developed in the context of travel mode inference with uni-dimensional smartphone trajectory data. We use data from a large-scale smartphone travel survey in Montreal, Canada. We convert GPS trajectories into fixed-sized segments with five channels (variables).…
New method tackles high-dimensional contextual bandits with flexible kernel models.
problem Maximizing rewards in decision-making scenarios with many features.
method Introduces stochastic assumptions and no-regret learning for Gaussian kernels.
result Achieves no-regret learning even with feature dimensions growing with samples.
Transformers learn low-dimensional target functions efficiently in-context.
problem Efficiently learning nonlinear target functions in-context using transformers.
method Nonlinear MLP layer in transformers optimized by gradient descent, focusing on single-index target functions.
result Transformers can learn target functions with low-dimensional structures efficiently in-context.
Transformer models waste resources on long-context tasks.
problem Redundant attention computations in Transformer models for long-context tasks.
method Reformulate sequence modeling as supervised learning, analyze attention sparsity, formulate attention optimization as linear coding problem, propose Dynamic Group Attention.
result DGA reduces computational costs while maintaining performance.
Predicts clinical events using a landmark approach with machine learning for large biomarker histories.
problem Dynamic prediction of clinical events from large biomarker histories.
method Landmark approach extended to endogenous markers history combined with machine learning methods for survival data.
result Superlearner combining regularized regressions and random survival forests outperforms standard survival models.
Noise Sensitivity Exponent controls statistical-computational gaps in learning.
problem Understanding when learning is statistically possible yet computationally hard in high-dimensional statistics.
method Investigating statistical-computational gaps in single- and multi-index models using Noise Sensitivity Exponent.
result Noise Sensitivity Exponent governs statistical-computational gaps in high-dimensional learning.
Some selected applications of KT and HKT geometries in string theory, supergravity, black hole moduli spaces and hermitian geometry are reviewed. It is shown that the moduli spaces of a large class of five-dimensional supersymmetric black holes are HKT spaces. In hermitian geometry, it is shown that a compact, conforma…
We compress large neural networks for quick adaptation to specific contexts.
problem How to quickly adapt a pretrained large neural network to specific contexts.
method Propose a Bayesian hypernetwork framework to compress the network and encourage sparsity.
result Generated compressed networks are significantly smaller than baseline methods.
This paper improves context-aware recommender systems by selecting and incorporating relevant low-dimensional contextual information.
problem Generating accurate recommendations is not enough; contextual information can cause issues like battery drain and privacy.
method Developed a feature-selection algorithm based on genetic algorithms to reduce context dimensions while maintaining explainability.
result The approach improves accuracy and transparency in recommendations, outperforming state-of-the-art models.
The paper analyzes bootstrap ensemble classifiers in high-dimensional settings.
problem Performance of bootstrap ensemble classifiers in high-dimensional data.
method Random Matrix Theory applied to LSSVM ensemble.
result Strategies to optimize performance of LSSVM ensemble.
Gradient flow in a potential energy (or Euclidean action) landscape provides a natural set of paths connecting different saddle points. We apply this method to General Relativity, where gradient flow is Ricci flow, and focus on the example of 4-dimensional Euclidean gravity with boundary S^1 x S^2, representing the can…
We present a structural clustering algorithm for large-scale datasets of small labeled graphs, utilizing a frequent subgraph sampling strategy. A set of representatives provides an intuitive description of each cluster, supports the clustering process, and helps to interpret the clustering results. The projection-based…
We study compressing empirical measures in finite RKHSs using convex optimization.
problem Efficiently approximating empirical measures in high-dimensional spaces.
method Convex optimization and lower bounds on ball size.
result High probability lower bounds on ball size under various conditions.
In the sentence classification task, context formed from sentences adjacent to the sentence being classified can provide important information for classification. This context is, however, often ignored. Where methods do make use of context, only small amounts are considered, making it difficult to scale. We present a …
The paper argues for interpreting neural networks as approximating the true posterior, enhancing in-context learning.
problem The limitations of traditional MLE interpretation in large-scale, single-epoch training setups.
method Demonstrates the power of interpreting neural networks as approximations of the true posterior, using experiments to predict generalizations.
result Models become robust in-context learners by effectively composing knowledge from their training data, revealing surprising generalizations.
Proposes an iterative algorithm for optimizing attention mechanisms in large language models.
problem Optimizing attention mechanisms in large language models.
method Iterative algorithm for rescaled hyperbolic functions regression.
result Efficiency and generalizability of the rescaled softmax regression framework.
ICEE learns new RL tasks in less time with a Transformer model.
problem Efficient in-context policy learning for reinforcement learning.
method In-context Exploration-Exploitation (ICEE) algorithm that optimizes efficiency without explicit Bayesian inference.
result ICEE solves Bayesian optimization problems as efficiently as Gaussian process biased methods but in significantly less time.
Approximation of Riemannian Distances and Applications to Distance-Based Learning on Manifoldsmath.DG
Several important algorithms for machine learning and data analysis use pairwise distances as input. On Riemannian manifolds these distances may be prohibitively costly to compute, in particular for large datasets. To tackle this problem, we propose a distance approximation which requires only a linear number of geodes…
Locally Homogeneous Spaces, Induced Killing Vector Fields and Applications to Bianchi Prototypesmath.DG
An answer to the question: Can, in general, the adoption of a given symmetry induce a further symmetry, which might be hidden at a first level? has been attempted in the context of differential geometry of locally homogeneous spaces. Based on E. Cartan's theory of moving frames, a methodology for finding all symmetries…
A new algorithm for differential privacy in kernelized contextual bandits reduces error rate.
problem Joint differential privacy in kernelized contextual bandits.
method Proposes a novel algorithm with a specific error rate and privacy parameter dependence.
result Achieves an error rate of $\mathcal{O}\left(\sqrt{\frac{γ_T}{T}} + \frac{γ_T}{T \varepsilon}
ight)$ after queries.
The paper introduces contexture theory to characterize representation learning from contexts.
problem Lack of systematic characterization of representation learning methods.
method Characterizes representation learning as learning from the association between input and context variable.
result Contexture theory shows that representations can be approximated by top singular functions of the context.
This paper proposes a fast and accurate method for sparse regression in the presence of missing data. The underlying statistical model encapsulates the low-dimensional structure of the incomplete data matrix and the sparsity of the regression coefficients, and the proposed algorithm jointly learns the low-dimensional s…