Semantic representations of words have been successfully extracted from unlabeled corpuses using neural network models like word2vec. These representations are generally high quality and are computationally inexpensive to train, making them popular. However, these approaches generally fail to approximate out of vocabul…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,657 papers · 148 categories
Trend · papers per month
48 results for “Context Clues”
LLMs can be tricked into recalling facts based on context clues.
problem Manipulation of LLMs' factual recall through context changes.
method Mathematical exploration of transformers' associative memory properties.
result Transformers use self-attention and value matrix for associative memory.
New methods improve uncertainty explanations for models.
problem Improving interpretation of uncertainty estimates from probabilistic models.
method Developed new methods to generate diverse and global explanations for uncertain model predictions.
result Generated diverse and global explanations for uncertain model predictions, addressing previous limitations.
δ-CLUE generates diverse explanations for model uncertainty.
problem Lack of constraints in generating explanations for uncertainty estimates.
method Augmenting CLUE approach to provide a set of plausible explanations.
result Returns a set of diverse inputs that yield confident predictions.
Cryptonite tests NLP models with cryptic crossword clues.
problem Ambiguity in language poses a challenge for NLP models.
method Cryptic crossword dataset based on cryptic clues.
result Fine-tuning T5-Large on Cryptonite achieves only 7.6% accuracy.
CLUE method interprets uncertainty from BNNs by showing how inputs change to increase confidence.
problem Lack of work on interpreting uncertainty estimates from probabilistic models.
method CLUE method uses counterfactual explanations to interpret uncertainty from BNNs.
result CLUE outperforms baselines and helps practitioners understand predictive uncertainty.
Resting-state functional MRI (rs-fMRI) is a rich imaging modality that captures spontaneous brain activity patterns, revealing clues about the connectomic organization of the human brain. While many rs-fMRI studies have focused on static measures of functional connectivity, there has been a recent surge in examining th…
Understanding and interpreting how machine learning (ML) models make decisions have been a big challenge. While recent research has proposed various technical approaches to provide some clues as to how an ML model makes individual predictions, they cannot provide users with an ability to inspect a model as a complete e…
For an embedded singly periodic minimal surface M with genus bigger than or equal to 4 and annular ends, some weak symmetry hypotheses imply its congruence with one of the Hoffman-Wohlgemuth examples. We give a very geometrical proof of this fact, along which they come out many valuable clues for the understanding of t…
Animals (especially humans) have an amazing ability to learn new tasks quickly, and switch between them flexibly. How brains support this ability is largely unknown, both neuroscientifically and algorithmically. One reasonable supposition is that modules drawing on an underlying general-purpose sensory representation a…
Detecting adversarial examples currently stands as one of the biggest challenges in the field of deep learning. Adversarial attacks, which produce adversarial examples, increase the prediction likelihood of a target class for a particular data point. During this process, the adversarial example can be further optimized…
In reinforcement learning (RL), temporal abstraction still remains as an important and unsolved problem. The options framework provided clues to temporal abstraction in the RL, and the option-critic architecture elegantly solved the two problems of finding options and learning RL agents in an end-to-end manner. However…
Study of rotation angles in a rotating disc model.
problem Understanding geometric phase in rotating systems.
method Analyzes a simple kinematic model of rotating discs.
result Explicit form of geometric phase found using Baumkuchen lemma.
SmoothGrad and VarGrad are techniques that enhance the empirical quality of standard saliency maps by adding noise to input. However, there were few works that provide a rigorous theoretical interpretation of those methods. We analytically formalize the result of these noise-adding methods. As a result, we observe two …
Performing supervised learning from the data synthesized by using Generative Adversarial Networks (GANs), dubbed GAN-synthetic data, has two important applications. First, GANs may generate more labeled training data, which may help improve classification accuracy. Second, in scenarios where real data cannot be release…
In human cognition, the expansion of perceived between-category distances and compression of within-category distances is known as categorical perception (CP). There are several hypotheses about the causes of CP (e.g., language, learning, evolution) but no functional model. Whether CP is essential to categorisation or …
We provide a theoretical explanation of the role of the number of nodes at each layer in deep neural networks. We prove that the largest variation of a deep neural network with ReLU activation function arises when the layer with the fewest nodes changes its activation pattern. An important implication is that deep neur…
Study of characteristic numbers in 24-dimensional String manifolds.
problem Characterizing and understanding characteristic numbers of 24-dimensional String manifolds.
method Using Pontryagin numbers, integral basis of String cobordism group, and divisibility results.
result Established 2- and 3-primary divisibilities of characteristic numbers.
While training error of most deep neural networks degrades as the depth of the network increases, residual networks appear to be an exception. We show that the main reason for this is the Lyapunov stability of the gradient descent algorithm: for an arbitrarily chosen step size, the equilibria of the gradient descent ar…
The study preserves lower bounds of total scalar curvature under specific metric convergence.
problem Preserving lower bounds of total scalar curvature on smooth manifolds.
method Used stability of Ricci flow and heat flow with Ricci flow background.
result Lower bound of weighted total scalar curvature is preserved under specified convergence conditions.
It is oftentimes impossible to understand how machine learning models reach a decision. While recent research has proposed various technical approaches to provide some clues as to how a learning model makes individual decisions, they cannot provide users with ability to inspect a learning model as a complete entity. In…
Neural networks struggle with periodic functions, a new activation fixes this.
problem Neural networks fail to learn simple periodic functions.
method Proposed a new activation function, , to learn periodic functions.
result The new activation function successfully learns and predicts periodic functions.
We introduce a formal language IE that is a variant of the language PAL developed in [van Benthem 2011] by adding a belief operator and a common belief operator,specializing to stochastic analysis. A constant symbol in the language denotes a stochastic process so that we can represent several financial events as formul…
A new RL framework handles autocorrelated actions for better learning and stability.
problem Improving reinforcement learning algorithms for better stability and efficiency.
method Introduces a new algorithm that optimizes policies with autocorrelated actions.
result The new algorithm outperforms existing methods in four simulated control problems.
Boosting improves data fitting while maintaining fairness guarantees.
problem Ensuring fairness in data preprocessing.
method Boosting algorithm to learn sufficient statistics of exponential families.
result The learned distribution maintains fairness guarantees while fitting the data better.
Convolutional Neural Networks (CNNs) have become the state-of-the-art method to learn from image data. However, recent research shows that they may include a texture and colour bias in their representation, contrary to the intuition that they learn the shapes of the image content and to human biological learning. Thus,…
We study the performance of the euro/Swiss franc exchange rate in the extraordinary period from September 6, 2011 and January 15, 2015 when the Swiss National Bank enforced a minimum exchange rate of 1.20 Swiss francs per euro. Based on the analogy between Brownian motion in finance and physics, the first-order effect …
A bilevel optimization method for reweighting data leads to suboptimal results.
problem Reweighting data for better model performance on a different distribution.
method Formalizes data reweighting as a bilevel optimization problem and analyzes the limitations of classical solvers.
result The final data weights are often very sparse, indicating suboptimal solutions.
A group-category is an additively semisimple category with a monoidal product structure in which the simple objects are invertible. For example in the category of representations of a group, 1-dimensional representations are the invertible simple objects. This paper gives a detailed exploration of "topological quantum …
Complex classification performance metrics such as the F-measure and Jaccard index are often used, in order to handle class-imbalanced cases such as information retrieval and image segmentation. These performance metrics are not decomposable, that is, they cannot be expressed in a per-example manner, which hinder…
Computer generated academic papers have been used to expose a lack of thorough human review at several computer science conferences. We assess the problem of classifying such documents. After identifying and evaluating several quantifiable features of academic papers, we apply methods from machine learning to build a b…
Although deep learning has produced dazzling successes for applications of image, speech, and video processing in the past few years, most trainings are with suboptimal hyper-parameters, requiring unnecessarily long training times. Setting the hyper-parameters remains a black art that requires years of experience to ac…
Gradient descent on ReLU networks with square loss implicitly favors balanced weights.
problem Understanding implicit regularization in nonlinear neural networks with regression losses.
method Analyzing gradient descent dynamics on ReLU networks with square loss.
result It is impossible to characterize the implicit regularization of ReLU networks with square loss by any explicit function of model parameters.
We propose the use of the functional determinant of geometric operators in constructing an entropy functional associated to geometric flows. Our approach is based on the direct computation of the partition function, with a well-defined set of microstates and macrostates in the canonical ensemble. The approach is motiva…
The primary goal of this study is doing a meta-analysis research on two groups of published studies. First, the ones that focus on the evaluation of the United States Department of Agriculture (USDA) forecasts and second, the ones that evaluate the market reactions to the USDA forecasts. We investigate four questions. …
Genome-wide association studies (GWAS) have emerged as a rich source of genetic clues into disease biology, and they have revealed strong genetic correlations among many diseases and traits. Some of these genetic correlations may reflect causal relationships. We developed a method to quantify causal relationships betwe…
This paper extends stability analysis to non-convergent neural network training.
problem Generalization of neural networks whose training does not converge to fixed points.
method Introduces statistical algorithmic stability (SAS) to study non-convergent algorithms and their generalization.
result Stability of non-convergent training dynamics correlates with generalization performance.
DARTS- improves robustness by factoring out skip connections' advantage.
problem Performance collapse in DARTS architecture search.
method Factor out skip connections' advantage with an auxiliary skip connection.
result Significantly improved robustness across various datasets.
Layered neural networks have greatly improved the performance of various applications including image processing, speech recognition, natural language processing, and bioinformatics. However, it is still difficult to discover or interpret knowledge from the inference provided by a layered neural network, since its inte…
CMLE reduces spurious correlations in deep models.
problem Spurious correlations in deep learning models.
method Counterfactual Maximum Likelihood Estimation (CMLE) on interventional distribution.
result CMLE outperforms regular MLE in out-of-domain generalization and spurious correlation reduction.
Depression is one of the most common mental health disorders, and a large number of depressed people commit suicide each year. Potential depression sufferers usually do not consult psychological doctors because they feel ashamed or are unaware of any depression, which may result in severe delay of diagnosis and treatme…
We consider the dynamic linear regression problem, where the predictor vector may vary with time. This problem can be modeled as a linear dynamical system, with non-constant observation operator, where the parameters that need to be learned are the variance of both the process noise and the observation noise. While var…
Heavy-tailed regularization improves deep neural network performance.
problem Improving generalization of deep neural networks.
method Introducing Heavy-Tailed Regularization, using differentiable penalty terms and Bayesian statistics.
result Heavy-tailed regularization outperforms conventional regularization techniques.
Paper tackles rDR classification and lesion segmentation using self-supervised equivariant learning and attention-based MIL.
problem Classifying rDR and segmenting lesions from image-level labels.
method Integrates self-supervised equivariant attention mechanism (SEAM) with attention-based multi-instance learning (MIL).
result Achieved AU ROC of 0.958 on Eyepacs dataset, outperforming state-of-the-art.
GNN-CL model improves financial fraud detection using graph neural networks and reinforcement learning.
problem Complex financial fraud detection with high false positive rates.
method Combines GNN, CNN, LSTM, and MLPS to analyze transaction patterns and uses reinforcement learning to filter out noise.
result GNN-CL outperforms existing methods in financial fraud detection, reducing false positives.
Anomaly detection algorithms are often thought to be limited because they don't facilitate the process of validating results performed by domain experts. In Contrast, deep learning algorithms for anomaly detection, such as autoencoders, point out the outliers, saving experts the time-consuming task of examining normal …
Reconstruction-based learning produces uninformative features for perception tasks.
problem Misalignment between reconstruction-based learning and perception tasks.
method Investigated the impact of input space reconstruction on feature learning for perception tasks.
result Reconstruction-based learning allocates model capacity to a subspace with uninformative features for perception tasks.
We propose a novel approach for semantic segmentation that uses an encoder in the reverse direction to decode. Many semantic segmentation networks adopt a feedforward encoder-decoder architecture. Typically, an input is first downsampled by the encoder to extract high-level semantic features and continues to be fed for…