arXiv research

Task hinting improves transformer performance on longer tasks.

SAGE enhances reinforcement learning by injecting hints to prevent model stagnation.

New algorithms reduce forecasting errors by leveraging optimistic learning and hinting.

New algorithms reduce regret in online learning with imperfect hints.

Algorithm achieves logarithmic regret with sublinear hints.

Detecting structure in noisy time series is a difficult task. One intuitive feature is the notion of trend. From theoretical hints and using simulated time series, we empirically investigate the efficiency of standard recurrent neural networks (RNNs) to detect trends. We show the overall superiority and versatility of …

Trained recurrent networks are powerful tools for modeling dynamic neural computations. We present a target-based method for modifying the full connectivity matrix of a recurrent network to train it to perform tasks involving temporally complex input/output transformations. The method introduces a second network during…

We propose a novel method for imputing missing data by adapting the well-known Generative Adversarial Nets (GAN) framework. Accordingly, we call our method Generative Adversarial Imputation Nets (GAIN). The generator (G) observes some components of a real data vector, imputes the missing components conditioned on what …

Oracle-efficient algorithms for online learning with smoothed and hint-adversaries.

Due to the unparallelizable nature of the autoregressive factorization, AutoRegressive Translation (ART) models have to generate tokens sequentially during decoding and thus suffer from high inference latency. Non-AutoRegressive Translation (NART) models were proposed to reduce the inference time, but could only achiev…

Problems for the graduate students who want to improve problem-solving skills in geometry. Every problem has a short elegant solution -- this gives a hint which was not available when the problem was discovered.

Paper proposes hybrid modeling to improve surrogate accuracy using multiple data sources.

In recent years several trading platforms appeared which provide a backtest engine to calculate historic performance of self designed trading strategies on underlying candle data. The construction of a correct working backtest engine is, however, a subtle task as shown by Maier-Paape and Platen (cf. arXiv:1412.5558 [q-…

Many recent invertible neural architectures are based on coupling block designs where variables are divided in two subsets which serve as inputs of an easily invertible (usually affine) triangular transformation. While such a transformation is invertible, its Jacobian is very sparse and thus may lack expressiveness. Th…

Knowledge distillation (KD) is a popular method for reducing the computational overhead of deep network inference, in which the output of a teacher model is used to train a smaller, faster student model. Hint training (i.e., FitNets) extends KD by regressing a student model's intermediate representation to a teacher mo…

In unsupervised data generation tasks, besides the generation of a sample based on previous observations, one would often like to give hints to the model in order to bias the generation towards desirable metrics. We propose a method that combines Generative Adversarial Networks (GANs) and reinforcement learning (RL) in…

We review a resent {\em time-dependent} performance measure for economical time series -- the (optimal) investment horizon approach. For stock indices, the approach shows a pronounced gain-loss asymmetry that is {\em not} observed for the individual stocks that comprise the index. This difference may hint towards an sy…

AI bias arises from human-defined goals, not algorithmic flaws.

New algorithms adapt to both gradient norms and comparator norms in online learning.

The abstract proves the existence of universal lottery tickets without needing further training.

Why do large neural network generalize so well on complex tasks such as image classification or speech recognition? What exactly is the role regularization for them? These are arguably among the most important open questions in machine learning today. In a recent and thought provoking paper [C. Zhang et al.] several au…

Improved MCMC sampling for expensive, irregular likelihoods.

We describe GTApprox - a new tool for medium-scale surrogate modeling in industrial design. Compared to existing software, GTApprox brings several innovations: a few novel approximation algorithms, several advanced methods of automated model selection, novel options in the form of hints. We demonstrate the efficiency o…

This paper explores GNN functions on random graphs, highlighting the importance of node Positional Encodings.

Local Hebbian learning is believed to be inferior in performance to end-to-end training using a backpropagation algorithm. We question this popular belief by designing a local algorithm that can learn convolutional filters at scale on large image datasets. These filters combined with patch normalization and very steep …

In this paper, we consider data consisting of multiple networks, each comprised of a different edge set on a common set of nodes. Many models have been proposed for the analysis of such multi-view network data under the assumption that the data views are closely related. In this paper, we provide tools for evaluating t…

Explores new perspectives in transverse index theory for Lie group actions.

CNT leverages noisy targets to guide model learning.

Study finds chirally cosmetic surgeries on knots and manifolds, contradicting previous conjectures.

Observational data hints at a finite universe, with spherical manifolds such as the Poincare dodecahedral space tentatively providing the best fit. Simulating the physics of a model universe requires knowing the eigenmodes of the Laplace operator on the space. The present article provides explicit polynomial eigenmodes…

A test measures artificial agents' human-like behavior in video games.

Optimizes predictions by recalibrating online forecasts with minimal error.

Despite the high interest for Machine Learning (ML) in academia and industry, many issues related to the application of ML to real-life problems are yet to be addressed. Here we put forward one limitation which arises from a lack of adaptation of ML models and datasets to specific applications. We formalise a new notio…

New algorithms optimize without tuning, matching tuned SGD performance.

Unified framework for SSL methods linking contrastive and non-contrastive approaches.

Explearn learns to explain predictions using Gaussian Processes.

The Riemannian Bures metric on the space of (normalized) complex positive matrices is used for parameter estimation of mixed quantum states based on repeated measurements just as the Fisher information in classical statistics. It appears also in the concept of purifications of mixed states in quantum physics. Here we d…

New scaling laws explain deep learning performance growth.

Quantum models generalize well with little data, challenging traditional generalization theories.

In this paper, we develop a method for constructing left-orders on the fundamental groups of rational homology 3-spheres. We begin by constructing the holonomy extension locus of a rational homology solid torus $M$, which encodes the information about peripherally hyperbolic $\widetilde{\text{PSL}_2\mathbb{R}}$ represe…

We establish a theoretical link between adversarial training and operator norm regularization for deep neural networks. Specifically, we prove that $\ell_p$-norm constrained projected gradient ascent based adversarial training with an $\ell_q$-norm loss on the logits of clean and perturbed inputs is equivalent to data-…

Characterization of the American put option price is still an open issue. From the beginning of the nineties there exists a non-closed formula for this price but nontrivial numerical computations are required to solve it. Strong efforts have been done to propose methods more and more computationally efficient but most …

Survey of robust streaming techniques and their relationships.

Enhances clinical trial predictions by quantifying uncertainty.

Quantum RNG improves financial risk metrics estimation.

New insights into training deep networks with low rank layers.

The paper identifies and critiques problems with risk matrices using ordinal scales.

Paper verifies RNNs using automata learning and model checking.