Graph Neural Networks (GNNs) are based on repeated aggregations of information across nodes' neighbors in a graph. However, because common neighbors are shared between different nodes, this leads to repeated and inefficient computations. We propose Hierarchically Aggregated computation Graphs (HAGs), a new GNN graph re…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,695 papers · 148 categories
Trend · papers per month
48 results for “repeated training”
Repeated self-distillation improves model performance significantly.
problem How much gain is possible by applying multiple steps of self-distillation?
method Investigated linear regression tasks, applied multiple steps of self-distillation, analyzed excess risk reduction.
result Multi-step self-distillation reduces excess risk by a factor as large as , where is the input dimension.
The paper analyzes how repeating epochs affects data scaling in linear regression.
problem Understanding how to scale data for multi-epoch training in linear regression.
method Theoretical analysis of effective reuse rate (E(K, N)) under strong convexity or Zipf-distributed data.
result The effective reuse rate E(K, N) plateaus at a problem-dependent value that grows with N, indicating diminishing marginal gains.
A new machine learning method for Bayesian inverse problems in function spaces.
problem Bayesian inverse problems in function spaces with incompatibility of white noise sources.
method One-step generative transport with amortized neural operator and prior-aligned Gaussian random field.
result Generative operator trained on prior samples and noisy observations generates posterior samples efficiently.
Optimal strategies are found for a repeated betting game using diffusion approximation.
problem Finding optimal strategies for a repeated betting game with i.i.d. outcomes.
method Constructing a diffusion approximation of the repeated game and analyzing the wealth share process.
result Necessary and sufficient conditions for the wealth share process to be transient or recurrent are derived.
RENT selects stable features for robust model interpretation.
problem Stability in feature selection for robust model interpretation.
method RENT uses an ensemble of elastic net models trained on distinct subsets of data.
result RENT selects features with high stability improving model robustness.
Retraining stabilizes model influence on data.
problem Performativity in predictive models leads to feedback loops.
method Developed the stable signal principle to address retraining dynamics.
result Repeated risk minimization converges geometrically to stable signal direction.
CRAFT improves on existing methods for sampling complex distributions.
problem Sampling from complex probability distributions.
method Combines SMC with variational inference using normalizing flows.
result Improves on Annealed Flow Transport Monte Carlo and MCMC-based Stochastic Normalizing Flows.
Convolutional networks struggle with repeating patterns in ECGs.
problem Modeling repeating patterns in electrocardiogram signals.
method Demonstrated through ECG examples, highlighting systemic issues in deep learning.
result Counterintuitive effects on generalization in deep networks.
Zoetrope Genetic Programming improves symbolic regression performance.
problem Evolutionary symbolic regression for complex mathematical expressions.
method Zoetropic representation, repeated fusion operations, linear combination, crossover, mutation, selection.
result Zoetrope Genetic Programming achieves state-of-the-art performance and low computational time.
For many large undirected models that arise in real-world applications, exact maximumlikelihood training is intractable, because it requires computing marginal distributions of the model. Conditional training is even more difficult, because the partition function depends not only on the parameters, but also on the obse…
We study two systems of tangle equations that arise when modeling the action of the Integrase family of proteins on DNA. These two systems--direct and inverted repeats--correspond to two different possibilities for the initial DNA sequence. We present one new class of solutions to the tangle equations. In the case of i…
Study explores algorithmic collusion in repeated games using various learning dynamics.
problem Understanding algorithmic collusion in repeated games with different learning dynamics.
method Examines -learning, gradient learning, and other dynamics in a general repeated game setting.
result Characterizes the set of payoff vectors achievable by these dynamics, revealing possibilities for collusion.
Scorio.jl ranks systems from repeated tasks using various methods.
problem Evaluating and ranking systems from repeated responses to shared tasks.
method Common tensor-based interface for multiple ranking methods.
result Pilot experiments show stability and runtime scaling.
Why does Deep Learning work? What representations does it capture? How do higher-order representations emerge? We study these questions from the perspective of group theory, thereby opening a new approach towards a theory of Deep learning. One factor behind the recent resurgence of the subject is a key algorithmic step…
We investigate the question of when distinct branched surfaces in the complement of a 2-bridge knot support essential surfaces with identical boundary slopes. We determine all instances in which this occurs and identify an infinite family of knots for which no boundary slopes are repeated.
LAFF algorithm balances adaptability and non-exploitability in repeated games.
problem Low regret in repeated games against unknown opponent classes.
method LAFF algorithm searches within sub-algorithms optimal for each opponent class and uses a punishment policy for exploitation.
result LAFF guarantees sublinear regret uniformly over possible opponents, except exploitative ones, for which it guarantees linear regret.
Link concordance and Whitney towers linked to Milnor invariants.
problem Link concordance and Whitney towers classification.
method Clasper surgeries, Whitney towers, and Milnor invariants.
result Link concordance and Whitney towers classified in terms of Milnor invariants.
This work studies how an AI-controlled dog-fighting agent with tunable decision-making parameters can learn to optimize performance against an intelligent adversary, as measured by a stochastic objective function evaluated on simulated combat engagements. Gaussian process Bayesian optimization (GPBO) techniques are dev…
Proposes a strategy to train models with minimal labeled data.
problem Scarce and expensive labeled data for medical tasks.
method Recursive training strategy to use image-level annotations for pixel-level segmentation.
result Improved segmentation of intracranial hemorrhage in CT scans.
Risk is part of the fabric of every business; surprisingly, there is little work on establishing best practices for systematic, repeatable risk identification, arguably the first step of any risk management process. In this paper, we present a proposal that constitutes a more holistic risk management approach, a method…
It has long been known that a Milnor invariant with no repeated index is an invariant of link homotopy. We show that Milnor's invariants with repeated indices are invariants not only of isotopy, but also of self C_k-moves. A self C_k-move is a natural generalization of link homotopy based on certain degree k clasper su…
Influence maximization (IM) is one of the most important problems in social network analysis. Its objective is to find a given number of seed nodes that maximize the spread of information through a social network. Since it is an NP-hard problem, many approximate/heuristic methods have been developed, and a number of th…
New models learn stable latent clusters without side info.
problem Stability of non-linear ICA representations without side information.
method Deep generative models with latent clusterings, compared to standard VAEs and auxiliary labeled models.
result Deep generative models with latent clusterings are as stable as models with side information.
We investigate some geometric properties of the real algebraic variety of symmetric matrices with repeated eigenvalues. We explicitly compute the volume of its intersection with the sphere and prove a Eckart-Young-Mirsky-type theorem for the distance function from a generic matrix to points in . We exhibit conne…
This paper tackles the reduction of redundant repeating generation that is often observed in RNN-based encoder-decoder models. Our basic idea is to jointly estimate the upper-bound frequency of each target vocabulary in the encoder and control the output words based on the estimation in the decoder. Our method shows si…
Machine learning models for repeated measurements are limited. Using topological data analysis (TDA), we present a classifier for repeated measurements which samples from the data space and builds a network graph based on the data topology. When applying this to two case studies, accuracy exceeds alternative models wit…
We introduce a training method for both better word representation and performance, which we call GROVER (Gradual Rumination On the Vector with maskERs). The method is to gradually and iteratively add random noises to word embeddings while training a model. GROVER first starts from conventional training process, and th…
New algorithm estimates task affinities without repeated training, improving model performance and efficiency.
problem Efficiently estimating task affinities among multiple tasks for model training.
method Grad-TAG algorithm: trains a base model for all tasks and uses gradient-based linearization to estimate task affinities.
result Estimates task affinities with high accuracy and low computational cost.
We present Sequential Attend, Infer, Repeat (SQAIR), an interpretable deep generative model for videos of moving objects. It can reliably discover and track objects throughout the sequence of frames, and can also generate future frames conditioning on the current frame, thereby simulating expected motion of objects. Th…
Study analyzes broker's gain from trade in repeated context-based trading.
problem Maximizing traders' net utility in repeated context-based brokerage.
method Proposes algorithms achieving tight regret bounds in full and limited feedback settings.
result Achieves tight 1/2-approximation result for gain from trade.
A new method combines machine learning with mixed-effects models for better repeated measurement analysis.
problem Inference of linear coefficients in partially linear mixed-effects models with complex interactions and high-dimensional variables.
method Double machine learning approach to estimate nonparametrically nonlinear variables, then use standard linear mixed-effects techniques to estimate the linear coefficient.
result The estimated fixed effects coefficient converges at the parametric rate and is semiparametrically efficient.
Why does Deep Learning work? What representations does it capture? How do higher-order representations emerge? We study these questions from the perspective of group theory, thereby opening a new approach towards a theory of Deep learning. One factor behind the recent resurgence of the subject is a key algorithmic step…
A common problem machine learning developers are faced with is overfitting, that is, fitting a pipeline too closely to the training data that the performance degrades for unseen data. Automated machine learning aims to free (or at least ease) the developer from the burden of pipeline creation, but this overfitting prob…
FSNet improves online time series forecasting by balancing fast adaptation and old knowledge.
problem Online time series forecasting challenges in handling abrupt and recurring patterns.
method Inspired by CLS theory, FSNet uses a dynamic balance between fast adaptation and old knowledge retrieval.
result FSNet achieves robustness to both new and recurring patterns through dynamic balancing and associative memory.
We present a general framework for training deep neural networks without backpropagation. This substantially decreases training time and also allows for construction of deep networks with many sorts of learners, including networks whose layers are defined by functions that are not easily differentiated, like decision t…
A fast method for LOOCV in k-NN regression reduces computation time.
problem Efficient computation of LOOCV for k-NN regression.
method Identical LOOCV estimate to (k+1)-NN MSE on training data.
result LOOCV computation can be done with (k+1)-NN regression once.
Whether stochastic or parametric, the Pareto/NBD model can only be utilized for an in-sample prediction rather than an out-of-sample prediction. This research thus provides a neural network based extension of the Pareto/NBD model to estimate the out-of-sample parameters, which overrides the estimation burden and the ap…
Deep convolutional neural networks have achieved great success in various applications. However, training an effective DNN model for a specific task is rather challenging because it requires a prior knowledge or experience to design the network architecture, repeated trial-and-error process to tune the parameters, and …
The study analyzes deep linear networks from random initialization, capturing dynamics and hyperparameter effects.
problem Understanding training dynamics in deep linear networks from random initialization.
method Theoretical analysis of gradient descent dynamics in deep linear networks with random initialization and large data.
result Captures the 'wider is better' effect and hyperparameter transfer effects, contrasting with neural-tangent parameterization.
XSPNs combine SPNs and MEVMs for efficient inference in data with repeated parts.
problem Efficient inference in data with repeated interchangeable parts.
method Introducing Exchangeability-Aware Sum-Product Networks (XSPNs) that combine SPNs and MEVMs.
result XSPNs can be more accurate than conventional SPNs when data contains repeated parts.
Neyman Meets Causal Machine Learning: Experimental Evaluation of Individualized Treatment Rulesstat.ME
Neyman's framework evaluates personalized treatment rules using experiments.
problem Evaluating the efficacy of individualized treatment rules derived by machine learning.
method Neyman's repeated sampling framework applied to cross-fitted ITRs.
result Ex-post evaluation of ITRs can be more efficient than random assignment.
Variational autoencoders (VAEs) with an auto-regressive decoder have been applied for many natural language processing (NLP) tasks. The VAE objective consists of two terms, (i) reconstruction and (ii) KL regularization, balanced by a weighting hyper-parameter β. One notorious training difficulty is that the KL term ten…
Convolutional autoregressive models have recently demonstrated state-of-the-art performance on a number of generation tasks. While fast, parallel training methods have been crucial for their success, generation is typically implemented in a naïve fashion where redundant computations are unnecessarily repeated. This res…
A new neural network initialization method is proposed for faster and more accurate training.
problem Efficient initialization for training multi-layer feedforward neural networks.
method Initialization based on Stein's identity, using eigenvectors of cross-moment matrix.
result The SteinGLM method is faster and more accurate than other initialization methods.
Federated learning has a variety of applications in multiple domains by utilizing private training data stored on different devices. However, the aggregation process in federated learning is highly vulnerable to adversarial attacks so that the global model may behave abnormally under attacks. To tackle this challenge, …
This paper optimizes deep learning training by efficiently sharding weight updates across replicas.
problem Redundant weight update computation on all replicas in data-parallel training.
method Automatic sharding of weight updates using static analysis and transformations on the training graph.
result Substantial speedups achieved on large-scale models using Cloud TPUs.
Deep RL agents suffer from transient non-stationarity, which ITER mitigates.
problem Transient non-stationarity in deep RL agents affects generalization.
method Iterated Relearning (ITER) transfers knowledge between networks to reduce non-stationarity.
result ITER improves deep RL agents' performance on generalization benchmarks.