Algorithm learns to solve tasks by gradually expanding policy space.
problem Reinforcement learning difficulty progression.
method Decreasingly constrained policy space expansion.
result Superior learning rate in Tetris tasks.
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,742 papers · 148 categories
Trend · papers per month
48 results for “difficulty progression”
RLVR training dynamics reveal an implicit curriculum that shapes learning progression.
problem Understanding how RLVR overcomes the long-horizon barrier.
method Developed a theory of training dynamics for RLVR on transformers, using Fourier analysis on finite groups.
result Mixed-difficulty training naturally follows an implicit curriculum, shaping the learning progression from easy to hard.
New method learns optimal environment and goal difficulty for reinforcement learning.
problem Lack of robust transfer in reinforcement learning.
method Self-Supervised Active Domain Randomization (SS-ADR).
result Optimal domain randomization strategy improves transfer in goal-directed tasks.
In this paper, we investigate a new form of automated curriculum learning based on adaptive selection of accuracy requirements, called accuracy-based curriculum learning. Using a reinforcement learning agent based on the Deep Deterministic Policy Gradient algorithm and addressing the Reacher environment, we first show …
These notions in the title are of fundamental importance in any branch of physics. However, there have been great difficulties in finding physically acceptable definitions of them in general relativity since Einstein's time. I shall explain these difficulties and progresses that have been made. In particular, I shall i…
This research formalizes inductive generalization and proposes a new learning paradigm called Inductive Learning.
problem Generalization from easy to hard tasks, especially out-of-domain generalization.
method Formalizes inductive generalization, introduces Inductive Learning, and outlines steps to adapt techniques for learning model successors.
result A new learning paradigm (Inductive Learning) that emphasizes induction and universal properties of learning and computation.
Proposes PIC and POIC for measuring task difficulty in RL.
problem Lack of metrics to measure task difficulty in RL.
method Introduces policy information capacity (PIC) and policy-optimal information capacity (POIC) as metrics based on mutual information.
result Empirically shows PIC and POIC correlate with task solvability better than alternatives.
RLVR learning dynamics naturally create an implicit curriculum for transformers.
problem How rewards based on final outcomes help overcome the long-horizon barrier in reasoning models.
method Developed a theory of training dynamics for RLVR on transformers, using Fourier analysis on finite groups.
result Mixed-difficulty training induces an implicit curriculum that shapes the learning progression from easy to hard.
Rapid progress in deep learning has spurred its application to bioinformatics problems including protein structure prediction and design. In classic machine learning problems like computer vision, progress has been driven by standardized data sets that facilitate fair assessment of new methods and lower the barrier to …
Study shows code-level optimizations significantly impact deep RL algorithms.
problem Understanding the impact of implementation details on deep RL algorithms.
method Case study on PPO and TRPO, investigating the effects of code-level optimizations.
result Code-level optimizations are crucial for performance in deep RL algorithms.
Recent studies have significantly improved the state-of-the-art on common-sense reasoning (CSR) benchmarks like the Winograd Schema Challenge (WSC) and SWAG. The question we ask in this paper is whether improved performance on these benchmarks represents genuine progress towards common-sense-enabled systems. We make ca…
Bounding the generalization error of learning algorithms has a long history, which yet falls short in explaining various generalization successes including those of deep learning. Two important difficulties are (i) exploiting the dependencies between the hypotheses, (ii) exploiting the dependence between the algorithm'…
The paper evaluates samplers on multi-modal targets, focusing on mode separation and recovery.
problem Handling multi-modality in sampling.
method Synthetic experimental setting focusing on mode relative importance recovery.
result Illustrates the challenges and potential of samplers in multi-modality.
It is widely believed that the success of deep convolutional networks is based on progressively discarding uninformative variability about the input with respect to the problem at hand. This is supported empirically by the difficulty of recovering images from their hidden representations, in most commonly used network …
New method reduces high-dimensional data to key features.
problem Challenges of high-dimensional data analysis and interpretability.
method Randomized search to produce subspaces, ensemble of models for variable selection.
result Outperforms existing methods in prediction and variable selection.
PS-KD distills a model's own knowledge to soften hard targets during training.
problem Improving generalization of deep neural networks by softening hard targets.
method Progressive self-knowledge distillation (PS-KD) that progressively distills a model's own knowledge to soften hard targets.
result PS-KD improves accuracy and provides high quality of confidence estimates in terms of calibration and ordinal ranking.
Proposes a curriculum-based scheme to smooth CNN feature embeddings.
problem Distortion artifacts in early training stages of CNNs.
method Smoothes feature embedding using Gaussian kernels to control high-frequency information.
result Significant performance improvements on various vision tasks.
The paper develops a theory for iterative self-improvement of models, proving conditions for better performance with easy-to-hard curricula.
problem Lack of theoretical foundation for iterative self-improvement in practical settings.
method Modeling self-improvement as maximum-likelihood fine-tuning on reward-filtered distributions and proving finite-sample guarantees.
result Explicit feedback loop and conditions for better performance with easy-to-hard curricula.
Instance-Dependent Near-Optimal Policy Identification in Linear MDPs via Online Experiment Designcs.LG
Algorithm extsc{Pedel} learns near-optimal policies efficiently on specific problems.
problem Learning near-optimal policies in linear MDPs with minimal samples.
method Online experiment design to focus exploration on relevant directions.
result Achieves instance-dependent complexity, outperforming minimax-optimal algorithms.
Recent progress in the field of reinforcement learning has been accelerated by virtual learning environments such as video games, where novel algorithms and ideas can be quickly tested in a safe and reproducible manner. We introduce the Google Research Football Environment, a new reinforcement learning environment wher…
New method identifies critical states to improve RL agent explainability and speed.
problem Challenges in RL agent explainability and action selection timing.
method Identify critical states based on action-based variance in Q-function, prioritize exploitation on these states.
result Identified critical states accelerate RL in grid worlds and deep RL tasks.
Variational autoencoders (VAEs) with an auto-regressive decoder have been applied for many natural language processing (NLP) tasks. The VAE objective consists of two terms, (i) reconstruction and (ii) KL regularization, balanced by a weighting hyper-parameter β. One notorious training difficulty is that the KL term ten…
Supervised learning from training data with imbalanced class sizes, a commonly encountered scenario in real applications such as anomaly/fraud detection, has long been considered a significant challenge in machine learning. Motivated by recent progress in curriculum and self-paced learning, we propose to adopt a semi-s…
This paper studies a theoretical pruning method for RNNs to reduce computational costs.
problem High computational costs in recurrent neural networks (RNNs).
method Spectral pruning inspired approach for RNNs.
result Generalization error bounds for compressed RNNs are provided.
In this article, we investigate when the set of primitive geodesic lengths on a Riemannian manifold have arbitrarily long arithmetic progressions. We prove that in the space of negatively curved metrics, a metric having such arithmetic progressions is quite rare. We introduce almost arithmetic progressions, a coarsific…
The paper explores the Thomas-Yau conjecture using holomorphic curves and Floer theory.
problem Proving the Thomas-Yau conjecture in the context of Lagrangian branes.
method Using holomorphic curves and Floer theory to construct and analyze bordism currents and moduli spaces.
result Established Floer theoretic obstructions and variational framework for finding special Lagrangians.
As integrated circuits have become progressively more complex, constrained random stimulus has become ubiquitous as a means of stimulating a designs functionality and ensuring it fully meets expectations. In theory, random stimulus allows all possible combinations to be exercised given enough time, but in practice with…
Inverse reinforcement learning (IRL) is the problem of inferring the reward function of an agent, given its policy or observed behavior. Analogous to RL, IRL is perceived both as a problem and as a class of methods. By categorically surveying the current literature in IRL, this article serves as a reference for researc…
Time-continuous emotion prediction has become an increasingly compelling task in machine learning. Considerable efforts have been made to advance the performance of these systems. Nonetheless, the main focus has been the development of more sophisticated models and the incorporation of different expressive modalities (…
Paper introduces a method to explain deep learning models and identify good generalization.
problem Limited interpretability of neural networks hinders progress and real-world applications.
method Polytope interpolation method for local explainability and generalization assessment.
result Developed a method to identify deep learning models with good generalization properties.
Recent progress in deep latent variable models has largely been driven by the development of flexible and scalable variational inference methods. Variational training of this type involves maximizing a lower bound on the log-likelihood, using samples from the variational posterior to compute the required gradients. Rec…
Lazy vs hasty: linearization in deep networks impacts learning schedule based on example difficultycs.LG
Deep networks prioritize easier examples over harder ones, leading to faster training.
problem Understanding how deep networks prioritize examples of varying difficulty.
method Investigated the effect of linear vs non-linear learning modes on example difficulty.
result Non-linear dynamics tend to sequentialize the learning of examples of increasing difficulty.
SentRNA: Improving computational RNA design by incorporating a prior of human design strategiesq-bio.QM
Solving the RNA inverse folding problem is a critical prerequisite to RNA design, an emerging field in bioengineering with a broad range of applications from reaction catalysis to cancer therapy. Although significant progress has been made in developing machine-based inverse RNA folding algorithms, current approaches s…
Survey of machine learning methods for Windows malware classification.
problem Difficulties in malware classification through data collection, labeling, feature creation, and selection.
method Review of current methods and challenges in malware classification.
result Discussion of constraints and unaddressed problems for machine learning in cybersecurity.
Joint Stochastic Approximation and Its Application to Learning Discrete Latent Variable Modelsstat.ML
New method uses joint stochastic approximation to improve learning of discrete latent models.
problem Challenges in learning discrete latent variable models, especially with inference model gradients and log-likelihood optimization.
method Proposes a new method based on stochastic approximation theory that directly maximizes the target log-likelihood and minimizes the posterior-inference model divergence.
result Consistently outperforms recent competitive algorithms in generative modeling and structured prediction tasks.
A new method called TemperFlow tackles multimodality in sampling from unnormalized distributions.
problem Sampling from unnormalized distributions with isolated modes.
method TemperFlow learns a sequence of tempered distributions to progressively approach the target distribution.
result TemperFlow overcomes the limitations of existing methods and achieves superior performance.
Dynamic networks are a general language for describing time-evolving complex systems, and discrete time network models provide an emerging statistical technique for various applications. It is a fundamental research question to detect the community structure in time-evolving networks. However, due to significant comput…
Which factor dominates the industry evolution? A synergy analysis based on China's ICT industryphysics.soc-ph
Industry evolution caused by various reasons, among which technology progress driving industry development has been approved, but with the new trend of industry convergence, inter-industry convergence also plays an increasing important role. This paper plans to probe the industry synergetic evolution mechanism based on…
Paper introduces a new curriculum generation method for reinforcement learning.
problem Improving reinforcement learning performance and speed through curriculum learning.
method The paper proposes a novel curriculum generation paradigm based on progression and mapping functions.
result Empirical results show the new approach outperforms state-of-the-art algorithms.
Lie groups applied to tech progress in economic growth.
problem Understanding the impact of technical progress on economic growth.
method Application of Lie theory and economic modeling.
result Estimation of GDP function for Viet Nam, highlighting tech progress impact.
DKLM learns adaptive kernels for robust nonlinear subspace clustering.
problem Nonlinear structures in data and challenges with kernel-based clustering.
method Data-driven kernel learning with adaptive weighting and optimal block-diagonal affinity matrix.
result DKLM enhances robustness and preserves manifold structure in nonlinear space.
SpatialSim benchmarks machine learning in recognizing object spatial configurations.
problem Machine learning in recognizing precise geometrical configurations of groups of objects.
method SpatialSim benchmark with tasks of Identification and Comparison, using Graph Neural Networks (MPGNNs).
result MPGNNs outperform baselines in recognizing spatial configurations, highlighting current limits.
This paper improves disentanglement in VAEs by progressively learning hierarchical representations.
problem Compromised disentanglement in VAEs due to high-level abstraction extraction.
method Progressive learning of independent hierarchical representations from high to low levels.
result Improved disentanglement demonstrated on two benchmark datasets using new metrics.
Paper presents a GAN model for realistic river image synthesis.
problem Generating high-quality river images for hydrological research.
method Used a Progressive Growing GAN (PGGAN) architecture to overcome training challenges.
result Demonstrated the effectiveness of GANs in generating high-resolution river images.
Bayesian model identifies health disparities in disease progression.
problem Health disparities bias disease progression models.
method Interpretable Bayesian model accounting for three disparities.
result Model identifies and corrects for health disparities.
New algorithm outperforms existing ones by focusing on mastering rate.
problem Low sample efficiency in learning progress-based algorithms.
method Proposes a new algorithm based on mastering rate.
result Significantly outperforms learning progress-based algorithms.
Some statistical models are specified via a data generating process for which the likelihood function cannot be computed in closed form. Standard likelihood-based inference is then not feasible but the model parameters can be inferred by finding the values which yield simulated data that resemble the observed data. Thi…
Feature Quantization improves GAN training stability.
problem Stability issues in GAN training.
method Feature Quantization (FQ) for the discriminator, embedding true and fake data into a shared discrete space.
result FQ-GAN achieves new state-of-the-art performance on various GAN tasks.