This work uses QPGPs to improve ILC performance in repetitive tasks.
problem Performance degradation in repetitive motion tasks due to environmental changes and robot wear.
method Incorporates Quasi-Periodic Gaussian Processes into a predictive ILC framework.
result The proposed approach achieves faster convergence and robustness under disturbances.
Generative model captures repetitive industrial processes with varying durations and dynamics.
problem Capturing repetitive industrial processes with varying durations and dynamics using Gaussian Processes.
method Posterior-weighted Gaussian Process with a novel kernel to decouple intra-repetition and inter-repetition variability.
result Generative model produces realistic synthetic trajectories from toy datasets.
Bayesian optimization outperforms other methods in hyperparameter tuning for reinforcement learning.
problem Finding optimal hyperparameters that generalize across random seeds in reinforcement learning.
method Benchmarked Successive Halving, Random Search, and Bayesian Optimization with and without repetitions on PPO2 algorithms for Cartpole and Inverted Pendulum tasks.
result Bayesian optimization with noise robust acquisition function is the best choice.
This study investigates abrupt learning dynamics in Transformers, revealing plateau formation and internal representation collapse.
problem Abrupt learning in Transformers, particularly during the loss plateau.
method Investigates mechanisms of abrupt learning in shallow Transformers, focusing on attention maps and hidden states.
result Reveals plateau formation, internal representation collapse, and strong repetition bias in outputs.
The paper uses persistent homology to estimate recurrence times in multi-variate time series.
problem Estimating recurrence times in multi-variate time series with different cyclic behaviors.
method Persistent homology framework with three specialized methods.
result Validated methods on real-world data, including a new benchmark dataset.
This paper considers the problem of characterizing the simplest discrete point sets that are aperiodic, using invariants based on topological dynamics. A Delone set whose patch-counting function N(T), for radius T, is finite for all T is called repetitive if there is a function M(T) such that every ball of radius M(T)+…
UTE improves reinforcement learning by measuring action uncertainty, enhancing policy learning efficiency.
problem Degrading performance of action repetition in reinforcement learning, especially with sub-optimal actions.
method UTE uses ensemble methods to measure uncertainty during action extension, allowing strategic exploration or certainty.
result UTE outperforms existing action repetition algorithms, significantly enhancing policy learning efficiency.
Telescope detects LLM generated text by measuring token repetition probability.
problem Distinguishing LLM generated text from human writing.
method Telescope Perplexity, evaluating token repetition probability.
result Telescope Perplexity enables effective zero-shot LLM detection.
Optimizes learning schedules for better memory retention.
problem Finding the best review schedule for spaced repetition.
method Flexible representation of spaced repetition using marked temporal point processes and optimal control for stochastic differential equations with jumps.
result Optimal reviewing schedule is the recall probability of content.
CorrCA identifies reliable dimensions in multivariate data across repetitions.
problem Finding consistent dimensions in multivariate data across trials, subjects, or raters.
method Maximizes the ratio of between-repetition to within-repetition covariance.
result CorrCA leads to repeat-reliability maximization and is equivalent to Linear Discriminant Analysis for zero-mean signals.
Meta-learning agents forget tasks; this work creates a memory system to help them remember.
problem Meta-learning agents forget previously learned tasks when new ones appear.
method Developed a meta-learning architecture that combines LSTM and a differentiable neural episodic memory.
result Agents with the episodic LSTM architecture can solve tasks more effectively when they recur.
Study shows how repetition affects learning in bandit settings, providing algorithms with sublinear regret.
problem Effect of persistence of engagement on learning in stochastic multi-armed bandit settings.
method Novel algorithms that achieve sublinear regret under temporal constraints.
result Additive effect of priming on regret upper bound, matching popular algorithms in absence of priming.
This paper improves the accuracy of estimating parameters from experiments using D-optimality criterion.
problem Estimating an unknown vector β from linear measurements with Gaussian noise. method Developed approximation algorithms for D-optimal design problems with and without repetitions. result First constant factor approximation for D-optimal design problems. Proposes a new framework for semi-supervised learning with theoretical support.
problem Lack of theoretical support for using predictions as pseudo-labels in deep SSL methods.
method D2 framework with repetitive reprediction (R2) strategy.
result R2-D2 method outperforms state-of-the-art methods by 5 percentage points on ImageNet.
A new neural network separates vocals from music accompaniment.
problem Separating vocals from music accompaniment in recordings.
method Self-attention convolutional neural network (CNN) with densely-connected blocks.
result 19.5% relative improvement in vocals separation.
This paper evaluates AutoML tools for ML tasks.
problem Efficiency in machine learning for ML engineers.
method Evaluation and comparison of AutoML tools on various datasets.
result Performance and advantages/disadvantages of AutoML tools.
Game aims to improve social interactions for teenagers with ASD.
problem Improving social interactions for teenagers with ASD.
method Presentation of game structure, skeleton detection, and imitation learning methods.
result Game structure and support tools for skeleton detection and imitation learning presented.
DAS3H optimizes skill-based spaced repetition schedules.
problem Optimizing adaptive and personalized spaced repetition schedules for skill-based learning.
method Developed a new student learning and forgetting model (DAS3H) that considers memory decay and multiple skills.
result DAS3H outperforms other models on real-world educational datasets.
Improves generative Visual Dialog by asking diverse questions.
problem Generative Visual Dialog models degrade after a few rounds of interaction.
method Introduce a simple auxiliary objective to incentivize Qbot to ask diverse questions.
result Better dialog diversity, consistency, fluency, and detail with improved image relevance.
New method improves text generation by assigning lower probability to repetitive sequences.
problem Dull and repetitive text generated by standard likelihood training.
method Unlikelihood training, which forces unlikely generations to be assigned lower probability.
result Unlikelihood training produces less repetitive, less dull text compared to standard techniques.
Iterative method learns unknown constraints for MPC control.
problem Learning to satisfy unknown polyhedral state constraints in iterative MPC.
method Collects and improves estimates of unknown constraints using collected data, designs an MPC controller to satisfy the estimated constraints.
result Robust and probabilistic guarantees of constraint satisfaction as a function of task iterations.
New method detects conflicts in police body-worn audio.
problem Poor metrics for conflict detection in police interactions.
method Adaptive noise removal, non-speech filtering, and new phrase-based measures.
result Demonstrated effectiveness on LAPD body-worn audio data.
LLMs are compared to Markov chains for natural language processing.
problem Theoretical analysis of LLMs' generalization capabilities.
method Equivalence between LLMs and Markov chains, studying multi-step inference.
result Derives generalization bounds for LLMs, capturing their behavior in practice.
Tiny episodic memory significantly improves continual learning performance.
problem Transfer knowledge between tasks in continual learning.
method Store and replay a small number of examples from previous tasks.
result A simple baseline outperforms CL approaches with episodic memory.
Data repetition improves SGD's learning of high-dimensional functions.
problem Learning pertinent features in multi-index models with high-dimensional noisy data.
method Investigation of two-layer shallow neural networks trained with gradient-based algorithms, focusing on data repetition.
result Data repetition significantly improves the computational efficiency of SGD, learning all directions with at most O(dlogd) steps. Study proposes an ad creative selection algorithm considering user fatigue.
problem Ad creative effectiveness varies with user exposure.
method Proposes an algorithm that considers user fatigue in ad selection.
result The proposed algorithm outperforms baseline algorithms in real-world testing.
Improved few-shot visual reasoning with image preprocessing.
problem Few-shot classifiers struggle with abstract visual reasoning tasks.
method Spectral feature removal to emphasize unique image parts.
result Combining spectral preprocessing with Relational Networks improves accuracy nearly 40%.
Estimates linear model from noisy covariates and instruments using spectral regularization.
problem Estimating a linear model from many noisy covariates and instruments.
method Two-stage least squares with spectral regularization of canonical correlations.
result Upper and lower bounds on estimation error, proving optimality of the method with noisy data.
Task-adaptive clustering improves few-shot learning with unlabeled data.
problem Handling unseen tasks with limited labeled data.
method Task-conditioned clustering in a new projection space.
result State-of-the-art semi-supervised few-shot classification performance.
Determinantal point processes (DPPs), which arise in random matrix theory and quantum physics, are natural models for subset selection problems where diversity is preferred. Among many remarkable properties, DPPs offer tractable algorithms for exact inference, including computing marginal probabilities and sampling; ho…
A new method optimizes neural sequence models for better task performance.
problem Training neural sequence models with maximum likelihood estimation ignores task losses.
method Maximum likelihood guided parameter search (MGS) in the parameter space.
result MGS optimizes sequence-level losses, reducing repetition and non-termination.
Study improves understanding of what makes machine learning explanations human-interpretable.
problem Understanding what makes explanations human-interpretable in machine learning systems.
method Controlled human-subject experiments to identify regularizers for interpretability across three tasks.
result Cognitive chunks affect performance more than variable repetitions, suggesting common design principles.
Neural Bootstrapper reduces bootstrapping cost for deep neural networks.
problem Computational burden in bootstrapping deep neural networks.
method Neural Bootstrapper learns to generate bootstrapped neural networks through single model training.
result Neural Bootstrapper outperforms bagging methods with lower computational cost.
We introduce COCO, an open source platform for Comparing Continuous Optimizers in a black-box setting. COCO aims at automatizing the tedious and repetitive task of benchmarking numerical optimization algorithms to the greatest possible extent. The platform and the underlying methodology allow to benchmark in the same f…
PG-IM uses neural-symbolic programs to manipulate images.
problem Creating holistic image representations and manipulations.
method PG-IM detects patterns, induces symbolic programs, and manipulates images using a neural network.
result PG-IM learns from a single image and achieves superior performance.
Introduces Motion Programs for better video analysis of human motion.
problem Current video analysis focuses on raw pixels or keypoints, missing higher-level motion primitives.
method Introduces Motion Programs as a neuro-symbolic representation of motions as a composition of high-level primitives.
result Motion Programs accurately describe diverse human motions and improve downstream tasks.
Project infinite time series graphs to finite marginal models using number theory.
problem Handling infinite time series graphs for causal inference.
method Projection method using number theory to find common ancestors in infinite graphs.
result Developed algorithm to project infinite graphs to finite marginal models.
Improved rates for continual learning using SGD and last-iterate analysis.
problem Forgetting in overparameterized models after fitting multiple tasks.
method Developed novel SGD upper bounds for continual linear models and analyzed their performance.
result Established universal forgetting rates for continual learning.
Paper proposes RCD method to discover causal structure with latent confounders.
problem Causal discovery from data with latent confounders.
method Repetitive causal discovery (RCD) method to infer causal directions between observed variables.
result RCD effectively identifies latent confounders and causal directions between observed variables.
This paper guides practical solutions for neural text generation issues.
problem Undesired behavior in neural text generation models.
method Tuning end-to-end neural network models with encoder and decoder components.
result Resolving issues like truncated, repetitive, bland, and ungrammatical outputs.
New algorithms mitigate impairment effect in stochastic bandits.
problem Impairment effect in stochastic multi-armed bandits.
method Developed two novel algorithms with bucketing to address impairment and temporal constraints.
result Achieved sublinear regret, explicitly capturing impairment cost.
Variance-Calibrated Modulation (VCM) addresses the likelihood trap in LLMs by reshaping the probability distribution before truncation.
problem LLMs fall into the likelihood trap, leading to repetitive degeneration and vocabulary dullness.
method VCM reshapes the probability distribution before truncation through Contextual Searchlight and Adaptive Self-Debiasing.
result VCM mitigates the likelihood trap across open-ended generation, factual QA, and mathematical reasoning.
Machine learning predicts x-ray pulse properties from XFEL parameters.
problem Characterizing XFEL pulses for sorting data due to large fluctuations.
method Applied machine learning to predict x-ray pulse properties using electron beam and x-ray parameters.
result Mean errors below 0.3 eV for photon energy and below 1.6 fs for delay between pulses at 530 eV.
Riemannian manifolds can be realized as leaf spaces of matchbox manifolds.
problem Realizing Riemannian manifolds as leaf spaces of matchbox manifolds.
method Graph coloring techniques to prove realization of manifolds as leaves.
result Any repetitive Riemannian manifold of bounded geometry can be realized as a leaf of a minimal Riemannian matchbox manifold without holonomy.
HRL improves open-domain dialog models by optimizing long-term conversational goals.
problem Challenges in open-domain dialog generation, including repetitive outputs, difficulty tracking conversational goals, and inappropriate text.
method Proposes VHRL, a hierarchical reinforcement learning approach using policy gradients to tune utterance-level embeddings of a variational sequence model.
result Significant improvements in human evaluation and automatic metrics over state-of-the-art dialog models.
NAS improves blockchain-based cryptocurrency predictions.
problem Efficiently designing and optimizing neural networks for cryptocurrency prediction.
method Customized Neural Architecture Search (NAS) with network morphism and Bayesian optimization.
result NAS algorithms can achieve results comparable to manually designed models.
Extends V-IP framework to use LLMs for generating task-relevant concepts, improving interpretability and performance.
problem Limited applicability of V-IP to small-scale tasks due to manual data annotation.
method Integrates Foundational Models with Large Language and Multimodal Models to generate and annotate concepts.
result FM+V-IP achieves better test performance with fewer concepts/queries compared to other frameworks.
Characterizes Milnor invariants with limited repetitions.
problem Understanding Milnor invariants with constraints on repetition.
method Algebraic and diagrammatic approaches using k-reduced free groups and welded knot theory. result Algebraic and diagrammatic characterizations of Milnor invariants.