New technique improves imitation learning by preventing local minima and exploring states.
problem Behavioral cloning gets stuck in local minima and lacks effective exploration.
method Two-phase model with sampling mechanisms and self-attention modules.
result Significantly outperforms previous state-of-the-art in various environments.
The Exploration-Exploitation tradeoff arises in Reinforcement Learning when one cannot tell if a policy is optimal. Then, there is a constant need to explore new actions instead of exploiting past experience. In practice, it is common to resolve the tradeoff by using a fixed exploration mechanism, such as ε-greedy ex…
This paper investigates the effectiveness of adversarial training in enhancing the robustness of Deep Q-Network (DQN) policies to state-space perturbations. We first present a formal analysis of adversarial training in DQN agents and its performance with respect to the proportion of adversarial perturbations to nominal…
Survey explores cohomology's roles in applied math and sciences.
problem Understanding cohomology's role in solving differential equations.
method Examining differential complexes and structure-preserving discretizations.
result Various fundamental concepts in mechanics are formulated using differential complexes.
A new method boosts exploration in bandit algorithms, reducing regret.
problem Improving exploration in bandit algorithms with bounded or unbounded rewards.
method Residual Bootstrap Exploration (ReBoot) method that injects data-driven randomness.
result Proves logarithmic regret in Gaussian multi-armed bandits with appropriate variance inflation.
Expands differential privacy mechanisms to include the Generalized Gaussian mechanism for improved private machine learning.
problem Improving privacy in machine learning algorithms while maintaining utility.
method Introduces and analyzes the Generalized Gaussian (GG) mechanism for differential privacy.
result The GG mechanism provides better performance than the Laplace and Gaussian mechanisms across various values of β.
In this paper, we demonstrate how to do automated theorem proving in the presence of a large knowledge base of potential premises without learning from human proofs. We suggest an exploration mechanism that mixes in additional premises selected by a tf-idf (term frequency-inverse document frequency) based lookup in a d…
This work uses statistical mechanics to explain AI learning.
problem Understanding the statistical principles behind AI learning.
method Starting from sample concentration behaviors, the study applies statistical mechanics principles to AI and machine learning.
result Exponential families and statistical quantities are key in AI and machine learning.
Study symmetry breaking in quantum mechanics to understand many-body physics.
problem Understanding many-body physics from quantum mechanics.
method Analyzing potentials with unstable critical points and local minima.
result Emergence of many-body physics from spontaneous symmetry breaking.
Study explores grokking in neural networks, revealing transition from memorization to generalization.
problem Understanding the transition from memorization to generalization in over-parameterized neural networks.
method Extensive experiments and exploration of various viewpoints on grokking mechanism.
result Sharp transition from no generalization to perfect generalization observed during prolonged training.
Two-stage mechanism designs reduce regret in recommender systems with stochastic covariates.
problem Designing effective recommender systems with user covariates sampled online.
method Two-stage algorithm integrating incentivized exploration with offline learning methods.
result Achieves sublinear regret while maintaining incentive compatibility.
This paper studies the potential of the return distribution for exploration in deterministic reinforcement learning (RL) environments. We study network losses and propagation mechanisms for Gaussian, Categorical and Gaussian mixture distributions. Combined with exploration policies that leverage this return distributio…
Paper explores Bayes rule for Gaussian mixtures with missing data, outperforming supervised classifiers.
problem Improving classification accuracy in partially classified samples with missing data.
method Generative model framework with missing-data mechanism, Bayes rule allocation.
result Bayes rule classifier with missing-data mechanism outperforms fully supervised classifiers in various conditions.
New integrators for mechanical systems on Lie groups simplify based on group properties.
problem Designing numerical integrators for mechanical systems on Lie groups.
method Leverage retraction maps and Lie group properties to design structure-preserving integrators.
result Simplified design of integrators for Euler-Poincare and Lie-Poisson equations.
The paper explores invariant subbundles in nonholonomic mechanics.
problem Determining invariant affine subbundles in nonholonomic and constrained variational mechanics.
method Using Spencer cohomology and iterative formulae, the paper formalizes the integrability of linear partial differential equations and determines the largest invariant affine subbundle.
result Iterative formulae for determining the largest invariant affine subbundle are provided.
Partial Differential Equations (PDE) are fundamental to model different phenomena in science and engineering mathematically. Solving them is a crucial step towards a precise knowledge of the behaviour of natural and engineered systems. In general, in order to solve PDEs that represent real systems to an acceptable degr…
A new algorithm balances exploration and exploitation in online decision-making.
problem Balancing exploration and exploitation in online decision-making.
method Proposed C4-UCB algorithm incorporating conservative mechanism. result Proved n-step upper regret bound for two situations.
The classical tools which ensure the completeness of vector fields and second order differential equations for mechanical systems are revisited. Possible extensions in three directions are discussed: infinite dimensional Banach and Hilbert manifolds, Finsler metrics and pseudo-Riemannian spaces, including links with so…
AttViz offers online visualizations of neural language model attention mechanisms.
problem Limited interpretability of neural language models.
method Online toolkit for exploring self-attention mechanisms.
result Visualizations help understand model decision-making.
LNUCB-TA improves MAB performance by dynamically adjusting exploration rates and recognizing spatiotemporal patterns.
problem Suboptimal performance in environments with rapidly changing reward structures and static exploration rates.
method Hybrid model combining linear and nonlinear estimation, with adaptive k-NN for temporal attention.
result Significantly outperforms state-of-the-art algorithms in cumulative and mean reward, convergence, and robustness.
New RL approach learns dynamic VCG mechanisms in unknown MDP environments.
problem Learning dynamic VCG mechanisms in unknown MDP environments.
method Reward-free online RL for exploration, combined with function approximation.
result Regret bound of O~(T2/3) for dynamic VCG mechanism learning. The abstract explores a new wave equation linking quantum mechanics and complex adaptive systems.
problem Understanding the underlying mechanism of distribution formation in complex quantum entanglement.
method Exploring the logical relationship between Schrödinger's wave equation and Shi's trading volume-price wave equation in finance.
result A non-localized wave equation in quantum mechanics reveals the invariance of interaction as a universal law.
Paper develops efficient mechanisms for estimating variance and covariance under differential privacy in the add-remove model.
problem Estimating variance and covariance under differential privacy in the add-remove model.
method Developed mechanisms based on the Bézier mechanism, a novel moment-release framework.
result Proved minimax optimality of the Bézier-based estimator in the high-privacy regime and demonstrated its better utility in instance-wise analysis.
RP1 uses active learning to improve world model in fewest samples.
problem Improving sample efficiency in MBRL for continuous control tasks.
method RP1 views MBRL as active learning, using a hybrid objective function and principled termination.
result Statistically significant gains over existing approaches on continuous control tasks.
DiCE uses diverse agents to explore and learn, avoiding local minima.
problem Local minima in RL due to limited exploration and correlated behavior.
method DiCE employs a group of heterogeneous agents to explore simultaneously and share experiences, with a diversity regularization mechanism.
result DiCE achieves substantial improvement over baselines in MuJoCo locomotion tasks.
A novel incentive mechanism improves fairness and participation in federated learning.
problem Low-quality clients and lack of fairness in federated learning.
method Client selection process and money transfer mechanism to ensure fairness and participation.
result The proposed incentive mechanism improves the duration and fairness of federated learning.
First derived from human intuition, later adapted to machine translation for automatic token alignment, attention mechanism, a simple method that can be used for encoding sequence data based on the importance score each element is assigned, has been widely applied to and attained significant improvement in various task…
Improved risk-sensitive RL with exponential Bellman equation and better regret bounds.
problem Exponential gap between upper and lower bounds in risk-sensitive RL.
method Identified and addressed deficiencies in existing algorithms and analysis; developed novel analysis and exploration mechanism.
result Improved regret upper bounds over existing ones.
Paper develops a framework to discover bioprocessing regulatory mechanisms using symbolic and statistical learning.
problem Challenges in modeling complex intracellular regulation, stochastic system behavior, and limited experimental data.
method Symbolic and statistical learning framework based on stochastic differential equations and Bayesian learning.
result Improved sample efficiency and robust model selection compared to state-of-the-art approaches.
Self-regulating annealing improves sampling from heavy-tailed datasets.
problem Sampling from heavy-tailed distributions using diffusion models.
method Proposed an SDE-based sampler with a state-dependent diffusion coefficient.
result State dependence induces a self-regulating annealing mechanism.
New acquisition function for extreme rewards in bandits.
problem Online decision making with extreme payoffs in multi-armed bandits.
method Modeling payoffs as Gaussian processes and using a novel UCB acquisition function.
result Demonstrated benefits across synthetic and real-world benchmarks.
In this paper, we explore dynamics of the nonholonomic system called vakonomic mechanics in the context of Lagrange-Dirac dynamical systems using a Dirac structure and its associated Hamilton-Pontryagin variational principle. We first show the link between vakonomic mechanics and nonholonomic mechanics from the viewpoi…
This paper tackles time series imputation by identifying and modeling different missing mechanisms.
problem Different types of missing mechanisms (MAR, MNAR) in time series data.
method Proposes a framework for time series imputation by analyzing data generation processes and modeling latent variables via variational inference and normalizing flow.
result Establishes identifiability results for latent variables under nonlinear independent component analysis, showing that latent variables are identifiable.
Mechanism designs for unknown agent values in stochastic bandit settings.
problem Designing truthful mechanisms for maximizing social welfare in settings with unknown agent values and stochastic feedback.
method Developed a VCG-like mechanism with regret bounds for multi-round allocations, balancing agent and seller welfare.
result Achieved an $Ω(T^{ rac{2}{3}})$ lower bound for the maximum of welfare, agent utilities, and mechanism utility after T rounds. Meta-learning curiosity algorithms improves exploration across various tasks.
problem Generating curious behavior in reinforcement learning.
method Meta-learning approach to adapt reward signals dynamically.
result Two novel curiosity algorithms outperform human-designed ones.
Study explores strategies for randomized allocation in delayed rewards bandits.
problem Understanding the exploration-exploitation tradeoff in randomized strategies with delayed rewards.
method Examines two strategies: updating exploration sequence at every time point vs. updating only when a new reward is observed.
result The strategy updating only when a new reward is observed leads to strong consistency in allocation for a wider scope of situations.
In this work, we explore the impact of visual modality in addition to speech and text for improving the accuracy of the emotion detection system. The traditional approaches tackle this task by fusing the knowledge from the various modalities independently for performing emotion classification. In contrast to these appr…
Recent years have witnessed a tremendous improvement of deep reinforcement learning. However, a challenging problem is that an agent may suffer from inefficient exploration, particularly for on-policy methods. Previous exploration methods either rely on complex structure to estimate the novelty of states, or incur sens…
Deep generative neural networks (DGNNs) have achieved realistic and high-quality data generation. In particular, the adversarial training scheme has been applied to many DGNNs and has exhibited powerful performance. Despite of recent advances in generative networks, identifying the image generation mechanism still rema…
New strategies avoid forced exploration for unimodal bandits.
problem Optimal exploration of unimodal bandit problems.
method Adapting IMED strategy to unimodal structure without forced exploration.
result Proven optimal strategies for unimodal bandits.
Self-supervision improves GCNs' generalizability and robustness.
problem Improving graph convolutional networks' performance.
method Three mechanisms of self-supervision, multi-task learning, and graph adversarial training.
result Self-supervision enhances GCNs' robustness and generalizability.
Proposes a new imputation method using autoencoders and feedback mechanisms.
problem Missing data undermines the performance of downstream data products.
method Multiple Imputation with Denoising Autoencoders and Metamorphic Truth/Imputation Feedback.
result Outperforms other methods in various missingness mechanisms and data patterns.
In many environments, only a relatively small subset of the complete state space is necessary in order to accomplish a given task. We develop a simple technique using emergency stops (e-stops) to exploit this phenomenon. Using e-stops significantly improves sample complexity by reducing the amount of required explorati…
Adversarial training is one of the strongest defenses against adversarial attacks, but it requires adversarial examples to be generated for every mini-batch during optimization. The expense of producing these examples during training often precludes adversarial training from use on complex image datasets. In this study…
In this work, we study abstractive text summarization by exploring different models such as LSTM-encoder-decoder with attention, pointer-generator networks, coverage mechanisms, and transformers. Upon extensive and careful hyperparameter tuning we compare the proposed architectures against each other for the abstractiv…
A new variational method speeds up Bayesian phylogenetic inference.
problem Slow and inefficient MCMC methods in Bayesian phylogenetic inference.
method Combining subsplit Bayesian networks with variational inference for efficient tree topology and branch length estimation.
result Variational approach provides competitive performance with significantly fewer iterations.
Investigates the benefits of multi-head attention in Transformers, deriving convergence and generalization guarantees.
problem Underexplored dynamics of multi-head attention in Transformer training and generalization.
method Derives convergence and generalization guarantees for gradient-descent training of a multi-head self-attention model.
result Establishes conditions for initialization that ensure multi-head attention's realizability.
The paper develops an economic foundation for multi-agent learning in markets.
problem Learning dynamics in markets with strategic externalities.
method A two-phase incentive mechanism that estimates and uses implementable transfers to steer long-run dynamics.
result The mechanism achieves sublinear social-welfare regret and asymptotically optimal welfare under mild rationality and exploration conditions.