Paper tackles continual reinforcement learning challenges with diversity exploration and adversarial self-correction.
problem Challenges in learning new tasks sequentially due to catastrophic forgetting.
method Develops an end-to-end framework (CDAN) combining unsupervised diversity exploration and adversarial self-correction.
result Final result outperforms baseline by 18.35% in NSD and 0.61 in average reward.
Sparse reward is one of the biggest challenges in reinforcement learning (RL). In this paper, we propose a novel method called Generative Exploration and Exploitation (GENE) to overcome sparse reward. GENE automatically generates start states to encourage the agent to explore the environment and to exploit received rew…
CAM-GAN improves GANs for continual learning with efficient feature map transformations.
problem Efficient continual learning for GANs with reduced parameter growth.
method Designing and leveraging parameter-efficient feature map transformations, including global and task-specific parameters, residual bias, and Fisher information matrix.
result Significantly improved model performance and high-quality samples with fewer parameters.
Deep network solves maze path planning without training.
problem Efficient path planning in large mazes with obstacles.
method Max pooling layers without training.
result Solves mazes with over half a billion nodes in short time.
PBCS combines RL and motion planning for better exploration.
problem RL algorithms struggle with versatile exploration in complex environments.
method PBCS uses motion planning to find a good trajectory, then trains RL on a curriculum derived from it.
result PBCS outperforms state-of-the-art RL algorithms in 2D maze environments.
MAZE attacks a model without data, using synthetic inputs.
problem Model Stealing attacks that require data to replicate a target model.
method MAZE uses zeroth-order gradient estimation to create synthetic inputs.
result MAZE achieves high clone accuracy without data.
Model learns evolving network relationships over time.
problem Dimension reduction for dynamic network data.
method Metric functional space for vector-valued functions.
result Effective link prediction and role identification in dynamic networks.
Study of ants' movement rules on a 6D space, revealing distribution structures and singular trajectories.
problem Understanding the movement patterns of ants in a 6D space.
method Analyzing mechanical system rules to derive distribution structures and singular trajectories.
result Distributions and singular trajectories of ants' movement rules in a 6D space.
ANT learns sparse embeddings for large vocabularies efficiently.
problem Lack of scalable methods for embedding large vocabularies in neural networks.
method Anchor & Transform (ANT) algorithm that learns a small set of anchor embeddings and a sparse transformation matrix.
result ANT achieves stronger performance with fewer parameters (up to 40x compression) compared to existing methods.
Blind Source Separation (BSS) has proven to be a powerful tool for the analysis of composite patterns in engineering and science. We introduce Convex Analysis of Mixtures (CAM) for separating non-negative well-grounded sources, which learns the mixing matrix by identifying the lateral edges of the convex data scatter p…
Deep learning has achieved remarkable successes in solving challenging reinforcement learning (RL) problems when dense reward function is provided. However, in sparse reward environment it still often suffers from the need to carefully shape reward function to guide policy optimization. This limits the applicability of…
A new framework for robust transfer learning that avoids negative transfer in domains with unequal information.
problem Negative transfer in unsupervised domain adaptation, especially when source and target domains have different levels of informativeness.
method Decision-theoretic framework based on Le Cam's theory of statistical experiments, using constructive approximations to replace strict invariance with directional simulability.
result Le Cam Distortion achieves near-perfect frequency estimation and zero source utility loss in various domains, demonstrating superior performance compared to traditional methods.
We propose a technique for making Convolutional Neural Network (CNN)-based models more transparent by visualizing input regions that are 'important' for predictions -- or visual explanations. Our approach, called Gradient-weighted Class Activation Mapping (Grad-CAM), uses class-specific gradient information to localize…
Ant colonies and boosting algorithms both reduce bias and variance through adaptive mechanisms.
problem Understanding the mathematical principles behind ensemble learning and ant colony behavior.
method Developed a formal mapping between AdaBoost's adaptive reweighting and ant recruitment dynamics.
result Proved that the fundamental theorem of weak learnability has a direct analog in colony decision-making.
Ant colony optimization for clustering with improved K-means.
problem Clustering a set of objects to minimize intra-variance.
method Constructive approach using ant-like solutions with visibility and pheromone trails, improved with K-means.
result Encouraging results in nearly all benchmark real-data sets.
NetDP predicts loan defaults using network data, addressing cold-start issues.
problem Cold-start problem in default prediction for new users.
method Combines unsupervised and supervised network representations, using parameter-server for scalability.
result Effectiveness in cold-start problem, especially for new users.
Logical neural networks solve mazes by filling dead ends, but not all methods generalize well.
problem Understanding how logical neural networks extrapolate solutions to mazes.
method Examined recurrent and implicit neural networks trained on maze-solving tasks.
result Models fail to generalize well to diverse maze sizes, suggesting limitations in learning scalable algorithms.
The paper identifies regions where investment strategies match expected performance.
problem Inconsistent performance of Markowitz efficient portfolios.
method Density forecasting to measure ex-ante accuracy and identify the consistency region.
result Investment strategies based on consistent portfolios outperform efficient ones.
Enhanced visibility forecasts using CAMS data improve accuracy.
problem Improving the accuracy of visibility predictions in weather forecasts.
method Statistical post-processing with historical observations and CAMS forecasts.
result Post-processed forecasts with CAMS data are substantially superior to raw and climatological predictions.
This paper presents a problem of model learning for the purpose of learning how to navigate a ball to a goal state in a circular maze environment with two degrees of freedom. The motion of the ball in the maze environment is influenced by several non-linear effects such as dry friction and contacts, which are difficult…
HRM-Agent learns to navigate dynamic mazes using reinforcement learning.
problem Training HRM in dynamic, uncertain, partially observable environments.
method Reinforcement learning to train HRM-Agent.
result HRM-Agent successfully learns to navigate dynamic mazes.
Functional groups (FGs) are molecular substructures that are served as a foundation for analyzing and predicting chemical properties of molecules. Automatic discovery of FGs will impact various fields of research, including medicinal chemistry and material sciences, by reducing the amount of lab experiments required fo…
TitAnt detects online transaction fraud in milliseconds.
problem Real-time detection of online transaction fraud in e-commerce.
method Feature extraction, real-time prediction, deployment in Ant Financial.
result Efficient real-time fraud detection in milliseconds.
This paper improves reinforcement learning policies in a scalable way.
problem Ensuring monotonic policy improvement in entropy-regularized RL.
method Derives an entropy-aware lower bound and proposes a novel RL algorithm.
result Demonstrates effectiveness in continuous-state tasks using a linear function approximator.
In this paper we approach two relevant deep learning topics: i) tackling of graph structured input data and ii) a better understanding and analysis of deep networks and related learning algorithms. With this in mind we focus on the topological classification of reachability in a particular subset of planar graphs (Maze…
We introduce a conceptually simple and scalable framework for continual learning domains where tasks are learned sequentially. Our method is constant in the number of parameters and is designed to preserve performance on previously encountered tasks while accelerating learning progress on subsequent problems. This is a…
LAAT detects multiple low-density manifolds in noisy data.
problem Detecting multiple low-density manifolds in noisy data.
method Locally Aligned Ant Technique (LAAT) based on Ant Colony Optimization.
result LAAT recovers multiple manifolds in extremely noisy data.
This paper investigates how realized and option implied volatilities are related to the future quantiles of commodity returns. Whereas realized volatility measures ex-post uncertainty, volatility implied by option prices reveals the market's expectation and is often used as an ex-ante measure of the investor sentiment.…
Deep Reinforcement Learning has shown tremendous success in solving several games and tasks in robotics. However, unlike humans, it generally requires a lot of training instances. Trajectories imitating to solve the task at hand can help to increase sample-efficiency of deep RL methods. In this paper, we present a simp…
ANT improves TS diffusion models by automatically determining noise schedules.
problem Suboptimal performance of TS diffusion models due to lack of domain-specific noise schedules.
method ANT proposes an adaptive noise schedule that automatically determines proper noise schedules for TS datasets based on their statistics.
result ANT achieves state-of-the-art performance on various TS tasks, including forecasting, refinement, and generation.
Extends causal additive models to include higher-order interactions.
problem Inferring causal insights from data with higher-order mechanisms.
method Introduces directed acyclic hypergraphs to represent higher-order interactions in causal structure learning.
result Learning more complex hypergraphs can lead to better empirical results.
New RL approach builds short ancestral recombination graphs.
problem Building short ancestral recombination graphs (ARGs).
method Reinforcement Learning applied to genetic sequences.
result RL can build ARGs as short as heuristic algorithms.
Study exact minimax rates for density estimation over convex classes, extending previous work.
problem Deriving minimax rates for density estimation over convex density classes.
method Building on Le Cam's work, determine exact minimax rates using local metric entropy.
result Exact minimax rates derived for any convex density class, including nonparametric and parametric cases.
For distributed computing environment, we consider the empirical risk minimization problem and propose a distributed and communication-efficient Newton-type optimization method. At every iteration, each worker locally finds an Approximate NewTon (ANT) direction, which is sent to the main driver. The main driver, then, …
The ability to use a 2D map to navigate a complex 3D environment is quite remarkable, and even difficult for many humans. Localization and navigation is also an important problem in domains such as robotics, and has recently become a focus of the deep reinforcement learning community. In this paper we teach a reinforce…
The paper proposes a structure learning model for efficient reinforcement learning.
problem Efficient sample-based decision making in complex environments.
method Temporal Gaussian Mixture Model with structure learning.
result The model discovers the number of states and transition probabilities.
AutoDIME automates design of multi-agent environments for RL.
problem Designing multi-agent environments for reinforcement learning is challenging.
method Developed intrinsic teacher rewards for multi-agent settings and evaluated them in various tasks.
result Value disagreement was found to be most consistent and effective across tasks.
The study examines conditions for achieving a simple lower bound in estimating mean from samples.
problem Achieving a simple lower bound for estimating the mean of a distribution.
method Analyzes conditions for nearly attaining Le Cam's two-point testing lower bound for mean estimation.
result An algorithm nearly attains the two-point testing rate for mixtures of symmetric, log-concave distributions with a common mean.
Adaptive Nucleus Truncation Improves Long-Form Reasoning
problem Improving long-form reasoning in language models
method Adaptive Nucleus Truncation Sampling (ANTS)
result Significant performance gains across various benchmarks
Deep neural features identify unique vehicles from dash-cam feeds.
problem Identifying unique vehicles in dash-cam feeds for self-driving cars.
method Used pretrained YOLO network feature maps to create deep integrated feature signatures (DIFS) for 700 images of 35 vehicles and 340 images of 17 vehicles.
result Correctly identified unique vehicles at 96.7% for high resolution data and 86.8% for lower resolution data.
The paper extends statistical estimation techniques under differential privacy.
problem Establishing sample complexity bounds for estimation tasks under differential privacy.
method Proposes analogues of Le Cam's method, Fano's inequality, and Assouad's lemma under central differential privacy.
result Optimal sample complexity bounds for discrete distribution estimation under total variation and ℓ2 distances. CSAC enables cooperative reinforcement learning for multi-stage tasks.
problem Coordinating consecutive reinforcement learning agents for long-term multi-stage tasks.
method CSAC modifies each agent's policy to maximize both current and next agent's critic.
result CSAC outperforms uncooperative policies and single-agent training in multi-room maze domain.
Paper proposes methods to discover causal models with unobserved variables.
problem Discovering causal relationships in data with unobserved variables.
method Two methods leveraging prior knowledge for causal discovery in CAM-UV models.
result Accuracy of causal discovery improves with more prior knowledge.
CAMS selects best pre-trained model for unlabeled data points.
problem Efficiently utilizing pre-trained models and unlabeled data.
method Contextual active model selection algorithm with two components: contextual model selection and active query.
result CAMS requires less than 10% labeling effort compared to existing methods, achieving similar or better accuracy.
This paper introduces a new approach to active inference using constrained Bethe Free Energy.
problem Tackling the limitations of existing epistemic behavior models in active inference.
method Introducing a constrained Bethe Free Energy (CBFE) perspective to optimize epistemic behavior in generative models.
result CBFE optimization leads to more robust and flexible epistemic behavior compared to existing methods.
Value Iteration Networks (VINs) are effective differentiable path planning modules that can be used by agents to perform navigation while still maintaining end-to-end differentiability of the entire architecture. Despite their effectiveness, they suffer from several disadvantages including training instability, random …
This study shows how social insects and machine learning methods share a common mathematical framework.
problem Understanding how decentralized systems achieve optimal decision-making.
method Developed a rigorous mathematical framework to show isomorphism between ant colonies and ensemble machine learning.
result Demonstrated that ant colony decision-making and random forest learning implement identical variance reduction strategies through decorrelation of identical units.
Regularizes model-based planning using energy-based models for efficient learning.
problem Challenges in using learned dynamics models for accurate planning.
method Regularization using energy estimates of state transitions.
result Proposes effective regularization method for planning with pre-trained dynamics models.