This work analyzes the gap between off-policy and on-policy policy gradient methods and provides conditions to reduce this gap.
problem The gap between off-policy and on-policy policy gradient methods and conditions to reduce it.
method Theoretical analysis and empirical evidence of conditions to reduce the on-off gap.
result Conditions to reduce the on-off gap between off-policy and on-policy policy gradient methods.
New research shows that the dimension gap between intrinsic and ambient dimensions affects adversarial vulnerability of machine learning models.
problem The mystery of adversarial attacks on machine learning models.
method Introducing two types of adversarial attacks and proving their relationship to the dimension gap.
result The dimension gap between intrinsic and ambient dimensions makes clean-trained models more vulnerable to off-manifold adversarial perturbations.
Non-intrusive load monitoring (NILM), also known as energy disaggregation, is a blind source separation problem where a household's aggregate electricity consumption is broken down into electricity usages of individual appliances. In this way, the cost and trouble of installing many measurement devices over numerous ho…
Hybrid RL algorithm combines offline and online data for robust and efficient policy learning.
problem Combining robust on-policy methods with efficient offline data for hybrid RL.
method Integrates off-policy training on offline data into on-policy NPG framework.
result Achieves state-of-the-art theoretical guarantees and maintains on-policy NPG guarantees.
Continuum Dropout improves neural differential equations by preventing overfitting.
problem Overfitting in Neural Differential Equations (NDEs).
method Introduces Continuum Dropout, a regularization technique based on alternating renewal processes.
result Continuum Dropout outperforms existing methods in various tasks, improving generalization and uncertainty quantification.
Energy disaggregation in a non-intrusive way estimates appliance level electricity consumption from a single meter that measures the whole house electricity demand. Recently, with the ongoing increment of energy data, there are many data-driven deep learning architectures being applied to solve the non-intrusive energy…
TIM framework uses LLMs and domain experts to infer DeFi user transaction intents.
problem Challenges in understanding user intent in DeFi transactions due to complex interactions and opaque logs.
method TIM framework leverages a DeFi intent taxonomy, multi-agent LLM system, and a Meta-Level Planner.
result TIM significantly outperforms existing methods in inferring user transaction intents.
This study analyzes the effects of lifting lockdowns on Brazil's COVID-19 spread.
problem Understanding the impact of lifting lockdowns on Brazil's COVID-19 spread.
method Extended SEIR model with on/off strategy and developed a neural regressor.
result Lifting lockdowns could lead to a more severe second wave of COVID-19.
This article studies the financial integration between the six main Latin American markets and the US market in a nonlinear framework. Using the threshold cointegration techniques of Hansen and Seo (2002), we show significant threshold stock market linkages between Mexico, Chile and the US. Thus, the dynamics of these …
Investigates numerical issues in GP interpolation parameter estimation.
problem Numerical issues in maximum likelihood parameter estimation for Gaussian process interpolation.
method Investigates and proposes strategies to improve open-source software implementations.
result Improves reliability and reproducibility of studies relying on GP implementations.
In the last few years, deep multi-agent reinforcement learning (RL) has become a highly active area of research. A particularly challenging class of problems in this area is partially observable, cooperative, multi-agent learning, in which teams of agents must learn to coordinate their behaviour while conditioning only…
This paper reviews off-policy evaluation methods in reinforcement learning.
problem Efficiency and accuracy of off-policy evaluation methods in reinforcement learning.
method Discussion of existing OPE methods, their statistical properties, and related research directions.
result Efficiency bounds and state-of-the-art OPE methods in reinforcement learning.
This paper investigates the problem of online prediction learning, where learning proceeds continuously as the agent interacts with an environment. The predictions made by the agent are contingent on a particular way of behaving, represented as a value function. However, the behavior used to select actions and generate…
The paper analyzes the role of ReLU gates in deep learning networks.
problem Understanding the role of gates in deep learning networks.
method Developed neural path features (NPF) and neural path values (NPV) to characterize the active sub-networks during training.
result The neural path kernel associated with NPFs is a fundamental quantity that characterizes the information stored in the gates of a DNN.
We introduce a new approach to incorporate uncertainty into the decision to invest in a commodity reserve. The investment is an irreversible one-off capital expenditure, after which the investor receives a stream of cashflow from extracting the commodity and selling it on the spot market. The investor is exposed to pri…
Study of dynamic competition between old and new technologies using a nonlinear map.
problem Dynamic competition between old and new technologies in market share.
method Simulations with three key parameters: scientific-technological, purely technological, and economic.
result Different outcomes in terms of technology prevalence based on interaction of key parameters.
Recent studies have shown that the aggregated dynamic flexibility of an ensemble of thermostatic loads can be modeled in the form of a virtual battery. The existing methods for computing the virtual battery parameters require the knowledge of the first-principle models and parameter values of the loads in the ensemble.…
Paper tackles policy selection in offline RL without hyperparameters.
problem Selecting between policies and value functions in offline RL.
method Designs hyperparameter-free algorithms based on BVFT for policy selection.
result Demonstrates effectiveness in discrete-action benchmarks like Atari.
Recurrent neural networks (RNNs) achieve cutting-edge performance on a variety of problems. However, due to their high computational and memory demands, deploying RNNs on resource constrained mobile devices is a challenging task. To guarantee minimum accuracy loss with higher compression rate and driven by the mobile r…
Emission from a class of benzene-based molecules known as Polycyclic Aromatic Hydrocarbons (PAHs) dominates the infrared spectrum of star-forming regions. The observed emission appears to arise from the combined emission of numerous PAH species, each with its unique spectrum. Linear superposition of the PAH spectra ide…
New metrics help rebuild trust in Active Learning for industry practitioners.
problem Misalignment of objectives between research and industry in Active Learning.
method Presentation of actionable metrics to interpret AL strategies.
result Actionable metrics improve AL performance and interpretability.
Despite the recent progress in hyperparameter optimization (HPO), available benchmarks that resemble real-world scenarios consist of a few and very large problem instances that are expensive to solve. This blocks researchers and practitioners not only from systematically running large-scale comparisons that are needed …
We propose to execute deep neural networks (DNNs) with dynamic and sparse graph (DSG) structure for compressive memory and accelerative execution during both training and inference. The great success of DNNs motivates the pursuing of lightweight models for the deployment onto embedded devices. However, most of the prev…
Adapts GRPO for off-policy RL, improving reward.
problem Improving training stability and efficiency in RL.
method Adapts GRPO to off-policy setting, uses clipped surrogate objectives.
result Off-policy GRPO outperforms on-policy GRPO in empirical tests.
Paper tackles efficient evaluation of natural stochastic policies in offline RL.
problem Efficiency issues in evaluating natural stochastic policies due to unknown evaluation policy.
method Derive efficiency bounds for tilting and modified treatment policies, propose nonparametric estimators.
result Proposed estimators attain efficiency bounds under lax conditions and enjoy partial double robustness.
Paper introduces risk assessment for contextual bandits without experiments.
problem Evaluate policies using logged data in context bandits.
method Lipschitz risk functionals and Off-Policy Risk Assessment (OPRA) framework.
result OPRA provides finite sample guarantees for various risk estimates.
New sketches for weighted ℓp sampling without replacement improve accuracy and efficiency.
problem Efficiently sampling weighted data with high accuracy and minimal redundancy.
method Design of novel composable sketches for WOR ℓp sampling, based on CountSketch. result First to provide WOR sampling for p>1 and signed updates. The paper explores gaps in curvature-related metrics and rigidity.
problem Understanding gaps in curvature-related metrics and rigidity.
method Analyzes three types of gaps: spectral, metric-rigidity, and topological-rigidity.
result Proposes open problems in the field.
New method detects adversarial attacks in industrial systems.
problem Adversarial attacks compromise machine learning models in industrial control systems.
method Introduces domain adaptation layer and Monte Carlo simulation for proactive defense.
result Approximates Bayes optimal mitigation for improved detection model health.
Study gap-dependent regret bounds for risk-sensitive RL.
problem Risk-sensitive reinforcement learning with entropic risk measure.
method Propose cascaded gaps to adapt to problem structures, derive regret bounds.
result Exponential improvement over existing bounds in appropriate settings.
Cramming method evaluates learned policies from contextual bandits efficiently.
problem Evaluating final learned policies from contextual bandit algorithms.
method On-policy evaluation using a single pass of data, ensuring consistency and asymptotic normality.
result Cramming method reduces evaluation standard error by approximately 40% compared to off-policy methods.
Machine learning has recently emerged as a fruitful area for finding potential quantum computational advantage. Many of the quantum enhanced machine learning algorithms critically hinge upon the ability to efficiently produce states proportional to high-dimensional data points stored in a quantum accessible memory. Eve…
The paper explores how to extrapolate from limited data points using causal mechanisms.
problem Handling distribution shifts with limited target samples.
method Formulates the extrapolation problem with a latent-variable model embodying the minimal change principle in causal mechanisms, and identifies conditions for identification.
result Theoretical understanding and practical methods for extrapolation without requiring an on-support target distribution.
CoCoPIE shows AI can run on regular devices without special hardware.
problem Real-time AI on mobile devices without special hardware.
method Compression-compiler co-design using CoCoPIE framework.
result Real-time AI applications on off-the-shelf mobile devices.
Paper analyzes origami slope gaps and their distribution, finding a unique pattern.
problem Analyzing slope gaps in origami surfaces.
method Derived slope gap distribution of a specific origami by considering return times under the horocycle flow.
result Found a unique distribution of origami slope gaps, not a sum of scaled Hall distributions.
PaRoT simplifies robust training for deep neural networks.
problem Training deep neural networks to be robust to small input changes.
method Developed a practical framework on TensorFlow for robust training without code modifications.
result PaRoT's performance is comparable to existing methods and is easy to use on real-world models.
Random hyperbolic surfaces have nearly optimal spectral gaps.
problem Proving the nearly optimal spectral gap conjecture for random Belyi surfaces.
method Using the Brooks-Makover model, the authors show a spectral gap greater than 1/4 - c/log(n).
result A random hyperbolic surface in the Brooks-Makover model has a spectral gap greater than 1/4 - c/log(n).
Paper proposes an active multi-step TD algorithm for reinforcement learning.
problem Challenging decision making and control tasks in reinforcement learning.
method Active stepsize learning and adaptive multi-step TD algorithm with context-aware mechanism.
result Competitive results compared to other reinforcement learning baselines on discrete and continuous space tasks.
We present a method that trains large capacity neural networks with significantly improved accuracy and lower dynamic computational cost. We achieve this by gating the deep-learning architecture on a fine-grained-level. Individual convolutional maps are turned on/off conditionally on features in the network. To achieve…
Paper improves volume gap between minimal submanifolds and unit spheres.
problem Volume gap between minimal submanifolds and unit spheres.
method Modified Cheng-Li-Yau coefficients and applied Cheng-Yang eigenvalue estimate for Laplacian.
result Enhanced volume gap between minimal submanifolds and unit spheres.
A new framework detects anomalous inputs to DNNs.
problem Detecting adversarial and out-of-distribution inputs in DNNs.
method Unsupervised meta-algorithm with configurable components based on statistical testing.
result The proposed method effectively detects adversarial and OOD inputs.
The article proves a conjecture about the fundamental gap for horoconvex domains in hyperbolic space.
problem Proving a conjecture about the fundamental gap for horoconvex domains in hyperbolic space.
method Establishing conformal log-concavity estimates for the first eigenfunction.
result Proves a conjecture about the fundamental gap for horoconvex domains in hyperbolic space.
Kahler-Einstein metrics linked to eigenvalue gaps on Fano manifolds.
problem Existence of Kahler-Einstein metrics on Fano manifolds.
method Characterization via eigenvalue gaps of Cauchy-Riemann and Hamiltonian vector fields.
result Existence of Kahler-Einstein metrics linked to eigenvalue gaps.
Researchers compute gap distributions for saddle connection directions on specific translation surfaces.
problem Computing gap distributions for saddle connection directions on translation surfaces.
method Translation to dynamical question of return times to a transversal under the horocycle flow.
result Gap distributions have support at 0 and quadratic tail decay.
Local gaps in Ricci shrinkers depend only on dimension.
problem Understanding local properties of Ricci shrinkers.
method Proved local versions of Ricci curvature and entropy gap theorems.
result Local gaps depend only on dimension, not global entropy.
Study shows gaps in Bitcoin order book are linked to returns but only in the short term.
problem Understanding the relationship between gaps and returns in Bitcoin order books.
method Examined the dynamics of gaps and returns in a Bitcoin order book without considering long-term causation.
result The causal relationship between gaps and returns is limited to instantaneous causation.
Proves gap rigidity theorem for Hermitian symmetric spaces.
problem Gap rigidity problems in compact Hermitian symmetric spaces.
method Dual analogy to Mok's noncompact case theorem, theorem on higher dimensional submanifolds.
result Proves gap rigidity theorem for diagonal curves in tube type spaces.
The paper introduces gapped scale-sensitive dimensions to improve learning rate bounds.
problem Improving lower bounds on rates of convergence in statistical and online learning.
method Introducing and analyzing gapped scale-sensitive dimensions for function classes.
result Gapped dimensions lead to stronger lower bounds on offset Rademacher averages.