Adjoint Matching improves flow and diffusion models with reward fine-tuning.
problem Improving generative models with reward fine-tuning.
method Casting reward fine-tuning as stochastic optimal control (SOC) and enforcing a specific noise schedule.
result Adjoint Matching outperforms existing SOC algorithms.
Conservation laws improve diffusion model training by optimizing likelihood.
problem Training diffusion models with denoising objectives.
method Developed conservation laws based on GEXIT functions for memoryless noise processes.
result Unified characterization of diffusion model likelihood, reducing training to learning marginal posteriors.
Optimizer memory affects learning rate sensitivity in shuffle order, impacting fine-tuning noise.
problem Optimizer memory affects the learning rate sensitivity in shuffle order, leading to fine-tuning noise.
method Isolated the mechanism of fixed-clock optimizer memory affecting the learning rate sensitivity in shuffle order, deriving a fit-free way to size the noise.
result Fixed-clock optimizers like AdamW produce a larger first-order noise channel compared to memoryless optimizers, affecting fine-tuning comparisons.
New method evaluates LLMs fairness in universal prediction.
problem Evaluating fairness of large language models in universal prediction.
method Introducing batch regret as a modification of average regret for LLMs.
result Asymptotical value of batch regret for add-constant predictors on memoryless and first-order Markov sources.
The paper derives a new theorem for predicting batches of data.
problem Finding lower bounds on minimal batch regret.
method Derives a conditional version of the regret-capacity theorem.
result Reveals a connection between conditional Rényi divergence and conditional Sibson's mutual information.
BestChanID identifies the channel with maximal capacity using training sequences.
problem Identifying the channel with maximal capacity among several discrete memoryless channels.
method Formulated as a multi-armed bandit problem, proposed a capacity estimator, and developed gap-elimination algorithms.
result Guaranteed to output the DMC with the largest capacity with a desired confidence.
Off-policy evaluation (OPE) in reinforcement learning allows one to evaluate novel decision policies without needing to conduct exploration, which is often costly or otherwise infeasible. We consider for the first time the semiparametric efficiency limits of OPE in Markov decision processes (MDPs), where actions, rewar…
We propose a new reinforcement learning algorithm for partially observable Markov decision processes (POMDP) based on spectral decomposition methods. While spectral methods have been previously employed for consistent learning of (passive) latent variable models such as hidden Markov models, POMDPs are more challenging…
We propose a new reinforcement learning algorithm for partially observable Markov decision processes (POMDP) based on spectral decomposition methods. While spectral methods have been previously employed for consistent learning of (passive) latent variable models such as hidden Markov models, POMDPs are more challenging…
The problem of distributed representation learning is one in which multiple sources of information X1,…,XK are processed separately so as to learn as much information as possible about some ground truth Y. We investigate this problem from information-theoretic grounds, through a generalization of Tishby's ce…
dtControl uses decision trees to represent controllers efficiently and explainably.
problem Representing controllers concisely and explainably.
method dtControl uses decision tree learning algorithms to represent controllers. Novel techniques for determinizing controllers are introduced.
result Novel techniques for determinizing controllers during decision tree construction are extremely efficient, yielding small decision trees.
Study language generation with limited memory, showing different impacts on achievable densities and convergence.
problem Language generation with bounded memory constraints.
method Analyzed memoryless generators, sliding windows, and adaptive past examples; revisited identification in the limit.
result Achievable densities and convergence properties differ based on the size of the target language collection.
A technique identifies memoryless algorithms approximating memory-dependent optimization methods.
problem Understanding how memory in optimization algorithms affects loss and generalization.
method Introducing a general technique to replace past iterates with the current one and adding a correction term.
result Lion does not have the same implicit anti-regularization as AdamW, explaining its better generalization performance.
In the Bayesian approach to sequential decision making, exact calculation of the (subjective) utility is intractable. This extends to most special cases of interest, such as reinforcement learning problems. While utility bounds are known to exist for this problem, so far none of them were particularly tight. In this pa…
Unified framework for blending ML and mechanistic models in dynamical systems.
problem Learning dynamical systems from noisy, partially observed data.
method A unifying framework that combines mechanistic and machine learning approaches.
result Proves that hybrid models can learn memory-dependent model error.
Market activity scales near a constant of 0.632 in intrinsic time.
problem Understanding the stability of market scaling laws.
method Modeling market directional changes as a memoryless exponential hazard process and identifying the intrinsic time scaling constant.
result The intrinsic time scaling constant is 1−1/e=0.632. We review the decomposition method of stock return cross-correlations, presented previously for studying the dependence of the correlation coefficient on the resolution of data (Epps effect). Through a toy model of random walk/Brownian motion and memoryless renewal process (i.e. Poisson point process) of observation ti…
A new DVAE architecture improves channel estimation by incorporating temporal correlations.
problem Improving the estimation of time-varying channels.
method Introducing k-MemoryMarkovVAE (k-MMVAE) architecture to learn temporal correlations.
result The k-MMVAE aided channel estimator outperforms other ML aided estimators.
\begin{abstract} We model individual T2DM patient blood glucose level (BGL) by stochastic process with discrete number of states mainly but not solely governed by medication regimen (e.g. insulin injections). BGL states change otherwise according to various physiological triggers which render a stochastic, statisticall…
New CTBNs with clocks allow for non-exponential survival times.
problem Modeling phenomena with non-exponential survival times in continuous time.
method Introduced node-wise clocks to construct graph-coupled semi-Markov chains, enabling non-exponential survival times without auxiliary states.
result Parameter and structure inference algorithms provided, demonstrating advantages over current CTBN extensions.
We propose an algorithm for deterministic continuous Markov Decision Processes with sparse rewards that computes the optimal policy exactly with no dependency on the size of the state space. The algorithm has time complexity of O(∣R∣3×∣A∣2) and memory complexity of O(∣R∣×∣A∣), where ∣R∣ is the…
We characterize value functions in partially observable MDPs as semi-algebraic sets.
problem Understanding feasible value functions in partially observable Markov decision processes.
method Characterization of feasible value functions as semi-algebraic sets defined by polynomial inequalities.
result The feasible set of value functions in POMDPs is a semi-algebraic set, not a polytope as in MDPs.
In this paper, the `Approximate Message Passing' (AMP) algorithm, initially developed for compressed sensing of signals under i.i.d. Gaussian measurement matrices, has been extended to a multi-terminal setting (MAMP algorithm). It has been shown that similar to its single terminal counterpart, the behavior of MAMP algo…
New method calibrates false detection rates in sequential change detection.
problem Challenges in setting time-invariant thresholds for false positives.
method Simulation-based approach to time-varying thresholds.
result Accurately targets desired expected runtime while keeping false positive rate constant.
Disease progression models are instrumental in predicting individual-level health trajectories and understanding disease dynamics. Existing models are capable of providing either accurate predictions of patients prognoses or clinically interpretable representations of disease pathophysiology, but not both. In this pape…
Recurrent neural networks (RNNs) have drawn interest from machine learning researchers because of their effectiveness at preserving past inputs for time-varying data processing tasks. To understand the success and limitations of RNNs, it is critical that we advance our analysis of their fundamental memory properties. W…
We investigate the Heston model with stochastic volatility and exponential tails as a model for the typical price fluctuations of the Brazilian São Paulo Stock Exchange Index (IBOVESPA). Raw prices are first corrected for inflation and a period spanning 15 years characterized by memoryless returns is chosen for the ana…
A new hierarchy quantifies agency in systems based on information processing.
problem Lack of a measurable, universal definition for agency in intelligent systems.
method Developed a bottom-up framework based on information processing hierarchy.
result Identified three orders of information processing (I, II, III) as necessary for agency.
New methods for evaluating and optimizing policies in offline RL with unobserved confounders.
problem Evaluating and optimizing policies in the presence of unobserved confounders.
method Characterized settings and algorithms for consistent value estimates and lower bounds, with sample complexity guarantees.
result Proved local convergence guarantees for offline policy improvement.
Study on future-dependent value functions for off-policy evaluation in complex environments.
problem Exponential dependence on horizon in off-policy evaluation for complex observations.
method Developed novel coverage assumptions for POMDPs to achieve polynomial bounds.
result Achieved polynomial bounds on previously exponential quantities, improving off-policy evaluation.
We introduce the "NoBackTrack" algorithm to train the parameters of dynamical systems such as recurrent neural networks. This algorithm works in an online, memoryless setting, thus requiring no backpropagation through time, and is scalable, avoiding the large computational and memory cost of maintaining the full gradie…
The episodic, irregular and asynchronous nature of medical data render them difficult substrates for standard machine learning algorithms. We would like to abstract away this difficulty for the class of time-stamped categorical variables (or events) by modeling them as a renewal process and inferring a probability dens…
Fine-grained analysis of gradient descent with momentum provides modified loss equations.
problem Understanding the dynamics of gradient descent with momentum.
method Fine-grained analysis and derivation of modified loss equations.
result Global approximation bounds and continuous modified equations for HB.
We propose a neural superstatistics method to estimate dynamic cognitive models from time series data.
problem Memoryless cognitive models ignore parameter fluctuations, leading to inaccurate predictions.
method Developed a simulation-based deep learning method for Bayesian inference of superstatistical models.
result Deep learning method efficiently recovers time-varying and time-invariant parameters.
New insights into image compression trade-offs with private randomness.
problem Trade-off between compression rate and perceptual quality in image compression.
method Characterization of rate-distortion trade-off with private randomness under different realism constraints.
result Encoder private randomness is not useful if compression rate is below source entropy, even with limited common and decoder private randomness.
ASBS improves sampling from Boltzmann distributions without importance weighting.
problem Sampling from Boltzmann distributions with known energies but unknown samples.
method Adjoint Schrödinger Bridge Sampler using kinetic-optimal transportation.
result ASBS achieves scalable and efficient sampling without importance weighting.
We study a variant of the source identification game with training data in which part of the training data is corrupted by an attacker. In the addressed scenario, the defender aims at deciding whether a test sequence has been drawn according to a discrete memoryless source X∼PX, whose statistics are known to hi…
New training algorithm enhances SNNs for temporal signal processing.
problem Lack of robust training algorithms for large-scale SNNs.
method Formulated SNN as IIR filters, proposed training algorithm for optimal synapse filter kernels and weights.
result Model and training algorithm outperform state-of-the-art approaches in accuracy.
Bayesian inference and superstatistics model financial volatility dynamics across different timescales.
problem Modeling correlated volatility in financial time series with heavy tails and long memory.
method Superstatistical dynamics, Bayesian Inference, Metropolis-Hasting sampling.
result The log-Normal model is reliable for short timescales, while inverse-Gamma is preferred for long timescales.
This work investigates the case of a network of agents that attempt to learn some unknown state of the world amongst the finitely many possibilities. At each time step, agents all receive random, independently distributed private signals whose distributions are dependent on the unknown state of the world. However, it m…
Stochastic defense improves natural classifiers against adversarial attacks.
problem Vulnerability of deep networks to adversarial attacks.
method Long-run MCMC sampling with Energy-Based Model for adversarial purification.
result Balancing memoryless and metastable behavior leads to effective purification and robust classification.
Measures three types of noise in LLM evaluations.
problem Separating signal from noise in LLM experiments.
method Defined and measured three types of noise: prediction, data, and total noise. Proposed the all-pairs paired method for statistical power.
result Total noise level is characteristic and predictable across all model pairs.
The study investigates noise effects on parameter estimation for Ornstein-Uhlenbeck processes.
problem Impact of noise on parameter fitting for Ornstein-Uhlenbeck processes.
method Proposed algorithms to distinguish between thermal and multiplicative noise.
result Effective methods to estimate parameters even when multiplicative noise dominates.
Noise in SGD affects overparameterized models, favoring sparse solutions.
problem Understanding and mitigating implicit bias in SGD with parameter-dependent noise.
method Theoretical analysis of a quadratically-parameterized model with label noise and Gaussian noise.
result SGD with label noise recovers sparse ground-truth solutions, while SGD with Gaussian noise overfits dense solutions.
L2R learns to denoise images without needing noise distribution knowledge.
problem Traditional denoising methods require noise distribution knowledge, limiting their applicability.
method L2R uses a learnable monotonic neural network to learn recorruption without distribution knowledge.
result L2R achieves state-of-the-art performance across various noise distributions.
Estimating the entropy based on data is one of the prototypical problems in distribution property testing and estimation. For estimating the Shannon entropy of a distribution on S elements with independent samples, [Paninski2004] showed that the sample complexity is sublinear in S, and [Valiant--Valiant2011] showed…
Interpolating label noise makes models vulnerable to adversarial attacks.
problem Adversarial vulnerability of models trained on noisy labels.
method Theoretical analysis of label noise and adversarial risk relationship.
result Uniform label noise induces adversarial risk similar to worst-case poisoning.
Generative adversarial networks (GANs) are neural networks that learn data distributions through adversarial training. In intensive studies, recent GANs have shown promising results for reproducing training images. However, in spite of noise, they reproduce images with fidelity. As an alternative, we propose a novel fa…