This work tackles online memory selection in continual learning using information theory.
problem Online selection of a representative replay memory from data streams.
method Information-theoretic criteria (surprise, learnability) and Bayesian model for efficient computation.
result InfoRS improves robustness against data imbalance compared to reservoir sampling.
Modyn automates continuous ML model training on growing datasets.
problem Continuous model retraining is costly and impractical.
method Data-centric ML platform with policies for continuous training.
result Modyn enables high throughput training with sample-level data selection.
Paper proves method for calculating NML code length works for continuous models.
problem Uncertainty in calculating NML code length for continuous models.
method Introduced a novel decomposition approach based on the coarea formula to prove correctness for continuous cases.
result Method accurately calculates NML code length for continuous models.
This thesis studies two problems in modern statistics. First, we study selective inference, or inference for hypothesis that are chosen after looking at the data. The motiving application is inference for regression coefficients selected by the lasso. We present the Condition-on-Selection method that allows for valid s…
RL approach for continuous-time mean-variance portfolio selection with empirical validation.
problem Continuous-time mean-variance portfolio selection in unknown market coefficients.
method Reinforcement learning for diffusion processes, sublinear regret bound derivation.
result RL strategy consistently outperforms model-based counterparts, especially in volatile markets.
Stock selection improved with a novel neural model capturing continuous stock dynamics.
problem Lack of continuous stock dynamics prediction and implicit cross-domain dependencies.
method StockODE, a latent variable model with NRODEs and hierarchical hypergraph for continuous stock volatility and inter-domain dependencies.
result Significantly outperforms baselines, improving Sharpe Ratio by up to 18.57%.
Detects model changes in data streams using Ddim.
problem Early detection of model changes in data streams.
method Continuous model selection based on Ddim.
result Early warning signals of model changes.
A hybrid model for Bayesian optimization handles mixed variables using MCTS for categorical and GP for continuous.
problem Optimizing functions with mixed variable types (continuous, integer, categorical).
method Merges MCTS for categorical and GP for continuous variables, integrates UCTS search strategy, and dynamically selects kernels.
result Hybrid models outperform traditional methods in Bayesian optimization.
PEAKS selects key training examples incrementally based on prediction error and kernel similarity.
problem Dynamic data selection in deep learning models.
method Prediction Error Anchored by Kernel Similarity (PEAKS) for incremental data selection.
result PEAKS outperforms existing selection strategies and yields better performance returns as training data size grows.
Method estimates treatment effects with continuous values, correcting for confounding.
problem Estimating treatment effects with continuous values, dealing with confounding.
method Two-stage kernel ridge regression: first stage learns response, second stage corrects for distribution shift.
result Optimal learning bounds achieved without estimating treatment density, adapts to unknown overlap and kernel spectral decay.
Novel optimization method detects change points in Gaussian data.
problem Detecting change points in univariate Gaussian data sequences.
method Continuous optimization for best subset selection (COMBSS) applied to a reformulated statistical inverse problem.
result Adaptation and evaluation of COMBSS for offline normal mean multiple change-point detection.
Naive investors make riskier choices than optimal strategies in continuous-time finance.
problem Continuous-time Markowitz portfolio selection with naive reoptimization.
method Analytical derivation of naive policies from discretely naive policies.
result Naive policies are always riskier and less efficient than equilibrium policies.
Bayesian model selection improves multivariate causal discovery without restrictive assumptions.
problem Real-world causal discovery requires flexible assumptions to avoid restrictive model assumptions.
method Continuous relaxation of discrete model selection problem, using Causal Gaussian Process Conditional Density Estimator (CGP-CDE).
result Bayesian approach outperforms traditional methods in multivariate causal discovery.
Efficient optimisation of black-box problems that comprise both continuous and categorical inputs is important, yet poses significant challenges. We propose a new approach, Continuous and Categorical Bayesian Optimisation (CoCaBO), which combines the strengths of multi-armed bandits and Bayesian optimisation to select …
Proves FR-NGD optimally approximates evolutionary dynamics and continuous Bayesian inference.
problem Optimizing continuous time replicator equations and continuous Bayesian inference.
method Fisher-Rao natural gradient descent (FR-NGD) and its correspondence with evolutionary dynamics.
result FR-NGD optimally approximates continuous time replicator equations and continuous Bayesian inference.
AGS-CL selectively updates penalties based on node importance for continual learning.
problem Catastrophic forgetting in continual learning.
method Adaptive Group Sparsity (AGS) with proximal gradient descent.
result Significantly outperforms baselines on various continual learning benchmarks.
Study on optimal portfolio selection with varying borrowing and saving rates in continuous-time markets.
problem Optimal portfolio selection in markets with different borrowing and saving rates.
method Hamilton-Jacobi-Bellman equation, partial differential equation, verification argument.
result Existence and smoothness of the value function, identification of trading regions and strategies.
This survey covers in our opinion the most important results in the theory of continuous selections of multivalued mappings (approximately) from 2002 through 2012. It extends and continues our previous such survey which appeared in Recent Progress in General Topology, II, which was published in 2002. In comparison, our…
Continual lifelong learning is essential to many applications. In this paper, we propose a simple but effective approach to continual deep learning. Our approach leverages the principles of deep model compression, critical weights selection, and progressive networks expansion. By enforcing their integration in an itera…
Perplexity fails to distinguish correct predictions from incorrect ones in some cases.
problem Perplexity's limitations in model selection.
method Analyzing Transformer continuity and iso-perplexity plots.
result Perplexity does not always select the correct model.
Continual lifelong learning requires an agent or model to learn many sequentially ordered tasks, building on previous knowledge without catastrophically forgetting it. Much work has gone towards preventing the default tendency of machine learning models to catastrophically forget, yet virtually all such work involves m…
It has long been observed that for practically any computational problem that has been intensely studied, different instances are best solved using different algorithms. This is particularly pronounced for computationally hard problems, where in most cases, no single algorithm defines the state of the art; instead, the…
The paper is devoted to generalization of well-known Michael's Selection theorem on the case of extension dimension.
VAIOM models financial returns using continuous input and categorical output.
problem Modeling continuous, noisy, and heterogeneous financial data.
method VAIOM is a decoder-only Transformer that separates input representation from output likelihood.
result VAIOM models outperform fixed single-bar LightGBM baseline in both Test halves.
ecpc R-package improves high-dimensional prediction with co-data.
problem High-dimensional prediction with more variables than samples.
method Adaptive ridge penalised models with co-data, including continuous co-data.
result Improved variable selection and prediction performance.
Develops a framework for modeling set-valued data in continuous-time.
problem Handling sequences where each event is associated with a set of items.
method General framework for modeling set-valued data, developed inference methods, and importance sampling techniques.
result Orders-of-magnitude improvements in efficiency for probabilistic queries over direct sampling.
Study optimal portfolios in a non-Markovian regime-switching model with random time horizon.
problem Optimal portfolio selection in a market with non-Markovian regime-switching and random time horizon.
method Formulated as a constrained stochastic linear-quadratic optimal control problem, derived closed-form expressions for optimal portfolios and efficient frontier.
result Closed-form expressions for optimal portfolios and efficient frontier derived under non-Markovian regime-switching and random time horizon.
New method protects whistleblowers from retaliation by ensuring their reports remain private.
problem Whistleblowers face retaliation, and current protections are insufficient.
method Formalizes protection against strong-adversary threat model as per-report (0,δ)-differential privacy, and provides a generic mechanism to reduce private auditing to private continual counting. result Demonstrates a reduction in selection error and improved utility over randomized response.
This paper presents the R package gRapHD for efficient selection of high-dimensional undirected graphical models. The package provides tools for selecting trees, forests and decomposable models minimizing information criteria such as AIC or BIC, and for displaying the independence graphs of the models. It has also some…
PRISM infers model structures and parameters from simulations, controlling complexity at test time.
problem Choosing among large model families for scientific discovery.
method Simulation-based encoder-decoder that infers model structures and parameters, with test-time complexity control.
result PRISM scales to large model families and performs model selection in biophysical diffusion MRI.
fastHDMI improves neuroimaging variable selection in high-dimensional data.
problem Efficient variable screening in high-dimensional neuroimaging datasets.
method Three mutual information estimation methods implemented in fastHDMI.
result FFTKDE-based method superior for continuous nonlinear outcomes.
Decomposes smooth manifolds into algebraic submanifolds.
problem Understanding the structure of smooth manifolds induced by continuous selections.
method Generic continuous selection of smooth functions provides stratification of the manifold.
result Stratification leads to local topological structure with nondegenerate critical points.
Paper proposes a new method for selective inference in robust regression.
problem Statistical inference after removing outliers identified by robust methods.
method Conditional SI using piecewise-linear homotopy continuation.
result Proposed method is applicable to a wide class of robust regression and outlier detection methods.
In this paper, we develop a new sequential regression modeling approach for data streams. Data streams are commonly found around us, e.g in a retail enterprise sales data is continuously collected every day. A demand forecasting model is an important outcome from the data that needs to be continuously updated with the …
A new method optimizes MMD test power by dynamically selecting kernels, overcoming traditional trade-offs.
problem Fixed kernels fail to distinguish certain distributions, leading to overfitting and variance collapse.
method Complexity-Penalized MMD (CP-MMD) criterion, derived from concentration inequality, optimizes kernel selection.
result CP-MMD maximizes true test power while ensuring unconditional Type-I validity, matching or exceeding state-of-the-art performance.
New model considers wealth and time affecting risk aversion in portfolio selection.
problem Optimal investment strategy and consumption process depend on wealth and future income balance.
method Proposed a new mean-variance-utility framework with time and state-dependent risk aversion, solved using game theory.
result Equilibrium investment and consumption policies derived, aligning with investor behavior.
Develops a more powerful selective inference method for stepwise feature selection.
problem Loss of power in existing conditional SI methods due to over-conditioning.
method Uses homotopy continuation approach to overcome over-conditioning.
result Shows improved power and efficiency in selective inference for feature selection.
A continual learning agent learns online with a non-stationary and never-ending stream of data. The key to such learning process is to overcome the catastrophic forgetting of previously seen data, which is a well known problem of neural networks. To prevent forgetting, a replay buffer is usually employed to store the p…
Selective inference for group lasso estimators across various distributions and covariates.
problem Developing selective inference methods for group lasso estimators.
method Randomized group-regularized optimization problem with post-selection likelihood.
result Selective point estimator and Wald-type confidence regions for regression parameters.
In this paper, we build upon previous work on designing informative and efficient Exploratory Landscape Analysis features for characterizing problems' landscapes and show their effectiveness in automatically constructing algorithm selection models in continuous black-box optimization problems. Focussing on algorithm pe…
Investment strategies for rank-dependent utility agents are derived in a continuous-time market.
problem Time inconsistency in rank-dependent utility models.
method Study of consistent planners seeking intra-personal equilibrium strategies.
result Explicit final wealth profile replicating equilibrium strategies, with scaling function derived.
Bayesian approach controls FDR in high-dimensional models.
problem High-dimensional variable selection and inference.
method Adapted Mirror Statistic to Bayesian framework for FDR control.
result Effective FDR control without data splitting.
Continuous-time mean-variance portfolio selection model with nonlinear wealth equations and bankruptcy prohibition is investigated by the dual method. A necessary and sufficient condition which the optimal terminal wealth satisfies is obtained through a terminal perturbation technique. It is also shown that the optimal…
Model selection based on classical information criteria, such as BIC, is generally computationally demanding, but its properties are well studied. On the other hand, model selection based on parameter shrinkage by ℓ1-type penalties is computationally efficient. In this paper we make an attempt to combine their st…
One of the crucial problems in mathematical finance is to mitigate the risk of a financial position by setting up hedging positions of eligible financial securities. This leads to focusing on set-valued maps associating to any financial position the set of those eligible payoffs that reduce the risk of the position to …
This paper models AMM positions using CI options to calculate LVR and provide actionable guidance.
problem Calculating and managing adverse-selection costs in automated market makers (AMMs).
method Modeling AMM positions as perpetual American CI options to replicate delta and calculate LVR.
result LVR is identical to theta of CI options, and AMM positions have approximately constant LVR over long windows.
Paper proposes a new method to select memory data for online class-incremental learning.
problem Selecting which buffered images to replay for online class-incremental learning.
method Adversarial Shapley value scoring method to preserve latent decision boundaries.
result Proposed ASER method provides competitive or improved performance compared to state-of-the-art methods.
A new method selects representative subdata without assuming underlying models.
problem Selecting representative subdata for fast and efficient analysis.
method Uses decision trees to partition data and selects samples from each partition.
result PED subdata results in higher classification accuracy than other methods.