New algorithms minimize regret in SSP with optimal sparse updates.
problem Minimizing regret in Stochastic Shortest Path models.
method Implicit finite-horizon approximation for analysis, model-free and model-based algorithms developed.
result Minimax optimal regret for both model-free and model-based algorithms.
Gradient filters track moving parameters under noisy data and misspecification.
problem Tracking multidimensional time-varying parameters under noisy observations and model misspecification.
method Gradient-based filters update parameters using the gradient of a postulated objective function, evaluated at either the predicted or updated parameters.
result Novel sufficient conditions for exponential stability of the filtered parameter path, and finite-sample and asymptotic mean squared error bounds.
Update rules for learning in dynamic time warping spaces are based on optimal warping paths between parameter and input time series. In general, optimal warping paths are not unique resulting in adverse effects in theory and practice. Under the assumption of squared error local costs, we show that no two warping paths …
Study finds non-IID data causes FL performance issues.
problem Reduced performance in federated learning due to non-IID data.
method Investigated from IID to non-IID settings, categorized methods into two strategies.
result Inconsistencies in client loss landscapes are the primary cause of performance degradation.
A new method reduces Monte Carlo variance for financial payoffs.
problem Reducing variance in Monte Carlo estimators for financial payoffs.
method Path-dependent importance sampling using neural networks.
result Significant variance reduction (2-9 times) for various financial payoffs.
We propose a new algorithm for estimating NARMAX models with L1 regularization for models represented as a linear combination of basis functions. Due to the L1-norm penalty the Lasso estimation tends to produce some coefficients that are exactly zero and hence gives interpretable models. The novelty of the contri…
We develop a normative framework for hierarchical model-based policy optimization based on applying second-order methods in the space of all possible state-action paths. The resulting natural path gradient performs policy updates in a manner which is sensitive to the long-range correlational structure of the induced st…
Adaptive learning, also known as adaptive teaching, relies on learning path recommendation, which sequentially recommends personalized learning items (e.g., lectures, exercises) to satisfy the unique needs of each learner. Although it is well known that modeling the cognitive structure including knowledge level of lear…
Particle MCMC is a class of algorithms that can be used to analyse state-space models. They use MCMC moves to update the parameters of the models, and particle filters to propose values for the path of the state-space model. Currently the default is to use random walk Metropolis to update the parameter values. We show …
LR-EDNN reduces PDE solver complexity by limiting network weights to low-rank subspace.
problem Efficiently solving time-dependent PDEs with deep neural networks.
method Low-rank constraint on network weights using SVD for efficient parameter updates.
result LR-EDNN achieves comparable accuracy to full EDNN with fewer parameters and lower cost.
EnKF's update is shown to be similar to Matheron's method in Gaussian process regression.
problem Data assimilation in high-dimensional systems.
method Empirical Matheron update applied to EnKF.
result Ensemble Kalman Filter's update is equivalent to an empirical Matheron update.
In this paper, we address the challenging problem of selecting tuning parameters for high-dimensional sparse regression. We propose a simple and computationally efficient method, called path thresholding (PaTh), that transforms any tuning parameter-dependent sparse regression algorithm into an asymptotically tuning-fre…
Novel method for SDE calibration from sparse data using neural flows.
problem Calibrating SDEs from sparse, noisy observations.
method Characterization of posterior SDE using neural networks trained to solve a PDE with multiplicative updates.
result Significant improvement in scalability and accuracy compared to classical methods.
Improves inference-time alignment for diffusion models without updating weights.
problem Aligning diffusion models without updating weights for high-reward outputs.
method Trust-Region Iterative Twisted Sequential Monte Carlo (TRI-TSMC) for variance reduction and efficiency.
result Improves primary alignment objectives on text generation tasks.
Path integral method calculates PDBS option prices with time-dependent parameters.
problem Pricing proportional double-barrier step options with time-dependent interest rates and volatilities.
method Path integral method applied to a quantum mechanical analogy of barrier options.
result Derivation of pricing kernel for PDBS options with time-dependent parameters.
We propose a unified modelling framework that theoretically justifies the main empirical regularities characterizing the international trade network. Each country is associated to a Polya urn whose composition controls the propensity of the country to trade with other countries. The urn composition is updated through t…
Training large machine learning (ML) models with many variables or parameters can take a long time if one employs sequential procedures even with stochastic updates. A natural solution is to turn to distributed computing on a cluster; however, naive, unstructured parallelization of ML algorithms does not usually lead t…
Due to the inherent uncertainty of data, the problem of predicting partial ranking from pairwise comparison data with ties has attracted increasing interest in recent years. However, in real-world scenarios, different individuals often hold distinct preferences. It might be misleading to merely look at a global partial…
A new method for efficient neural network fine-tuning using queryable low-rank update atoms.
problem Rigidity of static low-rank adaptation methods when input and depth-wise computation vary.
method A shared queryable memory of low-rank update atoms, allowing dynamic and context-sensitive adaptation.
result Improves final test performance and training stability compared to standard low-rank adaptation.
For a variety of regularized optimization problems in machine learning, algorithms computing the entire solution path have been developed recently. Most of these methods are quadratic programs that are parameterized by a single parameter, as for example the Support Vector Machine (SVM). Solution path algorithms do not …
The study identifies volatility models from path geometry using signature-based methods.
problem Identifying different stochastic volatility models from observed data.
method Mapping volatility trajectories into a feature space via truncated path signatures and applying a gradient boosting classifier.
result The method achieves high classification accuracy across various volatility dynamics and parameter settings.
A new algorithm for faster model selection in twin multi-class SVM.
problem Challenges in effective solution of multi-classification and fast model selection in twin multi-class SVM.
method Sample data set partition strategy, Lagrangian multipliers, piecewise linear update, initialization algorithm, and event-based iteration.
result Comparable classification performance achieved without solving quadratic programming problems.
This paper presents the asymptotic behavior of a linear instrumental variables (IV) estimator that uses a ridge regression penalty. The regularization tuning parameter is selected empirically by splitting the observed data into training and test samples. Conditional on the tuning parameter, the training sample creates …
Two algorithms for linear contextual bandits with rare updates achieve optimal regret and efficiency.
problem Linear contextual bandits with infrequent parameter updates.
method Two practical algorithms with O(loglogT) updates, BLCE-G and BLCE. result Minimax-optimal regret with low computational complexity.
Many convex problems in machine learning and computer science share the same form: \begin{align*} \min_{x} \sum_{i} f_i( A_i x + b_i), \end{align*} where fi are convex functions on Rni with constant ni, Ai∈Rni×d, bi∈Rni and ∑ini=n. This probl…
A new framework predicts hidden Markov model regimes online.
problem Efficiently identify hidden Markov model regimes in streaming data.
method Develops a predictive-first optimisation framework for streaming HMMs, approximating the full posterior predictive distribution.
result The method provides competitive prequential performance compared to Online EM and Sequential Monte Carlo.
PathNNs improve graph neural networks by distinguishing non-isomorphic graphs.
problem Limitations of standard GNNs in distinguishing non-isomorphic graphs.
method Proposes PathNNs that aggregate paths to update node representations.
result Two variants of PathNNs are strictly more powerful than 1-WL algorithm.
New Max-Plus neural network exploits subgradient sparsity for efficient training.
problem Training Max-Plus neural networks is challenging due to dense subgradients.
method Proposes a sparse subgradient algorithm tailored to Max-Plus models.
result Achieves more efficient updates while retaining theoretical guarantees.
Bayesian method improves few-shot classification accuracy.
problem Few-shot classification with small labeled datasets.
method Gaussian process classifier with Pólya-Gamma augmentation and one-vs-each softmax.
result Improved accuracy and uncertainty quantification.
The paper introduces a new method to improve model generalization by routing model copies through permutations.
problem Improving model generalization in machine learning.
method The method replicates a model \(M\) times and rewire the contexts in which local learning messages are computed using permutations.
result The method improves generalization by structured message sharing rather than coupling parameters.
Paper proposes method for generating paths of stochastic volatility CGMY process for option pricing.
problem Generating accurate sample paths for stochastic volatility models for option pricing.
method Monte-Carlo method for European and American options, least square regression for calibration.
result Calibrated model parameters to S\&P 100 index options market using path-dependent options.
Study path-dependent affine models under uncertain parameters for financial applications.
problem Valuation of path-dependent financial derivatives under parameter uncertainty.
method Developed path-dependent setting for value function, established dynamic programming principle, approximated functional derivatives with neural networks.
result Efficient numerical methods for valuation of complex financial derivatives under parameter uncertainty.
Tensor-EM method learns MoLDS from complex, noisy data.
problem Modeling diverse temporal dynamics in neural data.
method Tensor-based moment method followed by EM updates.
result Tensor-EM achieves more reliable recovery and robustness.
SigMA uses signatures and attention to estimate parameters in fBm-driven SDEs.
problem Estimating parameters in SDEs driven by fBm is challenging due to non-Markovian and semimartingale issues.
method SigMA integrates path signatures with multi-head self-attention, using convolutional and MLP layers.
result SigMA outperforms other methods in accuracy, robustness, and model compactness.
Solves complex equation for specific geometric solitons.
problem Solving complex Monge-Ampère equation for specific geometric solitons.
method Aubin continuity path and continuity method.
result Initial value of the path parameter has a solution and is open to all.
Framework for safely updating machine learning models.
problem Continuous updates to machine learning models can lead to unintended consequences.
method Formalizes the problem as computing the largest locally invariant domain (LID), uses tractable primal-dual formulation.
result Matches or exceeds heuristic baselines for avoiding forgetting while providing formal safety guarantees.
In several recently proposed stochastic optimization methods (e.g. RMSProp, Adam, Adadelta), parameter updates are scaled by the inverse square roots of exponential moving averages of squared past gradients. Maintaining these per-parameter second-moment estimators requires memory equal to the number of parameters. For …
Study on rich regime training in deep learning, finding active parameters in bottom layers.
problem Understanding the practical success of deep learning models.
method Empirical study on rich regime training with benchmark datasets, re-initialization analysis, and probabilistic Layer-Wise Sparse SGD.
result Probabilistic Layer-Wise Sparse SGD matches vanilla SGD's generalization performance with improved efficiency.
A new FL algorithm reduces communication overhead by selectively updating model parameters.
problem Data heterogeneity and communication overhead in federated learning.
method Uses age of information metric to selectively update model parameters and group clients with similar data.
result Our method can expedite training and surpass other communication-efficient strategies in efficiency.
Paper tackles non-Markovian control problems with new learning methods.
problem Non-Markovian stochastic control problems with unknown parameters.
method Off-model training and importance sampling for deep neural network approximation.
result Quantitative error bounds for adaptive learning under model uncertainty.
PSiLON Net uses L1 weight normalization and 1-path-norm regularization for efficient learning and sparsity.
problem Efficient learning and sparsity in neural networks with limited data.
method PSiLON Net employs L1 weight normalization and 1-path-norm regularization to simplify the 1-path-norm and achieve efficient learning and near-sparse parameters. result PSiLON Net achieves reliable optimization and strong performance in the small data regime.
Capsule network (CapsNet) was introduced as an enhancement over convolutional neural networks, supplementing the latter's invariance properties with equivariance through pose estimation. CapsNet achieved a very decent performance with a shallow architecture and a significant reduction in parameters count. However, the …
A very simple event frequency approximation algorithm that is sensitive to event timeliness is suggested. The algorithm iteratively updates categorical click-distribution, producing (path of) a random walk on a standard n-dimensional simplex. Under certain conditions, this random walk is self-similar and corresponds …
The paper deals with learning probability distributions of observed data by artificial neural networks. We suggest a so-called gradient conjugate prior (GCP) update appropriate for neural networks, which is a modification of the classical Bayesian update for conjugate priors. We establish a connection between the gradi…
Method learns dynamics from noisy partial observations.
problem Reconstructing stochastic dynamical systems from indirect noisy data.
method Amortized path generation method for nonlinear stochastic filtering.
result Learned conditional path generator quantifies uncertainty.
Left invariant metrics induced by the p-norms of the trace in the matrix algebra are studied on the general lineal group. By means of the Euler-Lagrange equations, existence and uniqueness of extremal paths for the length functional are established, and regularity properties of these extremal paths are obtained. Minimi…
New model predicts implied volatility using past asset price paths.
problem Forecasting implied volatility surfaces and asset prices.
method Proposes a new model using past asset price trajectories to predict implied volatility.
result Large part of implied volatility movements can be explained by past returns and squares.
GIST adapts HMC by tuning parameters based on position and momentum.
problem Locally adaptive sampling in Hamiltonian Monte Carlo.
method GIST uses Gibbs sampling to adaptively tune HMC parameters.
result GIST improves sampling efficiency for high-dimensional models.