arXiv research

New method explains predictive uncertainty by focusing on second-order effects.

Paper discusses optimal CP for second-order predictions.

Proposes a new stock prediction method that accounts for market dynamics.

Second-order economic theory considers new variables to improve price volatility predictions.

A new framework for measuring uncertainty in machine learning models.

Improved prediction algorithm for 'easy' sequences with reduced regret.

New findings show second-order scoring rules can't accurately represent epistemic uncertainty.

With the rapid adoption of machine learning systems in sensitive applications, there is an increasing need to make black-box models explainable. Often we want to identify an influential group of training samples in a particular test prediction for a given machine learning model. Existing influence functions tackle this…

Loss minimisation fails to capture epistemic uncertainty in second-order predictors.

Second-order methods fail to fully quantify epistemic uncertainty, leading to biased predictions.

Paper introduces variance-based measures for second-order uncertainty quantification in classification problems.

Data whitening and second order optimization harm generalization by reducing access to dataset information.

Optimistic method adapted for faster convex-concave min-max problems.

New method speeds up deep learning optimization.

Dynamic Boltzmann Machine (DyBM) has been shown highly efficient to predict time-series data. Gaussian DyBM is a DyBM that assumes the predicted data is generated by a Gaussian distribution whose first-order moment (mean) dynamically changes over time but its second-order moment (variance) is fixed. However, in many fi…

We present Ordinary Differential Equation Variational Auto-Encoder (ODE$^2$VAE), a latent second order ODE model for high-dimensional sequential data. Leveraging the advances in deep generative models, ODE$^2$VAE can simultaneously learn the embedding of high dimensional trajectories and infer arbitrarily complex conti…

This paper investigates different vector step-size adaptation approaches for non-stationary online, continual prediction problems. Vanilla stochastic gradient descent can be considerably improved by scaling the update with a vector of appropriately chosen step-sizes. Many methods, including AdaGrad, RMSProp, and AMSGra…

Virtually any model we use in machine learning to make predictions does not perfectly represent reality. So, most of the learning happens under model misspecification. In this work, we present a novel analysis of the generalization performance of Bayesian model averaging under model misspecification and i.i.d. data usi…

In this paper, we consider the problem of prediction with expert advice in dynamic environments. We choose tracking regret as the performance metric and develop two adaptive and efficient algorithms with data-dependent tracking regret bounds. The first algorithm achieves a second-order tracking regret bound, which impr…

Derives formulae for general permutation equivariant layers and presents a second order graph variational encoder.

Improved robustness in optimization methods using second-order information.

Efficient Bayesian FL method improves predictive accuracy and uncertainty estimates.

Paper introduces new metrics for evaluating model accuracy.

We aim to design strategies for sequential decision making that adjust to the difficulty of the learning problem. We study this question both in the setting of prediction with expert advice, and for more general combinatorial decision tasks. We are not satisfied with just guaranteeing minimax regret rates, but we want …

New method improves model risk prediction using cross-audit projection.

GAPA method provides efficient uncertainty quantification for pretrained networks.

Solves Merton's investment-consumption problem with certainty equivalent approach.

Paper introduces arctan pinball loss for XGBoost quantile regression.

Second-order optimizers retain residual information after data deletion, affecting machine unlearning.

Exact second-order optimization for deep learning reduces computational cost and improves performance.

SOLBP extends efficient inference to uncertain Bayesian networks.

Developed a theory of local convexity for second order differential equations on Lie algebroids.

Factorization machines (FMs) are a supervised learning approach that can use second-order feature combinations even when the data is very high-dimensional. Unfortunately, despite increasing interest in FMs, there exists to date no efficient training algorithm for higher-order FMs (HOFMs). In this paper, we present the …

This paper proposes a stochastic model using the concept of Markov chains for the inter-state transitions of the millisecond order quasi-stable phase synchronized patterns or synchrostates, found in multi-channel Electroencephalogram (EEG) signals. First and second order transition probability matrices are estimated fo…

Paper examines risk measure expansions under FGM dependence, improving accuracy at extreme levels.

We study online aggregation of the predictions of experts, and first show new second-order regret bounds in the standard setting, which are obtained via a version of the Prod algorithm (and also a version of the polynomially weighted average algorithm) with multiple learning rates. These bounds are in terms of excess l…

Negative step sizes improve second-order methods for neural networks.

Simplified argument for second order estimate in quaternionic Calabi-Yau problem.

Paper studies second order tail probabilities in risk models.

A dynamical system on the total space of the fibre bundle of second order accelerations, $T^2M$, is defined as a third order vector field $S$ on $T^2M$, called semispray, which is mapped by the second order tangent structure into one of the Liouville vector field. For a regular Lagrangian of second order we prove that …

Paper studies second order symmetric parallel tensors in generalized f.pk-space forms.

We introduce a new recursive aggregation procedure called Bernstein Online Aggregation (BOA). The exponential weights include an accuracy term and a second order term that is a proxy of the quadratic variation as in Hazan and Kale (2010). This second term stabilizes the procedure that is optimal in different senses. We…

Paper generalizes connections between Lie groups and affine connections.

A method for inferring motility models and heterogeneity from particle trajectories.

In this present paper, we study geometric structures of rank two prolongations of implicit second-order partial differential equations (PDEs) for two independent and one dependent variables and characterize the type of these PDEs by the topology of fibers of the rank two prolongations. Moreover, by using properties of …

In this paper, we study stochastic non-convex optimization with non-convex random functions. Recent studies on non-convex optimization revolve around establishing second-order convergence, i.e., converging to a nearly second-order optimal stationary points. However, existing results on stochastic non-convex optimizatio…

AdamQLR optimizes Adam with K-FAC heuristics, achieving comparable performance to tuned benchmarks.

The quantification of diversification benefits due to risk aggregation plays a prominent role in the (regulatory) capital management of large firms within the financial industry. However, the complexity of today's risk landscape makes a quantifiable reduction of risk concentration a challenging task. In the present pap…