arXiv research

Bayesian neural networks learn weights with closed-form updates.

Paper introduces robust Gaussian process regression without sacrificing computational efficiency.

In this letter, we generalize the convolutional NMF by taking the $β$-divergence as the contrast function and present the correct multiplicative updates for its factors in closed form. The new updates unify the $β$-NMF and the convolutional NMF. We state why almost all of the existing updates are inexact and approximat…

Regression problems that have closed-form solutions are well understood and can be easily implemented when the dataset is small enough to be all loaded into the RAM. Challenges arise when data is too big to be stored in RAM to compute the closed form solutions. Many techniques were proposed to overcome or alleviate the…

Improved HGF networks avoid negative precision errors in volatility updates.

Paper proposes SSSE, an efficient method to erase samples from trained models.

New algorithm closes empirical gap in PFSGD performance.

This paper deals with the concept of curvature of framed space curves, their higher-order derivatives, variations, and co-rotational derivatives. We realize that parametrizing rotation tensor using the Gibbs vector is effective in deriving a closed form formula to obtain any order derivative of the curvature tensor as …

Optimizes neural networks' last layer with closed-form solutions.

Expectation-Maximization (EM) is a prominent approach for parameter estimation of hidden (aka latent) variable models. Given the full batch of data, EM forms an upper-bound of the negative log-likelihood of the model at each iteration and updates to the minimizer of this upper-bound. We first provide a "model level" in…

A new method for machine learning updates reduces complexity and improves robustness.

We present ARU, an Adaptive Recurrent Unit for streaming adaptation of deep globally trained time-series forecasting models. The ARU combines the advantages of learning complex data transformations across multiple time series from deep global models, with per-series localization offered by closed-form linear models. Un…

Iterative tilting fine-tunes diffusion models for reward-tilted distributions.

Robust state-space radio interferometric imaging using Stochastic Approximation Expectation Maximization

Nonnegative matrix factorization (NMF) has attracted much attention in the last decade as a dimension reduction method in many applications. Due to the explosion in the size of data, naturally the samples are collected and stored distributively in local computational nodes. Thus, there is a growing need to develop algo…

LF-IBIS learns optimal policies online without explicit likelihood.

The paper analyzes dynamics of momentum in high dimensions with sparse updates.

We define and discuss the first sparse coding algorithm based on closed-form EM updates and continuous latent variables. The underlying generative model consists of a standard `spike-and-slab' prior and a Gaussian noise model. Closed-form solutions for E- and M-step equations are derived by generalizing probabilistic P…

To ensure stability of learning, state-of-the-art generalized policy iteration algorithms augment the policy improvement step with a trust region constraint bounding the information loss. The size of the trust region is commonly determined by the Kullback-Leibler (KL) divergence, which not only captures the notion of d…

E$^2$M optimizes tensor density estimation by relaxing $α$-divergence to KL-divergence.

ITSPACE improves covariance alignment faster than other methods.

Stochastic variational inference offers an attractive option as a default method for differentiable probabilistic programming. However, the performance of the variational approach depends on the choice of an appropriate variational family. Here, we introduce automatic structured variational inference (ASVI), a fully au…

New algorithm for estimating MLR parameters with non-Gaussian noise.

We shed new insights on the two commonly used updates for the online $k$-PCA problem, namely, Krasulina's and Oja's updates. We show that Krasulina's update corresponds to a projected gradient descent step on the Stiefel manifold of the orthonormal $k$-frames, while Oja's update amounts to a gradient descent step using…

We propose a scalable stochastic variational approach to GP classification building on Polya-Gamma data augmentation and inducing points. Unlike former approaches, we obtain closed-form updates based on natural gradients that lead to efficient optimization. We evaluate the algorithm on real-world datasets containing up…

We propose a new algorithm for solving the graph-fused lasso (GFL), a method for parameter estimation that operates under the assumption that the signal tends to be locally constant over a predefined graph structure. Our key insight is to decompose the graph into a set of trails which can then each be solved efficientl…

We introduce a minorization-maximization approach to optimizing common measures of discovery significance in high energy physics. The approach alternates between solving a weighted binary classification problem and updating class weights in a simple, closed-form manner. Moreover, an argument based on convex duality sho…

T-PSDA improves speaker recognition accuracy on toroidal submanifolds.

Gradient-based methods for games suffer from discrete update steps that cause drift, affecting performance.

A new method for speaker recognition on hyperspheres improves on PLDA's limitations.

Geodesic descent optimizes likelihood in dually flat spaces.

Efficient algorithm for mobile health provides timely physical activity suggestions.

Efficiently identifies important variables in binary outcomes using variational Bayes.

Online learning makes sequence of decisions with partial data arrival where next movement of data is unknown. In this paper, we have presented a new technique as multiple times weight updating that update the weight iteratively forsame instance. The proposed technique analyzed with popular state-of-art algorithms from …

A new framework for PPLS combines noise estimation, optimization, and calibration.

CILF learns adaptive embeddings for class-incremental learning with novel class detection and model update.

Variational inference (VI) has become the method of choice for fitting many modern probabilistic models. However, practitioners are faced with a fragmented literature that offers a bewildering array of algorithmic options. First, the variational family. Second, the granularity of the updates e.g. whether the updates ar…

NeuralSurv models survival analysis with Bayesian uncertainty.

BaM improves BBVI by optimizing a score-based divergence, leading to faster convergence.

This paper tackles efficient federated learning for generalized linear bandits.

A new method for efficient neural network fine-tuning using queryable low-rank update atoms.

AdvPCA uses robust optimization to achieve sparse PCA without tuning.

This paper proposes an online tree-based Bayesian approach for reinforcement learning. For inference, we employ a generalised context tree model. This defines a distribution on multivariate Gaussian piecewise-linear models, which can be updated in closed form. The tree structure itself is constructed using the cover tr…

In this paper we consider the problem of minimizing composite objective functions consisting of a convex differentiable loss function plus a non-smooth regularization term, such as $L_1$ norm or nuclear norm, under Rényi differential privacy (RDP). To solve the problem, we propose two stochastic alternating direction m…

Paper introduces a fast, robust, scalable method for detecting changes in data streams.

Matrix Factorization is a popular non-convex optimization problem, for which alternating minimization schemes are mostly used. They usually suffer from the major drawback that the solution is biased towards one of the optimization variables. A remedy is non-alternating schemes. However, due to a lack of Lipschitz conti…

Recently, considerable research attention has been paid to network embedding, a popular approach to construct feature vectors of vertices. Due to the curse of dimensionality and sparsity in graphical datasets, this approach has become indispensable for machine learning tasks over large networks. The majority of existin…

Paper addresses online alignment of large language models under uncertain preference feedback.