arXiv research

MTAdam optimizes multiple loss terms in neural models, balancing gradients dynamically.

New scalable algorithm for non-negative linear regression with entropy-regularized OT loss.

We stabilize MI-based losses by adding a regularization term, improving their performance and stability.

New approach shapes error distribution in long-term forecasting.

Model shows how banks' fears of future defaults can cause immediate financial stress.

Malware detection is a popular application of Machine Learning for Information Security (ML-Sec), in which an ML classifier is trained to predict whether a given file is malware or benignware. Parameters of this classifier are typically optimized such that outputs from the model over a set of input samples most closely…

New insights into training machine learning models with momentum.

Study on learning to defer with multiple experts using new surrogate losses.

We study online aggregation of the predictions of experts, and first show new second-order regret bounds in the standard setting, which are obtained via a version of the Prod algorithm (and also a version of the polynomially weighted average algorithm) with multiple learning rates. These bounds are in terms of excess l…

We present a TTS neural network that is able to produce speech in multiple languages. The proposed network is able to transfer a voice, which was presented as a sample in a source language, into one of several target languages. Training is done without using matching or parallel data, i.e., without samples of the same …

While momentum-based accelerated variants of stochastic gradient descent (SGD) are widely used when training machine learning models, there is little theoretical understanding on the generalization error of such methods. In this work, we first show that there exists a convex loss function for which the stability gap fo…

Improved SVRG with a coefficient reduces training loss in deep learning.

Method estimates multiple related Gaussian distributions using Laplacian regularization.

This work analyzes two methods for combining multiple binary labels in bipartite ranking.

Extended LSTMs improve volatility prediction by 20%.

HFformer outperforms LSTM in high-frequency trading with multiple signals.

This paper presents a new mathematical framework to analyze the loss functions of deep neural networks with ReLU functions. Furthermore, as as application of this theory, we prove that the loss functions can reconstruct the inputs of the training samples up to scalar multiplication (as vectors) and can provide the numb…

We propose a generic framework to calibrate accuracy and confidence of a prediction in deep neural networks through stochastic inferences. We interpret stochastic regularization using a Bayesian model, and analyze the relation between predictive uncertainty of networks and variance of the prediction scores obtained by …

This work includes a number of novel contributions for the multiple-source adaptation problem. We present new normalized solutions with strong theoretical guarantees for the cross-entropy loss and other similar losses. We also provide new guarantees that hold in the case where the conditional probabilities for the sour…

This paper tackles deferral learning with multiple experts, providing strong theoretical guarantees.

We introduce negative binomial matrix factorization (NBMF), a matrix factorization technique specially designed for analyzing over-dispersed count data. It can be viewed as an extension of Poisson matrix factorization (PF) perturbed by a multiplicative term which models exposure. This term brings a degree of freedom fo…

Paper uses deep Ritz method for solving stationary Schrödinger equation, proving convergence and feature emergence.

In this paper we study a class of insurance products where the policy holder has the option to insure $k$ of its annual Operational Risk losses in a horizon of $T$ years. This involves a choice of $k$ out of $T$ years in which to apply the insurance policy coverage by making claims against losses in the given year. The…

New algorithms avoid a dominant lower-order term in heavy-tailed loss settings.

Multi-task learning is a method for improving the generalizability of multiple tasks. In order to perform multiple classification tasks with one neural network model, the losses of each task should be combined. Previous studies have mostly focused on multiple prediction tasks using joint loss with static weights for tr…

In this paper, we propose a new loss function called generalized end-to-end (GE2E) loss, which makes the training of speaker verification models more efficient than our previous tuple-based end-to-end (TE2E) loss function. Unlike TE2E, the GE2E loss function updates the network in a way that emphasizes examples that ar…

This paper addresses the problem of time series forecasting for non-stationary signals and multiple future steps prediction. To handle this challenging task, we introduce DILATE (DIstortion Loss including shApe and TimE), a new objective function for training deep neural networks. DILATE aims at accurately predicting s…

Explains deep learning models and their geometric properties.

Study of estimation errors in surrogate loss minimizers, providing stronger guarantees than existing methods.

Unpaired image-to-image translation has attracted significant interest due to the invention of CycleGAN, a method which utilizes a combination of adversarial and cycle consistency losses to avoid the need for paired data. It is known that the CycleGAN problem might admit multiple solutions, and our goal in this paper i…

This paper analyzes optimal consumption strategies for loss-averse investors with multiplicative habit formation.

Study minimax rates for online learning with time-varying dynamics.

VCL adds uncertainty to contrastive learning models.

Proposes a method to ensure low losses across all subpopulations in large datasets.

We study long-term growth-optimal strategies on a simple market with linear proportional transaction costs. We show that several problems of this sort can be solved in closed form, and explicit the non-analytic dependance of optimal strategies and expected frictional losses of the friction parameter. We present one der…

Study on Bayesian deep linear networks with multiple outputs and convolutional layers.

A novel method detects multiple mitosis events and mitigates annotation gaps in phase-contrast microscopy.

The paper decomposes probabilistic scores into reliability, uncertainty, and information loss.

Proposes squentropy loss for improved classification accuracy and model calibration.

The paper presents Imbalance-XGBoost, a Python package that combines the powerful XGBoost software with weighted and focal losses to tackle binary label-imbalanced classification tasks. Though a small-scale program in terms of size, the package is, to the best of the authors' knowledge, the first of its kind which prov…

Study on learning to defer to multiple experts with consistent surrogates and confidence calibration.

Paper proposes a new anomaly detection method using Random Forest with Mallows-like criterion.

SGD's escape rate depends on log loss barrier, not linear loss barrier.

Paper develops upper-bounds for target general loss in multiple source DA and DG settings.

A hybrid loss framework improves time series forecasting by balancing global and component errors.

In this manuscript we propose two objective terms for neural image compression: a compression objective and a cycle loss. These terms are applied on the encoder output of an autoencoder and are used in combination with reconstruction losses. The compression objective encourages sparsity and low entropy in the activatio…

Unified framework for binary responses using AUC loss and low-rank constraint.

Proposes sigmoidF1 loss for multilabel classification, improving performance metrics.