arXiv research

A new GAN model $α$-GAN with tunable loss function addresses gradient vanishing and mode collapse issues.

We present $α$-loss, $α\in [1,\infty]$, a tunable loss function for binary classification that bridges log-loss ($α=1$) and $0$-$1$ loss ($α= \infty$). We prove that $α$-loss has an equivalent margin-based form and is classification-calibrated, two desirable properties for a good surrogate loss function for the ideal y…

Improved GAN training stability through tunable classification losses.

We introduce a tunable loss function called $α$-loss, parameterized by $α\in (0,\infty]$, which interpolates between the exponential loss ($α= 1/2$), the log-loss ($α= 1$), and the 0-1 loss ($α= \infty$), for the machine learning setting of classification. Theoretically, we illustrate a fundamental connection between $…

The recently proposed "generalized min-max" (GMM) kernel can be efficiently linearized, with direct applications in large-scale statistical learning and fast near neighbor search. The linearized GMM kernel was extensively compared in with linearized radial basis function (RBF) kernel. On a large number of classificatio…

Dual-objective GANs reduce training instabilities with tunable α-loss parameters.

TSSM splits neural networks for parallel training with minimal accuracy loss.

We analyze the optimization landscape of α-loss in logistic models.

PINNs struggle with increasingly complex ODEs, especially when parameters control their complexity.

Paper introduces MRCs that minimize worst-case 0-1 loss, providing tight performance guarantees.

While tree methods have been popular in practice, researchers and practitioners are also looking for simple algorithms which can reach similar accuracy of trees. In 2010, (Ping Li UAI'10) developed the method of "abc-robust-logitboost" and compared it with other supervised learning methods on datasets used by the deep …

This paper enhances the Random Survival Forest model for better predictive maintenance.

Introduces tunable basis functions for Gaussian processes.

Enhances multi-modular models by directing information flow between components.

We propose a method to build quantum memristors in quantum photonic platforms. We firstly design an effective beam splitter, which is tunable in real-time, by means of a Mach-Zehnder-type array with two equal 50:50 beam splitters and a tunable retarder, which allows us to control its reflectivity. Then, we show that th…

MLtuner automatically tunes settings for training tunables (such as the learning rate, the momentum, the mini-batch size, and the data staleness bound) that have a significant impact on large-scale machine learning (ML) performance. Traditionally, these tunables are set manually, which is unsurprisingly error-prone and…

Modern supervised machine learning algorithms involve hyperparameters that have to be set before running them. Options for setting hyperparameters are default values from the software package, manual configuration by the user or configuring them for optimal predictive performance by a tuning procedure. The goal of this…

When humans learn a new concept, they might ignore examples that they cannot make sense of at first, and only later focus on such examples, when they are more useful for learning. We propose incorporating this idea of tunable sensitivity for hard examples in neural network learning, using a new generalization of the cr…

AugLoss combines data augmentation and robust loss functions for robust DL models.

In this paper, we propose a Dual Focal Loss (DFL) function, as a replacement for the standard cross entropy (CE) function to achieve a better treatment of the unbalanced classes in a dataset. Our DFL method is an improvement on the recently reported Focal Loss (FL) cross-entropy function, which proposes a scaling metho…

Generalized dual discriminator GANs improve upon traditional GANs by using two discriminators and a flexible loss function.

Learning in Deep Neural Networks (DNN) takes place by minimizing a non-convex high-dimensional loss function, typically by a stochastic gradient descent (SGD) strategy. The learning process is observed to be able to find good minimizers without getting stuck in local critical points, and that such minimizers are often …

Meta-Semi learns to optimize SSL with minimal hyper-parameter tuning.

Approximate dynamic programming (ADP) has proven itself in a wide range of applications spanning large-scale transportation problems, health care, revenue management, and energy systems. The design of effective ADP algorithms has many dimensions, but one crucial factor is the stepsize rule used to update a value functi…

This paper shows DPPs can outperform random coresets in machine learning tasks.

Using unitary (instead of general) matrices in artificial neural networks (ANNs) is a promising way to solve the gradient explosion/vanishing problem, as well as to enable ANNs to learn long-term correlations in the data. This approach appears particularly promising for Recurrent Neural Networks (RNNs). In this work, w…

SC-InfoNCE improves InfoNCE for feature clustering in contrastive learning.

New research shows LLMs can't be explained by statistical generalization alone.

Common privacy enhancing technologies fail to effectively hide certain statistical aspects of encrypted traffic, namely individual packets length, packets direction and, packets timing. Recent researches have shown that using such attributes, an adversary is able to extract various information from the encrypted traffi…

Redundancy improves learning stability and generalization in structured systems.

Many optimization problems can be cast into the maximum satisfiability (MAX-SAT) form, and many solvers have been developed for tackling such problems. To evaluate a MAX-SAT solver, it is convenient to generate hard MAX-SAT instances with known solutions. Here, we propose a method of generating weighted MAX-2-SAT insta…

Our team of dance artists, physicists, and machine learning researchers has collectively developed several original, configurable machine-learning tools to generate novel sequences of choreography as well as tunable variations on input choreographic sequences. We use recurrent neural network and autoencoder architectur…

RETR improves indoor radar perception with a novel transformer model.

Bayesian method for semi-structured models accounts for both types of uncertainty.

Novel OTT method for cryptocurrency trading offers high annualized profit.

Quantum neural networks need both data-dependent and trainable unitaries for effective geometric deformation.

ARS visualization improves t-SNE dynamics with tunable attraction and repulsion.

Knowledge bases (KBs) are the backbone of many ubiquitous applications and are thus required to exhibit high precision. However, for KBs that store subjective attributes of entities, e.g., whether a movie is "kid friendly", simply estimating precision is complicated by the inherent ambiguity in measuring subjective phe…

Node embedding is the task of extracting informative and descriptive features over the nodes of a graph. The importance of node embeddings for graph analytics, as well as learning tasks such as node classification, link prediction and community detection, has led to increased interest on the problem leading to a number…

We present an algorithm to generate synthetic datasets of tunable difficulty on classification of Morse code symbols for supervised machine learning problems, in particular, neural networks. The datasets are spatially one-dimensional and have a small number of input features, leading to high density of input informatio…

Kernel ridgeless regression with random features shows good generalization without explicit regularization.

Paper defines hyperparameter importance for efficient tuning.

New Q-learning methods improve sample complexity for MDPs with continuous states.

Margin maximization in the hard-margin sense, proposed as feature elimination criterion by the MFE-LO method, is combined here with data radius utilization to further aim to lower generalization error, as several published bounds and bound-related formulations pertaining to lowering misclassification risk (or error) pe…

We develop a scalable method for Bayesian neural networks with stochastic differential equations.

Wiener-Granger causality is a widely used framework of causal analysis for temporally resolved events. We introduce a new measure of Wiener-Granger causality based on kernelization of partial canonical correlation analysis with specific advantages in the context of large high-dimensional data. The introduced measure is…

In the spirit of behavioral finance, we study the process of opinion formation among investors using a variant of the 2D Voter Model with a tunable social temperature. Further, a feedback acting on the temperature is introduced, such that social temperature reacts to market imbalances and thus becomes time dependent. I…

A l1-norm penalized orthogonal forward regression (l1-POFR) algorithm is proposed based on the concept of leaveone- out mean square error (LOOMSE). Firstly, a new l1-norm penalized cost function is defined in the constructed orthogonal space, and each orthogonal basis is associated with an individually tunable regulari…