arXiv research

Paper presents a probabilistic diagnostic model for identifying and treating supervised learning degradation issues.

Study on W2S generalization with spurious correlations, proposing remedies.

SUMER updates deployed models with new data, improving performance.

The problem of inhomogeneous cluster densities has been a long-standing issue for distance-based and density-based algorithms in clustering and anomaly detection. These algorithms implicitly assume that all clusters have approximately the same density. As a result, they often exhibit a bias towards dense clusters in th…

Develops algorithm to reduce real-world inequality.

Sigmoid-type networks avoid vanishing gradients with regularization and rescaling.

A brief historical perspective is first given concerning financial crashes, - from the 17th till the 20th century. In modern times, it seems that log periodic oscillations are found before crashes in several financial indices. The same is found in sand pile avalanches on Sierpinski gaskets. A discussion pertains to the…

REMEDI improves neural entropy estimation across various tasks.

In retail, there are predictable yet dramatic time-dependent patterns in customer behavior, such as periodic changes in the number of visitors, or increases in customers just before major holidays. The current paradigm of multi-armed bandit analysis does not take these known patterns into account. This means that for a…

In recent years, Deep Learning has become the go-to solution for a broad range of applications, often outperforming state-of-the-art. However, it is important, for both theoreticians and practitioners, to gain a deeper understanding of the difficulties and limitations associated with common approaches and algorithms. W…

Paper introduces a new gradient statistic to improve deep learning convergence.

Existing feature selection methods fail to properly account for interactions between features when evaluating feature subsets. In this paper, we attempt to remedy this issue by using orthogonal variance decomposition to evaluate features. The orthogonality of the decomposition allows us to directly calculate the total …

Nested Cavity Classifier (NCC) is a classification rule that pursues partitioning the feature space, in parallel coordinates, into convex hulls to build decision regions. It is claimed in some literatures that this geometric-based classifier is superior to many others, particularly in higher dimensions. First, we give …

We use automatic speech recognition to assess spoken English learner pronunciation based on the authentic intelligibility of the learners' spoken responses determined from support vector machine (SVM) classifier or deep learning neural network model predictions of transcription correctness. Using numeric features produ…

The amount of information in the form of features and variables avail- able to machine learning algorithms is ever increasing. This can lead to classifiers that are prone to overfitting in high dimensions, high di- mensional models do not lend themselves to interpretable results, and the CPU and memory resources necess…

New algorithm improves group fairness in social classification problems by exploiting performativity.

This paper has been withdrawn by the authors, because it has been made obsolete by the detailed expositions in our papers in arXiv:0812.4885 (the mathematics part) and arXiv:0812.4737 (the economics part).

Blind source separation algorithms such as independent component analysis (ICA) are widely used in the analysis of neuroimaging data. In order to leverage larger sample sizes, different data holders/sites may wish to collaboratively learn feature representations. However, such datasets are often privacy-sensitive, prec…

Paper proposes a modified fairness constraint to address shortcomings of counterfactual fairness.

Machine learning, especially deep neural networks, has been rapidly developed in fields including computer vision, speech recognition and reinforcement learning. Although Mini-batch SGD is one of the most popular stochastic optimization methods in training deep networks, it shows a slow convergence rate due to the larg…

The paper argues that normalized mutual information is biased in clustering and community detection.

Despite strong performance on a variety of tasks, neural sequence models trained with maximum likelihood have been shown to exhibit issues such as length bias and degenerate repetition. We study the related issue of receiving infinite-length sequences from a recurrent language model when using common decoding algorithm…

Recent work in metric learning has significantly improved the state-of-the-art in k-nearest neighbor classification. Support vector machines (SVM), particularly with RBF kernels, are amongst the most popular classification algorithms that uses distance metrics to compare examples. This paper provides an empirical analy…

Improved risk-sensitive RL with exponential Bellman equation and better regret bounds.

New methods improve temporal difference learning for policy evaluation in Markov decision processes.

Improved Frank-Wolfe algorithm speeds up training of differentially private LASSO models.

The paper studies randomized approximations of Tukey's depth for log-concave isotropic data.

We develop new techniques in the theory of convex surfaces to prove complete classification results for tight contact structures on lens spaces, solid tori, and T^2 X I. Erratum: In this note we seek to remedy errors which appeared in version 2 and were propagated in subsequent papers.

Latent class model (LCM), which is a finite mixture of different categorical distributions, is one of the most widely used models in statistics and machine learning fields. Because of its non-continuous nature and the flexibility in shape, researchers in practice areas such as marketing and social sciences also frequen…

Study on deleting user data in linear regression models to maintain limited memory.

Paper develops an online EM algorithm for graph signal inference from streaming data.

New fairness criteria for algorithmic recourse actions that consider causal relationships.

Clustering algorithms have become a popular tool in computer security to analyze the behavior of malware variants, identify novel malware families, and generate signatures for antivirus systems. However, the suitability of clustering algorithms for security-sensitive settings has been recently questioned by showing tha…

We present a method of discovering governing differential equations from data without the need to specify a priori the terms to appear in the equation. The input to our method is a dataset (or ensemble of datasets) corresponding to a particular solution (or ensemble of particular solutions) of a differential equation. …

Improved TSK fuzzy regression models with MBGD-RDA and rule pruning.

Rather than proposing a new method, this paper investigates an issue present in existing learning algorithms. We study the learning dynamics of reinforcement learning (RL), specifically a characteristic coupling between learning and data generation that arises because RL agents control their future data distribution. I…

COPOD detects outliers efficiently and interpretable using copulas.

Financial markets are complex environments that produce enormous amounts of noisy and non-stationary data. One fundamental problem is online portfolio selection, the goal of which is to exploit this data to sequentially select portfolios of assets to achieve positive investment outcomes while managing risks. Various al…

Deep Neural Network has proved its potential in various perception tasks and hence become an appealing option for interpretation and data processing in security sensitive systems. However, security-sensitive systems demand not only high perception performance, but also design robustness under various circumstances. Unl…

Every smooth fiber bundle admits a complete (Ehresmann) connection. This result appears in several references, with a proof on which we have found a gap, that does not seem possible to remedy. In this note we provide a definite proof for this fact, explain the problem with the previous one, and illustrate with examples…

The vector of periodic, compound returns of a typical investment portfolio is almost never a convex combination of the return vectors of the securities in the portfolio. As a result the ex post version of Harry Markowitz's "standard mean-variance portfolio selection model" does not apply to compound return data. We pro…

In reinforcement learning (RL) research, it is common to assume access to direct online interactions with the environment. However in many real-world applications, access to the environment is limited to a fixed offline dataset of logged experience. In such settings, standard RL algorithms have been shown to diverge or…

Adversarial example generation becomes a viable method for evaluating the robustness of a machine learning model. In this paper, we consider hard-label black-box attacks (a.k.a. decision-based attacks), which is a challenging setting that generates adversarial examples based on only a series of black-box hard-label que…

Adversarial attacks on deep learning models have compromised their performance considerably. As remedies, a lot of defense methods were proposed, which however, have been circumvented by newer attacking strategies. In the midst of this ensuing arms race, the problem of robustness against adversarial attacks still remai…

Improved knowledge gradient (iKG) outperforms the original KG algorithm in best arm identification problems.

Modern stochastic optimization methods often rely on uniform sampling which is agnostic to the underlying characteristics of the data. This might degrade the convergence by yielding estimates that suffer from a high variance. A possible remedy is to employ non-uniform importance sampling techniques, which take the stru…

This work examines robust MCMC for pathological distributions.

Naturalistic driving trajectories are crucial for the performance of autonomous driving algorithms. However, most of the data is collected in safe scenarios leading to the duplication of trajectories which are easy to be handled by currently developed algorithms. When considering safety, testing algorithms in near-miss…