arXiv research

We present a new anytime algorithm that achieves near-optimal regret for any instance of finite stochastic partial monitoring. In particular, the new algorithm achieves the minimax regret, within logarithmic factors, for both "easy" and "hard" problems. For easy problems, it additionally achieves logarithmic individual…

Model learns brevity by exposing to easy problems, improving efficiency without explicit length penalties.

In this paper, we consider the problem of minimizing the sum of two convex functions subject to linear linking constraints. The classical alternating direction type methods usually assume that the two convex functions have relatively easy proximal mappings. However, many problems arising from statistics, image processi…

LIBS2ML is a library based on scalable second order learning algorithms for solving large-scale problems, i.e., big data problems in machine learning. LIBS2ML has been developed using MEX files, i.e., C++ with MATLAB/Octave interface to take the advantage of both the worlds, i.e., faster learning using C++ and easy I/O…

A key problem in reinforcement learning for control with general function approximators (such as deep neural networks and other nonlinear functions) is that, for many algorithms employed in practice, updates to the policy or $Q$-function may fail to improve performance---or worse, actually cause the policy performance …

We define and study the problem of modular concept learning, that is, learning a concept that is a cross product of component concepts. If an element's membership in a concept depends solely on it's membership in the components, learning the concept as a whole can be reduced to learning the components. We analyze this …

We consider online learning problems where the aim is to achieve regret which is efficient in the sense that it is the same order as the lowest regret amongst K experts. This is a substantially stronger requirement that achieving $O(\sqrt{n})$ or $O(\log n)$ regret with respect to the best expert and standard algorithm…

Detection of dense cycles in graphs reveals a gap between easy detection and hard recovery.

In this paper we show all possible ramps where an object can move with constant speed under the effect of gravity and friction. The planar ramp are very easy to describe, just rotate a curve with velocity vector (tanh(as),sech(as)). Recall that tanh(as)^2+sech^2(as) = 1. Therefore, the solution of the planar constant s…

The major challenge of learning from multi-label data has arisen from the overwhelming size of label space which makes this problem NP-hard. This problem can be alleviated by gradually involving easy to hard tags into the learning process. Besides, the utilization of a diversity maintenance approach avoids overfitting …

For the existence of a branched covering Sigma~ --> Sigma between closed surfaces there are easy necessary conditions in terms of chi(Sigma~), chi(Sigma), orientability, the total degree, and the local degrees at the branching points. A classical problem dating back to Hurwitz asks whether these conditions are also suf…

Improved prediction algorithm for 'easy' sequences with reduced regret.

Model shows feature learning can improve neural scaling laws for hard tasks.

The paper explores when to prioritize easy or hard samples in learning tasks.

Most methods for decision-theoretic online learning are based on the Hedge algorithm, which takes a parameter called the learning rate. In most previous analyses the learning rate was carefully tuned to obtain optimal worst-case performance, leading to suboptimal performance on easy instances, for example when there ex…

Study how models represent features in naturalistic learning problems.

Neural model with parameterized algorithms improves graph CO problem solving.

We found an easy and quick post-learning method named "Icing on the Cake" to enhance a classification performance in deep learning. The method is that we train only the final classifier again after an ordinary training is done.

This paper will serve as an introduction to the body of work on robust subspace recovery. Robust subspace recovery involves finding an underlying low-dimensional subspace in a dataset that is possibly corrupted with outliers. While this problem is easy to state, it has been difficult to develop optimal algorithms due t…

We introduce a learning framework called learning using privileged information (LUPI) to the computer vision field. We focus on the prototypical computer vision problem of teaching computers to recognize objects in images. We want the computers to be able to learn faster at the expense of providing extra information du…

High-dimensional k-sample comparison is a common applied problem. We construct a class of easy-to-implement nonparametric distribution-free tests based on new tools and unexplored connections with spectral graph theory. The test is shown to possess various desirable properties along with a characteristic exploratory fl…

Solves local minima problems on smooth manifolds.

We aim to design strategies for sequential decision making that adjust to the difficulty of the learning problem. We study this question both in the setting of prediction with expert advice, and for more general combinatorial decision tasks. We are not satisfied with just guaranteeing minimax regret rates, but we want …

Study reveals DNNs prefer easy-to-learn cues over essential ones in image recognition.

We introduce GLR-klUCB, a novel algorithm for the piecewise iid non-stationary bandit problem with bounded rewards. This algorithm combines an efficient bandit algorithm, kl-UCB, with an efficient, parameter-free, changepoint detector, the Bernoulli Generalized Likelihood Ratio Test, for which we provide new theoretica…

Anomaly detection is not an easy problem since distribution of anomalous samples is unknown a priori. We explore a novel method that gives a trade-off possibility between one-class and two-class approaches, and leads to a better performance on anomaly detection problems with small or non-representative anomalous sample…

Motivated by applications in clinical trials and finance, we study the problem of online convex optimization (with bandit feedback) where the decision maker is risk-averse. We provide two algorithms to solve this problem. The first one is a descent-type algorithm which is easy to implement. The second algorithm, which …

Majority of state-of-the-art deep learning methods are discriminative approaches, which model the conditional distribution of labels given inputs features. The success of such approaches heavily depends on high-quality labeled instances, which are not easy to obtain, especially as the number of candidate classes increa…

The paper explores when linear system identification is hard or easy, especially for under-actuated systems.

In this note, we present a new averaging technique for the projected stochastic subgradient method. By using a weighted average with a weight of t+1 for each iterate w_t at iteration t, we obtain the convergence rate of O(1/t) with both an easy proof and an easy implementation. The new scheme is compared empirically to…

The paper develops a theory for iterative self-improvement of models, proving conditions for better performance with easy-to-hard curricula.

The elbow method for choosing k in k-means is unreliable and should be avoided.

In this paper, we propose a Dual Focal Loss (DFL) function, as a replacement for the standard cross entropy (CE) function to achieve a better treatment of the unbalanced classes in a dataset. Our DFL method is an improvement on the recently reported Focal Loss (FL) cross-entropy function, which proposes a scaling metho…

We consider a class of nonconvex nonsmooth optimization problems whose objective is the sum of a smooth function and a finite number of nonnegative proper closed possibly nonsmooth functions (whose proximal mappings are easy to compute), some of which are further composed with linear maps. This kind of problems arises …

A simple continuous relaxation for argsort improves performance and is easy to implement.

In Natural Language Processing (NLP) tasks, data often has the following two properties: First, data can be chopped into multi-views which has been successfully used for dimension reduction purposes. For example, in topic classification, every paper can be chopped into the title, the main text and the references. Howev…

This book is expository and is in Russian (sample English translation of two pages is given). It is shown how in the course of solution of interesting geometric problems (close to applications) naturally appear different notions of curvature, which distinguish given geometry from the 'ordinary' one. Direct elementary d…

We introduce elastic geodesic grids for easy-to-fabricate, deployable structures.

In this paper, we show that for every abelian subgroup $H$ of a Garside group, some conjugate $g^{-1}Hg$ consists of ultra summit elements and the centralizer of $H$ is a finite index subgroup of the normalizer of $H$. Combining with the results on translation numbers in Garside groups, we obtain an easy proof of the a…

Generative adversarial nets (GAN) has been successfully introduced for generating text to alleviate the exposure bias. However, discriminators in these models only evaluate the entire sequence, which causes feedback sparsity and mode collapse. To tackle these problems, we propose a novel mechanism. It first segments th…

We find optimal learning rate schedules for a random feature model.

A new method uses natural gradients for efficient distribution optimization.

Physics-informed neural networks simulate radiative transfer efficiently.

Simple method improves uncertainty estimation for distribution shifts.

Linear regression models are not as interpretable as commonly believed.

New algorithms solve word and conjugacy problems in braid group B3.

This note provides an easy construction of fake octagons.

Extends GENO framework for GPU optimization of constrained ML problems.