arXiv research

New estimator improves mutual information estimation.

This paper considers a multiple regression model and compares, under full model hypothesis, analytically as well as by simulation, the performance characteristics of some popular penalty estimators such as ridge regression, LASSO, adaptive LASSO, SCAD, and elastic net versus Least Squares Estimator, restricted estimato…

Estimating a large alphabet probability distribution from a limited number of samples is a fundamental problem in machine learning and statistics. A variety of estimation schemes have been proposed over the years, mostly inspired by the early work of Laplace and the seminal contribution of Good and Turing. One of the b…

We consider a problem of clustering a sequence of multinomial observations by way of a model selection criterion. We propose a form of a penalty term for the model selection procedure. Our approach subsumes both the conventional AIC and BIC criteria but also extends the conventional criteria in a way that it can be app…

We consider nonparametric estimation of $L_2$, Renyi-$α$ and Tsallis-$α$ divergences between continuous distributions. Our approach is to construct estimators for particular integral functionals of two densities and translate them into divergence estimators. For the integral functionals, our estimators are based on cor…

We propose an improved LASSO estimation technique based on Stein-rule. We shrink classical LASSO estimator using preliminary test, shrinkage, and positive-rule shrinkage principle. Simulation results have been carried out for various configurations of correlation coefficients ($r$), size of the parameter vector ($β$), …

Study improves regularity estimates for harmonic maps into ellipsoids.

We define and study the statistical models in exponential family form whose sufficient statistics are the degree distributions and the bi-degree distributions of undirected labelled simple graphs. Graphs that are constrained by the joint degree distributions are called $dK$-graphs in the computer science literature and…

Paper proposes CLAIR for efficient LLM fine-tuning across clients.

We introduce a novel aggregation method to efficiently perform image denoising. Preliminary filters are aggregated in a non-linear fashion, using a new metric of pixel proximity based on how the pool of filters reaches a consensus. We provide a theoretical bound to support our aggregation scheme, its numerical performa…

New fast estimation methods stemming from control theory lead to a fresh look at time series, which bears some resemblance to "technical analysis". The results are applied to a typical object of financial engineering, namely the forecast of foreign exchange rates, via a "model-free" setting, i.e., via repeated identifi…

A preliminary group classification of the class 2D nonlinear heat equations $u_t=f(x,y,u,u_x,u_y)(u_{xx}+u_{yy})$, where $f$ is arbitrary smooth function of the variables $x,y,u,u_x$ and $u_y$ using Lie method, is given. The paper is one of the few applications of an algebraic approach to the problem of group classific…

BYOV combines SSL and Bayesian methods for uncertainty estimation.

Preliminary group classification for a class of generalized inviscid Burger's equations in the general form $u_t+g(x, u)u_x = f(x, u)$ is given and additional equivalence transformations are found. Adduced results complete and essentially generalize recent works on the subject . A number of new interesting nonlinear in…

We consider stochastic zero-order optimization problems, which arise in settings from simulation optimization to reinforcement learning. We propose an adaptive sampling quasi-Newton method where we estimate the gradients of a stochastic function using finite differences within a common random number framework. We emplo…

Complexity measures for neural nets with general activations using path-based norms.

PQ-learning improves Q-learning by periodically updating target estimates.

New optimizer Eve uses examplewise gradients for better second-moment estimates.

Item-item collaborative filtering (CF) models are a well known and studied family of recommender systems, however current literature does not provide any theoretical explanation of the conditions under which item-based recommendations will succeed or fail. We investigate the existence of an ideal item-based CF method a…

The purpose of this article is to give a preliminary clarification on the relation between crossing number and crossing change. With a main focus on the span of X polynomial, we prove that, as our theorem claims, the crossing number of the link after crossing change can be estimated when certain conditions are met. At …

New Random Forest variants estimate heterogeneous treatment effects using Wasserstein distances.

A time series that represents daily values of the WIG index (the main index of Warsaw Stock Exchange) over last 5 years is examined. Non-Gaussian features of distributions of fluctuations, namely returns, over a time scale are considered. Some general properties like exponents of the long range correlation estimated by…

The PARAFAC tensor decomposition has enjoyed an increasing success in exploratory multi-aspect data mining scenarios. A major challenge remains the estimation of the number of latent factors (i.e., the rank) of the decomposition, which yields high-quality, interpretable results. Previously, we have proposed an automate…

We analyze the generalized Mallows model, a popular exponential model over rankings. Estimating the central (or consensus) ranking from data is NP-hard. We obtain the following new results: (1) We show that search methods can estimate both the central ranking pi0 and the model parameters theta exactly. The search is n!…

Automatically differentiable estimation for BLP model reduces bias in demand estimation.

In this paper, we consider the $V$-soliton equation which is a degenerate fully nonlinear equation introduced by La Nave and Tian in their work on Kähler-Ricci flow on symplectic quotients. One can apply the interpretation to study finite time singularities of the Kähler-Ricci flow. As in the case of Kähler-Einstein me…

Stochastic gradient algorithms have been the main focus of large-scale learning problems and they led to important successes in machine learning. The convergence of SGD depends on the careful choice of learning rate and the amount of the noise in stochastic estimates of the gradients. In this paper, we propose a new ad…

Principles of financial product synthesis from a few basic financial products constitute an interesting research topic inspired by Islamic finance. We make an effort to answer general questions that should be answered before starting to investigate the main issues concerning this topic with the formalization of financi…

We propose the BinaryGAN, a novel generative adversarial network (GAN) that uses binary neurons at the output layer of the generator. We employ the sigmoid-adjusted straight-through estimators to estimate the gradients for the binary neurons and train the whole network by end-to-end backpropogation. The proposed model …

A new imputation method estimates missing values by matching observed marginals from masked data.

Aleatoric uncertainty is an intrinsic property of ill-posed inverse and imaging problems. Its quantification is vital for assessing the reliability of relevant point estimates. In this paper, we propose an efficient framework for quantifying aleatoric uncertainty for deep residual learning and showcase its significant …

2D path planning in static environment is a well-known problem and one of the common ways to solve it is to 1) represent the environment as a grid and 2) perform a heuristic search for a path on it. At the same time 2D grid resembles much a digital image, thus an appealing idea comes to being -- to treat the problem as…

While machine learning systems show high success rate in many complex tasks, research shows they can also fail in very unexpected situations. Rise of machine learning products in safety-critical industries cause an increase in attention in evaluating model robustness and estimating failure probability in machine learni…

In machine learning and data mining, linear models have been widely used to model the response as parametric linear functions of the predictors. To relax such stringent assumptions made by parametric linear models, additive models consider the response to be a summation of unknown transformations applied on the predict…

Joint state and parameter estimation is a core problem for dynamic Bayesian networks. Although modern probabilistic inference toolkits make it relatively easy to specify large and practically relevant probabilistic models, the silver bullet---an efficient and general online inference algorithm for such problems---remai…

The paper uses facial keypoints to estimate post-surgical pain intensity.

A new estimator for state values in reinforcement learning reduces complexity and improves convergence.

A new RL algorithm POWR learns world models to estimate action-values.

Robo-advisors estimate clients' risk aversion using interactive questionnaires.

A relation between interest rates and inflation is presented using a two component economic model and a simple general principle. Preliminary results indicate a remarkable similarity to classical economic theories, in particular that of Wicksell.

In this paper, we present a new location fingerprinting database comprised of Wi-Fi received signal strength (RSS) and geomagnetic field intensity measured with multiple devices at a multi-floor building in Xi'an Jiatong-Liverpool University, Suzhou, China. We also provide preliminary results of localization and trajec…

In this paper we propose a new parameter-free method for trajectory classification which finds the best trajectory partition and dimension combination for robust trajectory classification. Preliminary experiments show that our approach is very promising.

New approach combines geometric and probabilistic methods to estimate manifold dimension in high-dimensional data.

Develops nonparametric regression for non-smooth functions using fractional Laplacian.

Game teaches ASD teenagers to imitate gestures.

We present a new algorithm for stochastic variational inference that targets at models with non-differentiable densities. One of the key challenges in stochastic variational inference is to come up with a low-variance estimator of the gradient of a variational objective. We tackle the challenge by generalizing the repa…

In this paper we study how to learn stochastic, multimodal transition dynamics in reinforcement learning (RL) tasks. We focus on evaluating transition function estimation, while we defer planning over this model to future work. Stochasticity is a fundamental property of many task environments. However, discriminative f…

New method bounds stochastic subgradient methods with heavy-tailed noise.