arXiv research

New reweighted losses improve diffusion model training and image quality.

Domain adaptation provides a powerful set of model training techniques given domain-specific training data and supplemental data with unknown relevance. The techniques are useful when users need to develop models with data from varying sources, of varying quality, or from different time ranges. We build CrossTrainer, a…

Paper explains learning property of logistic and softmax losses for balanced and imbalanced class data.

This paper improves multi-label ranking by reweighting univariate losses, enhancing consistency and performance.

Toy model study shows resampling/reweighting can improve feature learning in imbalanced classification.

Improved learning to reweight using deep interactions between student and teacher models.

Study binary choice with asymmetric loss, offering simple solutions.

New loss function handles uncertain constraints in CSLO problems.

New loss function restores importance weighting in overparameterized models.

Least squares kernel based methods have been widely used in regression problems due to the simple implementation and good generalization performance. Among them, least squares support vector regression (LS-SVR) and extreme learning machine (ELM) are popular techniques. However, the noise sensitivity is a major bottlene…

Reweighting improves GAN accuracy without sacrificing statistical power.

The paper justifies time-dependent loss reweighting schemes for flow matching and diffusion models.

Study efficient learning of halfspaces with constant noise tolerance.

Iteratively reweighted $\ell_1$ algorithm is a popular algorithm for solving a large class of optimization problems whose objective is the sum of a Lipschitz differentiable loss function and a possibly nonconvex sparsity inducing regularizer. In this paper, motivated by the success of extrapolation techniques in accele…

Algorithm corrects bias in classification data.

A new method improves adversarial robustness by optimizing importance weights.

Proposes active sampling for improving fairness in machine learning.

This paper investigates tradeoffs among optimization errors, statistical rates of convergence and the effect of heavy-tailed errors for high-dimensional robust regression with nonconvex regularization. When the additive errors in linear models have only bounded second moment, we show that iteratively reweighted $\ell_1…

Deep neural networks have been shown to be very powerful modeling tools for many supervised learning tasks involving complex input patterns. However, they can also easily overfit to training set biases and label noises. In addition to various regularizers, example reweighting algorithms are popular solutions to these p…

Iterative reweighted algorithms, as a class of algorithms for sparse signal recovery, have been found to have better performance than their non-reweighted counterparts. However, for solving the problem of multiple measurement vectors (MMVs), all the existing reweighted algorithms do not account for temporal correlation…

Support vector machines (SVMs) are an important tool in modern data analysis. Traditionally, support vector machines have been fitted via quadratic programming, either using purpose-built or off-the-shelf algorithms. We present an alternative approach to SVM fitting via the majorization--minimization (MM) paradigm. Alg…

Recent work shows GRW approaches do not improve over ERM in distributional shift.

In the early history of positive-unlabeled (PU) learning, the sample selection approach, which heuristically selects negative (N) data from U data, was explored extensively. However, this approach was later dominated by the importance reweighting approach, which carefully treats all U data as N data. May there be a new…

A novel approach reduces class imbalance in network traffic classification.

AFR simplifies reducing reliance on spurious features, improving model performance.

Deep neural networks (DNNs) have great expressive power, which can even memorize samples with wrong labels. It is vitally important to reiterate robustness and generalization in DNNs against label corruption. To this end, this paper studies the 0-1 loss, which has a monotonic relationship with an empirical adversary (r…

In this paper, we study a classification problem in which sample labels are randomly corrupted. In this scenario, there is an unobservable sample with noise-free labels. However, before being observed, the true labels are independently flipped with a probability $ρ\in[0,0.5)$, and the random label noise can be class-co…

We prove that, under low noise assumptions, the support vector machine with $N\ll m$ random features (RFSVM) can achieve the learning rate faster than $O(1/\sqrt{m})$ on a training set with $m$ samples when an optimized feature map is used. Our work extends the previous fast rate analysis of random features method from…

RR-GNN improves GNN prediction intervals by accounting for graph heteroscedasticity and structural biases.

This paper develops a novel deep recurrent neural network for sequential signal reconstruction.

Improved matching for multiple objects using a novel reweighting method.

Reweighted l1-algorithms have attracted a lot of attention in the field of applied mathematics. A unified framework of such algorithms has been recently proposed by Zhao and Li. In this paper we construct a few new examples of reweighted l1-methods. These functions are certain concave approximations of the l0-norm func…

Proposes a new method to improve regression models with reweighted samples.

Simplifies neural regression by combining two sub-networks for predictions and uncertainties.

New robust MPCA method handles casewise and cellwise outliers in tensor data.

Defense against user shilling attacks in collaborative filtering using edge reweighting.

A new convex loss function optimizes set predictions with balanced size and coverage.

We analyze a reweighted version of the Kikuchi approximation for estimating the log partition function of a product distribution defined over a region graph. We establish sufficient conditions for the concavity of our reweighted objective function in terms of weight assignments in the Kikuchi expansion, and show that a…

CurveRL optimizes large model reasoning by reweighting prompts based on their rank and density.

Unified analysis of reweighted least-squares algorithms for linear models.

Iteratively reweighted least squares (IRLS) is a widely-used method in machine learning to estimate the parameters in the generalised linear models. In particular, IRLS for L1 minimisation under the linear model provides a closed-form solution in each step, which is a simple multiplication between the inverse of the we…

Federated learning has a variety of applications in multiple domains by utilizing private training data stored on different devices. However, the aggregation process in federated learning is highly vulnerable to adversarial attacks so that the global model may behave abnormally under attacks. To tackle this challenge, …

New bounds for balanced classification improve understanding of imbalanced datasets.

New algorithm improves online binary classification with constant time complexity.

Recently, the paradigm of unfolding iterative algorithms into finite-length feed-forward neural networks has achieved a great success in the area of sparse recovery. Benefit from available training data, the learned networks have achieved state-of-the-art performance in respect of both speed and accuracy. However, the …

Resampling outperforms reweighting for correcting biased data in machine learning models.

Proposes a new robust expectile regression method for high-dimensional data.

This work presents a general framework for solving the low rank and/or sparse matrix minimization problems, which may involve multiple non-smooth terms. The Iteratively Reweighted Least Squares (IRLS) method is a fast solver, which smooths the objective function and minimizes it by alternately updating the variables an…