This paper develops a novel deep recurrent neural network for sequential signal reconstruction.
problem Sequential signal reconstruction from low-dimensional measurements.
method Unfolding a reweighted ℓ1-ℓ1 minimization algorithm to design a deep recurrent neural network. result The proposed reweighted-RNN significantly outperforms existing RNN models in sequential frame reconstruction.
Reweighting improves GAN accuracy without sacrificing statistical power.
problem Improving the fidelity of generative models.
method Post-hoc reweighting function applied to generated examples.
result Weighted GAN examples significantly improve accuracy.
Reweighted l1-algorithms have attracted a lot of attention in the field of applied mathematics. A unified framework of such algorithms has been recently proposed by Zhao and Li. In this paper we construct a few new examples of reweighted l1-methods. These functions are certain concave approximations of the l0-norm func…
Iterative reweighted algorithms, as a class of algorithms for sparse signal recovery, have been found to have better performance than their non-reweighted counterparts. However, for solving the problem of multiple measurement vectors (MMVs), all the existing reweighted algorithms do not account for temporal correlation…
New reweighted losses improve diffusion model training and image quality.
problem Training and improving diffusion models for image generation.
method Constructing a cascade of time-dependent variational lower bounds.
result Significant improvements in pixel-space image modeling quality.
Resampling outperforms reweighting for correcting biased data in machine learning models.
problem Correcting sampling bias in machine learning models trained on biased data sets.
method Compared resampling and reweighting techniques, focusing on their performance with stochastic gradient algorithms.
result Resampling outperforms reweighting when combined with stochastic gradient algorithms.
Improved matching for multiple objects using a novel reweighting method.
problem Current multi-object matching methods have limitations and are not robust.
method Proposes a novel iterative reweighting strategy using the graph connection Laplacian.
result Demonstrates superior performance over state-of-the-art methods.
Proposes a new method to improve regression models with reweighted samples.
problem Improves regression models' performance under low sample sizes and covariate perturbations.
method Reparametrizes sample weights using a doubly non-negative matrix and solves the reweighted estimate efficiently.
result Adversarial reweighting strategy delivers promising results on various datasets.
Improved learning to reweight using deep interactions between student and teacher models.
problem Limitation of existing learning to reweight methods in utilizing student model's internal states.
method Proposes an algorithm that uses the student model's internal states to the teacher model, which returns adaptive weights to enhance student model training.
result Significant improvement over previous methods in image classification and neural machine translation experiments.
Machine learning tools are commonly used in modern high energy physics (HEP) experiments. Different models, such as boosted decision trees (BDT) and artificial neural networks (ANN), are widely used in analyses and even in the software triggers. In most cases, these are classification models used to select the "signal"…
Defense against user shilling attacks in collaborative filtering using edge reweighting.
problem Vulnerability of collaborative filtering to profile injection attacks.
method Adversarial robustness based edge reweighting to attenuate non-robust edges.
result Effective defense against various types of attacks demonstrated through experiments.
We analyze a reweighted version of the Kikuchi approximation for estimating the log partition function of a product distribution defined over a region graph. We establish sufficient conditions for the concavity of our reweighted objective function in terms of weight assignments in the Kikuchi expansion, and show that a…
CurveRL optimizes large model reasoning by reweighting prompts based on their rank and density.
problem Improving large language model reasoning through context reweighting.
method Formulated prompt reweighting as a functional derivative, proposing CurveRL based on quantile coordinate transform.
result CurveRL consistently outperforms existing methods across multiple benchmarks.
The paper improves sample reweighting methods for adapting to covariate shifts.
problem Improving accuracy in reproducing kernel Hilbert spaces when data distributions differ.
method Combining known error bounds for reweighted kernel regression in RKHS to show reduced sample size needed for accuracy.
result Under weak smoothness conditions, fewer samples are needed for the same accuracy as standard supervised learning.
Proposes SVI for covariate-shift generalization with sparse variable independence.
problem Covariate-shift generalization with limited data and unstable variables.
method Introduces sparsity constraint and combines reweighting and selection in an iterative way.
result Improves covariate-shift generalization performance on synthetic and real-world datasets.
Unified analysis of reweighted least-squares algorithms for linear models.
problem Recovering unknown signals from linear measurements using reweighted least squares.
method Unified asymptotic analysis of IRLS, lin-RFM, and alternating minimization algorithms.
result The algorithms can achieve favorable performance in a few iterations with appropriate reweighting.
Algorithm samples polygons of fixed edge lengths in any dimension.
problem Sampling random closed polygons with fixed edge lengths in any dimension.
method Weighted edge vectors on unit sphere, Möbius transformation, reweighting factors.
result Algorithm samples polygons according to standard probability measures efficiently.
Federated learning has a variety of applications in multiple domains by utilizing private training data stored on different devices. However, the aggregation process in federated learning is highly vulnerable to adversarial attacks so that the global model may behave abnormally under attacks. To tackle this challenge, …
A bilevel optimization method for reweighting data leads to suboptimal results.
problem Reweighting data for better model performance on a different distribution.
method Formalizes data reweighting as a bilevel optimization problem and analyzes the limitations of classical solvers.
result The final data weights are often very sparse, indicating suboptimal solutions.
Iteratively reweighted ℓ1 algorithm is a popular algorithm for solving a large class of optimization problems whose objective is the sum of a Lipschitz differentiable loss function and a possibly nonconvex sparsity inducing regularizer. In this paper, motivated by the success of extrapolation techniques in accele…
Optimizes sample reweighting to match laws under covariate shift using Wasserstein distance.
problem Matching laws of samples with different distributions under covariate shift.
method Minimizes Wasserstein distance between empirical measures of samples using Nearest Neighbors weights.
result Consistent reweighting leads to asymptotic convergence of empirical measures.
Domain adaptation provides a powerful set of model training techniques given domain-specific training data and supplemental data with unknown relevance. The techniques are useful when users need to develop models with data from varying sources, of varying quality, or from different time ranges. We build CrossTrainer, a…
Q-MMR evaluates policies using reweighted rewards and moment matching.
problem Off-policy evaluation in finite-horizon MDPs.
method Q-MMR learns scalar weights for data points via a moment matching objective against a value-function discriminator class.
result Data-dependent finite-sample guarantee with a dimension-free error bound.
New algorithm improves online binary classification with constant time complexity.
problem Online binary classification with rebalancing.
method Non-iteratively reweighted recursive least-squares.
result Exacts converges to batch formulation and outperforms existing algorithms.
Recently, the paradigm of unfolding iterative algorithms into finite-length feed-forward neural networks has achieved a great success in the area of sparse recovery. Benefit from available training data, the learned networks have achieved state-of-the-art performance in respect of both speed and accuracy. However, the …
Biases in observational data of treatments pose a major challenge to estimating expected treatment outcomes in different populations. An important technique that accounts for these biases is reweighting samples to minimize the discrepancy between treatment groups. We present a novel reweighting approach that uses bi-le…
l1 reweighting algorithms are very popular in sparse signal recovery and compressed sensing, since in the practice they have been observed to outperform classical l1 methods. Nevertheless, the theoretical analysis of their convergence is a critical point, and generally is limited to the convergence of the functional to…
A new method of moments estimator goes beyond data reweighting.
problem Estimation of moment restrictions and conditional moment restrictions.
method Kernel Method of Moments (KMM) based on maximum mean discrepancy.
result KMM achieves competitive performance on conditional moment restriction tasks.
Ribbon: Scalable Approximation and Robust Uncertainty Quantification
problem Reliably quantifying predictive uncertainty for complex models
method Ribbon, a scalable approximation to Dirichlet-reweighted bootstrap uncertainty
result Asymptotically equivalent to a flat-prior Laplace approximation under correct likelihood specification, recovers robust sandwich covariance under misspecification
New method uses neural networks to estimate parameters without needing detector simulations.
problem Estimating parameters in high-energy physics with detector effects.
method Two-level fitting approach: SRGN (Simulation-level fit based on Reweighting Generator-level events with Neural networks).
result Demonstrated using simulated datasets, SRGN can estimate parameters without detector effects.
This paper improves multi-label ranking by reweighting univariate losses, enhancing consistency and performance.
problem Improving multi-label ranking performance while maintaining consistency.
method Systematic study of consistency and generalization error bounds for learning algorithms, proposing a reweighted univariate loss.
result Inconsistent pairwise losses can lead to better performance than consistent univariate losses in practice.
Improved GAN training stability through clipping and reweighting.
problem Inconsistent GAN training leading to inferior performance.
method Proposes a variational GAN framework with probability ratio clipping and sample reweighting.
result Significantly improved performance across various GAN tasks.
Recent work shows GRW approaches do not improve over ERM in distributional shift.
problem Improving robustness to distributional shift in machine learning models.
method Generalized Reweighting (GRW) algorithms, which iteratively update model parameters based on reweighting of training samples.
result GRW approaches do not significantly improve over ERM in real applications with distribution shift.
Toy model study shows resampling/reweighting can improve feature learning in imbalanced classification.
problem Improving feature learning in imbalanced classification problems.
method High-dimensional toy model with replica method, class-wise resampling/reweighting, and simplified model.
result No resampling/reweighting can sometimes give best feature learning performance.
Precise scientific analysis in collider-based particle physics is possible because of complex simulations that connect fundamental theories to observable quantities. The significant computational cost of these programs limits the scope, precision, and accuracy of Standard Model measurements and searches for new phenome…
Paper identifies problematic baselines in Shapley value explanations and proposes a reweighting mechanism.
problem Identifying and addressing the suboptimality of baselines in Shapley value feature importance analysis.
method Analyzed suboptimality of baselines, identified problematic baseline, generalized uninformativeness, and designed a reweighting mechanism.
result Proposed uncertainty-based reweighting mechanism effectively accelerates computation and improves explanation quality.
Principal component analysis (PCA) is often used to reduce the dimension of data by selecting a few orthonormal vectors that explain most of the variance structure of the data. L1 PCA uses the L1 norm to measure error, whereas the conventional PCA uses the L2 norm. For the L1 PCA problem minimizing the fitting error of…
Boosts causal discovery by dynamically reweighting samples to learn better DAGs.
problem Overfitting to easier-to-fit samples and violating homogeneity assumptions in causal discovery.
method Adaptive sample reweighting via ReScore function to upweight and downweight samples based on fitting quality.
result Consistent and significant boosts in structure learning performance on synthetic and real-world datasets.
This paper improves model robustness to underrepresented groups using ranking metrics and reweighting.
problem Underrepresented groups suffer from low accuracy in models trained via ERM.
method Proposes Discounted Cumulative Gain (DCG) and Discounted Rank Upweighting (DRU) methods.
result Models trained with DRU show superior generalization to unseen groups.
Method reweights instances and classes to improve robustness in noisy data.
problem Improving deep learning performance in the presence of label noise.
method Formulates constrained optimization problems to assign importance weights to instances and class labels.
result Significant performance gains observed in benchmark datasets with label noise.
ARL improves fairness without protected features, showing AUC improvements for worst-case groups.
problem Training fairness in ML without known protected features.
method Adversarially Reweighted Learning (ARL) using non-protected features and task labels.
result ARL improves Rawlsian Max-Min fairness with notable AUC improvements for worst-case groups.
Kernel methods can learn hierarchical polynomials efficiently.
problem Learning hierarchical structure from data.
method Iteratively reweighting kernel machines using derivatives.
result Efficient learning of hierarchical polynomials.
Method reweights auxiliary tasks to reduce data need for main task.
problem Limited labeled data for supervised learning.
method Formulates weighted likelihood function as surrogate prior, minimizing divergence to true prior.
result Effective use of limited labeled data with auxiliary tasks, improving performance.
In this paper we study general lp regularized unconstrained minimization problems. In particular, we derive lower bounds for nonzero entries of first- and second-order stationary points, and hence also of local minimizers of the lp minimization problems. We extend some existing iterative reweighted l1 (IRL1) a…
Graph Convolution Networks (GCNs) are becoming more and more popular for learning node representations on graphs. Though there exist various developments on sampling and aggregation to accelerate the training process and improve the performances, limited works focus on dealing with the dimensional information imbalance…
AFR simplifies reducing reliance on spurious features, improving model performance.
problem Reducing reliance on spurious features for out-of-distribution generalization.
method Automatic Feature Reweighting (AFR) updates the model with a weighted loss.
result AFR improves model performance on benchmarks with minimal compute.
In this paper, we propose a generalized scale mixture family of distributions, namely the Power Exponential Scale Mixture (PESM) family, to model the sparsity inducing priors currently in use for sparse signal recovery (SSR). We show that the successful and popular methods such as LASSO, Reweighted ℓ1 and Reweigh…
NRWS improves training of SBNs and HMs using natural gradient.
problem Training Sigmoid Belief Networks and Helmholtz Machines efficiently.
method Exploits block-diagonal structure of Fisher Information Matrices to use natural gradient.
result NRWS and NBiHM achieve better log-likelihood and faster convergence.