Federated learning is protected against adversarial attacks with residual-based reweighting.
problem Adversarial attacks on federated learning's aggregation process.
method Residual-based reweighting combined with iteratively reweighted least squares.
result Our aggregation algorithm outperforms other methods in label-flipping and backdoor attacks.
Develops methods to correct bias in AI feedback for more accurate alignment.
problem Systematic bias in AI feedback compared to human labels.
method Two debiased alignment methods: DDPO and DIPO.
result Methods improve alignment efficiency and performance close to human-labeled data.
We improve optimization for data with varying variance.
problem Optimizing data with varying variance.
method Generalized learning and optimization frameworks for data-driven optimization.
result Asymptotic and finite sample guarantees for stochastic programs.
A GPU-based workflow for building physics emulators of hypersonic flows
problem Resolving complex physical phenomena in hypersonic flows
method Fully GPU-based workflow integrating accelerated data generation and neural emulators
result Physics emulators remain reliable beyond their training distribution
Proposes a neural network method to correct residual distortions in coordinate transformations.
problem Nonlinear and spatially dependent distortions in coordinate transformation models.
method Residual-based neural network approach focusing on systematic distortions.
result The method improves accuracy and stability in challenging conditions.
In this paper, we consider a framework adapting the notion of cointegration when two asset prices are generated by a driftless Itô-semimartingale featuring jumps with infinite activity, observed regularly and synchronously at high frequency. We develop a regression based estimation of the cointegrated relations method …
RSmote improves PINNs accuracy with less memory usage.
problem Imbalanced learning in Physics-Informed Neural Networks (PINNs).
method Residual-based Smote (RSmote) for local adaptive sampling.
result RSmote achieves or exceeds accuracy of state-of-the-art methods while reducing memory usage.
Adaptive sampling detects local concept drift with limited labels.
problem Detecting local concept drift in dynamic environments with scarce labels.
method Combines residual-based exploration and exploitation with EWMA monitoring.
result Superior performance in label efficiency and drift detection accuracy.
New method for Sharpe ratio analysis in high dimensions using residual-based nodewise regression.
problem Consistency of Sharpe ratio estimators in high-dimensional portfolios.
method Residual-based nodewise regression for estimating precision matrix of errors and returns.
result Consistent Sharpe ratio estimators in various portfolio settings.
Iterative reweighted algorithms, as a class of algorithms for sparse signal recovery, have been found to have better performance than their non-reweighted counterparts. However, for solving the problem of multiple measurement vectors (MMVs), all the existing reweighted algorithms do not account for temporal correlation…
We present a new physics informed neural network (PINN) algorithm for solving brittle fracture problems. While most of the PINN algorithms available in the literature minimize the residual of the governing partial differential equation, the proposed approach takes a different path by minimizing the variational energy o…
This paper develops a novel deep recurrent neural network for sequential signal reconstruction.
problem Sequential signal reconstruction from low-dimensional measurements.
method Unfolding a reweighted ℓ1-ℓ1 minimization algorithm to design a deep recurrent neural network. result The proposed reweighted-RNN significantly outperforms existing RNN models in sequential frame reconstruction.
Improved matching for multiple objects using a novel reweighting method.
problem Current multi-object matching methods have limitations and are not robust.
method Proposes a novel iterative reweighting strategy using the graph connection Laplacian.
result Demonstrates superior performance over state-of-the-art methods.
Reweighted l1-algorithms have attracted a lot of attention in the field of applied mathematics. A unified framework of such algorithms has been recently proposed by Zhao and Li. In this paper we construct a few new examples of reweighted l1-methods. These functions are certain concave approximations of the l0-norm func…
Proposes a new method to improve regression models with reweighted samples.
problem Improves regression models' performance under low sample sizes and covariate perturbations.
method Reparametrizes sample weights using a doubly non-negative matrix and solves the reweighted estimate efficiently.
result Adversarial reweighting strategy delivers promising results on various datasets.
Improved learning to reweight using deep interactions between student and teacher models.
problem Limitation of existing learning to reweight methods in utilizing student model's internal states.
method Proposes an algorithm that uses the student model's internal states to the teacher model, which returns adaptive weights to enhance student model training.
result Significant improvement over previous methods in image classification and neural machine translation experiments.
Defense against user shilling attacks in collaborative filtering using edge reweighting.
problem Vulnerability of collaborative filtering to profile injection attacks.
method Adversarial robustness based edge reweighting to attenuate non-robust edges.
result Effective defense against various types of attacks demonstrated through experiments.
We analyze a reweighted version of the Kikuchi approximation for estimating the log partition function of a product distribution defined over a region graph. We establish sufficient conditions for the concavity of our reweighted objective function in terms of weight assignments in the Kikuchi expansion, and show that a…
CurveRL optimizes large model reasoning by reweighting prompts based on their rank and density.
problem Improving large language model reasoning through context reweighting.
method Formulated prompt reweighting as a functional derivative, proposing CurveRL based on quantile coordinate transform.
result CurveRL consistently outperforms existing methods across multiple benchmarks.
Unified analysis of reweighted least-squares algorithms for linear models.
problem Recovering unknown signals from linear measurements using reweighted least squares.
method Unified asymptotic analysis of IRLS, lin-RFM, and alternating minimization algorithms.
result The algorithms can achieve favorable performance in a few iterations with appropriate reweighting.
New reweighted losses improve diffusion model training and image quality.
problem Training and improving diffusion models for image generation.
method Constructing a cascade of time-dependent variational lower bounds.
result Significant improvements in pixel-space image modeling quality.
Domain adaptation provides a powerful set of model training techniques given domain-specific training data and supplemental data with unknown relevance. The techniques are useful when users need to develop models with data from varying sources, of varying quality, or from different time ranges. We build CrossTrainer, a…
Reweighting improves GAN accuracy without sacrificing statistical power.
problem Improving the fidelity of generative models.
method Post-hoc reweighting function applied to generated examples.
result Weighted GAN examples significantly improve accuracy.
New algorithm improves online binary classification with constant time complexity.
problem Online binary classification with rebalancing.
method Non-iteratively reweighted recursive least-squares.
result Exacts converges to batch formulation and outperforms existing algorithms.
Recently, the paradigm of unfolding iterative algorithms into finite-length feed-forward neural networks has achieved a great success in the area of sparse recovery. Benefit from available training data, the learned networks have achieved state-of-the-art performance in respect of both speed and accuracy. However, the …
Resampling outperforms reweighting for correcting biased data in machine learning models.
problem Correcting sampling bias in machine learning models trained on biased data sets.
method Compared resampling and reweighting techniques, focusing on their performance with stochastic gradient algorithms.
result Resampling outperforms reweighting when combined with stochastic gradient algorithms.
Toy model study shows resampling/reweighting can improve feature learning in imbalanced classification.
problem Improving feature learning in imbalanced classification problems.
method High-dimensional toy model with replica method, class-wise resampling/reweighting, and simplified model.
result No resampling/reweighting can sometimes give best feature learning performance.
Machine learning tools are commonly used in modern high energy physics (HEP) experiments. Different models, such as boosted decision trees (BDT) and artificial neural networks (ANN), are widely used in analyses and even in the software triggers. In most cases, these are classification models used to select the "signal"…
Precise scientific analysis in collider-based particle physics is possible because of complex simulations that connect fundamental theories to observable quantities. The significant computational cost of these programs limits the scope, precision, and accuracy of Standard Model measurements and searches for new phenome…
We revisit residual algorithms in both model-free and model-based reinforcement learning settings. We propose the bidirectional target network technique to stabilize residual algorithms, yielding a residual version of DDPG that significantly outperforms vanilla DDPG in the DeepMind Control Suite benchmark. Moreover, we…
Algorithm samples polygons of fixed edge lengths in any dimension.
problem Sampling random closed polygons with fixed edge lengths in any dimension.
method Weighted edge vectors on unit sphere, Möbius transformation, reweighting factors.
result Algorithm samples polygons according to standard probability measures efficiently.
The paper improves sample reweighting methods for adapting to covariate shifts.
problem Improving accuracy in reproducing kernel Hilbert spaces when data distributions differ.
method Combining known error bounds for reweighted kernel regression in RKHS to show reduced sample size needed for accuracy.
result Under weak smoothness conditions, fewer samples are needed for the same accuracy as standard supervised learning.
Principal component analysis (PCA) is often used to reduce the dimension of data by selecting a few orthonormal vectors that explain most of the variance structure of the data. L1 PCA uses the L1 norm to measure error, whereas the conventional PCA uses the L2 norm. For the L1 PCA problem minimizing the fitting error of…
A bilevel optimization method for reweighting data leads to suboptimal results.
problem Reweighting data for better model performance on a different distribution.
method Formalizes data reweighting as a bilevel optimization problem and analyzes the limitations of classical solvers.
result The final data weights are often very sparse, indicating suboptimal solutions.
Optimizes sample reweighting to match laws under covariate shift using Wasserstein distance.
problem Matching laws of samples with different distributions under covariate shift.
method Minimizes Wasserstein distance between empirical measures of samples using Nearest Neighbors weights.
result Consistent reweighting leads to asymptotic convergence of empirical measures.
Q-MMR evaluates policies using reweighted rewards and moment matching.
problem Off-policy evaluation in finite-horizon MDPs.
method Q-MMR learns scalar weights for data points via a moment matching objective against a value-function discriminator class.
result Data-dependent finite-sample guarantee with a dimension-free error bound.
Method reweights instances and classes to improve robustness in noisy data.
problem Improving deep learning performance in the presence of label noise.
method Formulates constrained optimization problems to assign importance weights to instances and class labels.
result Significant performance gains observed in benchmark datasets with label noise.
ARL improves fairness without protected features, showing AUC improvements for worst-case groups.
problem Training fairness in ML without known protected features.
method Adversarially Reweighted Learning (ARL) using non-protected features and task labels.
result ARL improves Rawlsian Max-Min fairness with notable AUC improvements for worst-case groups.
Kernel methods can learn hierarchical polynomials efficiently.
problem Learning hierarchical structure from data.
method Iteratively reweighting kernel machines using derivatives.
result Efficient learning of hierarchical polynomials.
Method reweights auxiliary tasks to reduce data need for main task.
problem Limited labeled data for supervised learning.
method Formulates weighted likelihood function as surrogate prior, minimizing divergence to true prior.
result Effective use of limited labeled data with auxiliary tasks, improving performance.
Iteratively reweighted ℓ1 algorithm is a popular algorithm for solving a large class of optimization problems whose objective is the sum of a Lipschitz differentiable loss function and a possibly nonconvex sparsity inducing regularizer. In this paper, motivated by the success of extrapolation techniques in accele…
Ribbon: Scalable Approximation and Robust Uncertainty Quantification
problem Reliably quantifying predictive uncertainty for complex models
method Ribbon, a scalable approximation to Dirichlet-reweighted bootstrap uncertainty
result Asymptotically equivalent to a flat-prior Laplace approximation under correct likelihood specification, recovers robust sandwich covariance under misspecification
Graph Convolution Networks (GCNs) are becoming more and more popular for learning node representations on graphs. Though there exist various developments on sampling and aggregation to accelerate the training process and improve the performances, limited works focus on dealing with the dimensional information imbalance…
Paper explains learning property of logistic and softmax losses for balanced and imbalanced class data.
problem Understanding and optimizing loss functions for deep neural networks with class imbalances.
method Analyzing necessary conditions for convergence of logistic and softmax losses in CNNs.
result Proposes a novel reweighted logistic loss function that improves performance over softmax loss.
Proposes SVI for covariate-shift generalization with sparse variable independence.
problem Covariate-shift generalization with limited data and unstable variables.
method Introduces sparsity constraint and combines reweighting and selection in an iterative way.
result Improves covariate-shift generalization performance on synthetic and real-world datasets.
A new method of moments estimator goes beyond data reweighting.
problem Estimation of moment restrictions and conditional moment restrictions.
method Kernel Method of Moments (KMM) based on maximum mean discrepancy.
result KMM achieves competitive performance on conditional moment restriction tasks.
Boosts causal discovery by dynamically reweighting samples to learn better DAGs.
problem Overfitting to easier-to-fit samples and violating homogeneity assumptions in causal discovery.
method Adaptive sample reweighting via ReScore function to upweight and downweight samples based on fitting quality.
result Consistent and significant boosts in structure learning performance on synthetic and real-world datasets.
Biases in observational data of treatments pose a major challenge to estimating expected treatment outcomes in different populations. An important technique that accounts for these biases is reweighting samples to minimize the discrepancy between treatment groups. We present a novel reweighting approach that uses bi-le…