This work explores Target Networks and Functional Regularization in deep Reinforcement Learning.
problem Stability and performance issues in deep Reinforcement Learning due to target value instability.
method Proposes and studies an explicit Functional Regularization approach as a replacement for Target Networks.
result Functional Regularization improves performance and stability compared to Target Networks.
New method targets sparsity to prevent overfitting in deep nets.
problem Overfitting in deep neural networks with small datasets.
method Targeted sparsity regularization to visualize and counteract overfitting.
result Significant increase in image classification performance without overfitting.
Via Gauge theory, we give a new proof of partial regularity for harmonic maps in dimension m>2 into arbitrary targets. This proof avoids the use of adapted frames and permits to consider targets of "minimal" C^2 regularity. The proof we present moreover extends to a large class of elliptic systems of quadratic growth.
Proposes a regularization method for unsupervised domain adaptation that aligns predictions with target data's top singular vectors.
problem Domain adaptation challenges in high joint error scenarios.
method Regularizes classifier to align with unsupervised target data guided by label alignment property (LAP).
result The method improves performance in MNIST-USPS domain adaptation and cross-lingual sentiment analysis.
The paper studies how different entropic regularizations affect GAN solutions.
problem Improving numerical convergence and sparsity in GAN solutions.
method Entropic regularization of Wasserstein distance and Sinkhorn divergence.
result Entropy regularization promotes sparsity, while Sinkhorn divergence recovers unregularized solution.
NAPP-ERM improves ERM with differential privacy guarantees by iteratively achieving target regularization and delivering strong convexity.
problem Over-regularization in privacy-preserving ERM approaches.
method Noise-Augmented Privacy-Preserving Empirical Risk Minimization (NAPP-ERM) with a dual-purpose l2 regularizer and privacy budget retrieval strategy.
result Mitigates over-regularization and achieves strong convexity through a single regularizer.
Kim-Milman flow map stable under regular target measures
problem Stability of Kim-Milman flow map under target measure variations
method Stability in relative entropy and 2-Wasserstein distance result Lipschitz stability up to logarithmic factor
Paper shows robust generative learning with minimal assumptions on target distributions.
problem Learning generative models with minimal assumptions on target distributions.
method Lipschitz-regularized α-divergences with minimal assumptions. result Stable learning across various target distributions with minimal assumptions.
GRAC improves reinforcement learning by self-guiding and self-regularizing.
problem Learning divergence and slow updates in reinforcement learning algorithms.
method Self-regularized TD-learning and self-guided policy improvement.
result Achieved or outperformed state-of-the-art results on OpenAI gym tasks.
Improved generalization with iterative self-distillation using weighted ground-truth targets.
problem Improving generalization accuracy in neural networks.
method Iterative kernel regression with weighted ground-truth targets and ℓ2 regularization. result Closed-form solution for optimal weighting parameter and efficient estimation.
Neural networks learn the support of the target function through SGD's implicit regularization effect.
problem Learning the support of the target function in neural networks.
method Investigation of mini-batch SGD's ability to learn the support in the first layer of a neural network.
result Mini-batch SGD effectively learns the support in the first layer by shrinking irrelevant weights, while vanilla GD requires an explicit regularization term.
Local graph clustering methods aim to find small clusters in very large graphs. These methods take as input a graph and a seed node, and they return as output a good cluster in a running time that depends on the size of the output cluster but that is independent of the size of the input graph. In this paper, we adopt a…
A new method uncovers intrinsic data structures for unsupervised domain adaptation.
problem Learning domain-aligned features can damage intrinsic target discrimination.
method Structurally Regularized Deep Clustering (H-SRDC) integrating structural source regularization.
result H-SRDC outperforms existing methods in image classification and semantic segmentation.
Study improves regularity estimates for harmonic maps into ellipsoids.
problem Independence of regularity estimates on harmonic maps with varying target dimensions.
method Analyzes harmonic maps into ellipsoids, uses Palais-Smale sequences, and critical metrics.
result Enhanced regularity estimates for Laplace harmonic eigenmaps.
A new method debiases multiple target parameters without IFs.
problem Debiasing multiple target parameters in nonparametric models.
method Kernel Debiased Plug-in Estimation (KDPE) using TMLE and reproducing kernel Hilbert spaces.
result KDPE simultaneously debiases all pathwise differentiable target parameters.
We propose Regularized Learning under Label shifts (RLLS), a principled and a practical domain-adaptation algorithm to correct for shifts in the label distribution between a source and a target domain. We first estimate importance weights using labeled source data and unlabeled target data, and then train a classifier …
Regularization helps resolve ambiguity in mean-variance models, improving predictive uncertainty quantification.
problem Signal-to-noise ambiguity in overparameterized mean-variance models.
method Statistical field theory framework to explain phase transition.
result Regularization reduces variability and improves predictive uncertainty quantification.
Develops regularity theory for Beckmann's optimal transport problem.
problem Minimizing total squared flux in continuous transport from source to target.
method Unconstrained Lagrangian formulation, variational first order optimality conditions, Schauder estimates.
result Exact Hölder regularity of potential, flux, and flow generating on bounded, regular domains.
A new gradient flow for MMD with closed-form implementation.
problem Existing gradient flows either lack tractable numerical implementation or require strong assumptions.
method Introduces a (de)-regularized Maximum Mean Discrepancy (DrMMD) and its gradient flow.
result Guarantees near-global convergence for a broad class of targets in both continuous and discrete time.
This paper identifies a problem with the usual procedure for L2-regularization parameter estimation in a domain adaptation setting. In such a setting, there are differences between the distributions generating the training data (source domain) and the test data (target domain). The usual cross-validation procedure requ…
New method flattens decision boundary by targeting shortcut-aligned axes in disentangled latent space.
problem Shortcut learning in neural networks, leading to poor out-of-distribution generalization.
method Injects targeted anisotropic noise to regularize classifier sensitivity along shortcut-aligned axes.
result Achieves state-of-the-art OOD performance without shortcut labels or conflicting samples.
Improves generalization with few samples using a new regularization method.
problem Training deep neural networks with limited data leads to overfitting.
method Sample-based regularization (SBR) to improve generalization without relying on source model knowledge.
result SBR outperformed existing methods in various configurations.
Combines structured inference and targeted learning to tackle causal inference challenges.
problem Treatment assignment heterogeneity and lack of counterfactual data.
method Factorizes joint distribution into risk, confounding, instrumental, and miscellaneous factors; applies regularizer derived from influence curve.
result TVAE demonstrates competitive and state-of-the-art performance on benchmark datasets.
Regularized target encoding beats traditional methods for high cardinality features in ML.
problem Efficiently encoding high cardinality categorical variables for ML algorithms.
method Regularized target encoding compared to traditional encodings like integer and one-hot encoding.
result Regularized target encoding consistently provided the best results in a large-scale benchmark experiment.
DMRL improves UDA by mixing source and target samples and enriching latent space structures.
problem Lack of class-aware information and insufficient samples for domain-invariant feature extraction.
method Dual Mixup Regularized Learning (DMRL) that conducts category and domain mixup regularizations.
result DMRL achieves state-of-the-art performance on domain adaptation benchmarks.
The paper develops a theory for one-step Wasserstein-guided models for PDE-induced measures.
problem Theoretical understanding of generative models' accuracy in scientific computing.
method Regularity theory for optimal transport between doubling measures, excess-risk bounds.
result One-step Wasserstein-guided generative models can approximate PDE-induced measures with Hölder continuity.
Proposes VCNet for estimating ADRFs of continuous treatments.
problem Estimating ADRFs of continuous treatments from observational data.
method VCNet for improved model expressiveness and continuity; targeted regularization for finite sample performance.
result Improves model expressiveness and continuity of ADRFs.
Proposes a method to improve regression model performance with limited target data using fused-regularizer.
problem Model shifts and covariate shifts in high-dimensional regression.
method Two-step method with fused-regularizer to leverage source data for target task.
result Robust to covariate shifts, minimax-optimal under certain conditions, and validated by numerical tests.
Proposes a method to adapt models in nonstationary environments using ℓ1 regularization.
problem Adapting models to nonstationary environments in machine learning.
method Integrates ℓ1 regularization of differences between source and target parameters.
result Effective balance of stability and plasticity in model adaptation.
PS-KD distills a model's own knowledge to soften hard targets during training.
problem Improving generalization of deep neural networks by softening hard targets.
method Progressive self-knowledge distillation (PS-KD) that progressively distills a model's own knowledge to soften hard targets.
result PS-KD improves accuracy and provides high quality of confidence estimates in terms of calibration and ordinal ranking.
Paper estimates EOT maps for non-compactly supported measures with subGaussian target.
problem Estimating EOT maps between non-compactly supported measures.
method Uses bias-variance decomposition, T1-transport inequalities, and concentration of measure results.
result Shows error decay rates for different cases of subGaussian measures.
Improves domain adaptation by aligning source and target distributions and mitigating noisy labels.
problem Improving performance on target images with different acquisition conditions.
method Combines optimal transport, MixUp regularization, and robust loss for noisy labels.
result Improves domain adaptation performance on various benchmarks and real-world problems.
Improved fine-tuning with regularization and robustness for noisy labels.
problem Fine-tuning pre-trained models on small datasets can lead to overfitting and memorization.
method PAC-Bayes generalization bound analysis, layer-wise regularization, self-label-correction, label-reweighting.
result Improves performance by 1.76% on average for image classification tasks and 0.75% for few-shot classification.
In recent years, hyper-complex deep networks (such as complex-valued and quaternion-valued neural networks) have received a renewed interest in the literature. They find applications in multiple fields, ranging from image reconstruction to 3D audio processing. Similar to their real-valued counterparts, quaternion neura…
Transfer learning is a very important tool in deep learning as it allows propagating information from one "source dataset" to another "target dataset", especially in the case of a small number of training examples in the latter. Yet, discrepancies between the underlying distributions of the source and target data are c…
Domain adaptation (DA) addresses the real-world image classification problem of discrepancy between training (source) and testing (target) data distributions. We propose an unsupervised DA method that considers the presence of only unlabelled data in the target domain. Our approach centers on finding matches between sa…
Nyström subsampling with Tikhonov regularization for covariate shift adaptation under misspecified case
problem Adaptation to misspecified covariate shift
method Regularized Nyström subsampling with Tikhonov regularization
result Upper bounds on excess risk
GraphTEE estimates treatment effects on graph-structured targets, mitigating bias.
problem Understanding treatment effects on graph-structured targets with observational bias.
method GraphTEE framework focusing on confounding variable sets and new regularization.
result GraphTEE mitigates bias better than previous methods.
Statistical characteristics of deep network representations, such as sparsity and correlation, are known to be relevant to the performance and interpretability of deep learning. When a statistical characteristic is desired, often an adequate regularizer can be designed and applied during the training phase. Typically, …
Generative algorithms learn high-dimensional data efficiently and generate new samples.
problem Learning from scarce high-dimensional data.
method Lipschitz-regularized gradient flows and particle-based algorithms.
result Correctly transports gene expression data points with high dimensionality.
Adapting \cite{strz3}, we define generalized p-harmonic maps into Riemannian homogeneous targets, a notion of solutions not belonging to the energy space. Restricting our attention to the subcritical range p greater than the domain dimension n, we show a uniform C1,α-regularity result for a sequence of such …
We study a simplification of GAN training: the problem of transporting particles from a source to a target distribution. Starting from the Sobolev GAN critic, part of the gradient regularized GAN family, we show a strong relation with Optimal Transport (OT). Specifically with the less popular dynamic formulation of OT …
New method improves generative modeling on convex domains using regularized mirror maps and Student-t priors.
problem Challenges in generative modeling on convex domains with heavy-tailed targets.
method Mirror Flow Matching with regularized mirror maps and Student-t priors.
result Empirically outperforms baselines and achieves competitive sample quality.
The paper studies the loss landscape of regularized deep matrix factorization, revealing unique and sharp minimizers.
problem Understanding the loss landscape and minimizers of regularized deep matrix factorization problems.
method Theoretical analysis of ℓ2-regularized deep matrix factorization/deep linear network training problems with squared-error loss. result The unique end-to-end minimizer exists for all target matrices except for a set of Lebesgue measure zero.
A new method matches measures across different spaces using cost-regularized optimal transport.
problem Matching measures in different spaces without aligned data.
method Cost-regularized optimal transport formulation to match measures across two Euclidean spaces.
result Demonstrated applicability to single-cell spatial transcriptomics/multiomics matching tasks.
A new method for linear regression using feature graphs and hierarchical shrinkage.
problem Estimating robust parameters for linear regression models.
method Hierarchical Feature Regression (HFR) estimator that constructs a supervised feature graph to shrink parameters towards group targets.
result Demonstrates good predictive accuracy and versatility compared to other regularization techniques.
The paper examines how gradient descent stabilizes low-rank matrix factorization in noisy conditions.
problem Stability of low-rank implicit regularization in perturbed deep matrix factorization.
method Derives spectral conditions for gradient descent to exhibit a low-rank phase in noiseless settings and analyzes perturbed dynamics.
result Gradient descent converges to a low-rank solution under perturbation, with explicit dependence on perturbation size.
The paper constructs new non-trivial harmonic maps into higher-dimensional target manifolds.
problem Existence of non-trivial harmonic maps into higher-dimensional target manifolds.
method Perturbative argument, refined neck-analysis, energy identity, min-max problems.
result Construction of an infinite family of new null-homotopic n-harmonic n-spheres.