Federated learning studies separate client data and distribution gaps.
problem Understanding performance differences in federated learning across different datasets.
method Proposed a framework to disentangle out-of-sample and participation gaps.
result Dataset synthesis strategy is crucial for realistic simulations of federated learning generalization.
New method explains ML performance gaps without causal knowledge.
problem Understanding why ML algorithms perform differently across domains.
method Nonparametric hierarchical decomposition framework.
result Detailed variable-level explanations for performance gaps.
New framework for RL with linear-convex models reduces performance gap.
problem Continuous-time episodic reinforcement learning with unknown coefficients and convex objectives.
method Probabilistic framework and phase-based learning algorithm for optimal exploration-exploitation trade-off.
result Sublinear regrets achieved, matching best possible results in literature.
We present a data-driven framework called generative adversarial privacy (GAP). Inspired by recent advancements in generative adversarial networks (GANs), GAP allows the data holder to learn the privatization mechanism directly from the data. Under GAP, finding the optimal privacy mechanism is formulated as a constrain…
This work tackles missing annotations in large sensor datasets.
problem Missing annotations in large sensor datasets negatively affect supervised learning system performance.
method Proposes and evaluates three paradigms to handle gaps: dropping, single label, and unique label, along with a hybrid combination.
result Evaluation of proposed paradigms and hybrid combination shows significant performance improvement.
Improves Bayesian predictive performance in misspecified models.
problem Misspecification gap between inferential and predictive risks.
method Develops a multi-sample loss (PACm) to bridge the gap. result Empirical study shows improved predictive distribution.
Study quantifies performance gap between tensor and matrix-based approaches in nested matrix-tensor model.
problem Estimating a planted signal in a nested matrix-tensor model.
method Comparing tensor-based and matrix-based approaches for best rank-one approximation of tensor data.
result Derives precise algorithmic threshold for the unfolding approach and shows BBP-type transition behavior.
Study proposes worst+gap measure for better DG evaluation.
problem Lack of comprehensive exploration of average measure in DG evaluation.
method Introduced worst+gap measure and compared it with average measure.
result Worst+gap measure provides a more accurate approximation of true DG performance.
Rewiring GNNs to optimize community and feature alignment improves their performance.
problem Improving GNNs' performance by addressing over-squashing and generalization issues.
method Three rewiring strategies: ComMa, FeaSt, and ComFy, targeting community structure, node labels, and their alignment.
result Rewiring strategies enhance GNNs' performance by optimizing label-community alignment.
Estimates generalization gap for overparameterized models using Langevin approximation.
problem Estimating the difference between training and generalization performance in overparameterized models.
method Functional variance and Langevin approximation of functional variance.
result Demonstrates efficient estimation of generalization gaps for overparameterized models.
The study examines how averaging data improves model performance.
problem Understanding the generalization gap in machine learning models.
method Data averaging, covariance analysis, and stochastic gradient descent (SGD) noise modeling.
result A modified generalization gap is always non-negative for a large class of model parameter distributions.
Graph partitioning is the problem of dividing the nodes of a graph into balanced partitions while minimizing the edge cut across the partitions. Due to its combinatorial nature, many approximate solutions have been developed, including variants of multi-level methods and spectral clustering. We propose GAP, a Generaliz…
New algorithm closes empirical gap in PFSGD performance.
problem Empirical performance gap between tuned SGD and PFSGD.
method Parameter-free algorithm based on Coin-Betting ODE updates.
result New algorithm outperforms tuned baselines and matches optimal performance.
New technique reduces imitation learning performance gap in finite samples.
problem Imitation learning performance gap in finite samples.
method Replay estimation to reduce empirical variance in finite samples.
result Achieves optimal performance gap of $\widetilde{O} \left( \min({H^{3/2}} / {N}, {H} / {\sqrt{N}}
ight)$.
Improved gap-dependent bounds for reinforcement learning with linear approximations.
problem Achieving nearly minimax-optimal performance with linear function approximation.
method Developed and analyzed the LSVI-UCB++ algorithm and its concurrent variant.
result First gap-dependent regret bound for nearly minimax-optimal algorithm LSVI-UCB++.
New computational lower bounds for clustering and related problems.
problem Statistical-computational gaps in high-dimensional clustering problems.
method Investigation of low-degree polynomials in latent space models to derive lower bounds.
result New and sharper computational lower bounds for clustering, sparse clustering, and biclustering.
KZImputer improves time series data quality with adaptive imputation for short to medium-sized gaps.
problem Missing data in time series analysis.
method Adaptive imputation method for univariate time series with tailored strategies for different gap positions.
result KZImputer achieves strong performance, especially for high missingness rates and high-sparsity regimes.
New PAC-Bayesian bounds explain few-shot learning performance gaps.
problem Gap between PAC-Bayesian theory and practice in few-shot learning.
method Developed new PAC-Bayesian bounds for few-shot learning, derived MAML and Reptile from these bounds, and introduced a new PACMAML algorithm.
result PAC-Bayesian bounds explain performance of MAML and Reptile, and outperform existing algorithms.
New method uses model's generalization gap to predict membership inference attacks.
problem Predicting membership inference attacks on machine learning models.
method Uses the model's generalization gap as a metric to estimate the vulnerability to membership inference attacks.
result Demonstrates that the model's generalization gap provides an upper bound on MIA security.
New bound limits generalization gap for large models, independent of model complexity.
problem Understanding generalization gap in large-scale machine learning models.
method Established a model-independent upper bound for generalization gap using Rényi entropy.
result Generalization gap can be maintained with arbitrarily large models if data entropy is sufficient.
New method improves image compression using bits-back coding.
problem Lossy image compression with deep latent variable models.
method Iterative inference, stochastic annealing, bits-back coding.
result New state-of-the-art performance on lossy image compression.
MO-GP models fill gaps in biophysical data with across-domain info transfer.
problem Gap filling of biophysical parameters LAI and fAPAR over rice areas.
method Multi-output Gaussian Processes (MO-GP) based on Linear Model of Coregionalization (LMC).
result MO-GP models successfully predict biophysical variables even in high missing data regimes.
Attributing forecast gaps to component models in complex model suites
problem Attributing forecast gaps between model-suite forecasts and realized outcomes
method Formalizing walk analysis and adapting order-independent attribution frameworks
result Deriving efficient formulas for elementwise and vectorized gap attribution
New analysis reveals gaps in selective classifiers, guiding improvements.
problem Improving selective classifiers to match perfect-ordering oracle performance.
method Formalized selective classification gap, decomposed into five sources of looseness.
result Monotone post-hoc calibration has limited impact on closing the gap.
Paper uses 2-step Gradient Boosting to predict VAT tax gap.
problem Estimating tax evasion and revenue loss from tax avoidance.
method 2-steps Gradient Boosting model to correct selection bias.
result Significantly improved prediction of VAT tax gap.
The paper tackles attributing forecast gaps in complex model suites.
problem Attributing forecast gaps to individual component models in complex model suites.
method Formalized walk analysis, adapted LMDI and Shapley value approaches.
result Developed efficient formulas for gap attribution in practical portfolio-scale examples.
GACELA fills long gaps in musical audio with a GAN and context conditioning.
problem Restoring long gaps in musical audio with varying complexity and duration.
method Generative adversarial network (GAN) with five parallel discriminators and context conditioning.
result Reduced artifacts in inpaintings from unacceptable to mildly disturbing.
New research shows that the dimension gap between intrinsic and ambient dimensions affects adversarial vulnerability of machine learning models.
problem The mystery of adversarial attacks on machine learning models.
method Introducing two types of adversarial attacks and proving their relationship to the dimension gap.
result The dimension gap between intrinsic and ambient dimensions makes clean-trained models more vulnerable to off-manifold adversarial perturbations.
DRL agents perform poorly at high decision frequencies, but a new algorithm improves performance.
problem DRL agents struggle at high decision frequencies, leading to poor performance.
method Proved that DRL agents' action-conditioned return distributions collapse to their policy's return distribution as decision frequency increases. Defined superiority as a probabilistic generalization of advantage for high-frequency value-based RL.
result Proper modeling of superiority distribution improves performance of controllers at high decision frequencies.
New framework tightens certified robustness gaps in machine learning models.
problem Persistent gap between theoretical certified robustness and empirical accuracy.
method Leverages Lipschitz continuity and novel confidence intervals.
result Improves robust accuracy, compressing the gap between theory and practice.
This work is a part of ICLR Reproducibility Challenge 2019, we try to reproduce the results in the conference submission PADAM: Closing The Generalization Gap of Adaptive Gradient Methods In Training Deep Neural Networks. Adaptive gradient methods proposed in past demonstrate a degraded generalization performance than …
Fine-grained gap-dependent regret bounds for reinforcement learning.
problem Achieving optimal regret bounds for reinforcement learning with suboptimality gaps.
method Developed novel analytical frameworks and refined algorithms for UCB-based and non-UCB-based reinforcement learning.
result Established the first fine-grained gap-dependent regret bounds for both UCB-based and non-UCB-based algorithms.
New algorithm optimizes multi-armed bandit performance in stochastic and adversarial settings.
problem Optimizing multi-armed bandit performance in both stochastic and adversarial environments.
method Follow-the-regularized-leader method with adaptive learning rates.
result First BOBW algorithm with gap-variance-dependent regret bounds in adversarial settings.
SMART training improves mask-predict translations.
problem Closing the performance gap between semi-autoregressive and autoregressive models.
method SMART training method for conditional masked language models.
result SMART-trained models produce higher-quality translations.
The study finds that firm membership in flagship indices and TCFD endorsement are strong predictors of a wider Disclosure-Performance Gap.
problem The Aggregate Confusion hypothesis and the measurement of greenwashing in environmental disclosures.
method The study uses a Disclosure-Performance Gap (DPG) model to measure the divergence between voluntary environmental disclosures and realised emissions performance for 200 large European firms. The model selection process involved multiple stages and robust standard errors.
result Firm membership in flagship indices and TCFD endorsement are strong predictors of a wider gap, while renewable energy use and environmental capital expenditure significantly narrow the gap.
Dropout, a simple and effective way to train deep neural networks, has led to a number of impressive empirical successes and spawned many recent theoretical investigations. However, the gap between dropout's training and inference phases, introduced due to tractability considerations, has largely remained under-appreci…
Deep neural networks perform poorly on smooth functions despite strong approximation theory.
problem The gap between deep learning theory and practical performance on smooth functions.
method Computational framework to study DNN performance, comparing against best-in-class methods.
result There is a crucial gap between DNN approximation theory and practical performance, but it can be closed.
This article presents the use of Answer Set Programming (ASP) to mine sequential patterns. ASP is a high-level declarative logic programming paradigm for high level encoding combinatorial and optimization problem solving as well as knowledge representation and reasoning. Thus, ASP is a good candidate for implementing p…
Using a proper model to characterize a time series is crucial in making accurate predictions. In this work we use time-varying autoregressive process (TVAR) to describe non-stationary time series and model it as a mixture of multiple stable autoregressive (AR) processes. We introduce a new model selection technique bas…
Adaptive optimization algorithms, such as Adam and RMSprop, have shown better optimization performance than stochastic gradient descent (SGD) in some scenarios. However, recent studies show that they often lead to worse generalization performance than SGD, especially for training deep neural networks (DNNs). In this wo…
The main task in oil and gas exploration is to gain an understanding of the distribution and nature of rocks and fluids in the subsurface. Well logs are records of petro-physical data acquired along a borehole, providing direct information about what is in the subsurface. The data collected by logging wells can have si…
Study reveals 1/f noise in signals made from nonoverlapping rectangular pulses.
problem Analyzing 1/f noise in signals composed of nonoverlapping pulses. method Derived a general formula for power spectral density, analyzed rectangular pulse case.
result Observed pure 1/f noise until very low frequencies with long pulse durations. In recent years, deep learning researchers have focused on how to find the interpretability behind deep learning models. However, today cognitive competence of human has not completely covered the deep learning model. In other words, there is a gap between the deep learning model and the cognitive mode. How to evaluate…
Graph pruning improves neural network performance by addressing squashing and smoothing issues.
problem Over-squashing and over-smoothing in Graph Neural Networks.
method Proposes edge deletions to simultaneously address over-squashing and over-smoothing, optimizing spectral gap.
result Edge deletions improve generalization and distinguishability of nodes of different classes.
This paper evaluates fairness in deep metric learning and proposes a method to reduce subgroup performance gaps.
problem The negative impact of deep metric learning representations on minority subgroup performance in downstream tasks.
method Definition of fairness in DML through inter-class, intra-class, and uniformity properties; finDML benchmark; Partial Attribute De-correlation (PARADE) method.
result Bias in DML representations propagates to downstream tasks, even with balanced training data.
Free adversarial training reduces the generalization gap compared to vanilla method.
problem Improving generalization in adversarial training.
method Analysis of algorithmic stability in free adversarial training.
result Free adversarial training shows a lower generalization gap.
A modern aircraft may require on the order of thousands of custom shims to fill gaps between structural components in the airframe that arise due to manufacturing tolerances adding up across large structures. These shims are necessary to eliminate gaps, maintain structural performance, and minimize pull-down forces req…
The paper analyzes RLVR's training dynamics, proving convergence depends on aligning update direction with Gradient Gap.
problem Understanding why RLVR works and its limitations.
method Analysis of RLVR's training process at trajectory and token levels, introducing Gradient Gap.
result Convergence depends on aligning update direction with Gradient Gap, with a sharp step-size threshold.