Current approaches for Knowledge Distillation (KD) either directly use training data or sample from the training data distribution. In this paper, we demonstrate effectiveness of 'mismatched' unlabeled stimulus to perform KD for image classification networks. For illustration, we consider scenarios where this is a comp…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,742 papers · 148 categories
Trend · papers per month
48 results for “Data Mismatch”
Paper addresses linear regression with partially mismatched data using local search with theoretical guarantees.
problem Linear regression with partially mismatched data.
method Optimization formulation and greedy local search algorithm with theoretical guarantees.
result Local search algorithm converges to nearly-optimal solution at a linear rate under certain conditions.
This paper considers the classification of linear subspaces with mismatched classifiers. In particular, we assume a model where one observes signals in the presence of isotropic Gaussian noise and the distribution of the signals conditioned on a given class is Gaussian with a zero mean and a low-rank covariance matrix.…
Ensemble models improve prediction calibration for mismatched distributions.
problem Calibration issues in deep neural networks with mismatched train and test distributions.
method Simple data augmentation and mixing techniques for ensemble models.
result Improves calibration and accuracy on CIFAR10 and CIFAR100 benchmarks.
The paper addresses score-mismatched diffusion models and zero-shot conditional samplers.
problem Theoretical guarantees for score-mismatched diffusion models in zero-shot conditional sampling.
method Theoretical analysis of score-mismatched diffusion models and zero-shot conditional samplers.
result Theoretical performance guarantees with explicit dimensional dependencies for score-mismatched diffusion samplers.
Paper identifies objective mismatch in MBRL, affecting control task performance.
problem Objective mismatch in MBRL framework affects control task performance.
method Proposes re-weighting dynamics model training to mitigate mismatch.
result Likelihood of one-step ahead predictions is not always correlated with control performance.
Recently, there has been significant interest in linear regression in the situation where predictors and responses are not observed in matching pairs corresponding to the same statistical unit as a consequence of separate data collection and uncertainty in data integration. Mismatched pairs can considerably impact the …
New method addresses error bounds for PnP-ULA under mismatched models.
problem Error bounds for PnP-ULA under mismatched measurement and prior models.
method Posterior-L2 pseudometric to quantify error bounds.
result Explicit error bound for PnP-ULA under mismatched posterior distribution.
Improved BN for better performance in imbalanced data.
problem Improving classification accuracy in imbalanced datasets.
method Combining weighted loss function with modified batch normalization.
result Modified batch normalization improves classification accuracy in imbalanced datasets.
Study on LMMSE estimation with model mismatch, quantifying MSE trade-offs.
problem Model mismatch in LMMSE estimation with undermodeling.
method Analyzing the average MSE of LMMSE estimation with random regressors.
result Performance improvement depends on sufficient samples and model complexity.
Probabilistic models analyze data by relying on a set of assumptions. Data that exhibit deviations from these assumptions can undermine inference and prediction quality. Robust models offer protection against mismatch between a model's assumptions and reality. We propose a way to systematically detect and mitigate mism…
New algorithm reduces performance loss in IRL with mismatched transition dynamics.
problem Performance degradation in inverse reinforcement learning due to mismatched transition dynamics.
method Proposed a robust Maximum Causal Entropy (MCE) IRL algorithm leveraging robust reinforcement learning insights.
result Empirically demonstrated stable performance improvement under transition dynamics mismatches.
A new method improves data generation quality by correcting score mismatches.
problem Score mismatch issue in conditional score-based data generation methods.
method Denoising Likelihood Score Matching (DLSM) loss for classifier training.
result The proposed method outperforms previous methods on Cifar-10 and Cifar-100 benchmarks.
New method reduces state distribution mismatch in off-policy RL.
problem State distribution mismatch in off-policy RL algorithms.
method Develops a novel constrained off-policy gradient objective to minimize state distribution shift.
result Minimizing state distribution shift improves performance in off-policy RL algorithms.
Study addresses covariate mismatch in federated learning, improving model accuracy.
problem Learning from clients with different feature sets in federated learning.
method Developed two approaches for linear prediction under covariate mismatch: plug-in estimator and impute-then-regress strategy.
result Proposed methods provide asymptotic and finite-sample learning rates, improving model accuracy.
A new BN method corrects size mismatch in WLF for imbalanced data.
problem Learning from imbalanced datasets in neural networks.
method Proposes weighted batch normalization (WBN) to correct size mismatch between BN and WLF.
result WBN corrects size mismatch and improves classification performance in imbalanced datasets.
Many sleep studies suffer from the problem of insufficient data to fully utilize deep neural networks as different labs use different recordings set ups, leading to the need of training automated algorithms on rather small databases, whereas large annotated databases are around but cannot be directly included into thes…
MixMOOD improves SSDL by selecting unlabelled data based on deep feature similarity.
problem Class distribution mismatch in semi-supervised learning.
method MixMOOD uses deep dataset dissimilarity measures to select unlabelled data.
result MixMOOD selects unlabelled data based on strong correlation with MixMatch accuracy.
The dictionary-aided sparse regression (SR) approach has recently emerged as a promising alternative to hyperspectral unmixing (HU) in remote sensing. By using an available spectral library as a dictionary, the SR approach identifies the underlying materials in a given hyperspectral image by selecting a small subset of…
KalmanNet uses neural networks to improve state estimation in systems with unknown dynamics.
problem State estimation of systems with non-linear dynamics and partial information.
method KalmanNet integrates a recurrent neural network with the Kalman filter to handle non-linearities and model mismatches.
result KalmanNet outperforms classic filtering methods in systems with both mismatched and accurate domain knowledge.
BinaryDuo: Reducing Gradient Mismatch in Binary Activation Network by Coupling Binary Activationscs.LG
BinaryDuo improves BNNs by coupling binary activations, outperforming state-of-the-art models.
problem Gradient mismatch in BNNs due to binarizing activations.
method Using gradient of smoothed loss function to estimate gradient mismatch, proposing BinaryDuo scheme with coupled ternary activations.
result BinaryDuo outperforms state-of-the-art BNNs on various benchmarks.
We study the problem of off-policy policy optimization in Markov decision processes, and develop a novel off-policy policy gradient method. Prior off-policy policy gradient approaches have generally ignored the mismatch between the distribution of states visited under the behavior policy used to collect data, and what …
Thompson Sampling For Combinatorial Bandits: Polynomial Regret and Mismatched Sampling Paradoxstat.ML
Thompson Sampling shows polynomial regret for combinatorial semi-bandits with subgaussian rewards.
problem Finding optimal solutions in combinatorial semi-bandits with suboptimal sampling.
method Proposes Thompson Sampling with polynomial regret for linear combinatorial semi-bandits.
result Demonstrates 'mismatched sampling paradox' where knowing distributions can lead to worse performance.
New method improves spatial prediction validation accuracy.
problem Validation methods fail for spatial prediction tasks due to mismatch between validation and test locations.
method Proposes a new validation method that adapts existing covariate-shift ideas to spatial settings.
result Proves and demonstrates the new method's superiority in spatial prediction validation.
We study the estimation capacity of the generalized Lasso, i.e., least squares minimization combined with a (convex) structural constraint. While Lasso-type estimators were originally designed for noisy linear regression problems, it has recently turned out that they are in fact robust against various types of model un…
PACMAN provides bounds for classification tasks considering accuracy vs. negative log-loss mismatch.
problem Mismatch between accuracy and negative log-loss in classification tasks.
method Point-wise PAC approach over generalization gap, using likelihood ratio and concentration inequalities.
result PACMAN provides point-wise PAC bounds for the generalization problem.
A tacit assumption in linear regression is that (response, predictor)-pairs correspond to identical observational units. A series of recent works have studied scenarios in which this assumption is violated under terms such as ``Unlabeled Sensing and ``Regression with Unknown Permutation''. In this paper, we study the s…
Our study employs sentiment analysis to evaluate the compatibility of Amazon.com reviews with their corresponding ratings. Sentiment analysis is the task of identifying and classifying the sentiment expressed in a piece of text as being positive or negative. On e-commerce websites such as Amazon.com, consumers can subm…
The Perils of Optimizing Learned Reward Functions: Low Training Error Does Not Guarantee Low Regretcs.LG
Learning reward functions can lead to poor policy performance despite low error.
problem Low error in learned reward functions does not guarantee low regret in policy performance.
method Mathematical analysis of reward learning and policy optimization.
result A low expected test error of the reward model guarantees low worst-case regret, but error-regret mismatch can occur with certain data distributions.
A new method for inventory control using in-context learning and generative models.
problem Inventory control with decision-dependent censoring, focusing on the censored newsvendor problem.
method In-context generative posterior sampling (ICGPS) combining modern generative models and in-context autoregressive generation.
result ICGPS achieves sublinear Bayesian regret for the censored newsvendor problem, outperforming existing methods.
Supervised learning based on a deep neural network recently has achieved substantial improvement on speech enhancement. Denoising networks learn mapping from noisy speech to clean one directly, or to a spectrum mask which is the ratio between clean and noisy spectra. In either case, the network is optimized by minimizi…
Subspace models play an important role in a wide range of signal processing tasks, and this paper explores how the pairwise geometry of subspaces influences the probability of misclassification. When the mismatch between the signal and the model is vanishingly small, the probability of misclassification is determined b…
Synthetic Augmentation in Imbalanced Learning: When It Helps, When It Hurts, and How Much to Addstat.ML
Synthetic augmentation helps but not always in imbalanced learning.
problem Imbalanced learning causes poor performance on rare classes.
method Developed a statistical framework for synthetic augmentation in imbalanced learning.
result Synthetic augmentation is not always beneficial and depends on the imbalance regime.
We study the tracking problem, namely, estimating the hidden state of an object over time, from unreliable and noisy measurements. The standard framework for the tracking problem is the generative framework, which is the basis of solutions such as the Bayesian algorithm and its approximation, the particle filters. Howe…
Mixup technique improved, reducing manifold mismatch for better calibration.
problem Improving calibration of models using Mixup.
method Dynamic adjustment of interpolation coefficients based on sample similarity.
result Improved predictive performance and calibration with reduced manifold mismatch.
A trade-off between accuracy and fairness is almost taken as a given in the existing literature on fairness in machine learning. Yet, it is not preordained that accuracy should decrease with increased fairness. Novel to this work, we examine fair classification through the lens of mismatched hypothesis testing: trying …
RealMix improves SSL performance with real-world data challenges.
problem Poor SSL performance when labeled and unlabeled data distributions differ.
method Develops RealMix, a new SSL algorithm.
result Achieves state-of-the-art results on standard benchmark datasets.
Combining Physically-Based Modeling and Deep Learning for Fusing GRACE Satellite Data: Can We Learn from Mismatch?physics.geo-ph
Global hydrological and land surface models are increasingly used for tracking terrestrial total water storage (TWS) dynamics, but the utility of existing models is hampered by conceptual and/or data uncertainties related to various underrepresented and unrepresented processes, such as groundwater storage. The gravity …
Training on mixed distributions improves test performance even when components are unrelated.
problem Improving test performance with mismatched training and test distributions.
method Analyzing mixture distributions with different training and test proportions.
result Distribution shift can be beneficial, improving test performance even when components are unrelated.
Constructing VAE Latent Spaces with Prescribed Topology
problem Resolving topological mismatch in VAEs for non-Euclidean data
method A constructive framework for product covering spaces
result Topology-aware latent representations with closed-form KL divergences
An Empirical Analysis of the Advantages of Finite- v.s. Infinite-Width Bayesian Neural Networksstat.ML
Empirical study compares finite- and infinite-width BNNs, revealing performance differences under model mismatch.
problem Comparing BNNs with different widths due to conflicting model properties and inference intractability.
method Empirical comparison of finite- and infinite-width BNNs, analyzing performance under model mismatch.
result Increasing width can hurt BNN performance when the model is mis-specified, and finite-width BNNs generalize better under model mismatch.
We characterize the performance of sequential information guided sensing, Info-Greedy Sensing, when there is a mismatch between the true signal model and the assumed model, which may be a sample estimate. In particular, we consider a setup where the signal is low-rank Gaussian and the measurements are taken in the dire…
The performance of automatic speech recognition (ASR) systems can be significantly compromised by previously unseen conditions, which is typically due to a mismatch between training and testing distributions. In this paper, we address robustness by studying domain invariant features, such that domain information become…
Most of the existing studies on voice conversion (VC) are conducted in acoustically matched conditions between source and target signal. However, the robustness of VC methods in presence of mismatch remains unknown. In this paper, we report a comparative analysis of different VC techniques under mismatched conditions. …
New algorithm solves Schrödinger bridge problem with mismatched channels.
problem Solving Schrödinger bridge problem with input and noise channel mismatch.
method Design of a Sinkhorn recursion with memory for nonlinear PDEs.
result Demonstrates solving control-affine Schrödinger bridge problem.
RFC enhances humanoid control to imitate complex human motions.
problem Dynamics mismatch between humanoid models and real humans.
method Residual Force Control (RFC) augments control policies with external forces.
result RFC outperforms state-of-the-art methods in convergence speed and motion quality.
Deep transfer learning boosts single-EEG arousal detection accuracy.
problem Challenges in machine learning due to channel mismatch across different EEG datasets.
method Transfer learning strategy to adapt a multivariate model to single-channel EEG data.
result Fine-tuned model achieves similar performance to baseline model and significantly outperforms a single-channel model.
A method to learn robust policies for environments with model mismatches.
problem Training agents in high-stakes scenarios with mismatched training and real environments.
method Formalizes the perturbation as a zero-sum game to find Nash Equilibrium, which corresponds to the robust policy.
result Our algorithm can find a near-optimal robust policy with high probability using polynomial samples.