New method uses SLL to create masks for PX in noisy optimization problems.
problem Effective optimization in noisy problems with hidden variable dependencies.
method Statistical Linkage Learning (SLL) for decomposition and mask construction.
result Proposed method maintains effectiveness in noisy conditions and outperforms state-of-the-art.
Masking diffusion outperforms other discrete diffusion models by incorporating jump times into the model.
problem Improving the performance of discrete diffusion models.
method Conditioning on the jump schedule of discrete Markov processes.
result Schedule-conditioned discrete diffusion (SCUD) models outperform classical and masking diffusion models.
It is important to learn various types of classifiers given training data with noisy labels. Noisy labels, in the most popular noise model hitherto, are corrupted from ground-truth labels by an unknown noise transition matrix. Thus, by estimating this matrix, classifiers can escape from overfitting those noisy labels. …
Reconstruction-based learning produces uninformative features for perception tasks.
problem Misalignment between reconstruction-based learning and perception tasks.
method Investigated the impact of input space reconstruction on feature learning for perception tasks.
result Reconstruction-based learning allocates model capacity to a subspace with uninformative features for perception tasks.
Novel bio-inspired masking for robust speech emotion recognition.
problem Noise degradation in speech emotion recognition.
method Cochlear cepstrogram-based contrastive learning with temporal and frequency masking.
result Improved speech emotion recognition performance on K-EmoCon benchmark.
MDMs train to decode tokens in a random order, which affects performance; we show they can be optimized for a favorable order.
problem Performance of MDMs is affected by the random order in which tokens are decoded.
method We show that MDMs can be optimized for a favorable decoding order by equipping their continuous-time variational objective with multivariate noise schedules.
result MDMs can be decomposed into a weighted auto-regressive losses over orders, making them auto-regressive models with learnable orders.
Perturbative GAN, which replaces convolution layers of existing convolutional GANs (DCGAN, WGAN-GP, BIGGAN, etc.) with perturbation layers that adds a fixed noise mask, is proposed. Compared with the convolu-tional GANs, the number of parameters to be trained is smaller, the convergence of training is faster, the incep…
We present an unsupervised training approach for a neural network-based mask estimator in an acoustic beamforming application. The network is trained to maximize a likelihood criterion derived from a spatial mixture model of the observations. It is trained from scratch without requiring any parallel data consisting of …
Dropout and RaM become equivalent in large ResNets as depth and width increase.
problem Improving performance in deep learning models.
method Comparing Dropout and Random Gradient Masking in ResNets.
result Dropout and RaM converge to the same large-scale limiting dynamics in ResNets.
Remasking improves the quality of discrete diffusion models for natural language and image generation.
problem Limited iterative refinement in masked discrete diffusion models.
method Introducing ReMDM sampler that allows remasking during inference.
result Remasking enables better quality outputs with increased sampling steps.
Graph Convolutional Networks (GCNs) have shown very powerful for graph data representation and learning tasks. Existing GCNs usually conduct feature aggregation on a fixed neighborhood graph in which each node computes its representation by aggregating the feature representations of all its neighbors which is biased by…
Framework improves marine mammal monitoring in noisy underwater environments.
problem Underwater bioacoustic monitoring challenges due to overlapping calls and variable noise.
method Multi-step attention-guided framework with segmentation and mid-level fusion.
result Improved signal discrimination, reduced false positives, reliable representations.
Improves generative models by optimizing rewards and sample editing.
problem Efficiently generating high-reward samples with structural constraints.
method Introduces MDM-VGB, a discrete diffusion sampler that augments unmasking generation with reward-guided remasking.
result MDM-VGB achieves quadratic complexity and robustness to noise, outperforming heuristics like best-of-N. PMI-Masking improves MLM pretraining by masking correlated spans efficiently.
problem Uniform token masking leads to inefficient and suboptimal performance in MLMs.
method PMI-Masking uses Pointwise Mutual Information to mask n-grams with high collocation.
result PMI-Masking reaches half the training time and improves performance.
Latent Noise Injection improves synthetic data generation for privacy and statistical alignment.
problem Slow convergence of generative models in high-dimensional settings.
method Latent Noise Injection using Masked Autoregressive Flows (MAF).
result Synthetic data closely reflects the underlying distribution, especially in high-dimensional settings.
Paper introduces SMM for forecasting multiple time series with missing values.
problem Forecasting multiple time series with missing and noisy values.
method Sliding Mask Method (SMM) using Non-negative Matrix Factorization (NMF).
result The method outperforms state-of-the-art methods in time series forecasting.
ST-MTM models complex time series by decomposing and masking seasonal and trend components.
problem Forecasting complex time series with intricate temporal variations.
method Seasonal-Trend Decomposition with Masking and Contrastive Learning.
result ST-MTM achieves superior forecasting performance compared to existing methods.
K-means fails in high dimensions with noise and few samples.
problem Clustering in high-dimensional data with noise and limited samples.
method Simple Gaussian Mixture Model (GMM) analysis.
result Almost every partition becomes a fixed point of k-means in high dimensions.
Expands MLM by masking token positions, improving performance and convergence.
problem Improving language model performance and convergence.
method Masking token positions along with [MASK] tokens, using a fully connected classifier stage.
result Shows .3% improvement and 50% faster convergence for BERT Base with position masking.
SMART training improves mask-predict translations.
problem Closing the performance gap between semi-autoregressive and autoregressive models.
method SMART training method for conditional masked language models.
result SMART-trained models produce higher-quality translations.
In high-dimensional data, structured noise caused by observed and unobserved factors affecting multiple target variables simultaneously, imposes a serious challenge for modeling, by masking the often weak signal. Therefore, (1) explaining away the structured noise in multiple-output regression is of paramount importanc…
The α-Alternator adapts to varying noise levels in sequences, improving robustness and performance.
problem Current models assume uniform noise levels, limiting performance on noisy temporal data.
method Introduces α-Alternator using Vendi Score to dynamically adjust noise sensitivity. result Outperforms Alternators and state-of-the-art models in trajectory prediction, imputation, and forecasting.
New method reduces diffusion model function evaluations for discrete data.
problem High computational burden in generating samples from masked diffusion models.
method Modified causal attention mask and speculative sampling mechanism for non-factorized predictions.
result Achieved ~2x reduction in required network forward passes.
Proposes a proportional masking strategy for better tabular data imputation.
problem Heterogeneity of tabular data disrupts uniform random masking in MAEs.
method Computes missingness statistics, generates proportional masks, uses MLP token mixing.
result Proportional masking preserves missingness distribution, improves imputation performance.
Transformer model removes noise from light curves efficiently.
problem Challenges in processing astrophysical light curves due to noise.
method Denoising Time Series Transformer (DTST) model trained with masked objective.
result DTST model excels at removing noise and outliers in time series datasets.
FMix improves model performance by distorting learned functions without distorting data distribution.
problem Improving model robustness and generalization performance in mixed sample data augmentation.
method Proposes FMix, an MSDA using random binary masks generated from Fourier space.
result FMix outperforms MixUp and CutMix, achieving state-of-the-art results on CIFAR-10.
New analysis reveals masked self-supervised learning's effectiveness in extracting data structure.
problem Analyzing masked self-supervised learning in high-dimensional data.
method Developed precise high-dimensional analysis of masked modeling objectives.
result Identified phase transitions and structured regimes for masked self-supervised learning.
Study resolves the Korean LVRP puzzle by showing HVRP exists but is masked by investor heterogeneity and improper intensity normalization.
problem Puzzling Low Volume Return Premium (LVRP) in Korea, contradicting global High Volume Return Premium (HVRP) evidence.
method Used Korean market data (2020-2024) to demonstrate HVRP exists but is masked by investor heterogeneity and improper intensity normalization. Normalized institutional buying intensity by market capitalization rather than trading value.
result Demonstrated a perfect monotonic relationship between highest-conviction institutional buying and positive cumulative abnormal returns, while lowest-intensity trades yield modest returns.
This paper connects masked pre-training to Bayesian model selection.
problem Understanding the success of masked pre-training and its generalization.
method The paper shows masked pre-training corresponds to maximizing the marginal likelihood.
result Masked pre-training with a suitable scoring function maximizes the marginal likelihood.
Optimizes pruning masks for neural networks using probabilistic fine-tuning and PAC-Bayes bounds.
problem Improving neural network performance through adaptive pruning of weights.
method Optimizes stochastic pruning masks by minimizing expected loss, considering data-adaptive regularization and feature alignment.
result Probabilistic fine-tuning leads to improved test error over baseline methods in neural networks.
We consider the problem of selecting an optimal mask for an image manifold, i.e., choosing a subset of the pixels of the image that preserves the manifold's geometric structure present in the original data. Such masking implements a form of compressive sensing through emerging imaging sensor platforms for which the pow…
Recent object detectors use four-coordinate bounding box (bbox) regression to predict object locations. Providing additional information indicating the object positions and coordinates will improve detection performance. Thus, we propose two types of masks: a bbox mask and a bounding shape (bshape) mask, to represent t…
The paper investigates the effects of invalid action masking in policy gradient algorithms.
problem Invalid actions in policy gradient algorithms can lead to suboptimal performance.
method The paper provides theoretical justification and empirical demonstrations of the importance of invalid action masking.
result Invalid action masking is crucial as the number of invalid actions increases.
Improved YOLOv5 model detects mask-wearing with enhanced accuracy.
problem Detecting mask-wearing in high traffic public places.
method Improved YOLOv5l with Multi-Head Attentional Self-Convolution, Swin Transformer Block, I-CBAM module, and enhanced feature fusion.
result 1.1% improvement in mAP(0.5) and 1.3% improvement in mAP(0.5:0.95) compared to YOLOv5l.
Simplified masked diffusion models improve discrete data generation.
problem Complex model formulations and unclear relationships hinder discrete data generative modeling.
method Developed a simple and general framework for masked diffusion models.
result Models trained on OpenWebText surpass prior diffusion language models and outperform autoregressive models.
Language models trained on chess board states outperform those on moves, even with causal masking.
problem Applying causal masking to spatial data for training unimodal language models.
method Trained bidirectional and causal self-attention models on both spatial (board-based) and sequential (move-based) chess data.
result Models trained on spatial board states achieve stronger playing strength than those trained on sequential data, even with causal masking.
We analyse the structure of the distribution of eigenvalues of the stock market correlation matrix with increasing length of the time series representing the price changes. We use 100 highly-capitalized stocks from the American market and relate result to the corresponding ensemble of Wishart random matrices. It turns …
We formalize the notion of a pseudo-ensemble, a (possibly infinite) collection of child models spawned from a parent model by perturbing it according to some noise process. E.g., dropout (Hinton et. al, 2012) in a deep neural network trains a pseudo-ensemble of child subnetworks generated by randomly masking nodes in t…
Study different masking schemes for a universal marginaliser.
problem Understand how well a neural approximator learns conditional distributions.
method Compare networks trained with various masking schemes.
result Neural approximators perform differently based on the masking scheme.
We develop mask iterative hard thresholding algorithms (mask IHT and mask DORE) for sparse image reconstruction of objects with known contour. The measurements follow a noisy underdetermined linear model common in the compressive sampling literature. Assuming that the contour of the object that we wish to reconstruct i…
The ConditionaL Neural Networks (CLNN) and the Masked ConditionaL Neural Networks (MCLNN) exploit the nature of multi-dimensional temporal signals. The CLNN captures the conditional temporal influence between the frames in a window and the mask in the MCLNN enforces a systematic sparseness that follows a filterbank-lik…
Unified framework for ICL in causal and masked models.
problem Understanding ICL in masked language models and comparing it to causal models.
method Developed a statistical learning framework representing context by empirical measure and predicting using context and query.
result Upper bounds for masked and autoregressive objectives under Wasserstein-type regularity conditions.
VLSI mask optimization is one of the most critical stages in manufacturability aware design, which is costly due to the complicated mask optimization and lithography simulation. Recent researches have shown prominent advantages of machine learning techniques dealing with complicated and big data problems, which bring p…
Most machine translation systems generate text autoregressively from left to right. We, instead, use a masked language modeling objective to train a model to predict any subset of the target words, conditioned on both the input text and a partially masked target translation. This approach allows for efficient iterative…
This paper analyzes MaskGIT sampler and introduces a moment sampler for faster masked diffusion sampling.
problem Efficiently sampling from masked diffusion models.
method Theoretical analysis of MaskGIT sampler, introduction of moment sampler, and two innovations for improving choose-then-sample efficiency.
result The moment sampler is an asymptotically equivalent, more interpretable alternative to MaskGIT.
Modulating masks improve lifelong reinforcement learning.
problem Catastrophic forgetting and task interference in lifelong reinforcement learning.
method Adapted modulating masks for deep lifelong reinforcement learning (LRL) with PPO and IMPALA agents.
result Superior performance in both discrete and continuous RL tasks compared to LRL baselines.
Enhances DSN with multi-family wavelet transforms and sparsity.
problem Improving interpretability and expressive power of DSN.
method Multi-family wavelet transforms and optimal thresholding.
result Enhanced robustness and diversity of scattering coefficients.
BNCR-GAN improves GANs to generate clean images from degraded inputs.
problem Generating clean images from blurred, noisy, and compressed degraded inputs.
method Multiple-generator model with image, blur-kernel, noise, and quality-factor generators, using masking architectures and adaptive consistency losses.
result BNCR-GAN effectively learns clean image generators from degraded images without degradation parameters.