Channel normalization prevents vanishing gradients in convolutional neural networks.
problem Vanishing gradients in convolutional neural networks during optimization.
method Channel normalization, which centers and normalizes each channel individually.
result Channel normalization avoids vanishing gradients, enabling efficient optimization.
CN normalizes channels for better time series model performance.
problem Improper channel identification in time series models.
method Channel Normalization (CN) and its variants (ACN, PCN) using distinct affine transformations.
result Significant performance gains across various time series models.
A new method prunes neural network channels based on operation characteristics.
problem Compressing deep neural networks efficiently and maintaining accuracy.
method Differentiable masks for channel pruning considering BN and ReLU.
result Outstanding performance in accuracy with less resources compared to state-of-the-art methods.
New exact tests detect changepoints in binary and count data, especially when normal approximations fail.
problem Detecting changepoints in multichannel binary and count data.
method Exact tests combining two-sample conditional tests with multiplicity correction.
result Exact tests are much more powerful than asymptotic tests in various settings.
Model predicts stock price changes and forecasts using tokenized data.
problem Challenges in stock price forecasting and prediction due to dynamic data and statistical differences.
method Introduces PCIE model with tokenization to handle both forecasting and prediction.
result PCIE model outperforms state-of-the-art models in forecast and prediction tasks.
Generative diffusion models improve channel sampling from limited data.
problem Challenges in channel modelling and data collection for wireless systems.
method Diffusion model with U-Net architecture for frequency domain synthesis.
result Stable training and diverse high-fidelity samples generated from true channel distribution.
New technique trains deep neural networks without normalization or minibatch statistics.
problem Training deep neural networks at high learning rates without normalization.
method Channel-wise zero-mean initialization and gradient modification to maintain common mode rejection.
result Achieves higher accuracy compared to batch normalization and shows minibatches are unnecessary.
Batch normalization has become ubiquitous in many state-of-the-art nets. It accelerates training and yields good performance results. However, there are various other alternatives to normalization, e.g. orthonormalization. The objective of this paper is to explore the possible alternatives to channel normalization with…
Inspired by recent advances in deep learning, we propose a novel iterative BP-CNN architecture for channel decoding under correlated noise. This architecture concatenates a trained convolutional neural network (CNN) with a standard belief-propagation (BP) decoder. The standard BP decoder is used to estimate the coded b…
New ML models improve VVLC channel characterization for vehicular OWC.
problem Inaccurate channel models for VVLC due to mobility effects.
method Machine learning (ML) models incorporating ambient light, turbulence, and reflection effects.
result ML models predict VVLC channel loss and CFR more accurately than existing methods.
Dual U-net models improve multi-channel MRI image reconstruction.
problem Improving MRI image reconstruction from multi-channel data.
method Two-element U-nets (W-nets) in k-space and image domains, evaluated for four configurations.
result Dual domain methods are more advantageous for simultaneous reconstruction of all channels.
Quantum channels' contraction under privacy constraints studied.
problem Understanding the privacy constraints on quantum channel contractions.
method Established upper bounds on contraction coefficients for specific divergences under QLDP constraints.
result Upper bounds and full characterization of contraction coefficients for specific quantum distances.
A fall is an abnormal activity that occurs rarely, so it is hard to collect real data for falls. It is, therefore, difficult to use supervised learning methods to automatically detect falls. Another challenge in using machine learning methods to automatically detect falls is the choice of engineered features. In this p…
A new method normalizes activations to match batch normalization without batch dependence.
problem Performance degradation with batch-independent normalization techniques.
method Proxy-Normalizing Activations
result Proxy-Normalization technique emulates batch normalization's behavior and performance.
A-MMSE uses attention to learn efficient OFDM channel estimation.
problem Accurate OFDM channel estimation requires second-order statistics, which are hard to obtain in practice.
method A-MMSE is a model-based DNN framework that learns linear MMSE filters via Attention Transformer, reducing inference complexity.
result A-MMSE outperforms other methods in normalized MSE across various SNR conditions.
Finet uses FBN for efficient, lightweight neural networks.
problem Building efficient neural networks with limited computational resources.
method Introduces Fine-grained Batch Normalization (FBN) and a novel light-weight network (Finet) that combines FBN with standard convolution.
result Finet achieves state-of-the-art performance on ImageNet classification with reduced computational complexity.
We consider a setup in which confidential i.i.d. samples X1,…,Xn from an unknown finite-support distribution p are passed through n copies of a discrete privatization channel (a.k.a. mechanism) producing outputs Y1,…,Yn. The channel law guarantees a local differential privacy of ε. …
ProFITi model forecasts irregular time series with missing values using conditional flows.
problem Probabilistic forecasting of irregularly sampled multivariate time series with missing values.
method ProFITi model uses conditional normalizing flows and invertible layers to learn joint distributions conditioned on past observations and queried channels and times.
result ProFITi model provides 4 times higher likelihood than the previous best model.
We characterize the singular values of the linear transformation associated with a standard 2D multi-channel convolutional layer, enabling their efficient computation. This characterization also leads to an algorithm for projecting a convolutional layer onto an operator-norm ball. We show that this is an effective regu…
Locally private online quantile regression method addresses privacy constraints.
problem Estimating and inferring quantile regression under local differential privacy constraints.
method Developed a finite-alphabet channel where users compute local contributions, apply randomized response, and send reports. A public decoder corrects distortion and reconstructs inputs for averaging.
result Established local privacy, decoder unbiasedness, consistency, asymptotic normality, and inference for scalar contrasts.
We investigate how the final parameters found by stochastic gradient descent are influenced by over-parameterization. We generate families of models by increasing the number of channels in a base network, and then perform a large hyper-parameter search to study how the test error depends on learning rate, batch size, a…
IEBN normalizes noise by enhancing instance-specific information, improving deep learning performance.
problem Improving deep learning performance by regulating noise in batch normalization.
method Integrates self-attention mechanism to recalibrate channel information in BN.
result IEBN outperforms BN with improved generalization and stability.
Deep learning predicts V2I channel responses for high-mobility wireless communications.
problem Efficient channel estimation for high-mobility V2I communications.
method Developed a deep neural network-based channel prediction method.
result Deep neural networks can learn and predict V2I channel properties in real-time.
Attention mechanism is a hot spot in deep learning field. Using channel attention model is an effective method for improving the performance of the convolutional neural network. Squeeze-and-Excitation block takes advantage of the channel dependence, selectively emphasizing the important channels and compressing the rel…
The Expectation-Maximization (EM) algorithm for mixture models often results in slow or invalid convergence. The popular convergence proof affirms that the likelihood increases with Q; Q is increasing in the M -step and non-decreasing in the E-step. The author found that (1) Q may and should decrease in some E-steps; (…
Proposes dynamic channel pruning during neural network training.
problem Pruning neural networks during training to reduce computational cost and improve efficiency.
method Dynamic channel propagation to update channel utility values and selectively prune channels.
result Our scheme trains and prunes neural networks simultaneously, achieving superior performance.
A new DVAE architecture improves channel estimation by incorporating temporal correlations.
problem Improving the estimation of time-varying channels.
method Introducing k-MemoryMarkovVAE (k-MMVAE) architecture to learn temporal correlations.
result The k-MMVAE aided channel estimator outperforms other ML aided estimators.
Study uses neural networks to predict wall quantities in turbulent flows.
problem Predicting wall quantities in turbulent open channel flows.
method Training convolutional neural networks (FCN) and a proposed R-Net architecture to predict wall-shear-stress and wall pressure.
result R-Net architecture performs better and predicts wall quantities with around 10% error.
BestChanID identifies the channel with maximal capacity using training sequences.
problem Identifying the channel with maximal capacity among several discrete memoryless channels.
method Formulated as a multi-armed bandit problem, proposed a capacity estimator, and developed gap-elimination algorithms.
result Guaranteed to output the DMC with the largest capacity with a desired confidence.
A privacy-preserving method for transmitting data over a wiretap channel using generative networks.
problem Privacy protection in communication over a wiretap channel.
method Data-driven approach using variational autoencoder (VAE)-based joint source channel coding (JSCC).
result The approach provides high reconstruction quality at the receiver while confusing the eavesdropper about the latent sensitive attribute.
New adaptive SGD algorithms for federated learning over physical channels.
problem Reducing communication cost in federated learning over physical channels.
method Proposed adaptive federated SGD algorithms considering channel noise and hardware constraints.
result Demonstrated convergence rates adaptive to stochastic gradient noise level.
We present a definition of discrete channel surfaces in Lie sphere geometry, which reflects several properties for smooth channel surfaces. Various sets of data, defined at vertices, on edges or on faces, are associated with a discrete channel surface that may be used to reconstruct the underlying particular discrete L…
FRN layer eliminates batch dependence in deep learning, improving performance across various tasks.
problem Batch Normalization's dependency on mini-batch elements can degrade performance for small batches.
method Filter Response Normalization (FRN) operates independently on each activation channel of each batch element.
result FRN layer outperforms BN and other alternatives in various settings for all batch sizes.
VAE leverages MMSE channel estimation with data-driven modeling.
problem Data-driven channel estimation for wireless communications.
method Variational autoencoder (VAE) modeling of channel distribution and LMMSE approximation.
result VAE-based channel estimators approximate MMSE performance with practical training methods.
Paper improves channel charting using autoencoders with spatial constraints.
problem Improving logical positioning of UEs using channel-state information.
method Representation-constrained autoencoders to enhance channel charts.
result Improved quality of learned channel charts for UE positioning.
Interpretation of electroencephalogram (EEG) signals can be complicated by obfuscating artifacts. Artifact detection plays an important role in the observation and analysis of EEG signals. Spatial information contained in the placement of the electrodes can be exploited to accurately detect artifacts. However, when few…
This paper proposes a new approach to Transformers by integrating hierarchical associative memory with MetaFormers.
problem Theoretical framework for Transformers and MLP-Mixers remains underdeveloped.
method Integrating hierarchical associative memory with MetaFormers to create a parallelized MLP-Mixer.
result Symmetry-breaking effects improve the performance of the MLP-Mixer in image recognition tasks.
New DL algorithm estimates OFDM channels without pilots.
problem Estimating OFDM channels in deep fading conditions.
method Deep learning (DL) for blind channel estimation.
result First theory on MSE performance of DL-based estimator.
New method improves ResNet performance without batch normalization.
problem Improving ResNet performance without batch normalization.
method Adapted Weight Standardization to maintain signal propagation.
result Highly performant ResNets achieve state-of-the-art performance on ImageNet.
Paper proposes online learning for MIMO channel estimation using neural networks.
problem Challenges in MIMO channel estimation due to physical model simplifications and system configuration requirements.
method Unfold a channel estimation algorithm as a neural network, allowing online learning and correction of imperfect models.
result Achieves channel estimation error close to that of a perfectly calibrated system.
Novel channel pruning method accelerates deep CNNs by removing redundant, similar features.
problem Limitations of existing magnitude-based pruning algorithms in similar magnitude cases.
method Hierarchical clustering of channels based on probabilistic similarity metrics, without sparsity training.
result 30% reduction in FLOPs for pruned ResNet-50 on ImageNet, outperforming baseline.
MGLM models all possible language channel factorizations for improved multilingual generation.
problem Generating multilingual text with flexibility and quality.
method Generative joint distribution model over language channels, marginalizing all possible factorizations.
result MGLM outperforms traditional models in multilingual generation tasks.
A group of transition probability functions form a Shannon's channel whereas a group of truth functions form a semantic channel. Label learning is to let semantic channels match Shannon's channels and label selection is to let Shannon's channels match semantic channels. The Channel Matching (CM) algorithm is provided f…
Proposes GPCA module for channel attention in CNNs using Gaussian processes.
problem Improving performance in visual tasks through effective channel selection.
method Integrates Gaussian processes into channel attention mechanisms for probabilistic modeling of channel correlations.
result Demonstrates improved performance of GPCA module in end-to-end CNN training.
SOR-Mamba improves Mamba for robust time series forecasting by minimizing channel order bias.
problem Robust time series forecasting with Mamba's sequential order bias.
method SOR-Mamba incorporates regularization to minimize channel order discrepancy and introduces CCM for channel correlation preservation.
result SOR-Mamba enhances robustness to channel order and improves forecasting accuracy.
ReQuestNet simplifies 5G channel estimation with a unified model.
problem Complex channel estimation in 5G systems with varying conditions.
method Unified neural architecture that handles dynamic resource blocks and transmit layers.
result Significantly outperforms legacy methods, achieving up to 10dB gain at high SNRs.
We propose a joint source and channel coding (JSCC) technique for wireless image transmission that does not rely on explicit codes for either compression or error correction; instead, it directly maps the image pixel values to the complex-valued channel input symbols. We parameterize the encoder and decoder functions b…
We propose channel charting (CC), a novel framework in which a multi-antenna network element learns a chart of the radio geometry in its surrounding area. The channel chart captures the local spatial geometry of the area so that points that are close in space will also be close in the channel chart and vice versa. CC w…