Deep learning improves 3D microscopy resolution without matched target images.
problem Anisotropic resolution in volumetric fluorescence microscopy.
method Cycle-consistent generative adversarial network trained on unpaired 2D images.
result Enhanced axial resolution and restored details between imaging planes.
With super-resolution optical microscopy, it is now possible to observe molecular interactions in living cells. The obtained images have a very high spatial precision but their overall quality can vary a lot depending on the structure of interest and the imaging parameters. Moreover, evaluating this quality is often di…
Develops statistical guarantees for image-to-image regression models.
problem Current image-to-image regression models lack statistical guarantees for model mistakes and hallucinations.
method Uncertainty quantification techniques with rigorous statistical guarantees for image-to-image regression problems.
result Derives uncertainty intervals around each pixel with formal mathematical guarantees.
Local minimax analysis for Poisson deconvolution of discrete signals.
problem Estimating a discrete uniform signal from Poisson convolutions.
method Local minimax risk analysis of a broad class of kernels.
result Sharp estimation rates as a function of signal clustering.
Super-resolution fluorescence microscopy, with a resolution beyond the diffraction limit of light, has become an indispensable tool to directly visualize biological structures in living cells at a nanometer-scale resolution. Despite advances in high-density super-resolution fluorescent techniques, existing methods stil…
Single-molecule localization fluorescence microscopy constructs super-resolution images by sequential imaging and computational localization of sparsely activated fluorophores. Accurate and efficient fluorophore localization algorithms are key to the success of this computational microscopy method. We present a novel l…
Memory-efficient learning for large-scale imaging systems.
problem Memory limitations in GPUs for real-world large-scale inverse problems.
method Exploits reversibility of network layers to enable data-driven design.
result Demonstrated on small-scale and large-scale real-world systems.
A central problem in neuroscience is reconstructing neuronal circuits on the synapse level. Due to a wide range of scales in brain architecture such reconstruction requires imaging that is both high-resolution and high-throughput. Existing electron microscopy (EM) techniques possess required resolution in the lateral p…
New method learns optimal variance schedule for diffusion models.
problem Diffusion models' sensitivity to variance schedule.
method Probabilistic conditioning, learning schedule during training.
result Comparable or superior results in super-resolution microscopy and quantitative phase imaging.
Fluorescence microscopy has enabled a dramatic development in modern biology. Due to its inherently weak signal, fluorescence microscopy is not only much noisier than photography, but also presented with Poisson-Gaussian noise where Poisson noise, or shot noise, is the dominating noise source. To get clean fluorescence…
Signal processing is rich in inherently continuous and often nonlinear applications, such as spectral estimation, optical imaging, and super-resolution microscopy, in which sparsity plays a key role in obtaining state-of-the-art results. Coping with the infinite dimensionality and non-convexity of these problems typica…
Convolutional neural networks (CNN) have achieved state of the art performance on both classification and segmentation tasks. Applying CNNs to microscopy images is challenging due to the lack of datasets labeled at the single cell level. We extend the application of CNNs to microscopy image classification and segmentat…
Method estimates section thickness and XY anisotropy in ssEM images.
problem Accurate 3D reconstructions require precise section thickness and XY anisotropy estimates.
method Non-parametric Bayesian regression of image statistics.
result Method has lower estimation error compared to existing methods.
Study shows zero-shot super-resolution in neural operators is impossible in many cases.
problem Understanding the theoretical limits of zero-shot super-resolution in neural operators.
method Systematic theoretical study including information-theoretic and generalization bounds analysis.
result Zero-shot super-resolution is information-theoretically impossible in many settings.
We discuss recently emerging applications of the state-of-art deep learning methods on optical microscopy and microscopic image reconstruction, which enable new transformations among different modes and modalities of microscopic imaging, driven entirely by image data. We believe that deep learning will fundamentally ch…
Unified method for multi-defect microscopy image restoration with limited training data.
problem Challenges in applying deep learning methods due to limited training data for multi-defect microscopy images.
method Two-stage approach: data augmentation with GAN and conditional GAN training.
result Proposed method gives comparable or superior results to existing methods in image quality restoration.
Neural DEs improve single image super-resolution.
problem Challenging tasks in image super-resolution.
method Applied Neural Differential Equations to image super-resolution, using variational methods and backpropagation.
result Differential models match state-of-the-art performance.
A dataset of 10 molecule types for machine learning studies.
problem Lack of suitable datasets for machine learning in molecular imaging.
method Generated 2D cross-sectional projections of 10 molecule types from Molecular Dynamics trajectories.
result Benchmark dataset for machine learning, deep learning, and image processing in scattering, imaging, and microscopy.
Machine learning classifies colorectal tissue using photoacoustic microscopy.
problem Traditional diagnostic methods for colorectal cancer are limited in detail and painful.
method Machine learning applied to acoustic resolution photoacoustic microscopy.
result Machine learning accurately classified benign and malignant tissue.
DISPR uses diffusion models to predict 3D cell shapes from 2D images.
problem Predicting 3D cell shapes from 2D microscopy images.
method Diffusion model trained to predict 3D shapes from 2D microscopy images as a prior.
result Adding DISPR predictions to minority cell classes improves classification accuracy.
Machine learning speeds up FLIM analysis in biomedical research.
problem Complex, slow, and computationally expensive FLIM analysis.
method Machine learning techniques for faster and smarter FLIM data extraction and interpretation.
result Higher accuracy in classifying and segmenting FLIM images compared to conventional methods.
Bayesian framework for solar magnetogram super-resolution with uncertainty quantification.
problem Uncertainty in super-resolving solar magnetic field images.
method Bayesian decomposition of uncertainties into epistemic and aleatoric.
result Generation of maps measuring the range of possible high-resolution explanations.
Deconvolution microscopy has been extensively used to improve the resolution of the widefield fluorescent microscopy. Conventional approaches, which usually require the point spread function (PSF) measurement or blind estimation, are however computationally expensive. Recently, CNN based approaches have been explored a…
To improve the performance of classical generative adversarial network (GAN), Wasserstein generative adversarial networks (W-GAN) was developed as a Kantorovich dual formulation of the optimal transport (OT) problem using Wasserstein-1 distance. However, it was not clear how cycleGAN-type generative models can be deriv…
Multi-StyleGAN simulates live cell microscopy imagery.
problem Costly and complex live cell experiments.
method Generative adversarial network (GAN) synthesizing multi-domain time-lapse images.
result Captures biophysical factors and time dependencies in cell imagery.
Malaria is a life-threatening disease affecting millions. Microscopy-based assessment of thin blood films is a standard method to (i) determine malaria species and (ii) quantitate high-parasitemia infections. Full automation of malaria microscopy by machine learning (ML) is a challenging task because field-prepared sli…
New method extracts biological concepts from cell microscopy images.
problem Extracting meaningful concepts from vision foundation models trained on cell microscopy images.
method Sparse dictionary learning (DL) combined with PCA whitening pre-processing.
result Successfully retrieved biologically meaningful concepts like cell types and genetic perturbations.
Study improves pollen detection in optical and holographic images using deep learning.
problem Improving pollen detection accuracy in holographic microscopy images.
method Used YOLOv8s for detection and MobileNetV3L for classification, addressing performance gaps through dataset expansion and automated labeling.
result Significant improvement in detection and classification performance on holographic images.
Prototype for adaptive electron microscopy scans reduces dose and time.
problem Reduce electron microscopy scan time and dose with minimal loss.
method Adaptive partial scanning with reinforcement learning.
result Reinforcement learning trained neural network optimizes scan paths.
UDVD uses deep learning to denoise videos without supervision.
problem Lack of clean video data for training deep learning models.
method UDVD is a CNN trained solely on noisy video data, adapting to local motion.
result UDVD performs as well as supervised methods, even with limited training data.
A novel method detects multiple mitosis events and mitigates annotation gaps in phase-contrast microscopy.
problem Detecting multiple mitosis events and handling annotation gaps in closely placed cells.
method Estimating a spatiotemporal likelihood map via 3DCNN to detect multiple mitosis events and mitigate annotation gaps.
result Our method outperformed compared methods in terms of F1-score using a challenging dataset.
New deep learning method improves 4D Flow MRI super-resolution under domain shift.
problem Domain shift in low-resolution 4D Flow MRI data.
method Distributional deep learning framework for domain generalization.
result Framework significantly outperforms traditional methods in real data applications.
Improves spline quality and accuracy in computational microscopy.
problem Detecting slender, overlapping structures in microscopy images.
method Differentiable rendering approach for spline refinement.
result Achieves high reliability and sub-pixel accuracy.
A new method uses conformal prediction to create reliable confidence masks for image super-resolution.
problem Uncertainty quantification in image super-resolution using generative models.
method Conformal prediction techniques applied to a confidence mask for reliable uncertainty communication.
result Strong theoretical guarantees and empirical solid performance in image super-resolution.
New framework compresses and recovers scientific data efficiently.
problem Efficiently managing and recovering from large scientific datasets.
method Grounded in learning exponential families, preserves uncertainty and supports trade-offs.
result Preserves physical features and quantities of interest in compressed representations.
Generative adversarial networks generate realistic, time-evolving high-resolution atmospheric fields.
problem Improving spatial resolution of low-resolution atmospheric images.
method Recurrent, stochastic super-resolution GAN for generating ensembles of time-evolving high-resolution atmospheric fields.
result The GAN produces realistic, temporally consistent super-resolution sequences for radar-measured precipitation and cloud optical thickness.
Deep learning speeds up whole heart MRI to 30 seconds.
problem Long acquisition times in whole heart MRI.
method Deep learning, specifically a 3D residual U-Net, to reconstruct high-resolution images from low-resolution data.
result Super-resolution images show better edge sharpness and fewer artefacts than low-resolution images.
Super-resolution is a classical problem in image processing, with numerous applications to remote sensing image enhancement. Here, we address the super-resolution of irregularly-sampled remote sensing images. Using an optimal interpolation as the low-resolution reconstruction, we explore locally-adapted multimodal conv…
MeshfreeFlowNet generates high-resolution spatio-temporal solutions from low-resolution inputs.
problem Generating high-resolution spatio-temporal solutions from low-resolution inputs.
method Physics-constrained deep learning framework using fully convolutional encoders.
result Significantly outperforms existing baselines in super-resolution of turbulent flows.
Paper proposes SMFN for high-res spherical video super-resolution.
problem Super-resolution of 360-degree panoramic videos is expensive and challenging.
method Deformable convolutions, mixed attention mechanism, dual learning strategy, weighted mean square error loss function.
result The proposed SMFN method improves super-resolution of equatorial regions in 360-degree videos.
A cost-effective method to generate high-resolution images using wavelet-based super-resolution.
problem High compute resources required for training state-of-the-art image generation models.
method Wavelet-based two-step training framework: low-frequency sampling followed by super-resolution.
result Achieves FID of 10.59 on ImageNet 512x512 with half the compute of BigGAN.
GANs can learn hierarchical distributions in real-world images efficiently.
problem Understanding and efficiently learning complex, real-world distributions with GANs.
method Formally studying how GANs can learn hierarchically generated distributions close to real-life image distributions using SGDA.
result Training GANs via SGDA can efficiently learn distributions with a 'forward super-resolution' structure, both in sample and time complexities.
SRFRN accelerates image super-resolution using shallow residual units.
problem High computational complexity and time in deep learning image super-resolution.
method SRFRN uses a bicubic interpolated low-resolution image and residual representative units (RFR) for faster and more efficient high-resolution image reconstruction.
result SRFRN achieves superior performance and faster execution time compared to existing methods.
AI enhances pollen recognition in veterinary imaging using holographic microscopy.
problem Challenges in recognizing pollen in holographic images due to speckle noise and artifacts.
method Training YOLOv8s and MobileNetV3L on dual-modality dataset, employing WGAN-SN for synthetic data augmentation.
result GAN-based augmentation improves object detection and classification in holographic images, closing the performance gap.
This paper improves image super-resolution by integrating cross-scale non-local attention.
problem Improving image super-resolution by leveraging long-range and cross-scale feature correlations.
method Proposes a Cross-Scale Non-Local (CS-NL) attention module integrated into a recurrent neural network.
result Significantly improved performance on SISR benchmarks.
Cryo-electron microscopy (cryo-EM) is an emerging experimental method to characterize the structure of large biomolecular assemblies. Single particle cryo-EM records 2D images (so-called micrographs) of projections of the three-dimensional particle, which need to be processed to obtain the three-dimensional reconstruct…
Enhances VAEs for sharper image synthesis.
problem Blurriness in generated images from VAEs.
method Integrates a downscaled version of the original image into the VAE framework and uses it as input to the decoder.
result Improves FID score in image synthesis while maintaining similar log-likelihood performance.
The need for labour intensive pixel-wise annotation is a major limitation of many fully supervised learning methods for segmenting bioimages that can contain numerous object instances with thin separations. In this paper, we introduce a deep convolutional neural network for microscopy image segmentation. Annotation iss…