Proposes a new multi-scale architecture for generative flows to improve log-likelihood and sampling quality.
problem Challenges of high-dimensional latent space in flow models.
method Data-dependent dimension factorization based on likelihood contribution heuristic.
result Improvements in log-likelihood score and sampling quality on image benchmarks.
Recently, neural network based approaches have achieved significant improvement for solving large, complex, graph-structured problems. However, their bottlenecks still need to be addressed, and the advantages of multi-scale information and deep architectures have not been sufficiently exploited. In this paper, we theor…
Improved recurrent neural networks learn long-term dependencies through multi-scale memory.
problem Capturing long-term dependencies in recurrent neural networks.
method Incremental training of a modular RNN architecture with multi-scale hidden states.
result Incremental training and multi-scale memory enhance RNNs' ability to learn long-term dependencies.
Modern audio source separation techniques rely on optimizing sequence model architectures such as, 1D-CNNs, on mixture recordings to generalize well to unseen mixtures. Specifically, recent focus is on time-domain based architectures such as Wave-U-Net which exploit temporal context by extracting multi-scale features. …
New PINN architectures learn high-frequency features using Fourier features.
problem PINNs struggle with high-frequency or multi-scale features.
method Employ spatio-temporal and multi-scale random Fourier features.
result Effective PINN models for multi-scale PDEs.
Paper introduces multi-scale methods to improve CATE estimation from EO data.
problem Challenges in balancing fine-grained and contextual information in EO-based causal inference.
method Multi-Scale Representation Concatenation, combining Vision Transformer and Causal Forests.
result Multi-scale approach captures effect heterogeneity better than single-scale models.
We present a novel high frequency residual learning framework, which leads to a highly efficient multi-scale network (MSNet) architecture for mobile and embedded vision problems. The architecture utilizes two networks: a low resolution network to efficiently approximate low frequency components and a high resolution ne…
Deep CNN predicts disruptions in fusion plasmas with high accuracy.
problem Predicting plasma events in fusion devices with multi-scale, multi-physics characteristics.
method Deep convolutional neural networks (CNN) with dilated convolutions trained on ECEi diagnostic data.
result Deep CNN achieves an F1-score of ~91% on disruption prediction.
Transformer-based multi-scale model outperforms traditional methods in solving PDEs on irregular domains.
problem Solving partial differential equations on irregular domains using deep learning.
method Introduces Multi-Scale Attention Transformer (\msat{}) for solving PDEs.
result Achieves state-of-the-art generalization on complex geometry problems with significant speedup.
Paper improves multi-step chord prediction in jazz music.
problem Chord sequence prediction accuracy is poor in multi-step scenarios.
method Aggregated multi-scale encoder-decoder network architecture.
result Our model outperforms state-of-the-art methods in accuracy and perplexity.
Paper tackles leverage effect estimation from noisy data.
problem Estimating leverage effect from high-frequency data with microstructure noise.
method Holistic multi-scale framework operating directly on leverage effect, using Subsampling-and-Averaging Leverage Effect (SALE) and Multi-Scale Leverage Effect (MSLE) estimators.
result Holistic multi-scale framework achieves substantial efficiency gains over existing benchmarks.
The design of neural network architectures is an important component for achieving state-of-the-art performance with machine learning systems across a broad array of tasks. Much work has endeavored to design and build architectures automatically through clever construction of a search space paired with simple learning …
MCFNet recovers spatial detail and fuses it with semantic information for real-time segmentation.
problem Recovering spatial detail information and fusing it with semantic information in real-time.
method Proposes a new architecture (MCFNet) with feature refinement and fusion modules, and a gating unit.
result Achieves competitive performance with high speed (75.5% mIOU, 151.3 FPS on Cityscapes).
Neural HMM with AGA captures multi-scale dynamics in financial markets.
problem Capturing multi-scale temporal dynamics in financial markets.
method Parallel multi-resolution encoders, adaptive gating, and multi-head attention.
result Outperforms fixed-resolution baselines in predicting price movements and liquidity shocks.
Federated learning improves CRC grading accuracy and privacy.
problem Inter-observer variability and data privacy in CRC grading.
method Multi-scale federated learning framework integrating ResNetRS50.
result Framework achieves 83.5% accuracy, outperforming centralized models.
Proposes a deep multi-scale neural network for EEG signal representation learning.
problem Capturing multi-frequency properties in EEG signals for better brain-computer interface.
method A novel deep multi-scale neural network that discovers feature representations in multiple frequency/time ranges and extracts spatial relationships.
result Improved performance in various active/passive BCI datasets compared to state-of-the-art methods.
Novel neural network solves PDEs with multi-scale resolution.
problem Solving time-dependent PDEs with varying spatial and temporal scales.
method Multi-scale message passing neural network with temporal and spatial gating modules.
result Outperforms baselines on PDEs with diverse scales.
This study presents a new lossy image compression method that utilizes the multi-scale features of natural images. Our model consists of two networks: multi-scale lossy autoencoder and parallel multi-scale lossless coder. The multi-scale lossy autoencoder extracts the multi-scale image features to quantized variables a…
Tensor network architecture for classification and regression using wavelet transformations.
problem Efficiently performing classification and regression tasks on complex data.
method Tensor network layers based on MERA and MPS, with adaptive fine-graining.
result Adaptive fine-graining improves model performance without loss in accuracy.
CViT learns complex physical systems using vision transformer techniques.
problem Learning maps between infinite-dimensional function spaces in scientific machine learning.
method Combines vision transformer encoder, grid-based coordinate embedding, and cross-attention mechanism.
result Achieves state-of-the-art performance on multiple benchmarks, often surpassing larger models.
Proposes MscaleDNN for solving high-dimensional PDEs efficiently.
problem Solving high-dimensional PDEs efficiently.
method Radial scaling in frequency domain and compact support activation functions.
result Increased power in multi-scale resolution and high frequency capturing.
Boosting theory explains why multi-scale GNNs work.
problem Over-smoothing in graph neural networks.
method Gradient boosting and transductive learning analysis.
result Test error bound decreases with more node aggregations.
Many real-world time series, such as in health, have changepoints where the system's structure or parameters change. Since changepoints can indicate critical events such as onset of illness, it is highly important to detect them. However, existing methods for changepoint detection (CPD) often require user-specified mod…
A new UNet variant reduces spectral artifacts in image transformations.
problem Spectral artifacts caused by traditional UNet upsampling layers.
method Introduced a Guided UNet (GUNet) architecture using a novel upsampling module.
result GUNet produces higher fidelity outputs in image transformations.
3D U-Net improves kidney and tumor segmentation from CT scans.
problem Manual segmentation by clinicians is laborious and error-prone.
method Multi-scale supervised 3D U-Net with deep supervision and post-processing.
result MSS U-Net achieves high Dice coefficients (0.969 for kidney, 0.805 for tumor) on KiTS19 dataset.
ElasTST improves time-series forecasting across varying horizons.
problem Robust forecasting across different time horizons in varied industrial sectors.
method Elastic Time-Series Transformer (ElasTST) with non-autoregressive design, rotary position embedding, and multi-scale patching.
result ElasTST provides robust forecasts across varying horizons without retraining.
Neuro-inspired recurrent neural network algorithms, such as echo state networks, are computationally lightweight and thereby map well onto untethered devices. The baseline echo state network algorithms are shown to be efficient in solving small-scale spatio-temporal problems. However, they underperform for complex task…
CAST improves spectral clustering for multi-scale data by integrating reachability similarity.
problem Applying spectral clustering to multi-scale data where clusters vary in size and density.
method CAST integrates reachability similarity with distance-based similarity to derive a coefficient matrix, then applies trace Lasso regularization.
result CAST provides excellent performance and robustness across various multi-scale data test cases.
Learning to predict future images from a video sequence involves the construction of an internal representation that models the image evolution accurately, and therefore, to some degree, its content and dynamics. This is why pixel-space video prediction may be viewed as a promising avenue for unsupervised feature learn…
Efficient deep policy gradient method for continuous-time control problems.
problem Optimal control in continuous time with fine time discretization.
method Multi-scale deep policy gradient method with varying time discretization.
result Targeted efficiency in computational resources achieved through multi-scale approach.
Gatherings of thousands to millions of people frequently occur for an enormous variety of events, and automated counting of these high-density crowds is useful for safety, management, and measuring significance of an event. In this work, we show that the regularly accepted labeling scheme of crowd density maps for trai…
The analysis of temporal networks has a wide area of applications in a world of technological advances. An important aspect of temporal network analysis is the discovery of community structures. Real data networks are often very large and the communities are observed to have a hierarchical structure referred to as mult…
While Generative Adversarial Networks (GANs) have seen huge successes in image synthesis tasks, they are notoriously difficult to adapt to different datasets, in part due to instability during training and sensitivity to hyperparameters. One commonly accepted reason for this instability is that gradients passing from t…
A new multi-scale vector quantization method for unsupervised data.
problem Efficiently reconstructing unsupervised data with minimal distortion.
method Reconstruction trees, inspired by decision trees, explore data in a multi-scale fashion.
result Analysis of expected distortion under fixed unknown distribution, with asymptotic and finite sample results.
Bayesian deep learning counts crowds robustly despite occlusions and scale variations.
problem Accurately counting individuals in crowded scenes with occlusions and varying sizes.
method Proposes a Bayesian multi-scale neural network with a ResNet feature extractor, dilated convolutions, and a Perspective-aware Aggregation Module.
result Achieves superior performance on crowd counting benchmarks with uncertainty estimates.
BNAS improves neural architecture search with a scalable, fast, and efficient approach.
problem Efficiently searching for optimal neural architectures with high performance and low training time.
method Designing a broad scalable architecture (BCNN) with reinforcement learning and parameter sharing, and developing two variants.
result Significantly reduces training time and achieves state-of-the-art performance on CIFAR-10 and ImageNet.
Embed nodes with multi-scale attributes for robust network analysis.
problem Capturing complex node attributes across different scales.
method Multi-scale attributed node embedding (AE & MUSAE) using Skip-gram approach.
result Proves node-feature mutual information is implicitly factorized by embeddings.
WideBNet learns inverse scattering from wide-band data efficiently and stably.
problem Learning the inverse scattering map from wide-band scattering data.
method Combines butterfly factorization, FFT, and deep learning.
result WideBNet requires fewer training points and has stable training dynamics.
Framework for multi-scale clustering using phase transitions.
problem Clustering datasets with multi-scale structures.
method Cascade of phase transitions in simulated annealing of Expectation-Maximisation algorithm with weighted local covariance.
result Approximation of the number and size of clusters at different scales.
EvoMSN tackles time series forecasting under distribution shifts by evolving multi-scale normalization.
problem Accurate long-term time series forecasting under complex distribution shifts.
method EvoMSN framework with multi-scale statistics prediction and adaptive ensembling for collaborative updating.
result Improves forecasting performance of five mainstream methods on benchmark datasets.
Computer-aided diagnosis systems for classification of different type of skin lesions have been an active field of research in recent decades. It has been shown that introducing lesions and their attributes masks into lesion classification pipeline can greatly improve the performance. In this paper, we propose a framew…
Novel approach combines local and global brain changes for AD prediction.
problem Detecting Alzheimer's disease through local and global brain changes.
method Patch-based 3D-CNNs combined with global topological features for multi-scale brain tissue connectivity.
result Average precision score of 0.95 for classifying cognitively normal subjects and AD patients (prevalence ~55%).
Technological development aims to produce generations of increasingly efficient robots able to perform complex tasks. This requires considerable efforts, from the scientific community, to find new algorithms that solve computer vision problems, such as object recognition. The diffusion of RGB-D cameras directed the stu…
TelePiT improves S2S forecasting by integrating physics and teleconnections.
problem Challenges in subseasonal-to-seasonal climate forecasting due to chaotic dynamics and complex interactions.
method Integrates physics and teleconnections into a transformer architecture with spherical embedding and multi-scale physics-informed neural ODE.
result Significantly outperforms state-of-the-art methods across all forecast horizons.
MSD removes dequantization bottleneck in LLM inference by approximating high-precision activations.
problem Dequantization bottleneck in LLM inference on modern AI accelerators.
method MSD decomposes high-precision activations into multiple low-precision components for direct multiplication with quantized weights.
result MSD avoids INT8-to-BF16 weight conversion, reducing dequantization cycles and HBM traffic.
Compactifies moduli spaces of abelian differentials with specific zeroes and poles.
problem Constructing a compactification of moduli spaces of abelian differentials.
method Using a blowup of the incidence variety compactification, defining families of projectivized multi-scale differentials, and performing a real oriented blowup.
result The moduli space of multi-scale differentials is a complex orbifold with normal crossing boundary.
Proposes OC4Seq for detecting anomalies in discrete event sequences.
problem Challenges in detecting anomalies in discrete event sequences, including data imbalance, discrete events, and sequential nature.
method Integrates anomaly detection with recurrent neural networks (RNNs) to embed sequences into latent spaces and designs a multi-scale RNN framework to capture multi-scale sequential patterns.
result OC4Seq consistently outperforms various baselines on three benchmark datasets.
This work aims to improve semi-supervised learning in a neural network architecture by introducing a hybrid supervised and unsupervised cost function. The unsupervised component is trained using a differentiable estimator of the Maximum Mean Discrepancy (MMD) distance between the network output and the target dataset. …