Forecaster uses graph Transformers to forecast spatial and time-dependent data.
problem Complex spatial and temporal dependencies in data.
method Graph Transformer architecture with sparsification for spatial and temporal dependencies.
result Forecaster significantly outperforms state-of-the-art baselines in taxi demand forecasting.
This research shows adding regularization boosts accuracy and spatial robustness for adversarially transformed examples.
problem Improving accuracy and spatial robustness for adversarially transformed examples.
method Invariance-inducing regularization using worst-case transformations.
result Adding regularization on top of standard or adversarial training reduces relative error by 20% for CIFAR10 without increasing computational cost.
Proposes a non-autoregressive Transformer for time series forecasting.
problem Autoregressive errors and spatial-temporal dependencies in time series forecasting.
method Introduces a Non-Autoregressive Transformer with a learned temporal influence map.
result Demonstrates state-of-the-art performance on time series forecasting datasets.
Diffusion Transformer captures spatial-temporal dependencies in sequential data.
problem Capturing rich spatial and temporal dependencies in sequential data.
method Established theoretical guarantees for diffusion transformers learning Gaussian process data.
result Spatial-temporal dependencies are captured within attention layers of diffusion transformers.
This paper explores the trade-off between spatial and adversarial robustness in neural networks.
problem Understanding the trade-off between spatial and adversarial robustness in neural networks.
method Quantitative analysis and empirical testing with curriculum learning.
result Spatial robustness and adversarial robustness are quantitatively related and can be improved simultaneously.
This paper establishes that the Nahm transform sending spatially periodic instantons (instantons on the product of the real line and a three-torus) to singular monopoles on the dual three-torus is indeed a bijection as suggested by the heuristic. In the process, we show how the Nahm transform intertwines to a Fourier-M…
Recent studies show that widely used deep neural networks (DNNs) are vulnerable to carefully crafted adversarial examples. Many advanced algorithms have been proposed to generate adversarial examples by leveraging the Lp distance for penalizing perturbations. Researchers have explored different defense met…
Probabilistic STNs improve image classification and robustness.
problem Training and robustness issues in STNs.
method Probabilistic extension of STNs that estimates stochastic transformations.
result Improved classification performance, robustness, and model calibration.
Spatial graphs can be unknotted with region crossing changes.
problem Unknotted spatial graphs composed of theta-curves.
method Region crossing changes on regions of theta-curves.
result Spatial graphs of theta-curves can be unknotted.
Image clustering is an important but challenging task in machine learning. As in most image processing areas, the latest improvements came from models based on the deep learning approach. However, classical deep learning methods have problems to deal with spatial image transformations like scale and rotation. In this p…
Proposes a transformer model with geostatistical inductive bias for spatio-temporal forecasting.
problem Combining probabilistic rigor of geostatistics with flexible deep learning representations.
method Spatially-informed transformer with learnable covariance kernel.
result Successfully recovers spatial decay parameters end-to-end via backpropagation.
A model separates visual style from digit type on MNIST and facial features from shape on CelebA.
problem Learning compact, independent factors of data.
method Explicitly encoded in a generative model with two latent spaces: spatial transformations and intrinsic appearance.
result The model separates visual style from digit type on MNIST and facial features from shape on CelebA.
CutMix training technique improves spatial locality in Vision Transformers.
problem Improving spatial locality in Vision Transformers trained from scratch.
method Comparison of Baseline and Modern training protocols on CIFAR-10, CIFAR-100, and Tiny-ImageNet.
result CutMix training component significantly reduces Mean Attention Distance (MAD) in early layers of Vision Transformers.
Spatial graphs of non-Eulerian or proper Eulerian planar graphs are unknottable by region crossing changes.
problem Unknottability of spatial graphs by region crossing changes.
method Region crossing changes to switch over/under relations within regions of spatial graph diagrams.
result Spatial graphs of non-Eulerian or proper Eulerian planar graphs are unknottable by region crossing changes.
A new method detects financial fraud using graph transformers.
problem Detecting fraudulent transactions in financial data.
method Spatial-Temporal-Aware Graph Transformer (STA-GT) integrating GNNs and transformers.
result STA-GT outperforms general GNN models on financial fraud detection.
New method improves traffic forecasting models by adapting to spatial shifts.
problem Improving traffic forecasting models' ability to handle spatial shifts over years.
method Proposes a novel Mixture of Experts (MoE) framework for spatiotemporal models.
result Significant improvement in performance for handling spatial distribution shifts.
Backlund transformations of admissible curves in the Galilean 3-space and pseudo-Galilean 3-space and also spatial Backlund transformations of space curves in Galilean 4-space preserve the torsions under certain assumptions.
Develops deep models to handle nonstationary spatial extremal dependence.
problem Challenges in modeling nonstationary extremal dependence in spatial data.
method Deep compositional spatial models to capture nonstationarity in extremal dependence.
result Efficient estimation of warped space for nonstationary spatial data.
TK-GCN forecasts spatiotemporal dynamics using Koopman-enhanced graph convolutional networks.
problem Forecasting complex spatiotemporal dynamics over irregular domains.
method Two-stage framework: Koopman-enhanced Graph Convolutional Network (K-GCN) for spatial encoding and Transformer for temporal modeling.
result TK-GCN outperforms state-of-the-art methods in spatiotemporal cardiac dynamics forecasting.
Vision Transformers show different internal representations compared to CNNs.
problem Understanding how Vision Transformers solve image classification tasks.
method Comparative analysis of ViT and CNN architectures on image classification benchmarks.
result ViT has more uniform representations across all layers, while CNNs have more varied representations.
The complex wave representation (CWR) converts unsigned 2D distance transforms into their corresponding wave functions. Here, the distance transform S(X) appears as the phase of the wave function φ(X)---specifically, φ(X)=exp(iS(X)/τwhere τis a free parameter. In this work, we prove a novel result using the higher-orde…
Study develops ensemble machine learning framework for predicting groundwater heavy metal pollution.
problem Statistical complexity and spatial heterogeneity of heavy metal contamination in groundwater.
method Nested cross-validated ensemble machine learning with response transformations (raw, log, Gaussian copula).
result Copula-based models with DBSCAN clustering diagnostics provide the most reliable and interpretable assessments of groundwater contamination.
We introduce a general method of performing Residual Network inference and learning in the JPEG transform domain that allows the network to consume compressed images as input. Our formulation leverages the linearity of the JPEG transform to redefine convolution and batch normalization with a tune-able numerical approxi…
The main result is a computation of the Nahm transform of a SU(2)-instanton over RxT^3, called spatially-periodic instanton. It is a singular monopole over T^3, a solution to the Bogomolny equation, whose rank is computed and behavior at the singular points is described.
Kriformer uses graph transformers to estimate data in sparse sensor areas.
problem Sparse sensor deployment and unreliable data in spatiotemporal kriging tasks.
method Graph transformer model with positional encoding and attention mechanisms.
result Kriformer excels in representing unobserved locations in spatiotemporal kriging tasks.
Proposes new convex relaxations for certifying spatial robustness of neural networks.
problem Lack of provable guarantees for robustness against vector field transformations.
method Novel convex relaxations for certifying robustness against vector field transformations.
result First time providing a certificate of robustness against vector field transformations.
Hybrid framework merges data and domain knowledge for better spatial interpolation.
problem Spatial interpolation overlooks domain knowledge and limits to spatial coordinates.
method Integrates data-driven features with rule-assisted spatial dependency function mapping.
result Superior performance in two application scenarios, capturing localized features.
Transforms improve CNNs' invariance to image transformations.
problem Current CNN models lack robustness to spatial transformations.
method Randomly transform feature maps during training to learn invariant representations.
result Significant improvements on benchmark tasks, including image recognition and retrieval.
Classifies colored links and spatial graphs up to colored link-homotopy.
problem Classifying colored links and spatial graphs up to colored link-homotopy.
method Using Habegger-Lin theory for colored string links, and extending to colored links and spatial graphs.
result Classification of colored links and spatial graphs up to colored link-homotopy.
REST improves robustness of black-box models to geometric transformations.
problem Overconfident incorrect predictions on out-of-distribution samples.
method REinforcement Spatial Transform learner (REST) that transforms input data into in-distribution samples.
result Improves robustness to geometric transformations and sample efficiency.
Neural network iteratively refines image registration, achieving compactness and speed.
problem Non-compact representation of deformations in image registration.
method Recurrent registration neural network that computes local deformations iteratively.
result Our method achieves similar accuracy but is more compact and faster.
In this paper, we develop a novel Backtrackless Aligned-Spatial Graph Convolutional Network (BASGCN) model to learn effective features for graph classification. Our idea is to transform arbitrary-sized graphs into fixed-sized backtrackless aligned grid structures and define a new spatial graph convolution operation ass…
Time-aware deep learning methods improve spatial downscaling of atmospheric pollutants.
problem Transform coarse satellite data of atmospheric pollutants into high-resolution fields.
method Super-resolution deep residual networks and UNet architectures are extended with a temporal module encoding observation time.
result Temporal modules significantly improve downscaling performance and convergence speed.
We construct the Nahm transform from finite energy instantons on the product of a real line and a three dimensional torus to Dirac-type singular monopoles on the dual torus. Moreover, we show the correspondence between the data which handle the asymptotic behavior of instantons at infinity and one of monopoles at singu…
Planar neural networks learn image transformations from sequences.
problem Learning image transformations for mental simulation.
method Using planar neural networks, the study investigates various factors affecting the learning of image transformations.
result The approach can effectively learn and transfer image transformations, including translation, rotation, and scaling.
PixelHop++ improves image classification with a smaller model size.
problem Improving image classification models with smaller sizes.
method Decomposing input tensor, channel-wise Saab transform, successive subspace learning, feature ranking.
result PixelHop++ offers a flexible tradeoff between model size and performance.
The evolute of a smooth curve in an m-dimensional Euclidean space is the locus of centers of its osculating spheres, and the evolute of a spatial polygon is the polygon whose consecutive vertices are the centers of the spheres through the consecutive (m+1)-tuples of vertices of the original polygon. We study the iterat…
This paper explains a mechanism called phase collapse that improves image classification accuracy.
problem Understanding the role of non-linearities and convolutional filters in image classification.
method Demonstrates phase collapse as a mechanism that eliminates spatial variability and linearly separates classes.
result Phase collapse improves classification accuracy, while thresholding operators degrade performance.
Researchers develop a new spatial process model for non-Gaussian data.
problem Non-Gaussian spatial data with asymmetry and heavy-tailedness.
method Re-parameterized Unified Skew-Normal (SUN) distribution, GSUN process, neural Bayes inference with GATs.
result GSUN process captures non-Gaussian spatial data properties and outperforms conventional models.
Spacetimeformer learns spatiotemporal relationships from data alone.
problem Forecasting multivariate time series with distinct spatial relationships.
method Transformers with dynamic graph connections learning interactions between space, time, and value.
result Competitive results on various time series prediction benchmarks.
This note introduces a regression technique for finding a class of nonlinear integro-differential operators from data. The method parametrizes the spatial operator with neural networks and Fourier transforms such that it can fit a class of nonlinear operators without needing a library of a priori selected operators. We…
Proposes a feature transformation for spatio-temporal traffic models to improve performance and transferability.
problem Limited transferability of deep learning models for traffic flow prediction across different locations.
method Integrates Newell's traffic flow estimators to capture broader dynamics and incorporates spatial dependencies.
result Improves model performance in predicting traffic flows over different horizons.
Adversarial learning improves image augmentation for neural networks.
problem Improving data augmentation for neural networks with limited data.
method Adversarial learning using an encoder-decoder architecture with a spatial transformer network.
result Our approach outperforms previous generative data augmentation methods.
We propose a probabilistic model for refining coarse-grained spatial data by utilizing auxiliary spatial data sets. Existing methods require that the spatial granularities of the auxiliary data sets are the same as the desired granularity of target data. The proposed model can effectively make use of auxiliary data set…
Spatially aware ESN detects anomalies in chaotic time series.
problem Automated anomaly detection in chaotic time series, especially turbulent ocean simulations.
method Extended Echo State Network with spatially aware input maps and loss function.
result Spatial ESN reduces anomaly detection to thresholding of prediction error.
A new method matches measures across different spaces using cost-regularized optimal transport.
problem Matching measures in different spaces without aligned data.
method Cost-regularized optimal transport formulation to match measures across two Euclidean spaces.
result Demonstrated applicability to single-cell spatial transcriptomics/multiomics matching tasks.
New algorithm improves sparse-view tomography without needing ground-truth data.
problem Poor image reconstructions with sparse projections and non-uniform sensors.
method Unsupervised deep learning with CNN and STN modules.
result Significantly outperforms filtered backprojection in sparse-view scenarios.
GraphDINO learns neuronal morphologies from unlabeled data.
problem Unsupervised learning of neuronal morphologies from unlabeled data.
method Transformer-based approach with novel attention mechanism and data augmentation.
result GraphDINO yields morphological clusterings on par with expert classification.