CNNs can develop blind spots due to uneven padding in feature maps.
problem Spatial bias in convolutional networks leads to blind spots in certain tasks.
method Identified and analyzed the role of padding in convolutional networks, proposing solutions to mitigate bias.
result Mitigating spatial bias improves model accuracy, especially in tasks like small object detection.
Long Short-Term Memory (LSTM) Networks and Convolutional Neural Networks (CNN) have become very common and are used in many fields as they were effective in solving many problems where the general neural networks were inefficient. They were applied to various problems mostly related to images and sequences. Since LSTMs…
PAD offers a principled approach to malware detection against evasion attacks.
problem Machine Learning techniques for malware detection are vulnerable to evasion attacks.
method PAD proposes a new adversarial training framework with convergence guarantees for robust optimization.
result PAD significantly outperforms state-of-the-art defenses and can harden ML-based malware detection against 27 evasion attacks.
Proves DCNNs with expansive convolution are strongly universally consistent.
problem Theoretical consistency of deep convolutional neural networks (DCNNs).
method Empirical risk minimization on DCNNs with expansive convolution (with zero-padding).
result DCNNs with expansive convolution are strongly universally consistent.
Study improves forecasting in betting markets using novel neural networks.
problem Improving short-term price movement predictions in betting exchanges.
method Innovative convolutional attention mechanisms applied to recurrent neural networks and bi-dimensional layers.
result All proposed innovations positively impact classification task performance.
The problem of finding the vertex correspondence between two noisy graphs with different number of vertices where the smaller graph is still large has many applications in social networks, neuroscience, and computer vision. We propose a solution to this problem via a graph matching matched filter: centering and padding…
We introduce a guide to help deep learning practitioners understand and manipulate convolutional neural network architectures. The guide clarifies the relationship between various properties (input shape, kernel shape, zero padding, strides and output shape) of convolutional, pooling and transposed convolutional layers…
Heart disease is one of the most common diseases causing morbidity and mortality. Electrocardiogram (ECG) has been widely used for diagnosing heart diseases for its simplicity and non-invasive property. Automatic ECG analyzing technologies are expected to reduce human working load and increase diagnostic efficacy. Howe…
In surface mount technology (SMT), mounted components on soldered pads are subject to move during reflow process. This capability is known as self-alignment and is the result of fluid dynamic behaviour of molten solder paste. This capability is critical in SMT because inaccurate self-alignment causes defects such as ov…
This paper presents the concept of an adaptive safe padding that forces Reinforcement Learning (RL) to synthesise optimal control policies while ensuring safety during the learning process. Policies are synthesised to satisfy a goal, expressed as a temporal logic formula, with maximal probability. Enforcing the RL agen…
A deep-learning inference accelerator is synthesized from a C-language software program parallelized with Pthreads. The software implementation uses the well-known producer/consumer model with parallel threads interconnected by FIFO queues. The LegUp high-level synthesis (HLS) tool synthesizes threads into parallel FPG…
This paper shows how to approximate any log-concave distribution using well-conditioned affine coupling flows.
problem Understanding the representational power of affine coupling flows for log-concave distributions.
method Leveraging connections between affine coupling architectures, Langevin dynamics, and Hénon maps to prove log-concave approximation.
result Any log-concave distribution can be approximated using well-conditioned affine-coupling flows.
Study efficient pricing for barrier options in stochastic-volatility models with leverage correction.
problem Barrier options are sensitive to volatility dynamics, especially leverage, making accurate pricing difficult.
method Developed a class of continuous-path stochastic-clock volatility models and a systematic small-ρ expansion to incorporate leverage.
result Transform-only pricing formulas for barrier derivatives are fast and numerically stable, even for negative leverage.
2D CNNs approximate Korobov functions with near-optimal rates.
problem Approximating Korobov functions using 2D CNNs.
method Constructive approach for 2D CNNs with ReLU activations and fully connected layers.
result 2D CNNs achieve near-optimal approximation rates for Korobov functions.
Status prediction and anomaly detection are two fundamental tasks in automatic IT systems monitoring. In this paper, a joint model Predictor & Anomaly Detector (PAD) is proposed to address these two issues under one framework. In our design, the variational auto-encoder (VAE) and long short-term memory (LSTM) are joine…
Study on CNNs' learning rates and approximation capacities.
problem Learning rates and approximation capacities of CNNs.
method New approximation bound and covering number analysis for CNNs.
result Derives minimax optimal convergence rates for CNNs in various learning problems.
Deep neural networks (DNNs) have set benchmarks on a wide array of supervised learning tasks. Trained DNNs, however, often lack robustness to minor adversarial perturbations to the input, which undermines their true practicality. Recent works have increased the robustness of DNNs by fitting networks using adversarially…
A GPU framework speeds up BnB for discrete optimization problems.
problem Optimizing large-scale discrete problems with GPU limitations.
method Parallel BnB nodes in GPU batches, using padding and custom kernels.
result One to two orders of magnitude speedup and zero optimality gap.
Paper proposes a method to solve sparse Bayesian learning problems efficiently.
problem Finding sparsest solutions in high-dimensional settings.
method Sparse Bayesian learning with screening test to identify and remove zero coefficients.
result The method accelerates the solution process for sparse Bayesian learning problems.
In graph instance representation learning, both the diverse graph instance sizes and the graph node orderless property have been the major obstacles that render existing representation learning models fail to work. In this paper, we will examine the effectiveness of GRAPH-BERT on graph instance representation learning,…
Deep Neural Networks, and specifically fully-connected convolutional neural networks are achieving remarkable results across a wide variety of domains. They have been trained to achieve state-of-the-art performance when applied to problems such as speech recognition, image classification, natural language processing an…
Surface mount technology (SMT) is an enhanced method in electronic packaging in which electronic components are placed directly on soldered printing circuit board (PCB) and are permanently attached on PCB with the aim of reflow soldering process. During reflow process, once deposited solder pastes start melting, electr…
Predicting the health of components in complex dynamic systems such as an automobile poses numerous challenges. The primary aim of such predictive systems is to use the high-dimensional data acquired from different sensors and predict the state-of-health of a particular component, e.g., brake pad. The classical approac…
HOPE improves SSMs for long-memory tasks with robust initialization and training.
problem Improving state-space models for long-memory tasks with robust initialization and training.
method Developed a new parameterization scheme called HOPE using Hankel operators and Markov parameters.
result HOPE improves SSMs' performance on Long-Range Arena tasks and demonstrates non-decaying memory.
Proposes a new binary classification model inspired by fluid phase separation.
problem Binary classification challenges.
method Discretization of nonlinear reaction-diffusion equation coupled with ODE, inspired by fluid dynamics.
result PSBC model achieves comparable performance to traditional methods on MNIST.
Bond rating Transition Probability Matrices (TPMs) are built over a one-year time-frame and for many practical purposes, like the assessment of risk in portfolios or the computation of banking Capital Requirements (e.g. the new IFRS 9 regulation), one needs to compute the TPM and probabilities of default over a smaller…
GLSKF improves tensor completion by capturing both global and local variations.
problem Tensor completion with missing entries, especially in data with spatial or temporal side information.
method Integrates smoothness-constrained low-rank factorization with a locally correlated residual process.
result GLSKF achieves superior performance and scalability on real-world datasets.
DMF improves POI recommendation privacy and efficiency.
problem Privacy leaks and high computation/storage costs in centralized MF.
method Decentralized Matrix Factorization (DMF) with random walk training.
result DMF significantly improves recommendation performance.
Neural dialogue models, despite their successes, still suffer from lack of relevance, diversity, and in many cases coherence in their generated responses. These issues can attributed to reasons including (1) short-range model architectures that capture limited temporal dependencies, (2) limitations of the maximum likel…
Unified learning-rate scale for CNNs and ResNets, avoiding depth imbalance.
problem Challenges in choosing an appropriate learning rate for deep networks, especially as depth increases.
method Introduces Arithmetic-Mean μP (AM-μP), constraining network-wide average pre-activation second moment to a constant scale, combined with residual-aware He fan-in initialization. result Demonstrates a −3/2 scaling law for learning rates across depths, enabling zero-shot learning-rate transfer. HFNO enhances interpretability of turbulent flows through parallel wavenumber bin processing.
problem Opaque inner workings of Fourier Neural Operators (FNOs) hinder physical interpretability.
method Introduces HFNO, a novel FNO-based architecture that processes wavenumber bins in parallel, enhancing interpretability.
result HFNO decomposes turbulent flows across various scales, enabling increased interpretability and multiscale modeling.
New approach improves AI's handling of incomplete data.
problem Improving AI's ability to work with incomplete data.
method Proposes a new likelihood-free EM algorithm for faster, more efficient inference.
result More statistically efficient than masking approach and faster than conventional EM.
LipKernel adds robustness to CNNs by enforcing Lipschitz bounds.
problem Improving robustness of CNNs in real-time applications.
method Dissipative layers parameterized by LMIs and 2-D Roesser model.
result Orders of magnitude faster run-time compared to state-of-the-art methods.
We address representational challenges in normalizing flows, particularly depth and conditioning issues.
problem Challenges in training normalizing flows, including vanishing/exploding gradients and poor conditioning.
method Analyzes representational aspects of depth and conditioning in normalizing flows, proving theoretical bounds and investigating phenomena.
result Proves that shallow affine coupling networks are universal approximators in Wasserstein distance if ill-conditioning is allowed.
Improved weather forecasting using deep CNN on cubed-sphere grid.
problem Global weather prediction accuracy and speed.
method Deep convolutional neural network (CNN) on cubed-sphere grid, offline mapping, loss minimization.
result Significantly improved weather forecasts, indefinitely stable, realistic patterns at long lead times.
Chronicals boosts LLM fine-tuning speed by 3.51x over Unsloth.
problem Memory bottleneck in fine-tuning large language models.
method Four optimizations: fused kernels, cross-entropy reduction, LoRA+, and sequence packing.
result 3.51x speedup on A100-40GB with Qwen2.5-0.5B.
Study improves semantic segmentation of LiDAR point clouds for autonomous vehicles.
problem Semantic understanding of 3D LiDAR data for autonomous vehicles.
method Image-based semantic segmentation architectures, network size optimization, improved point cloud projection, soft Dice loss, reduced weight-sharing convolution.
result 3.2% increase in mIoU segmentation performance with 42% reduction in inference time.
pLSTM tackles long-range language modeling and computer vision tasks with parallelizable linear source transition mark networks.
problem Challenges of existing recurrent architectures in handling sequences and multi-dimensional data.
method Introduces pLSTM, a parallelizable linear source transition mark network for linear graphs and DAGs, addressing vanishing/exploding activation/gradient issues.
result pLSTM outperforms Transformers in long-range tasks like arrow-pointing extrapolation and image size extrapolation.
The paper tackles temporal coverage bias in financial panel data, proposing a structuring framework to correct for incomplete histories.
problem Incomplete histories of financial instruments lead to biased panel data.
method Formalizes the problem and proposes a coverage-aware structuring framework using structured metadata and an availability matrix.
result The framework reveals substantial distortions in return dynamics and volatility when naive temporal alignment is used.
Study finds no significant alignment between whitepaper claims and market structure.
problem Correlation between cryptocurrency whitepaper narratives and market behavior.
method Developed a contamination-aware pipeline for measuring structural correspondence, combining NLP classification and market statistics.
result No significant claims-market alignment detected in the sample.
PriRec preserves privacy in POI recommendation by keeping data and models on users' devices.
problem Privacy concerns in centralized POI recommendation models.
method Local differential privacy for sensitive data, secure decentralized gradient descent for linear models, secure aggregation for feature interactions.
result PriRec achieves comparable or better recommendation accuracy than FM while protecting user privacy.
Physics-informed neural networks improve surrogate modeling of turbulent Rayleigh-Bénard convection.
problem Modeling turbulent Rayleigh-Bénard convection with high accuracy and efficiency.
method Physics-informed neural networks (PINNs) with novel padding and regularization techniques.
result Significantly improved predictive accuracy of surrogate models at high Rayleigh numbers Ra = 2 × 10^9.