Generative Adversarial Networks create synthetic data for structural damage detection.
problem Data scarcity in structural damage detection.
method 1-D Wasserstein Deep Convolutional Generative Adversarial Networks (1-D WDCGAN-GP) for synthetic data generation.
result Generated synthetic data improves damage detection accuracy in 1-D Deep Convolutional Neural Networks.
The paper improves GANs' theoretical guarantees for low-dimensional data.
problem Theoretical guarantees for GANs' statistical accuracy remain pessimistic.
method Analytical derivation of statistical guarantees on estimated densities.
result Theoretical rates of convergence for GANs and BiGANs are derived.
Adversarial Regression is a proposition to perform high dimensional non-linear regression with uncertainty estimation. We used Conditional Generative Adversarial Network to obtain an estimate of the full predictive distribution for a new observation. Generative Adversarial Networks (GAN) are implicit generative models …
Study robust learning of Lipschitz functions under corrupted binary signals.
problem Learning a Lipschitz function with corrupted binary signals in a context of unknown corruption rounds.
method Introduced agnostic checking and new analysis techniques to design algorithms for symmetric and pricing losses.
result Achieved small cumulative loss for both symmetric and pricing losses.
High-dimensional smoothing techniques struggle with robustness guarantees against various attacks.
problem Challenges in extending randomized smoothing to other attack models in high-dimensional space.
method Analysis of isotropic and generalized Gaussian smoothing distributions, proving bounds on certified robustness radii.
result Certifiable robustness radii decrease as $O(1/d^{ rac{1}{2} - rac{1}{p}})$ with dimension d for p>2. Deep neural networks can approximate complex functions through repeated compositions of a fixed-size ReLU network.
problem Understanding the expressive power of deep neural networks through function compositions.
method Demonstrated the surprising expressive power of repeated compositions of a single fixed-size ReLU network.
result Repeated compositions of a single fixed-size ReLU network can approximate 1-Lipschitz continuous functions on [0,1]d with an error O(r−1/d). Deep neural nets on 1-D data are convex Lasso models with reflection features.
problem Training neural networks on 1-D data.
method Proving equivalence to convex Lasso problems with discrete, explicitly defined dictionary matrices.
result Reflection features in neural networks with certain activations.
New proof shows incremental flow models are essential for universal generation.
problem Understanding the universality of flow-based models in generating natural maps.
method Topological-dynamical argument and algebraic properties of flows.
result Incremental generation is necessary and sufficient for universal flow-based generation.
Deep ReLU networks can approximate and learn smooth functions efficiently.
problem Efficiently approximating and learning smooth functions using deep ReLU neural networks.
method Extending recent results to anisotropic and mixed smooth function classes, establishing approximation rates.
result Deep ReLU networks achieve minimax optimal rates up to logarithmic factors for various smooth function classes.
Deep ReLU networks can efficiently approximate Sobolev and Besov functions.
problem Approximating functions in Sobolev and Besov spaces using deep neural networks.
method Used deep ReLU neural networks with varied width and depth to approximate functions in Sobolev and Besov spaces.
result Generalized the approximation rate to hold under the Sobolev embedding condition.
1-D CNNs classify pupil size variations in scotopic conditions.
problem Handling wide inter-subjects variability in pupil analysis.
method 1-D Convolutional Neural Networks applied directly to raw pupil size data.
result 1-D CNNs provide high accuracy in classifying short-range pupil size sequences.
This paper optimizes ReLU networks for approximating Hölder continuous functions.
problem Optimizing the approximation rate of ReLU networks in terms of width and depth.
method Constructive proof of ReLU networks' approximation power with specific width and depth constraints.
result Optimal approximation rate of ReLU networks with width and depth constraints.
New neural network architecture with height adds expressive power.
problem Expressiveness of neural networks limited by width and depth.
method Introduces height as a new hyper-parameter in neural network architecture.
result Neural networks with height achieve significantly better approximation of functions.
This paper characterizes stable polynomial mappings in a specific set.
problem Characterizing stable polynomial mappings in a given set.
method Analyzing polynomial mappings with specific degrees and determining topological equivalence.
result Effective determination of mappings with generic topology.
Power-law spectrum of random feature model is preserved in neural networks.
problem Preserving power-law spectrum in neural networks through random feature model.
method Characterized eigenvalues of population random-feature covariance using dyadic head-tail decomposition and Wick chaos expansions.
result Power-law exponent α is inherited from input covariance, modified by a logarithmic correction. Minimum width for ReLU networks to approximate L^p functions is max(d_x+1, d_y).
problem Characterizing the minimum width for ReLU networks to approximate L^p functions.
method Analyzing networks with ReLU activation functions and proving the minimum width required.
result The minimum width required for the universal approximation of L^p functions is exactly max(d_x+1, d_y).
Study improves mortality prediction in ICU patients using feature engineering and 1D CNN.
problem Improving mortality prediction in ICU patients with high-dimensional, imbalanced, and missing data.
method Feature engineering, 1D Convolutional Neural Network (1D CNN), traditional machine learning algorithms.
result Best AUC of 0.848 achieved with 1D CNN model.
Where dealing with temporal sequences it is fair to assume that the same kind of deformations that motivated the development of the Dynamic Time Warp algorithm could be relevant also in the calculation of the dot product ("convolution") in a 1-D convolution layer. In this work a method is proposed for aligning the conv…
Constructs a path integral for fermionic SPTs, solving anomalies in 2+1D topological orders.
problem Anomalies in (2+1)D fermionic topological phases and their computation.
method Combining (2+1)D fermionic topological order with symmetry fractionalization data to construct a (3+1)D path integral.
result Reproduces the Z16 anomaly indicator for time-reversal symmetric topological superconductors. Deep ReLU networks can approximate smooth functions nearly optimally.
problem Approximating smooth functions with deep neural networks.
method Using Taylor expansions and deep ReLU network approximations, the paper establishes optimal approximation error bounds.
result Deep ReLU networks of width and depth O(NlnN) and O(LlnL) can approximate f∈Cs([0,1]d) with an error O(∥f∥Cs([0,1]d)N−2s/dL−2s/d). Gradient descent learns over-param neural nets better than NTK.
problem Learning over-parametrized neural networks with ReLU activations.
method Gradient descent from random initialization on a Gaussian input distribution.
result Gradient descent achieves population loss o(1/d), while NTK achieves Ω(1/d). Study on surfaces in flag threefold with constraints on twistor fibers.
problem Understanding the arrangement and existence of twistor fibers in surfaces of specific bidegree.
method Analyzing surfaces of bidegree (1,d) in the flag threefold, proving existence and non-existence of twistor fibers.
result Existence and non-existence of surfaces containing specific numbers of twistor fibers, with improved results for d=2 and d=3.
A classic problem in physics is the origin of fat tailed distributions generated by complex systems. We study the distributions of stock returns measured over different time lags τ. We find that destroying all correlations without changing the τ=1 d distribution, by shuffling the order of the daily returns, causes…
We propose a novel technique to make neural network robust to adversarial examples using a generative adversarial network. We alternately train both classifier and generator networks. The generator network generates an adversarial perturbation that can easily fool the classifier network by using a gradient of each imag…
Minimum width for ReLU networks on compact domain is exactly max{d_x, d_y, 2}
problem Characterizing the minimum width for ReLU networks to approximate functions on compact domains
method Analyzing the minimum width for Lp approximation of Lp functions from [0,1]d to Rdy using ReLU-like activation functions result The minimum width for Lp approximation on a compact domain is exactly max{d_x, d_y, 2} for ReLU-like activation functions New network approximates functions with error decreasing with network width and depth.
problem Approximating functions with high-dimensional data.
method Floor-ReLU networks with specific width and depth.
result Approximation error decreases as network width and depth increase.
An electrocardiogram (ECG) is a time-series signal that is represented by one-dimensional (1-D) data. Higher dimensional representation contains more information that is accessible for feature extraction. Hidden variables such as frequency relation and morphology of segment is not directly accessible in the time domain…
Shallow neural networks can represent polynomials efficiently.
problem Representing polynomials using shallow neural networks.
method Using shallow neural networks of width 2(R+d)d to represent d-variate polynomials of degree R. result Derives minimax optimal convergence rate for shallow networks to unknown univariate regression functions.
Deep learning methods are increasingly being used with neuroimaging data like structural and function magnetic resonance imaging (MRI) to predict the diagnosis of neuropsychiatric and neurological disorders. For psychiatric disorders in particular, it is believed that one of the most promising modality is the resting-s…
BPN defends against adversarial attacks by generating beneficial perturbations.
problem Adversarial attacks cause deep neural networks to misclassify clean inputs.
method BPN generates beneficial perturbations during training to neutralize future adversarial attacks.
result BPN is robust to adversarial examples and more efficient than classical adversarial training.
In recent years, neural networks have demonstrated outstanding effectiveness in a large amount of applications.However, recent works have shown that neural networks are susceptible to adversarial examples, indicating possible flaws intrinsic to the network structures. To address this problem and improve the robustness …
Ideally, what confuses neural network should be confusing to humans. However, recent experiments have shown that small, imperceptible perturbations can change the network prediction. To address this gap in perception, we propose a novel approach for learning robust classifier. Our main idea is: adversarial examples for…
Learning to approximate a separable function is hard, requiring many samples even with sparse networks.
problem Learning the separable function x↦∑i=1dxi2 with limited samples. method Sparse neural networks vs. dense neural networks, explicit regularization.
result The sample complexity for dense networks is O(d2.5) with explicit regularization, better than O(d4). We present Generative Adversarial Capsule Network (CapsuleGAN), a framework that uses capsule networks (CapsNets) instead of the standard convolutional neural networks (CNNs) as discriminators within the generative adversarial network (GAN) setting, while modeling image data. We provide guidelines for designing CapsNet…
New method beats volumetric barrier for manifold recovery.
problem Reconstructing latent geometry from noisy distances.
method Orthogonal Ring Distance Estimation Routine (ORDER).
result Achieves pointwise distance estimation of order n−2/(d+5). New framework improves efficiency in low-rank matrix bandit problems.
problem Stochastic contextual low-rank matrix bandit problem with unknown rank matrices.
method G-ESTT and G-ESTS frameworks using Stein's method and regularization.
result Achieved improved regret bounds for low-rank matrix bandit problems.
GNNs outperform NNs in interpolating bandlimited functions on Euclidean cubes.
problem Interpolating bandlimited functions on Euclidean cubes using GNNs vs. NNs.
method Investigates optimal GNN configurations and weights for function interpolation.
result GNNs require fewer weights and samples to interpolate bandlimited functions compared to NNs.
The study reveals how adversarial perturbations can include class features for generalization.
problem Understanding why adversarial examples deceive neural networks and transfer between networks.
method A one-hidden-layer network trained on mutually orthogonal samples.
result Adversarial perturbations, even of a few pixels, contain sufficient class features for generalization.
Paper provides robustness bounds for GNNs against adversarial attacks.
problem Adversarial robustness of GNNs for graph-related tasks.
method PAC-Bayesian framework applied to GCN and MP-GNN.
result Spectral norms of diffusion matrix and weights govern robustness.
The study finds rational points on specific types of hypersurfaces.
problem Identifying rational points on generic marked hypersurfaces.
method Analyzing hypersurfaces in projective spaces over various fields.
result Conditions for the existence of rational points on hypersurfaces.
SGD-trained neural networks generalize well even with adversarial label noise.
problem Generalization of neural networks trained on adversarial label noise.
method Training a one-hidden-layer neural network with SGD on arbitrary width networks.
result SGD-trained networks achieve classification accuracy competitive with the best halfspace over adversarial label noise.
One popular hypothesis of neural network generalization is that the flat local minima of loss surface in parameter space leads to good generalization. However, we demonstrate that loss surface in parameter space has no obvious relationship with generalization, especially under adversarial settings. Through visualizing …
New jet functors generalize classical notions in noncommutative geometry.
problem Defining and understanding jet functors in noncommutative settings.
method Constructing and proving properties of jet functors Jd(n), Jd[n], and Jdn. result Holonomic jet functor Jdn satisfies jet exact sequence under specific conditions. New method creates powerful, transferable adversarial attacks for deep learning networks.
problem Vulnerability of deep learning networks to adversarial attacks.
method Developed a method to generate strong, transferable adversarial attacks.
result Demonstrated that the proposed method can easily transfer strong adversarial attacks between different models.
Study reveals how neural network smoothness affects their vulnerability to adversarial attacks.
problem Understanding adversarial vulnerability in deep learning networks.
method Analysis of manifold smoothness and generalization capability of deep neural networks trained with local errors.
result High generalization accuracy requires a fast power-law decay of eigen-spectrum of hidden representations.
Quantum GAN improves volatility modeling in finance.
problem Improving volatility modeling in finance using GANs.
method Developed a quantum GAN for volatility modeling.
result Quantum GAN provides exponential advantage over classical methods.
Lazy neural networks are vulnerable to adversarial attacks.
problem Adversarial robustness issues in lazy training models.
method Extending recent work on adversarial examples to lazy training models.
result Over-parametrized neural networks that generalize well remain vulnerable to single-step gradient ascent attacks.
Deep neural networks with adversarial training achieve sup-norm convergence for nonparametric regression.
problem Achieving sup-norm convergence for deep neural network estimators in nonparametric regression.
method Developed an adversarial training scheme to address the sup-norm convergence issue.
result Deep neural network estimators achieve optimal sup-norm convergence with the proposed adversarial training.