The paper studies geometric properties of group equivariant operators and their Riemannian structure.
problem Understanding the geometric structure of group equivariant operators.
method Endowing the space of group equivariant non-expansive operators with a Riemannian manifold structure and using gradient descent methods.
result Gradient descent methods can be applied to minimize cost functions on the space of group equivariant non-expansive operators.
In this paper known results of symmetric orthogonality, as introduced by G. Birkhoff, and non-expansive nearest point projections are extended from the linear to the metric setting. If the space has non-positive curvature in the sense Busemann then it is shown that those concepts are actually equivalent. In the end it …
New analysis of stochastic approximation with non-expansive mappings.
problem Finite-time analysis of two-time-scale stochastic approximation with non-expansive mappings.
method Studied two-time-scale stochastic approximation algorithms with non-expansive mappings and projection steps.
result Last-iterate mean square residual error decays at a rate O(1/k1/4−ε). The aim of this paper is to provide a general mathematical framework for group equivariance in the machine learning context. The framework builds on a synergy between persistent homology and the theory of group actions. We define group-equivariant non-expansive operators (GENEOs), which are maps between function spaces…
Paper defines mathematical framework for neural network explainability.
problem Neural network explainability and equivariant operators.
method Mathematical framework based on Group Equivariant Non-Expansive Operators (GENEOs) and complexity measures.
result Formal properties and interpretability of Group Equivariant Operators (GEOs) defined.
New method recovers signals from compressed measurements using generative networks with contractive layers.
problem Signal recovery from compressed measurements with generative network priors.
method Developed a new matrix concentration inequality (R2WDC) to relax expansivity conditions for generative networks.
result Signals in the range of a Gaussian generative network can be recovered from few linear measurements with contractive layers.
Improved stochastic approximation method reduces residual error.
problem Reducing residual error in stochastic approximation algorithms.
method Fixed-schedule one-quarter barrier and bias-corrected acceleration.
result Achieves T−1/2+o(1) residual reduction with O(1) primitive samples. A single-vertex origami is a piece of paper with straight-line rays called creases emanating from a fold vertex placed in its interior or on its boundary. The Single-Vertex Origami Flattening problem asks whether it is always possible to reconfigure the creased paper from any configuration compatible with the metric, t…
We examine overlapping clustering schemes with functorial constraints, in the spirit of Carlsson--Memoli. This avoids issues arising from the chaining required by partition-based methods. Our principal result shows that any clustering functor is naturally constrained to refine single-linkage clusters and be refined by …
Convergence of the Kalman filter is best analyzed by studying the contraction of the Riccati map in the space of positive definite (covariance) matrices. In this paper, we explore how this contraction property relates to a more fundamental non-expansiveness property of filtering maps in the space of probability distrib…
Maximal concentration bounds for stochastic approximation with heavy-tailed noise.
problem Analyzing the convergence of stochastic approximation algorithms under heavy-tailed Markovian noise.
method Novel Lyapunov function and black-box truncation argument.
result Tail behavior of the error can be sub-Gaussian, sub-Weibull, or lighter than any Pareto but heavier than any Weibull.
Abstracts a theorem for non-smooth maps in infinite dimensions.
problem Generalizing inverse mapping theorem for non-smooth maps.
method Introduces property A and applies it to non-smooth maps.
result Generalized inverse mapping theorems for non-smooth maps.
Value function estimation is an important task in reinforcement learning, i.e., prediction. The Boltzmann softmax operator is a natural value estimator and can provide several benefits. However, it does not satisfy the non-expansion property, and its direct use may fail to converge even in value iteration. In this pape…
Deep networks are commonly used to model dynamical systems, predicting how the state of a system will evolve over time (either autonomously or in response to control inputs). Despite the predictive power of these systems, it has been difficult to make formal claims about the basic properties of the learned systems. In …
This work uses sampling theory to analyze smoothness and error bounds of finite neural networks.
problem Analyzing the function space of finite neural networks and providing error bounds.
method Applying sampling theory to finite neural networks with non-expansive activation functions, considering both deterministic and random sampling.
result Novel error bounds for univariate neural networks under band-limited input assumption, highlighting the advantage of deterministic uniform sampling.
Proposes a topological model for partial equivariance in neural networks.
problem Capturing partial equivariance in neural networks for data analysis.
method Introduces P-GENEOs and studies spaces of measurements and P-GENEOs between them.
result Spaces of measurements and P-GENEOs have convenient approximation and convexity properties.
A softmax operator applied to a set of values acts somewhat like the maximization function and somewhat like an average. In sequential decision making, softmax is often used in settings where it is necessary to maximize utility but also to hedge against problems that arise from putting all of one's weight behind a sing…
Framework for designing nonlinearities in neural networks with slope constraints.
problem Designing nonlinearities with specific properties for signal processing.
method Variational framework with regularization for slope constraints and optimization of adaptive splines.
result Adaptive nonuniform linear splines achieve global optimum in constrained optimization.
SCENE-Net improves 3D point cloud segmentation with low resource usage and transparency.
problem Lack of resources and transparency in 3D semantic segmentation models.
method SCENE-Net uses signature shapes identified via GENEOs to achieve semantic segmentation with minimal resources.
result SCENE-Net achieves comparable IoU to state-of-the-art methods with less data and computational resources.
Neural network quantization is becoming an industry standard to efficiently deploy deep learning models on hardware platforms, such as CPU, GPU, TPU, and FPGAs. However, we observe that the conventional quantization approaches are vulnerable to adversarial attacks. This paper aims to raise people's awareness about the …
Novel algorithm accelerates PnP methods for image deblurring and super-resolution.
problem Efficiently solving inverse problems and imaging with provable convergence guarantees.
method Incorporates quasi-Newton steps into provable PnP framework based on proximal denoisers.
result 2--8x faster convergence compared to other provable PnP methods with similar quality.
CCDF reduces diffusion sampling steps for inverse problems.
problem Slow sampling from diffusion models in inverse problems.
method Starting from a single forward diffusion step with better initialization, followed by stochastic contraction.
result Significantly reduced sampling steps for state-of-the-art reconstruction.
NAST generalizes scattering transform for non-stationary time series analysis.
problem Analyzing non-stationary time series data.
method Neural activation of scattering transform with various activation functions and high pass filters.
result Central and non-central limit theorems for NAST of Gaussian processes.
This work extends GNNs to handle multiple graphs with non-commuting operators, proving transferability.
problem Handling multiple graphs with non-commuting operators in graph neural networks.
method Developed a mathematical theory for graph-tuple neural networks (GtNNs) with non-commuting non-expansive operators.
result Proved universal transferability of GtNNs, ensuring no non-transferable energy under convergence.
We develop algorithms to learn non-linear dynamical systems without mixing assumptions.
problem Learning non-linear dynamical systems from dependent data.
method We introduce an offline algorithm and a one-pass streaming method with SGD-RER.
result Our methods achieve optimal or near-optimal performance for learning non-linear systems.