Adds layers to NNs to protect them from reverse engineering.
problem Extracting the underlying model of a Neural Network.
method Introducing parasitic layers that approximate a noisy identity mapping with a Convolutional NN.
result The protected NN's predictions remain mostly unchanged while making reverse-engineering more complex.
New invertible transformations improve flow-based generative models.
problem Improving flow-based generative models for better performance.
method Proposed new invertible transformations and coupling layers.
result New coupling layers achieve better results in IDF.
RevDEQs improve performance on tasks with exact gradients and fewer function evaluations.
problem Inexact gradient calculation in DEQs leads to unstable training and requires regularisation or many function evaluations.
method Introduce Reversible Deep Equilibrium Models (RevDEQs) that allow for exact gradient calculation, no regularisation, and far fewer function evaluations.
result RevDEQs significantly improve performance on language modelling and image classification tasks.
Reformer improves Transformer efficiency for long sequences.
problem Prohibitively costly training of large Transformer models.
method Locality-sensitive hashing for attention and reversible residual layers.
result Memory-efficient and faster performance on long sequences.
PETRA enables parallel training of deep models with reversible architectures.
problem Challenges in parallelizing deep model training.
method Introduces PETRA, a novel approach for parallelizing gradient computations in reversible architectures.
result Achieves competitive accuracies on CIFAR-10, ImageNet32, and ImageNet using ResNet models.
Recently, deep residual networks have been successfully applied in many computer vision and natural language processing tasks, pushing the state-of-the-art performance with deeper and wider architectures. In this work, we interpret deep residual networks as ordinary differential equations (ODEs), which have long been s…
We describe Bayesian Layers, a module designed for fast experimentation with neural network uncertainty. It extends neural network libraries with drop-in replacements for common layers. This enables composition via a unified abstraction over deterministic and stochastic functions and allows for scalability via the unde…
A new perspective on self-attention models using MLPs.
problem Improving sequence modeling with self-attention mechanisms.
method Introducing HyperMLP and HyperGLU, which use dynamic two-layer MLPs with reverse-offset layout.
result HyperMLP/HyperGLU consistently outperform softmax-attention baselines.
Flood extent mapping plays a crucial role in disaster management and national water forecasting. Unfortunately, traditional classification methods are often hampered by the existence of noise, obstacles and heterogeneity in spectral features as well as implicit anisotropic spatial dependency across class labels. In thi…
Memory-efficient learning for large-scale imaging systems.
problem Memory limitations in GPUs for real-world large-scale inverse problems.
method Exploits reversibility of network layers to enable data-driven design.
result Demonstrated on small-scale and large-scale real-world systems.
Self-supervised and supervised methods learn similar intermediate visual representations but diverge in final layers.
problem Comparing self-supervised and supervised methods for visual learning.
method Comparison of contrastive self-supervised and supervised methods on simple image data.
result Contrastive and supervised methods learn similar intermediate representations but diverge in final layers.
We analyse a multiplex of networks between OECD countries during the decade 2002-2010, which consists of five financial layers, given by foreign direct investment, equity securities, short-term, long-term and total debt securities, and five environmental layers, given by emissions of N O x, P M 10 SO 2, CO 2 equivalent…
Adversarial domain adaptation reduces sample bias in high energy physics classifier.
problem Sample bias in high energy physics classifier training.
method Adversarial domain adaptation using neural networks with gradient reversal layer.
result Successful bias removal on simulated events at the LHC.
A new method for image translation using disentangled style and content preservation.
problem Difficulty in maintaining original content during reverse diffusion in diffusion-based image translation.
method Disentangled style and content representation using intermediate keys from ViT model, CLIP loss, semantic divergence loss, and resampling strategy.
result Outperforms state-of-the-art models in text-guided and image-guided translation tasks.
Classifies reversible and strongly reversible elements in quaternionic groups.
problem Classifying reversible and strongly reversible elements in quaternionic groups.
method Proves elements are reversible if and only if they are products of skew-involutions (resp. involutions).
result Proves elements are reversible if and only if they are products of skew-involutions (resp. involutions).
The paper classifies reversible and strongly reversible elements in Hermitian isometry groups.
problem Classifying reversible and strongly reversible elements in Hermitian isometry groups.
method Classification through group theory and algebraic manipulation.
result New classification of strongly reversible elements in Sp(n).
Modeling brain dynamics to better understand and control complex behaviors underlying various cognitive brain functions are of interests to engineers, mathematicians, and physicists from the last several decades. With a motivation of developing computationally efficient models of brain dynamics to use in designing cont…
This paper classifies reversible and strongly reversible elements in affine groups.
problem Classifying reversible and strongly reversible elements in affine groups.
method Identifying affine transformations and using conjugacy by involutions.
result Classification of reversible and strongly reversible elements in affine groups.
New neural net learns time-reversible symplectic dynamics.
problem Lack of time-reversibility in neural networks for symplectic systems.
method Proposes a new neural network architecture for time-reversible symplectic systems.
result Demonstrates learning of time-reversible symplectic dynamics from data.
A new trading strategy using reinforcement learning for statistical arbitrage.
problem Traditional statistical arbitrage models rely on model assumptions and price deviations from a long-term mean.
method Empirical reversion time metric, reinforcement learning framework, and state space optimization.
result Optimal mean reversion strategy identified through reinforcement learning.
Algebraic method reveals criterion for quaternionic Möbius group reversibility.
problem Characterizing reversibility in quaternionic Möbius group elements.
method Purely algebraic approach using matrix entries and conjugacy invariants.
result Explicit criterion for reversibility in terms of matrix entries.
Recent studies have shown that online portfolio selection strategies that exploit the mean reversion property can achieve excess return from equity markets. This paper empirically investigates the performance of state-of-the-art mean reversion strategies on real market data. The aims of the study are twofold. The first…
A Finsler space is said to be geodesically reversible if each oriented geodesic can be reparametrized as a geodesic with the reverse orientation. A reversible Finsler space is geodesically reversible, but the converse need not be true. In this note, building on recent work of LeBrun and Mason, it is shown that a geodes…
Sharp stability results for reverse isoperimetric inequalities in 2D.
problem Reverse isoperimetric inequalities in the plane.
method Stability analysis of λ-convex bodies and convex bodies with smooth boundaries. result Sharp stability results for reverse isoperimetric inequalities, including inradius and Cheeger inequalities.
Let G be a group. An element g in G is called reversible if it is conjugate to g−1 within G, and called strongly reversible if it is conjugate to its inverse by an order two element of G. Let HHn be the n-dimensional quaternionic hyperbolic space. Let PSp(n,1) be the i…
On-line portfolio selection has attracted increasing interests in machine learning and AI communities recently. Empirical evidences show that stock's high and low prices are temporary and stock price relatives are likely to follow the mean reversion phenomenon. While the existing mean reversion strategies are shown to …
Boundary rigidity proven for non-reversible Finsler metrics.
problem Recovering non-reversible Finsler metrics from boundary distance data.
method Sum of reversible Finsler norm and closed 1-form, boundary rigidity results.
result 1-form can be uniquely recovered from boundary distance data.
We propose a novel approach for semantic segmentation that uses an encoder in the reverse direction to decode. Many semantic segmentation networks adopt a feedforward encoder-decoder architecture. Typically, an input is first downsampled by the encoder to extract high-level semantic features and continues to be fed for…
Conditional diffusion models can approximate target distributions well with Gaussian-mixture reverse kernels.
problem Approximating target distributions in conditional diffusion models.
method Using finite Gaussian mixtures with ReLU-network logits as reverse kernels, reducing the problem to static conditional density approximation.
result The resulting neural reverse-kernel class is dense in conditional KL divergence under exact terminal matching.
New knots not rationally concordant to their reverses found.
problem Identifying knots not rationally concordant to their reverses.
method Infinite family of knots constructed, rational knot concordance group analyzed.
result Infinite rank subgroup in rational knot concordance group.
The paper classifies reversible elements in Seifert-fibered spaces and braid groups.
problem Classifying reversible elements in Seifert-fibered spaces and braid groups.
method Classification of reversible elements in Fuchsian groups, application to Seifert-fibered groups, and analysis of 3-torsion elements.
result Classification and analysis of reversible and 3-torsion elements in Seifert-fibered spaces and braid groups.
Reverse annealing boosts quantum matrix factorization performance.
problem Improving quantum matrix factorization performance.
method Combining forward and reverse annealing for nonnegative/binary matrix factorization.
result Combination of forward and reverse annealing significantly improves performance.
TRS-ODENs learn dynamics with time-reversal symmetry for more efficient learning.
problem Learning dynamics with time-reversal symmetry for more efficient learning.
method Proposed a loss function and a new framework (TRS-ODENs) to learn dynamics efficiently.
result TRS-ODENs can learn dynamics from noisy and complex trajectories efficiently.
Paper proves rigidity theorems for geodesically reversible Finsler metrics.
problem Understanding geodesically reversible Finsler metrics in closed manifolds.
method Applied theory of volumes and areas on Finsler spaces to establish rigidity theorems.
result Partial explanation of the scarcity of geodesically reversible Finsler metrics in closed manifolds.
Unified understanding of neural networks on group operations verified.
problem Understanding and verifying neural networks trained on group operations.
method Investigated one-hidden-layer neural networks trained on binary operation of finite groups, revealing structure and providing a compact proof of model performance.
result Verified explanation applies to a large fraction of networks trained on the symmetric group S5, providing a >=95% accuracy bound for 45% of models.
Reduces identity testing of reversible Markov chains to simpler symmetric chain tests.
problem Testing identity of reversible Markov chains from a single trajectory.
method Using lumping-congruent Markov embeddings, the problem is simplified to testing symmetric chains over a larger state space.
result Achieves state-of-the-art sample complexity for identity testing.
Proof of reverse isoperimetric inequality for black holes.
problem Reverse isoperimetric inequality for black holes in Einstein gravity.
method Geometric-analytic approach.
result Reversal of the usual isoperimetric inequality is explained by curved backgrounds governed by Einstein's equations.
New algorithm learns bridged diffusion processes without time-reversals.
problem Learning bridged diffusion processes efficiently and accurately.
method Score matching with Doob's h-transform, avoiding time-reversals.
result Outperforms existing methods in learning bridged diffusion processes.
Consider the problem of pricing options on forwards in energy markets, when spot prices follow a geometric multi-factor model in which several rates of mean reversion appear. In this paper we investigate the role played by slow mean reversion when pricing and hedging options. In particular, we determine both upper and …
Characterizes isometries between non-reversible Finsler manifolds.
problem Understanding isometries in non-reversible Finsler manifolds.
method Generalization of Myers-Nakai Theorem for Riemannian manifolds, modification of function spaces to accommodate asymmetric structure.
result Functional characterization of isometries between non-reversible Finsler manifolds.
New method trains neural samplers to sample from multi-modal distributions efficiently.
problem Mode-seeking behavior of reverse KL divergence hinders effective sampling from multi-modal target distributions.
method Minimizing reverse diffusive KL divergence along diffusion trajectories of model and target densities.
result Demonstrated enhanced sampling performance across various multi-modal distributions.
State-of-the-art machine learning models frequently misclassify inputs that have been perturbed in an adversarial manner. Adversarial perturbations generated for a given input and a specific classifier often seem to be effective on other inputs and even different classifiers. In other words, adversarial perturbations s…
New ODE solvers improve training efficiency and accuracy.
problem Training Neural ODEs requires efficient and accurate gradient calculation.
method Presented algebraically reversible ODE solvers that are time and memory efficient, calculate exact gradients, and are numerically stable.
result Reversible solvers strictly improve upon previous architectures in efficiency and accuracy.
This paper describes an improvement in Deep Q-learning called Reverse Experience Replay (also RER) that solves the problem of sparse rewards and helps to deal with reward maximizing tasks by sampling transitions successively in reverse order. On tasks with enough experience for training and enough Experience Replay mem…
Classifies orientation-reversing homeomorphisms of even periods on surfaces.
problem Classifying orientation-reversing homeomorphisms of even periods on surfaces.
method Following the approach of [1] and correcting errors in [1] for the case of periods multiple of 4.
result Classification for orientation-reversing homeomorphisms of periods multiple of 4.
A new sampler speeds up Bayesian mixture models.
problem Sampling from Bayesian finite mixture models is slow and hard.
method Introduces a non-reversible sampling scheme for Bayesian finite mixture models.
result The new sampler outperforms classical samplers in many scenarios, especially during convergence.
We continue our study of geometric analysis on (possibly non-reversible) Finsler manifolds, based on the Bochner inequality established by the author and Sturm. Following the approach of the Γ-calculus a la Bakry et al, we show the dimensional versions of the Poincare--Lichnerowicz inequality, the logarithmic Sobolev…
A quandle orbit's orientation is problematic when reversed.
problem The natural orientation-reversal of quandle orbits is unsuitable for medial quandles.
method Defined the orientation-reversal of a quandle orbit by inverting translations, observed it's unsuitable for medial quandles.
result The natural orientation-reversal of quandle orbits is unsuitable for medial quandles.