DFRot improves LLMs by reducing outlier and massive activation effects.
problem Reducing outlier and massive activation effects in rotated LLMs.
method Weighted loss function and orthogonal Procrustes transforms for rotation matrix refinement.
result DFRot achieves dual free (Outlier-Free and Massive Activation-Free) with significant improvements in perplexity.
New method certifies images against transformations like rotations and translations.
problem Certifying robustness of images against transformations like rotations and translations.
method Randomized smoothing with three different kinds of defenses.
result Individual certificates can be obtained via statistical error bounds or efficient online inverse computation.
DeformRS certifies deep networks against various input deformations.
problem Vulnerability of deep networks to input deformations.
method Randomized smoothing reformulation for general deformations.
result Certifies rich deformations including translations, rotations, scaling, and affine.
Federated Learning enables mobile devices to collaboratively learn a shared inference model while keeping all the training data on a user's device, decoupling the ability to do machine learning from the need to store the data in the cloud. Existing work on federated learning with limited communication demonstrates how …
This note gives a simple analysis of a randomized approximation scheme for matrix multiplication proposed by Sarlos (2006) based on a random rotation followed by uniform column sampling. The result follows from a matrix version of Bernstein's inequality and a tail inequality for quadratic forms in subgaussian random ve…
Study optimizes estimation of orthogonal and rotation matrices from noisy data.
problem Estimating orthogonal and rotation matrices from noisy data.
method Iterative polar decomposition algorithm initialized by spectral methods.
result Algorithm achieves optimal error rate of $(1+o(1)) rac{σ^2 d(d-1)}{2np}$.
RENNs protect input privacy by rotating d-ary features.
problem Protecting input privacy from intermediate-layer features.
method Rotation-equivariant neural networks using d-ary vectors/tensors.
result RENNs effectively hide input information without degrading output accuracy.
In short, our experiments suggest that yes, on average, rotation forest is better than the most common alternatives when all the attributes are real-valued. Rotation forest is a tree based ensemble that performs transforms on subsets of attributes prior to constructing each tree. We present an empirical comparison of c…
Exact optimality achieved in distributed mean estimation with shared randomness.
problem Achieving optimal communication, privacy, and utility tradeoffs in distributed mean estimation.
method Utilization of a rotationally symmetric shared random codebook and a k-closest encoding mechanism. result Proposed mechanism achieves exact optimality for randomly rotated simplex codebook.
FibQuant improves KV-cache compression for long-context inference.
problem Memory traffic bottleneck in long-context inference due to KV cache growth.
method Introduces FibQuant, a universal vector quantizer that combines Beta-quantile radii, Fibonacci/Roberts-Kronecker directions, and Lloyd-Max refinement.
result FibQuant achieves high compression rates with minimal loss in attention cosine similarity.
The paper analyzes why Gaussianization slows down with higher dimensions and proposes a solution.
problem The convergence rate of Gaussianization slows down as the dimension increases.
method Analytical and empirical analysis of Gaussianization with random rotations.
result The number of required layers scales linearly with the dimension for Gaussian input.
We consider probabilistic PCA and related factor models from a Bayesian perspective. These models are in general not identifiable as the likelihood has a rotational symmetry. This gives rise to complicated posterior distributions with continuous subspaces of equal density and thus hinders efficiency of inference as wel…
Improved SRHT for linear SVM classification with higher accuracy.
problem Inefficient random projection methods for high-dimensional data.
method Importance sampling and deterministic top-r sampling for effective low-dimensional embedding. result Higher classification accuracy on real-life datasets.
In this work, a method of random parameters generation for randomized learning of a single-hidden-layer feedforward neural network is proposed. The method firstly, randomly selects the slope angles of the hidden neurons activation functions from an interval adjusted to the target function, then randomly rotates the act…
We propose a principled method for kernel learning, which relies on a Fourier-analytic characterization of translation-invariant or rotation-invariant kernels. Our method produces a sequence of feature maps, iteratively refining the SVM margin. We provide rigorous guarantees for optimality and generalization, interpret…
We demonstrate that, for a range of state-of-the-art machine learning algorithms, the differences in generalisation performance obtained using default parameter settings and using parameters tuned via cross-validation can be similar in magnitude to the differences in performance observed between state-of-the-art and un…
Most signal processing problems involve the challenging task of multidimensional probability density function (PDF) estimation. In this work, we propose a solution to this problem by using a family of Rotation-based Iterative Gaussianization (RBIG) transforms. The general framework consists of the sequential applicatio…
Study connects covariance cleaning theory to information theory for heavy-tailed distributions.
problem Optimizing covariance matrices for heavy-tailed distributions using information theory.
method Minimizing Frobenius norm and information loss between true and estimated covariance matrices.
result Asymptotic regime of large matrices minimizes information loss for Student's t distributions.
This chapter introduces quaternion machine learning for 3D rotations.
problem Lack of quaternion machine learning for 3D rotations.
method Augmented statistics, widely linear models, quaternion calculus, mean square estimation.
result Foundation for quaternion machine learning.
Expanding self-supervised learning to diverse domains reveals Rotation's semantic superiority.
problem Limited self-supervised learning experiments on diverse domains.
method Experimented on various domains (satellite, textural, biological) using popular self-supervised methods.
result Rotation task is semantically most meaningful, with other tasks relying on distribution rather than semantic understanding.
New TSER algorithms outperform existing methods in time series extrinsic regression.
problem Improving time series extrinsic regression models.
method Extended TSER archive, introduced two new algorithms (FreshPRINCE and DrCIF), compared with rotation forest.
result DrCIF and FreshPRINCE models significantly outperform existing methods.
Study on rotating surfaces in 4D space with matrices.
problem Understanding rotational surfaces in pseudo-Euclidean 4-space.
method Defined hyperbolic and elliptic rotational surfaces using curves and matrices in 4D semi-Euclidean space.
result Generated rotated surfaces using specific rotation matrices.
A new transform links rotating calorons to solutions of a differential equation.
problem Existence and characterization of rotating calorons.
method Formulated a Nahm transform to relate rotating calorons to solutions of a delayed-differential equation.
result Existence of an eight-parameter family of rotating calorons with nontrivial holonomy.
We define general rotational surfaces of elliptic and hyperbolic type in the pseudo-Euclidean 4-space with neutral metric which are analogous to the general rotational surfaces of C. Moore in the Euclidean 4-space. We study Lorentz general rotational surfaces with plane meridian curves and give the complete classificat…
The paper develops a new algorithm for RBMs using dynamical mean-field theory.
problem Learning in Restricted Boltzmann Machines (RBMs) with complex dependencies.
method Dynamical mean-field theory applied to RBMs with rectangular coupling matrices drawn from a bi-rotation invariant ensemble.
result The algorithm converges globally under a stability criterion, with rates matching numerical simulations.
The paper defines and analyzes homotopic rotation sets for surfaces of higher genus.
problem Defining and analyzing homotopic rotation sets for surfaces of higher genus.
method Developed a definition and proved several results using the theory of Le Calvez and Tal.
result Found that the homotopic rotation set can imply the existence of infinitely many periodic orbits under certain conditions.
Whitening, or sphering, is a common preprocessing step in statistical analysis to transform random variables to orthogonality. However, due to rotational freedom there are infinitely many possible whitening procedures. Consequently, there is a diverse range of sphering methods in use, for example based on principal com…
Study of timelike surfaces in Minkowski space with specific geometric properties.
problem Characterizing geometric properties of timelike surfaces in Minkowski space.
method Analytical study of two types of timelike general rotational surfaces.
result Explicit descriptions of minimal and surfaces with specific curvature properties.
The study characterizes loxodromes on specific rotational surfaces in 3D space.
problem Characterizing loxodromes on rotational surfaces with special geometric properties.
method Parametrizations and curvature/torsion calculations for loxodromes on various rotational surfaces.
result The loxodrome on a flat rotational surface is a general helix.
General rotational surfaces as a source of examples of surfaces in the four-dimensional Euclidean space have been introduced by C. Moore. In this paper we consider the analogue of these surfaces in the Minkowski 4-space. On the base of our invariant theory of spacelike surfaces we study general rotational surfaces with…
This paper explores the trade-off between spatial and adversarial robustness in neural networks.
problem Understanding the trade-off between spatial and adversarial robustness in neural networks.
method Quantitative analysis and empirical testing with curriculum learning.
result Spatial robustness and adversarial robustness are quantitatively related and can be improved simultaneously.
Rotation systems can't always be drawn in surfaces.
problem Rotation systems and simple drawings in surfaces.
method Extended the plane result to all fixed surfaces.
result Existence of rotation systems not arising from simple drawings in any fixed surface.
Invariance to nuisance transformations is one of the desirable properties of effective representations. We consider transformations that form a \emph{group} and propose an approach based on kernel methods to derive local group invariant representations. Locality is achieved by defining a suitable probability distributi…
This paper proposes a set of rules to revise various neural networks for 3D point cloud processing to rotation-equivariant quaternion neural networks (REQNNs). We find that when a neural network uses quaternion features under certain conditions, the network feature naturally has the rotation-equivariance property. Rota…
RotEqNet preserves rotation symmetry in fluid systems using high-order tensors.
problem Lack of rotational symmetry in machine learning models for fluid systems.
method Introduces RotEqNet, a network that guarantees rotation-equivariance for high-order tensors.
result RotEqNet reduces errors and maintains rotation-equivariance in fluid systems.
The rotation prediction (Rotation) is a simple pretext-task for self-supervised learning (SSL), where models learn useful representations for target vision tasks by solving pretext-tasks. Although Rotation captures information of object shapes, it hardly captures information of textures. To tackle this problem, we intr…
The paper applies Clairaut's theorem to rotational surfaces in pseudo-Euclidean 4-space.
problem Exploring geodesic curves on rotational surfaces in pseudo-Euclidean 4-space.
method Expressing Clairaut's theorem and deriving equations for geodesic curves.
result Characterization of geodesic curves on hyperbolic and elliptic surfaces of rotation.
Convolutional networks are successful due to their equivariance/invariance under translations. However, rotatable data such as images, volumes, shapes, or point clouds require processing with equivariance/invariance under rotations in cases where the rotational orientation of the coordinate system does not affect the m…
We consider n-dimensional discrete motions such that any two neighbouring positions correspond in a pure rotation ("rotating motions"). In the Study quadric model of Euclidean displacements these motions correspond to quadrilateral nets with edges contained in the Study quadric ("rotation nets"). The main focus of ou…
The study examines lower and upper bounds of Wasserstein distances for affine transformations of random vectors.
problem Understanding Wasserstein distances for affine transformations of random vectors.
method Lower and upper bounds for affine transformations of random vectors in Rn are derived using Bures metric and compositions of affine maps. result Concrete lower bounds and upper bounds for affine transformations are derived and applied to various distributions.
Minimal sets of moves for rotational Reidemeister diagrams are identified.
problem Understanding the minimal sets of moves for rotational Reidemeister diagrams.
method Detailed description and proof of minimal generating sets for rotational Reidemeister moves.
result Minimal generating sets for oriented, framed links contain 5 moves.
A new method for analyzing shapes using FDA techniques.
problem Statistical shape analysis of deformed contours.
method Functional Data Analysis (FDA) with basis expansion and principal component analysis.
result Successfully identifies deformation parameters and captures contour distributions.
New method studies moving points on curves using rotating frames.
problem Understanding the motion of points on curves.
method Constructing rotating frames for curves and analyzing the motion of points within these frames.
result A new binary mathematical formation mechanism for curves based on linear and rotational motion.
Study on rotational hypersurfaces with constant Gauss-Kronecker curvature.
problem Exploring hypersurfaces with constant Gauss-Kronecker curvature.
method Solving ODE for generating curves and analyzing geometric properties.
result Discovery of non-compact rotational hypersurfaces with negative Gauss-Kronecker curvature and finite volume.
In-plane drill rotations are impossible for smooth shells.
problem In-plane drill rotations on smooth shells are impossible.
method Analyzing the differential geometry of surfaces and isometries.
result Any isometry that coincides with the given surface at a portion of the boundary is the identity.
Positive factorization found for a specific map on surfaces.
problem Balanced superelliptic rotation on surfaces.
method Positive factorization approach.
result Positive factorization for balanced superelliptic rotation.
Study of rotation angles in a rotating disc model.
problem Understanding geometric phase in rotating systems.
method Analyzes a simple kinematic model of rotating discs.
result Explicit form of geometric phase Δg found using Baumkuchen lemma. The study disproves rotating ancient flows in 4D space.
problem The existence of rotating ancient flows in R4. method Analysis of ancient noncollapsed flows in R4. result Nonexistence of rotating ancient flows among ancient noncollapsed flows in R4.