Equivalent tests for SGD batch size selection found.
problem Finding equivalent tests for adaptive batch size selection in SGD.
method Norm and inner product/orthogonality tests equivalence demonstration.
result Norm and inner product/orthogonality tests are equivalent under specific conditions.
Norm-ranging LSH improves MIPS performance by addressing 2-norm distribution issues.
problem Long tails in 2-norm distribution of real datasets affect Simple-LSH performance.
method Norm-ranging LSH partitions datasets into sub-datasets and builds independent hash indexes.
result Norm-ranging LSH achieves an order of magnitude speedup over Simple-LSH.
Norm-range partition improves MIPS search efficiency by reducing query complexity.
problem Efficiently searching for maximum inner product in large datasets.
method Norm-range partition technique that divides datasets into sub-datasets with similar norms and builds independent hash indexes.
result Significantly reduces the number of probed buckets for LSH-based MIPS algorithms.
Estimates inner products between nonparametric distributions using Fourier basis.
problem Estimating inner products between two nonparametric distributions.
method Proposes estimators for inner products and induced norms, proves mean squared error bounds and minimax lower bounds.
result Proposed estimators are rate-optimal over Fourier ellipsoids.
Convex learning for diverse invariances in semi-inner-product space.
problem Efficiently learning invariant representations for a wide range of invariances.
method Developed a convex representation learning algorithm for generalized invariances modeled as semi-norms, introducing Euclidean embeddings for kernel representers in a semi-inner-product space.
result Accurate invariant representations learned efficiently and effectively, validated by experiments.
We study contractivity properties of gradient flows for functions on normed spaces or, more generally, on Finsler manifolds. Contractivity of the flows turns out to be equivalent to a new notion of convexity for the functions. This is different from the usual convexity along geodesics in non-Riemannian Finsler manifold…
Automated Bayesian inference for massive datasets with theoretical guarantees.
problem Intractable posterior inference in massive datasets.
method Hilbert coreset construction under log-likelihood space inner-product norm.
result Fully-automated, scalable Bayesian inference with theoretical guarantees.
The paper explores curvature and minimal surfaces in normed spaces.
problem Defining and studying curvature and minimal surfaces in normed spaces.
method Characterizing minimal surfaces and proving global theorems.
result Several characterizations of minimal surfaces and analogues of global theorems are derived.
New algorithms adapt to both gradient norms and comparator norms in online learning.
problem Adapting to both gradient norms and comparator norms in online learning.
method Developed parameter-free and scale-free algorithms for unbounded online convex optimization.
result Improved regret bounds for scale-invariant online prediction with linear models.
Adaptive sampling method reduces variance in stochastic optimization.
problem Reducing variance in stochastic optimization with limited gradient computations.
method Adaptive increase in sample size based on inner product test.
result Algorithm converges globally on nonconvex functions and linearly on strongly convex functions.
We study the spherical cap packing problem with a probabilistic approach. Such probabilistic considerations result in an asymptotic sharp universal uniform bound on the maximal inner product between any set of unit vectors and a stochastically independent uniformly distributed unit vector. When the set of unit vectors …
We consider a complete, totally umbilical hypersurface M of Riemannian space (R^n,g^) induced by a Minkowski space (Rn,F). Under certain conditions we prove that M is isometric to a "round" hypersphere of the (n+1)−dimensional Euclidean space. We also prove that the Minkowski norm F must be …
A new framework decouples CNN features into intra-class and semantic differences.
problem Learning visual representations in CNNs is challenging.
method Proposes a decoupled learning framework that models intra-class variation and semantic difference independently.
result Decoupled reparameterization leads to significant performance gains and easier convergence.
A fast algorithm for L1-norm kernel PCA with convergence analysis.
problem Finding an optimal solution for L1-norm kernel PCA due to its non-convexity and non-smoothness.
method A fixed-point type algorithm that iteratively computes binary weights for each observation, based on a geometrically interpretable reformulation of the problem.
result The algorithm converges to a local optimal solution in a finite number of steps and the sequence of objective values converges at a linear rate.
New method approximates complex kernel norms with random features, making learning tractable.
problem Complexity of learning with kernel methods in high dimensions.
method Random features approximations to Fp norms, focusing on p>1. result For p>1, the number of random features required is polynomial in the sample size, making learning tractable. PieClam autoencodes graphs into communities, improving graph anomaly detection.
problem Graph anomaly detection and universal graph autoencoding.
method Probabilistic graph model with overlapping inclusive and exclusive communities.
result PieClam is a universal autoencoder that uniformly approximates any graph.
Study higher rank inner products and their tilings to describe tori degenerations.
problem Understanding metric degenerations of tori.
method Introduce higher rank inner products and their tilings, use to describe degenerations.
result Describe metric degenerations of polarized tori and Hausdorff limits of tilings.
The abstract discusses a new type of space and its properties.
problem The abstract tackles the concept of non-Hilbertian (Lorentzian) length spaces.
method The abstract introduces a new type of space and analyzes its properties.
result The abstract finds that normed spaces without inner products have no sectional curvature bounds.
Latent space models are effective tools for statistical modeling and exploration of network data. These models can effectively model real world network characteristics such as degree heterogeneity, transitivity, homophily, etc. Due to their close connection to generalized linear models, it is also natural to incorporat…
Unified framework for constructing RKBSs with various norms and kernels.
problem Unclear relations among existing RKBS constructions.
method Generic definition of RKBS and reproducing kernel, continuous bilinear form, feature maps.
result Unified framework unifies existing RKBS constructions and develops representer theorems.
New method uses adaptive sampling for optimization in uncertain conditions.
problem Optimizing functions with unknown gradients in uncertain environments.
method Adaptive sampling quasi-Newton method with finite differences and norm tests.
result Potential performance benefits of the proposed method demonstrated in preliminary experiments.
Researchers prove inner product recovery is impossible in latent space models.
problem Recovering inner products in latent space models with random geometric graphs.
method Rate-distortion theory applied to Gaussian or spherical latent locations.
result Impossible to recover inner products if dimensionality exceeds nh(p), matching positive results' conditions. GRAIN: Group Aggregation via Min-Norm Objective
problem Learning instability in large models
method Replacing mean aggregation with min-norm convex combination
result Improves performance and reduces variance
Paper proposes a new method to optimize feature coordinates for better image classification.
problem Improving feature extraction for better machine learning classification.
method Mutual-energy inner product optimization method.
result The method enhances low-frequency features and suppresses high-frequency noise, leading to better classification results.
WIPS optimizes inner product weights to approximate various similarities.
problem Learning high-quality node representations and accurate similarities.
method Weighted inner product similarity (WIPS) with adjustable weights.
result WIPS can approximate arbitrary general similarities including positive definite and indefinite kernels.
Study of Gaussian distributions using entropic Gromov-Wasserstein and inner product Gromov-Wasserstein.
problem Optimal transportation between Gaussian distributions with different dimensions.
method Entropic Gromov-Wasserstein and inner product Gromov-Wasserstein, with closed-form expressions and von Neumann's trace inequality.
result Closed-form expressions for the entropic IGW and its unbalanced variant between Gaussian distributions.
Legendre curves are smooth plane curves which may have singular points, but still have a well defined smooth normal (and corresponding tangent) vector field. Because of the existence of singular points, the usual curvature concept for regular curves cannot be straightforwardly extended to these curves. However, Fukunag…
We propose a quantization based approach for fast approximate Maximum Inner Product Search (MIPS). Each database vector is quantized in multiple subspaces via a set of codebooks, learned directly by minimizing the inner product quantization error. Then, the inner product of a query to a database vector is approximated …
Study on kernel regression risk in high dimensions using Pinsker bound.
problem Kernel regression risk in high-dimensional inner product spaces.
method Investigation of Pinsker bound for kernel regression on sphere Sd with sample size n=αdγ(1+od(1)). result Exact minimax risk and Pinsker constant identified for kernel regression.
We point out that the Homfly polynomial (that is to say, Ocneanu's trace functional) contains two polynomial-valued inner products on the Hecke algebra representation of Artin's braid group. These bear a close connection to the Morton-Franks-Williams inequality. In these structures, the sets of positive, respectively n…
New tests for high-dimensional data improve on existing methods.
problem Testing mean vectors in high-dimensional data.
method Generalized multivariate sign transformation, using different norm functions.
result Tests using generalized signs have higher power than existing tests.
New metrics measure knots' shapes without changing their orientation.
problem Measuring knots without considering their orientation.
method Defined Möbius invariant metrics on knot space.
result Found conditions for Möbius invariant weighted inner products.
We present the first provably sublinear time algorithm for approximate \emph{Maximum Inner Product Search} (MIPS). Our proposal is also the first hashing algorithm for searching with (un-normalized) inner product as the underlying similarity measure. Finding hashing schemes for MIPS was considered hard. We formally sho…
Sobolev quantities (norms, inner products, and distances) of probability density functions are important in the theory of nonparametric statistics, but have rarely been used in practice, partly due to a lack of practical estimators. They also include, as special cases, L2 quantities which are used in many applicatio…
A new method estimates parameters in heavy-tailed corrupted regression with unknown covariance and heterogeneous noise.
problem Estimating parameters in regression with heavy-tailed errors and unknown covariance.
method Near-optimal computationally tractable estimator based on power method and Multiplicative Weight Update algorithm.
result The estimator achieves the optimal statistical rate and breakdown-point under near-optimal sample size.
Bounds eigenfunction inner products on curves and surfaces.
problem Estimating inner products of eigenfunctions on curves and surfaces.
method Using bounds on eigenfunctions and Fourier coefficients.
result Sharp bounds on inner products and Fourier coefficients.
New vector quantization method reduces relevance of parallel components in database points.
problem Scaling maximum inner product search to massive databases.
method Developed anisotropic vector quantization loss functions.
result Achieves state-of-the-art results on public benchmarks.
Estimates latent inner products from an anisotropic Gaussian graph with improved spectral method.
problem Recovering latent inner products from an anisotropic Gaussian random geometric graph.
method Doubly centered adjacency matrix, rank-d spectral approximation, Hermite expansion, decoupling argument.
result Estimator achieves mean squared error rate matching state of the art for isotropic case and ill-conditioned covariance matrices.
Being E a vector space with inner product and S the sphere of E, will be given a demonstration that every application of the sphere S itself it such that preserve inner product is the restriction of a linear isometry in E.
New attacks break robust aggregation methods for SGD in Byzantine-tolerant systems.
problem Breaking Byzantine-tolerant techniques in distributed machine learning.
method Inner product manipulation to break robust aggregation methods (median and Krum).
result Coordinate-wise median and Krum can be broken using new attack strategies.
SIPS extends graph embedding by approximating more types of similarities.
problem Graph embedding's limitation in approximating certain types of similarities.
method Shifted inner-product similarity (SIPS) with bias terms.
result SIPS can approximate PD and CPD similarities, improving graph embedding performance.
This study approximates neural network features for modeling relations and attention mechanisms.
problem Approximating neural network features for modeling relations and attention mechanisms.
method Analyzes inner products of multi-layer perceptrons for universal approximation of symmetric and asymmetric relation functions.
result Universal approximation of relation functions and attention mechanisms using inner products of neural networks.
The Bergman kernels of holomorphic vector bundles are studied to extend the Fubini-Study map.
problem Extending the Fubini-Study map to a closed range for general inner products.
method Associate Bergman kernels with general inner products on the dual space.
result FS is an injective immersion but not necessarily closed in the space of positive definite inner products.
New geometric interpretations reveal structure of AC integrands.
problem Understanding and verifying the atomic condition for integrands.
method Reinterpretation of atomic condition in convex geometry.
result Quantitative versions EC and QEC proposed; stability and regularity results.
Classifies metrics on specific Lie groups.
problem Classifying Riemannian metrics on nonunimodular Lie groups.
method Automorphism classification of inner products on Lie algebras.
result Classification of metrics on 4D nonunimodular Lie groups.
We propose a robust elastic net (REN) model for high-dimensional sparse regression and give its performance guarantees (both the statistical error bound and the optimization bound). A simple idea of trimming the inner product is applied to the elastic net model. Specifically, we robustify the covariance matrix by trimm…
We show that every unimodular Lie algebra, of dimension at most 4, equipped with an inner product, possesses an orthonormal basis comprised of geodesic elements. On the other hand, we give an example of a solvable unimodular Lie algebra of dimension 5 that has no orthonormal geodesic basis, for any inner product.
Signature Isolation Forest removes constraints from FIF by using rough path theory's signature transform.
problem Challenges in FIF's linear inner product and dictionary choices leading to unreliable results.
method Introduces Signature Isolation Forest using rough path theory's signature transform to remove linearity constraints.
result Demonstrates relevance of methods through numerical experiments and real-world applications.