arXiv research

There exists a plethora of techniques for inducing structured sparsity in parametric models during the optimization process, with the final goal of resource-efficient inference. However, few methods target a specific number of floating-point operations (FLOPs) as part of the optimization objective, despite many reporti…

Paper combines regularization and pruning to reduce FLOPs in DNNs.

sGPO optimizes RLVR training by balancing inference FLOPs and training efficiency.

In this paper, we study the behavior of Ricci-flat Kähler metrics on Calabi-Yau manifolds under algebraic geometric surgeries: extremal transitions or flops. We prove a version of Candelas and de la Ossa's conjecture: Ricci-flat Calabi-Yau manifolds related by extremal transitions and flops can be connected by a path c…

We define relative Gromov-Witten invariants and establish a general gluing theory of pseudo-holomorphic curves for symplectic cutting and contact surgery. Then, we use our general gluing theory to study the change of GW-invariants of Calabi-Yau 3-folds tranform under flops and extremal transitions. We prove a complete …

This article is concerned with an observation for proving non-existence of canonical Kahler metrics. The idea is to use a rather explicit type of degeneration that applies in many situations. Namely, in a variation on a theme introduced by Ross-Thomas, we consider flops of the deformation to the normal cone. This yield…

New method grows deep networks efficiently by dynamically pruning and growing layers.

RicciNets prunes neural networks by removing edges of low importance based on Ricci curvature, reducing FLOPs by 35%.

Funnel-Transformer reduces computation by compressing sequence data.

FLOP algorithm speeds up causal structure learning for linear models.

Speeds up training and inference by pruning entire channels before training.

This paper gives a description of the full space of Bridgeland stability conditions on the bounded derived category of a contraction algebra associated to a 3-fold flop. The main result is that the stability manifold is the universal cover of a naturally associated hyperplane arrangement, which is known to be simplicia…

LLoCa makes any network Lorentz-equivariant, achieving high accuracy and efficiency.

We propose a fast proximal Newton-type algorithm for minimizing regularized finite sums that returns an $ε$-suboptimal point in $\tilde{\mathcal{O}}(d(n + \sqrt{κd})\log(\frac{1}ε))$ FLOPS, where $n$ is number of samples, $d$ is feature dimension, and $κ$ is the condition number. As long as $n > d$, the proposed method…

Complex surfaces show non-simply connected diffeomorphism groups with non-homotopic loops.

Finet uses FBN for efficient, lightweight neural networks.

Structural pruning of neural network parameters reduces computation, energy, and memory transfer costs during inference. We propose a novel method that estimates the contribution of a neuron (filter) to the final loss and iteratively removes those with smaller scores. We describe two variations of our method using the …

Estimates MLP expected output without sampling, using fewer FLOPs.

Neural networks offer high-accuracy solutions to a range of problems, but are costly to run in production systems because of computational and memory requirements during a forward pass. Given a trained network, we propose a techique called Deep Learning Approximation to build a faster network in a tiny fraction of the …

We prove that the metric completion of a canonical Ricci-flat Kahler metric on the nonsingular part of a projective Calabi-Yau variety $X$ with ordinary double point singularities, is a compact metric length space homeomorphic to the projective variety $X$ itself. As an application, we prove a conjecture of Candelas an…

Deep neural networks with more parameters and FLOPs have higher capacity and generalize better to diverse domains. But to be deployed on edge devices, the model's complexity has to be constrained due to limited compute resource. In this work, we propose a method to improve the model capacity without increasing inferenc…

STR reparameterizes DNN weights with soft thresholds for better sparsity and accuracy.

DNAS disentangles neural architecture search for better interpretability and performance.

Ensembling smaller models can outperform larger models in terms of accuracy and efficiency.

This work proposes a method to learn sparse representations that are more efficient for large-scale data retrieval.

We introduce the fibred toric varieties as equivariant $\mathbb{C}P^r$ bundles over lower dimensional toric varieties. An equivalent characterization is that the natural morphisms on them degenerate to bundle projections in the context of variation of toric varieties as GIT quotients. Our main observation is that these…

This paper introduces channel gating, a dynamic, fine-grained, and hardware-efficient pruning scheme to reduce the computation cost for convolutional neural networks (CNNs). Channel gating identifies regions in the features that contribute less to the classification result, and skips the computation on a subset of the …

AlphaNet improves supernets training with alpha-divergence.

In 1995 D. Joyce explicitly constructed a series of self-dual metrics with torus action on the connected sums of complex projective planes. In this paper we explicitly construct the twistor spaces of some of Joyce's self-dual metrics. Starting from a fiber space whose fibers are compact singular toric surfaces, we appl…

This paper optimizes caching and model multiplexing for large model inference.

Long short-term memory (LSTM) has been widely used for sequential data modeling. Researchers have increased LSTM depth by stacking LSTM cells to improve performance. This incurs model redundancy, increases run-time delay, and makes the LSTMs more prone to overfitting. To address these problems, we propose a hidden-laye…

Joslim optimizes both width and weight configurations for slimmable neural networks, improving model efficiency.

Pruning CNNs by removing less important filters based on empirical loss changes.

Deep Convolutional Neural Networks (CNNs) have been widely used in various domains due to their impressive capabilities. These models are typically composed of a large number of 2D convolutional (Conv2D) layers with numerous trainable parameters. To reduce the complexity of a network, compression techniques can be appl…

We present Caffe con Troll (CcT), a fully compatible end-to-end version of the popular framework Caffe with rebuilt internals. We built CcT to examine the performance characteristics of training and deploying general-purpose convolutional neural networks across different hardware architectures. We find that, by employi…

Researchers find new $G_2$-conifolds in $M$-theory with potential field theory duals.

Estimates model performance based on compute budget and evaluates stability over time.

The paper shows how to efficiently generate large Gaussian process samples with reliability guarantees.

Bershadsky, Cecotti, Ooguri and Vafa constructed a real valued invariant for Calabi-Yau manifolds, which is called the BCOV invariant. In this paper, we consider a pair $(X,Y)$, where $X$ is a compact Kaehler manifold and $Y\in\big|K_X^m\big|$ with $m\in\mathbb{Z}\backslash\{0,-1\}$. We extend the BCOV invariant to suc…

Memory-augmented neural networks (MANNs) are designed for question-answering tasks. It is difficult to run a MANN effectively on accelerators designed for other neural networks (NNs), in particular on mobile devices, because MANNs require recurrent data paths and various types of operations related to external memory a…

Model-based compression is an effective, facilitating, and expanded model of neural network models with limited computing and low power. However, conventional models of compression techniques utilize crafted features [2,3,12] and explore specialized areas for exploration and design of large spaces in terms of size, spe…

Reducing the test time resource requirements of a neural network while preserving test accuracy is crucial for running inference on resource-constrained devices. To achieve this goal, we introduce a novel network reparameterization based on the Kronecker-factored eigenbasis (KFE), and then apply Hessian-based structure…

Deep neural networks (DNNs) have been proven to have many redundancies. Hence, many efforts have been made to compress DNNs. However, the existing model compression methods treat all the input samples equally while ignoring the fact that the difficulties of various input samples being correctly classified are different…

Paper uses surprisal to dynamically allocate computation between fast and slow models.

In this paper we propose the macroblock scaling (MBS) algorithm, which can be applied to various CNN architectures to reduce their model size. MBS adaptively reduces each CNN macroblock depending on its information redundancy measured by our proposed effective flops. Empirical studies conducted with ImageNet and CIFAR-…

Parallel computing has played an important role in speeding up convex optimization methods for big data analytics and large-scale machine learning (ML). However, the scalability of these optimization methods is inhibited by the cost of communicating and synchronizing processors in a parallel setting. Iterative ML metho…

To address the limitations of existing magnitude-based pruning algorithms in cases where model weights or activations are of large and similar magnitude, we propose a novel perspective to discover parameter redundancy among channels and accelerate deep CNNs via channel pruning. Precisely, we argue that channels reveali…

We present a filter correlation based model compression approach for deep convolutional neural networks. Our approach iteratively identifies pairs of filters with the largest pairwise correlations and drops one of the filters from each such pair. However, instead of discarding one of the filters from each such pair naï…