Recurrent Neural Networks (RNNs) are used in state-of-the-art models in domains such as speech recognition, machine translation, and language modelling. Sparsity is a technique to reduce compute and memory requirements of deep learning models. Sparse RNNs are easier to deploy on devices and high-end server processors. …
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,694 papers · 148 categories
Trend · papers per month
48 results for “blocking technique”
Proves conjecture linking WRT invariants and homological blocks for plumbed 3-manifolds.
problem Proving a conjecture about Witten-Reshetikhin-Turaev invariants and homological blocks for plumbed 3-manifolds.
method Developed a new technique for asymptotic expansions to compare WRT invariants and homological blocks, proving vanishing of weighted Gauss sums.
result Proved conjecture stating WRT invariants are radial limits of homological blocks.
Matrix Factorization (MF) on large scale matrices is computationally as well as memory intensive task. Alternative convergence techniques are needed when the size of the input matrix is higher than the available memory on a Central Processing Unit (CPU) and Graphical Processing Unit (GPU). While alternating least squar…
The desire to map neural networks to varying-capacity devices has led to the development of a wealth of compression techniques, many of which involve replacing standard convolutional blocks in a large network with cheap alternative blocks. However, not all blocks are created equally; for a required compute budget there…
Confidence Interval Construction for Multivariate time series using Long Short Term Memory Networkstat.ME
Paper proposes new method for time series confidence intervals using LSTM.
problem Constructing accurate confidence intervals for multivariate time series.
method Uses Long Short Term Memory Network (LSTM) and novel block bootstrap techniques.
result Demonstrates improved accuracy in constructing confidence intervals.
Spectral embedding is a popular technique for the representation of graph data. Several regularization techniques have been proposed to improve the quality of the embedding with respect to downstream tasks like clustering. In this paper, we explain on a simple block model the impact of the complete graph regularization…
This paper introduces GLT for better input data representation in BNN and proposes a compact topology with block pruning.
problem Improving input data representation for Binary Neural Networks (BNN).
method Generic Learned Thermometer (GLT) for encoding, block pruning and Knowledge Distillation for compact topology.
result Significant accuracy gains and lightweight fully-binarized models with limited accuracy degradation.
In this paper we analyze the randomized block-coordinate descent (RBCD) methods proposed in [8,11] for minimizing the sum of a smooth convex function and a block-separable convex function. In particular, we extend Nesterov's technique developed in [8] for analyzing the RBCD method for minimizing a smooth convex functio…
Deep learning has shown promising results on many machine learning tasks but DL models are often complex networks with large number of neurons and layers, and recently, complex layer structures known as building blocks. Finding the best deep model requires a combination of finding both the right architecture and the co…
An algorithm finds optimal covariates for blocking in randomized experiments.
problem Minimizing variance in causal effect estimates from heterogeneous data.
method Using causal graphs, an algorithm identifies optimal covariates for blocking.
result An efficient algorithm reduces variance in causal effect estimates.
Randomized block-diagonal preconditioning improves parallel learning convergence.
problem Improving convergence of gradient-based optimization methods in parallel settings.
method Randomization of coordinates during optimization to repartition tasks.
result Randomization significantly improves convergence of block-diagonal preconditioned methods.
A new method speeds up community detection in graphs.
problem Efficiently detecting communities in large graphs.
method Subsampled semidefinite programming for faster clustering.
result Statistical limits of sketching for community detection established.
Neural decoders were shown to outperform classical message passing techniques for short BCH codes. In this work, we extend these results to much larger families of algebraic block codes, by performing message passing with graph neural networks. The parameters of the sub-network at each variable-node in the Tanner graph…
Graph connection Laplacian (GCL) is a modern data analysis technique that is starting to be applied for the analysis of high dimensional and massive datasets. Motivated by this technique, we study matrices that are akin to the ones appearing in the null case of GCL, i.e the case where there is no structure in the datas…
Study risk-controlling prediction sets for single trajectory data from dynamical systems.
problem Performance guarantees for risk-controlling prediction sets in single trajectory data from unknown stochastic dynamical systems.
method Used blocking and decoupling techniques to analyze performance guarantees under different data generating processes.
result Performance guarantees similar to iid setting when data is stationary and contractive, with graceful degradation otherwise.
PatchUp: A Feature-Space Block-Level Regularization Technique for Convolutional Neural Networkscs.LG
PatchUp improves CNN robustness with mixed feature blocks.
problem High generalization gap in deep learning models with limited labeled data.
method Block-level regularization of hidden feature maps from mixed samples.
result PatchUp improves model robustness and generalization.
A novel bandit problem with context-dependent rewards and blocking.
problem Contextual bandit problem with blocking.
method Online bipartite matching, UCB algorithm, delayed exploitation, opportunistic subsampling.
result Guaranteed -regret in bandit setting.
Finding the best neural network architecture requires significant time, resources, and human expertise. These challenges are partially addressed by neural architecture search (NAS) which is able to find the best convolutional layer or cell that is then used as a building block for the network. However, once a good buil…
Gradient descent and its many variants, including mini-batch stochastic gradient descent, form the algorithmic foundation of modern large-scale machine learning. Due to the size and scale of modern data, gradient computations are often distributed across multiple compute nodes. Unfortunately, such distributed implement…
Simultaneous orthogonal matching pursuit (SOMP) and block OMP (BOMP) are two widely used techniques for sparse support recovery in multiple measurement vector (MMV) and block sparse (BS) models respectively. For optimal performance, both SOMP and BOMP require \textit{a priori} knowledge of signal sparsity or noise vari…
In this paper we consider the problem of minimizing a convex function using a randomized block coordinate descent method. One of the key steps at each iteration of the algorithm is determining the update to a block of variables. Existing algorithms assume that in order to compute the update, a particular subproblem is …
Simulating fluid flow in geological formations requires mesh generation, lithology mapping to the cells, and computing geometric properties such as normal vectors and volume of cells. The purpose of this research work is to compute and process the geometrical information required for performing numerical simulations in…
Extends co-clustering to mixed numerical and binary data.
problem Co-clustering of mixed data types (numerical and binary).
method Latent block models for mixed data types.
result Effectiveness of the proposed approach on simulated data.
The paper provides statistical theory and intuition for personalized PageRank (called "PPR"): a popular technique that samples a small community from a massive network. We study a setting where the entire network is expensive to obtain thoroughly or to maintain, but we can start from a seed node of interest and "crawl"…
In many state-of-the-art compression systems, signal transformation is an integral part of the encoding and decoding process, where transforms provide compact representations for the signals of interest. This paper introduces a class of transforms called graph-based transforms (GBTs) for video compression, and proposes…
Accelerating Speculative Diffusions via Block Verification
problem Adapting speculative decoding for continuous diffusion models
method Introducing a novel speculative sampling mechanism for diffusion models
result Improves acceptance rate and speeds up inference
Unified framework for solving linear systems with improved convergence rates.
problem Efficiently solving linear systems with randomized batch-sampling methods.
method Developed a unified randomized batch-sampling Kaczmarz framework with concentration inequalities for analysis.
result Derived new expected linear convergence rate bounds that are tighter and more reflective of empirical behavior.
This paper proposes a discrimination technique for vertices in a weighted network. We assume that the edge weights and adjacencies in the network are conditionally independent and that both sources of information encode class membership information. In particular, we introduce a edge weight distribution matrix to the s…
New algorithms tackle complex multi-block optimization problems in machine learning.
problem Non-convex multi-block bilevel optimization with hierarchical sampling challenges.
method Blockwise stochastic variance-reduced methods with parallel speedup.
result Achieves matching complexity to single-block problems with parallel speedup.
Resolving a conjecture of Abbe, Bandeira and Hall, the authors have recently shown that the semidefinite programming (SDP) relaxation of the maximum likelihood estimator achieves the sharp threshold for exactly recovering the community structure under the binary stochastic block model of two equal-sized clusters. The s…
ALMA improves clustering of multilayer networks.
problem Clustering multilayer networks with distinct layers and communities.
method Alternating minimization algorithm (ALMA) for simultaneous layer partition and community estimation.
result ALMA achieves higher accuracy than TWIST in clustering multilayer networks.
Efficient private algorithms for estimating block models and mixture models.
problem Estimating block models and mixture models in high-dimensional settings.
method General tools for designing efficient private estimation algorithms.
result First efficient private algorithms for weak and exact recovery of stochastic block models.
VEC-SBM detects communities using side information like texts and images.
problem Community detection in social networks with side information.
method Proposes a novel algorithm based on iterative refinement techniques.
result Optimally recovers latent communities with side information.
Block based Singular Value Decomposition approach to matrix factorization for recommender systemscs.LG
With the abundance of data in recent years, interesting challenges are posed in the area of recommender systems. Producing high quality recommendations with scalability and performance is the need of the hour. Singular Value Decomposition(SVD) based recommendation algorithms have been leveraged to produce better result…
Efficient algorithm for robust recovery in stochastic block models.
problem Robust recovery in stochastic block models.
method Convex optimization framework, addressing optimization landscape challenges.
result Achieves robust recovery without a price of robustness.
Perceptual ad-blocking is a novel approach that detects online advertisements based on their visual content. Compared to traditional filter lists, the use of perceptual signals is believed to be less prone to an arms race with web publishers and ad networks. We demonstrate that this may not be the case. We describe att…
WavPool improves deep neural networks with wavelet-based pooling.
problem Improving efficiency and performance of deep neural networks.
method Introducing WavPool, a wavelet-transform-based pooling layer.
result WavPool outperforms existing network architectures by 10% on CIFAR-10.
Paper proposes efficient methods for high-order clustering in tensor block models.
problem High-order clustering of multiway datasets in neuroimaging, genomics, etc.
method Tensor block model and computationally efficient algorithms (HLloyd, HSC)
result Achieves high-order exact clustering with statistical optimality and computational efficiency.
The central discovery of conformal theory was holomorphic factorization, which expressed correlation functions through bilinear combinations of conformal blocks, which are easily cut and joined without a need to sum over the entire huge Hilbert space of states. Somewhat similar, when a link diagram is glued from t…
Normalising flows (NFS) map two density functions via a differentiable bijection whose Jacobian determinant can be computed efficiently. Recently, as an alternative to hand-crafted bijections, Huang et al. (2018) proposed neural autoregressive flow (NAF) which is a universal approximator for density functions. Their fl…
Community detection is a fundamental unsupervised learning problem for unlabeled networks which has a broad range of applications. Many community detection algorithms assume that the number of clusters is known apriori. In this paper, we propose an approach based on semi-definite relaxations, which does not require…
New algorithm detects communities even with corrupted data, reaching Kesten-Stigum threshold.
problem Robust community detection in stochastic block model with node corruptions.
method Polynomial-time algorithm using Grothendieck norm of principal submatrices.
result First algorithm to achieve weak recovery at Kesten-Stigum threshold with node corruptions.
Learning Deep Representation Without Parameter Inference for Nonlinear Dimensionality Reductioncs.LG
Unsupervised deep learning is one of the most powerful representation learning techniques. Restricted Boltzman machine, sparse coding, regularized auto-encoders, and convolutional neural networks are pioneering building blocks of deep learning. In this paper, we propose a new building block -- distributed random models…
Glioma is one of the most common types of brain tumors; it arises in the glial cells in the human brain and in the spinal cord. In addition to having a high mortality rate, glioma treatment is also very expensive. Hence, automatic and accurate segmentation and measurement from the early stages are critical in order to …
Learning Enhanced Structural Representations with Block-Based Uncertainties for Ocean Floor Mappingcs.LG
Enhances ocean floor mapping with adaptive uncertainty estimates.
problem Inaccurate bathymetric data for precise ocean modeling.
method Block-based conformal prediction with VQ-VAE architecture.
result Significant improvements in reconstruction quality and uncertainty estimation reliability.
Randomized spectral co-clustering speeds up large-scale directed networks.
problem Co-clustering directed networks efficiently for large-scale data.
method Randomized spectral co-clustering algorithms using random-projection and random-sampling techniques.
result Theoretical and numerical validation of approximation and misclustering error rates.
New algorithms improve community detection in network data with strong consistency.
problem Challenges in effectively adapting spectral clustering techniques and achieving strong consistency in label recovery.
method Proposed Thresholded Cosine Spectral Clustering (TCSC) and one-step Refined TCSC algorithms, with strong consistency proofs.
result One-step Refined TCSC achieves strong consistency in community detection under PABM, correctly recovering all labels with high probability.
Improved graph clustering with modularity and coarsening for attributes and communities.
problem Inaccurate community detection and computational inefficiency in graph clustering.
method Integrates coarsening and modularity maximization, using a loss function with log-determinant, smoothness, and modularity components.
result Superior clustering outcomes, proven consistent under DC-SBM, and efficient algorithm integration with GNNs and VGAEs.