TOCO framework compresses neural networks based on tolerance analysis.
problem Deploying large neural networks on edge devices with limited resources.
method TOCO uses tolerance analysis to perform fine-grained compression, allowing flexibility to hardware changes.
result Fine-grained compression of neural networks on edge devices.
PCNN prunes CNN weights efficiently for hardware acceleration.
problem Efficiently compressing CNN models for hardware acceleration.
method PCNN uses a novel Sparsity Pattern Mask (SPM) to encode and prune weights.
result PCNN achieves up to 8.4X compression with minimal accuracy loss.
Pruning is an efficient model compression technique to remove redundancy in the connectivity of deep neural networks (DNNs). Computations using sparse matrices obtained by pruning parameters, however, exhibit vastly different parallelism depending on the index representation scheme. As a result, fine-grained pruning ha…
NeuroDiff improves neural network equivalence verification with fine-grained approximations.
problem Verifying the equivalence of compressed neural networks.
method Symbolic and fine-grained approximation technique for differential verification.
result NeuroDiff achieves up to 1000X speedup and 5X accuracy improvement.
In construction projects, estimation of the settlement of fine-grained soils is of critical importance, and yet is a challenging task. The coefficient of consolidation for the compression index (Cc) is a key parameter in modeling the settlement of fine-grained soil layers. However, the estimation of this parameter is c…
Recurrent Neural Networks (RNN) can be difficult to deploy on resource constrained devices due to their size.As a result, there is a need for compression techniques that can significantly compress RNNs without negatively impacting task accuracy. This paper introduces a method to compress RNNs for resource constrained e…
Model compression techniques, such as pruning and quantization, are becoming increasingly important to reduce the memory footprints and the amount of computations. Despite model size reduction, achieving performance enhancement on devices is, however, still challenging mainly due to the irregular representations of spa…
Fine-grained atlases improve fMRI analysis of brain activity.
problem Large fMRI datasets require scalable brain network summaries.
method Trained on millions of fMRI volumes, DiFuMo dictionaries of 64-1024 networks.
result Fine-grained atlases enhance classic fMRI analysis pipelines.
Deep convolutional neural networks (CNNs) are powerful tools for a wide range of vision tasks, but the enormous amount of memory and compute resources required by CNNs pose a challenge in deploying them on constrained devices. Existing compression techniques, while excelling at reducing model sizes, struggle to be comp…
Unified framework for distributed compressed SGD under (L0,L1)-smoothness.
problem Understanding the joint effect of batch noise, adaptivity, and compression in distributed stochastic optimization.
method Developed a unified theoretical framework using SDEs that incorporate curvature-dependent terms.
result Normalizing updates in DCSGD stabilizes convergence, with normalization degree determined by noise structure and landscape regularity.
Sparsity helps reduce the computational complexity of deep neural networks by skipping zeros. Taking advantage of sparsity is listed as a high priority in next generation DNN accelerators such as TPU. The structure of sparsity, i.e., the granularity of pruning, affects the efficiency of hardware accelerator design as w…
A very simple way to improve the performance of almost any machine learning algorithm is to train many different models on the same data and then to average their predictions. Unfortunately, making predictions using a whole ensemble of models is cumbersome and may be too computationally expensive to allow deployment to…
We introduce a new framework for unsupervised learning of representations based on a novel hierarchical decomposition of information. Intuitively, data is passed through a series of progressively fine-grained sieves. Each layer of the sieve recovers a single latent factor that is maximally informative about multivariat…
As entity type systems become richer and more fine-grained, we expect the number of types assigned to a given entity to increase. However, most fine-grained typing work has focused on datasets that exhibit a low degree of type multiplicity. In this paper, we consider the high-multiplicity regime inherent in data source…
Proposes a method to predict stock movements using fine-grained events from finance news.
problem Lack of specific semantic information in coarse-grained events for stock movement prediction.
method Built a finance event dictionary, extracted fine-grained events, combined with stock trade data, and used distant supervision for training.
result Method outperforms all baselines and shows good generalizability.
Rugby-Bot predicts multiple metrics from a single source using fine-grain data.
problem Complexity of sporting events requires multiple metrics for accurate analysis.
method Multi-task learning with fine-grain spatial data and wide-and-deep learning.
result Predictions are consistent and can be in distribution form.
Model compression techniques on Deep Neural Network (DNN) have been widely acknowledged as an effective way to achieve acceleration on a variety of platforms, and DNN weight pruning is a straightforward and effective method. There are currently two mainstreams of pruning methods representing two extremes of pruning reg…
Paper presents UrbanFM and UrbanPy models for inferring fine-grained urban flows.
problem Reduce cost of urban flow monitoring while maintaining data accuracy and granularity.
method Develops UrbanFM and UrbanPy models to infer fine-grained urban flows from coarse-grained observations.
result UrbanPy model demonstrates favorable performance for larger-scale inference tasks.
Fine-grained gap-dependent regret bounds for reinforcement learning.
problem Achieving optimal regret bounds for reinforcement learning with suboptimality gaps.
method Developed novel analytical frameworks and refined algorithms for UCB-based and non-UCB-based reinforcement learning.
result Established the first fine-grained gap-dependent regret bounds for both UCB-based and non-UCB-based algorithms.
This paper proposes a communication-efficient deep anomaly detection framework for industrial IoT.
problem Accurately detecting anomalies in time-series data from edge devices in industrial IoT.
method A federated learning-based approach with an Attention Mechanism-based Convolutional Neural Network-Long Short Term Memory (AMCNN-LSTM) model and gradient compression.
result The proposed framework accurately and timely detects anomalies with reduced communication overhead.
FIGARO generates symbolic music with fine-grained control.
problem Minimal control over generated music sequences.
method Description-to-sequence task, learning conditional distribution of sequences given high-level descriptions.
result State-of-the-art controllable symbolic music generation.
DistPre predicts traffic speeds efficiently for large networks.
problem Fine-grained, accurate speed prediction for large-scale transportation networks.
method Customizes LSTM models on a cluster, sharing trained models between detectors.
result Efficient and accurate fine-grained traffic-speed prediction.
Fine-grained pretraining improves neural network's ability to learn rare features.
problem Improving generalization in deep learning models.
method Introducing a hierarchical multi-view structure to confine input data distribution.
result Fine-grained pretraining leads to better accuracy on hard downstream test samples.
ECN framework improves training on noisy structured labels.
problem Structured errors in fine-grained annotations lead to biased models.
method Error-Correcting Networks (ECN) framework.
result ECN improves fine-grained annotation prediction.
Paper proposes a new model for fine-grained review classification using weak supervision.
problem Fine-grained analysis of reviews is challenging due to segment-level labels.
method Employed Multiple Instance Learning (MIL) with sigmoid attention mechanism for weak supervision.
result Proposed model outperforms state-of-the-art models in segment-level sentiment classification.
BERT improves fine-grained sentiment classification.
problem Fine-grained sentiment classification of text.
method Used BERT for fine-grained sentiment classification.
result BERT outperforms other models for fine-grained sentiment classification.
ViViT efficiently computes curvature for deep networks without approximations.
problem Efficiently computing curvature for deep networks without approximations.
method Leverages the GGN's low-rank structure without further approximations.
result ViViT allows for efficient computation of eigenvalues, eigenvectors, and directional derivatives.
Proposes a new method to generate unrestricted adversarial examples.
problem Generating unrestricted adversarial examples without norm constraints.
method Leveraging state-of-the-art generative models to manipulate image fine-grained aspects.
result Our adversarial images look indistinguishable from natural images and can bypass certified defenses.
Efficient algorithms learn from coarse labels instead of fine grained ones.
problem Learning from coarse labels when fine labels are unavailable.
method Formalized coarse label settings, used a reduction for SQs, and provided efficient algorithms.
result Any problem learnable from fine labels can be learned efficiently from coarse labels.
Proposes a method for weakly-supervised object localization to improve few-shot learning.
problem Challenges of few-shot learning, especially with fine-grained categories.
method Introduces a Self-Attention Based Complementary Module (SAC Module) for weakly-supervised object localization.
result Significantly outperforms state-of-the-art methods on benchmark datasets, especially for fine-grained few-shot tasks.
We propose a probabilistic model for refining coarse-grained spatial data by utilizing auxiliary spatial data sets. Existing methods require that the spatial granularities of the auxiliary data sets are the same as the desired granularity of target data. The proposed model can effectively make use of auxiliary data set…
L*ReLU improves deep learning for fine-grained image classification.
problem Fine-grained image classification requires specific AFs.
method Proposes L*ReLU, piece-wise linear AFs for deep learning.
result L*ReLU achieves superior results on FGVC datasets.
New method uses LLMs to generate detailed scientific hypotheses.
problem Generating detailed, actionable scientific hypotheses from coarse initial directions.
method Hierarchical search method that incrementally adds details to hypotheses.
result Hierarchical search method consistently outperforms strong baselines on expert-annotated hypotheses.
Improved investment performance with fine-grained LLM tasks.
problem Abstract financial trading systems often overlook real-world workflow intricacies, leading to degraded performance.
method Proposes a multi-agent LLM trading framework that decomposes investment analysis into fine-grained tasks.
result Fine-grained task decomposition significantly improves risk-adjusted returns compared to coarse-grained designs.
Predicting fine-grained interests of users with temporal behavior is important to personalization and information filtering applications. However, existing interest prediction methods are incapable of capturing the subtle degreed user interests towards particular items, and the internal time-varying drifting attention …
Solves dual imbalance in detecting sparse anomalies in MIL.
problem Detecting scarce and sparse anomalous samples in MIL.
method Reformulates MIL as a fine-grained PU learning problem, addressing imbalance at both macro and micro levels.
result Demonstrates effectiveness of BFGPU framework on synthetic and real-world datasets.
Paper tackles cross-granularity few-shot learning with meta-embedder.
problem Few-shot learning with coarse labels and fine-grained testing.
method Meta-embedder that optimizes visual and semantic discrimination across coarse and fine classes.
result Meta-embedder achieves effective cross-granularity few-shot classification.
Pipelined Backpropagation trains large models without batches efficiently.
problem Training large models efficiently on hardware with limited batch sizes.
method Fine-grained Pipelined Backpropagation with Spike Compensation and Linear Weight Prediction.
result Fine-grained Pipelined Backpropagation with a batch size of one matches the accuracy of SGD for multiple networks.
Hardware accelerations of deep learning systems have been extensively investigated in industry and academia. The aim of this paper is to achieve ultra-high energy efficiency and performance for hardware implementations of deep neural networks (DNNs). An algorithm-hardware co-optimization framework is developed, which i…
We analyze computational limits of modern Hopfield models based on pattern norms.
problem Understanding the efficiency of modern Hopfield models from a fine-grained complexity perspective.
method Fine-grained complexity analysis and upper bound criterion for pattern norms.
result Below a specific norm threshold, efficient variants of modern Hopfield models exist.
New framework learns labels at both bag and graph levels.
problem Learning multi-label classifiers from multi-graph bags.
method Designing scoring functions and rank-loss objective for graph and bag levels; developing sub-gradient descent algorithm.
result Superior performance over state-of-the-art algorithms.
Fine-grained event tagging system for SEC 8-K filings improves precision to 96%.
problem Coarse SEC item codes mislabel routine and significant events.
method Two-stage system tagging 8-K disclosures against a 119-event taxonomy.
result LLM judge finds precision rises to 96% with quality scores.
Generative profiling improves real-time task timing for varied resource contexts.
problem Inaccurate task timing analysis for complex hardware architectures.
method Nonparametric, conditional multi-marginal Schrödinger Bridge (MSB) formulation for synthesizing context-dependent timing profiles.
result Maximum likelihood accurate execution profiles for unseen resource contexts.
We introduce a hierarchical architecture for video understanding that exploits the structure of real world actions by capturing targets at different levels of granularity. We design the model such that it first learns simpler coarse-grained tasks, and then moves on to learn more fine-grained targets. The model is train…
We present the multiplicative recurrent neural network as a general model for compositional meaning in language, and evaluate it on the task of fine-grained sentiment analysis. We establish a connection to the previously investigated matrix-space models for compositionality, and show they are special cases of the multi…
Compared with global average pooling in existing deep convolutional neural networks (CNNs), global covariance pooling can capture richer statistics of deep features, having potential for improving representation and generalization abilities of deep CNNs. However, integration of global covariance pooling into deep CNNs …
Flexible framework compresses models using LC algorithm.
problem Efficiently compressing neural networks for resource constraints.
method Decouples learning and compression steps with alternating L and C phases.
result Compressed models maintain performance and accuracy.
Generative framework learns effective, lower-dimensional models from high-dimensional data.
problem Predicting long-term behavior of complex, multiscale systems with limited data.
method Physics-aware probabilistic model order reduction with latent variables.
result Guaranteed long-term stability and predictive accuracy in multiscale physical systems.