A new method prunes CNN channels recursively considering inter-layer dependency for significant acceleration.
problem NP-hard channel pruning due to inter-layer dependency in CNNs.
method Recursive Bayesian Pruning (RBP) using a Markov chain model and Dirac-like prior.
result Up to 5.0× FLOPs reduction with minimal accuracy loss on large datasets.
A new method compresses deep neural networks by predicting and quantizing weights between layers.
problem Resource constraints in deep neural networks.
method Inter-Layer Weight Prediction (ILWP) and quantization based on Smoothly Varying Weight Hypothesis (SVWH).
result The method achieves higher weight compression rates at the same accuracy level.
Recent work has studied the reasons for the remarkable performance of deep neural networks in image classification. We examine batch normalization on the one hand and the dynamical systems view of residual networks on the other hand. Our goal is in understanding the notions of stability and smoothness of the inter-laye…
A new method lifts training of input-convex neural networks to avoid dead weights and plateaued loss.
problem Training input-convex neural networks with non-negative weights.
method Introduces a hypernetwork that emits non-negative weights from a summary of the input batch, adding stochasticity to soften the loss landscape.
result The lift method achieves lower test loss than projected gradient descent and direct softplus reparametrization.
We introduce a principled approach for unsupervised structure learning of deep neural networks. We propose a new interpretation for depth and inter-layer connectivity where conditional independencies in the input distribution are encoded hierarchically in the network structure. Thus, the depth of the network is determi…
The rapid development of high-throughput technologies has enabled the generation of data from biological or disease processes that span multiple layers, like genomic, proteomic or metabolomic data, and further pertain to multiple sources, like disease subtypes or experimental conditions. In this work, we propose a gene…
Modern data analysis pipelines are becoming increasingly complex due to the presence of multi-view information sources. While graphs are effective in modeling complex relationships, in many scenarios a single graph is rarely sufficient to succinctly represent all interactions, and hence multi-layered graphs have become…
New model predicts financial connectedness via COVID-19 spread.
problem Predicting financial connectedness during COVID-19 spread.
method Semiparametric matrix regression model with Bayesian hierarchical mixture prior.
result Model captures heterogeneity in network responses to risk factors.
Model quantifies how global economic shocks propagate through interconnected trade and investment networks.
problem Understanding and predicting the global propagation of economic crises.
method Coupled epidemic and internal contagion dynamics on a multiplex network of trade and investment interactions.
result Linear relation between a country's shock magnitude and its global impact, influenced by internal contagion and intercountry propagation.
Using detailed statistical analyses of the size distribution of a universe of equity exchange-traded funds (ETFs), we discover a discrete hierarchy of sizes, which imprints a log-periodic structure on the probability distribution of ETF sizes that dominates the details of the asymptotic tail. This allows us to propose …
FOCA method prevents co-adaptation between feature extractor and classifier.
problem Co-adaptation between feature extractor and classifier degrades neural network performance.
method FOCA method uses randomly-generated, weak classifiers to optimize feature extractor without explicit co-adaptation.
result FOCA features form a point-like distribution within the same class under special conditions.
Sideways trains video models by overwriting activations as new frames arrive, potentially improving generalization.
problem Training deep video models synchronously slows down and requires storing activations, limiting parallelism.
method Sideways trains video models by overwriting activations as new frames arrive, breaking the precise correspondence between gradients and activations.
result Sideways training can converge and potentially generalize better than standard synchronized backpropagation.
In this work, we introduce a deep-structured conditional random field (DS-CRF) model for the purpose of state-based object silhouette tracking. The proposed DS-CRF model consists of a series of state layers, where each state layer spatially characterizes the object silhouette at a particular point in time. The interact…
FF algorithm uses goodness as a likelihood-ratio test for scalar normalization.
problem Training each layer locally with scalar goodness.
method FF algorithm uses a likelihood-ratio test with squared goodness as the sufficient statistic.
result The FF algorithm generalizes to anisotropic and heavy-tailed populations.
A framework for multilayer networks predicts links without shared structures.
problem Link prediction in multilayer networks without shared structures.
method Bi-level model averaging with K-fold cross-validation. result Framework outperforms existing methods in predictive accuracy and robustness.
FF algorithm uses goodness as a measure of input quality, derived from likelihood-ratio tests.
problem Training each layer locally with a goodness measure.
method FF algorithm uses a likelihood-ratio test to define goodness, which is the sum of squared activations normalized between layers.
result The goodness measure is a sufficient statistic for a likelihood-ratio test, explaining the FF algorithm's performance.
DeepLight accelerates CTR predictions in ad serving by 46X.
problem Significantly increased serving delay and high memory usage for ad serving.
method Explicitly searching feature interactions, pruning layers, promoting sparsity.
result Accelerates model inference by 46X on Criteo dataset.
The study characterizes how deep learning models transform data.
problem Understanding how deep learning models transform data and their performance.
method Multivariate nonparametric estimator of class separation (HP statistic) to analyze layer-induced representations.
result Characterization of the distributional change to class separation induced at each layer of the model.
IC-Network improves CNNs by integrating elastic collision units.
problem Designing more effective basic units in neural networks.
method Developed IC layer and IC block units combining the IC structure with convolution operations.
result Significant performance improvements in existing CNNs, reducing top-1 error from 22.85% to 21.49% on imagenet.
TMSCD detects multi-scale communities in temporal networks automatically.
problem Discovering multi-scale communities in large, evolving networks.
method Spectral multilayer formulation of MM method with automatic parameter selection.
result Automatic detection of multi-scale communities without manual parameter selection.
MLP residual networks implement a selective coarse-graining procedure governed by the spectral structure of the input distribution.
problem Understanding the coarse-graining procedure in MLP residual networks
method Analyzing a pure MLP residual stack on synthetic Markov chain sequences
result MLP residual networks implement a selective coarse-graining procedure governed by the spectral structure of the input distribution
TinyCNN accelerates CNN models on embedded FPGA with 15x speedup.
problem Limited memory on embedded FPGAs restricts CNN performance.
method Software and hardware design tool for FPGA resource-aware CNN accelerator.
result 3% accuracy loss with 15.75x speedup on image classification.
New tail dependence measures for stock indices.
problem Measuring tail dependence between financial variables.
method Introducing a new stochastic order and studying monotone tail dependence measures.
result Advantage of new tail dependence measures over classical ones.
Proposes a new dependency function for measuring non-linear relationships.
problem Need for a general-purpose measure of dependency between random variables.
method Revision of ideal properties and proposal of a new dependency function.
result Proposes a new dependency function that meets all desired properties.
The paper explores the relationship between joint mixability and negative dependence structures.
problem Understanding the connection between joint mixability and various negative dependence concepts.
method Analyzes the properties of joint mixes and their relation to negative dependence structures.
result Derives necessary and sufficient conditions for a joint mix to be negatively dependent.
Extend classical theory of affine processes to path-dependent setting
problem Path-dependent affine processes
method Introduce path-dependent coefficients and provide analytic formulas for their Fourier--Laplace transform
result Define path-dependent affine processes through their exponential-affine Fourier--Laplace transform and establish a characterization theorem
Extremal dependence between international stock markets is of particular interest in today's global financial landscape. However, previous studies have shown this dependence is not necessarily stationary over time. We concern ourselves with modeling extreme value dependence when that dependence is changing over time, o…
Forecaster uses graph Transformers to forecast spatial and time-dependent data.
problem Complex spatial and temporal dependencies in data.
method Graph Transformer architecture with sparsification for spatial and temporal dependencies.
result Forecaster significantly outperforms state-of-the-art baselines in taxi demand forecasting.
Predicts financial asset dependencies using spatiotemporal patterns.
problem Complex dependency structures in financial assets for risk mitigation.
method Proposes Asset Dependency Matrix (ADM) and Asset Dependency Neural Network (ADNN) with ConvLSTM for spatiotemporal asset dependency prediction.
result ADNN outperforms baselines in predicting asset dependencies and their applications.
ST-SAN predicts flow with spatial-temporal dependencies using self-attention.
problem Challenges in predicting flow due to spatial-temporal dependencies.
method Spatial-Temporal Self-Attention Network (ST-SAN) that addresses temporal and spatial dependencies.
result Significant improvement in flow prediction accuracy (9% in inflow, 4% in outflow) compared to state-of-the-art methods.
The paper develops methods to price and hedge options in path-dependent stock models.
problem Pricing and hedging options under complex stock models.
method Develops a path-dependent PDE for option pricing and differentiability of path-dependent SDE solutions.
result Provides formulas for option Greeks and differentiability of path-dependent SDE solutions.
New inequalities for learning from graph-dependent data with stability bounds.
problem Learning from dependent data with graph dependency.
method Proved McDiarmid-type concentration inequalities for graph-dependent variables, showed concentration relies on forest complexity.
result Proved stability bounds for learning from graph-dependent data.
Survey on learning with graph-dependent data, deriving new generalization bounds.
problem Traditional i.i.d. data assumption fails in many real-life applications.
method Collect and analyze graph-dependent concentration bounds, derive generalization bounds.
result New generalization bounds for graph-dependent data.
Paper tackles instance-dependent label noise by approximating it with part-dependent noise.
problem Learning with instance-dependent label noise is challenging.
method Approximate instance-dependent label noise with part-dependent noise. Use transition matrices for parts to model noise.
result Method outperforms state-of-the-art approaches for instance-dependent label noise.
Estimating the strength of dependency between two variables is fundamental for exploratory analysis and many other applications in data mining. For example: non-linear dependencies between two continuous variables can be explored with the Maximal Information Coefficient (MIC); and categorical variables that are depende…
In data science, it is often required to estimate dependencies between different data sources. These dependencies are typically calculated using Pearson's correlation, distance correlation, and/or mutual information. However, none of these measures satisfy all the Granger's axioms for an "ideal measure". One such ideal…
Deep signature algorithm for pricing path-dependent options.
problem Pricing path-dependent options with complex payoff functions.
method Extended backward scheme for state-dependent FBSDEs with reflections, incorporating signature layer for path-dependent FBSDEs.
result Convergence analysis of the algorithm with explicit dependence on truncation order and neural network approximation errors.
Model discovers dynamic dependencies in multivariate time series data.
problem Discovering non-linear and time-variant dependencies in multivariate time series.
method seq2graph model using multi-layer GRUs and multi-level attention.
result Jointly trained model discovers both lagged and inter-dependencies accurately.
Study on pairwise counter-monotonicity, a type of negative dependence.
problem Understanding and quantifying extremal negative dependence structures.
method Established stochastic representation and invariance property; showed implications and connections.
result Pairwise counter-monotonicity implies negative association and joint mix dependence.
New measures capture tail dependence and non-exchangeability in financial data.
problem Underestimation of tail dependence and inability to capture non-exchangeable tail dependence.
method Tail copulas and novel tail dependence measures (MTCM, ATCM) are proposed.
result Captures non-exchangeable tail dependence and provides analytical forms for various copulas.
This paper presents a new methodology for clustering multivariate time series leveraging optimal transport between copulas. Copulas are used to encode both (i) intra-dependence of a multivariate time series, and (ii) inter-dependence between two time series. Then, optimal copula transport allows us to define two distan…
Study on generalization for data-dependent hypothesis sets.
problem Understanding generalization in hypothesis sets dependent on data.
method Learning guarantee based on transductive Rademacher complexity and hypothesis set stability.
result Generalization bound for data-dependent hypothesis sets.
Study nearest-neighbor radii under dependent sampling, finding they remain informative.
problem Analyzing nearest-neighbor radii under dependent sampling.
method Consider strong mixing dependent observations, establish distribution-free almost sure convergence and sharp non-asymptotic moment bounds.
result Nearest-neighbor geometry remains informative under dependence sampling.
Study of time-dependent metrics and connections in geometry.
problem Understanding geodesics and connections in time-dependent Riemannian manifolds.
method Examine connections on product manifolds, explore parallel transport, geodesics, and torsion.
result Define the derivative of a one-parameter family of connections.
Estimating the dependency of variables is a fundamental task in data analysis. Identifying the relevant attributes in databases leads to better data understanding and also improves the performance of learning algorithms, both in terms of runtime and quality. In data streams, dependency monitoring provides key insights …
Estimates binary labels from dependent data using Markov Random Fields.
problem Statistical estimation from dependent data across spatial, temporal, and social domains.
method Modeling dependencies as Markov Random Fields and providing efficient estimation algorithms.
result Statistically efficient estimation rates for Ising models from a single sample.
Develops a numerical scheme for solving path-dependent FBSDEs and PDEs.
problem Solving path-dependent FBSDEs and PDEs numerically.
method Picard iteration method for FBSDEs, concentration inequality for estimator, supervised learning with neural networks for PDEs.
result Proves convergence and rate of convergence for the Picard iteration method.
We consider the problem of risk diversification of α-stable heavy tailed risks. We study the behaviour of the aggregated Value-at-Risk, with particular reference to the impact of different tail dependence structures on the limits to diversification. We confirm the large evidence of sub-additivity violations, particul…