We show that model compression can improve the population risk of a pre-trained model, by studying the tradeoff between the decrease in the generalization error and the increase in the empirical risk with model compression. We first prove that model compression reduces an information-theoretic bound on the generalizati…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,742 papers · 148 categories
Trend · papers per month
48 results for “risk compression”
Study connects compressed signal to AWGN model for risk estimation.
problem Estimating high-dimensional signals under compression constraints.
method Utilizes Gaussian approximation and Wasserstein distance to relate compressed and noisy signals.
result Establishes a connection between estimator risks under different conditions.
Paper ranks stocks by compression risk, not volatility.
problem Investment risk not correlated with stock price volatility.
method Binary-ternary compressive coding of price change time series.
result Compression risk is a better indicator of stock investment risk.
Paper uses neural networks to compress large portfolios of options, reducing risk and capital requirements.
problem Managing risk and capital requirements for large portfolios of financial options.
method Artificial neural network framework for portfolio compression, static hedging, and risk management.
result The compressed portfolio's risk profiles align closely with the target portfolio's, reducing capital requirements.
Optimal Network Compressionq-fin.RM
Paper tackles optimal network compression for financial systems.
problem Optimal network compression for financial systems under shocks.
method Formulated as an NP-hard problem, studied systemic risk measures, and analyzed specific networks.
result Systemic fragility results no longer hold generally under shocks and heterogeneous networks.
New theory controls compression change probability without prior knowledge.
problem Controlling compression change probability without prior knowledge.
method New theory on compression function and statistical risk.
result Cardinality of compressed set is a consistent estimator of probability of change of compression.
A Neural Network Model to Classify Liver Cancer Patients Using Data Expansion and Compressionstat.ML
We develop a neural network model to classify liver cancer patients into high-risk and low-risk groups using genomic data. Our approach provides a novel technique to classify big data sets using neural network models. We preprocess the data before training the neural network models. We first expand the data using wavel…
We describe a general framework -- compressive statistical learning -- for resource-efficient large-scale learning: the training collection is compressed in one pass into a low-dimensional sketch (a vector of random empirical generalized moments) that captures the information relevant to the considered learning task. A…
We establish a tight characterization of the worst-case rates for the excess risk of agnostic learning with sample compression schemes and for uniform convergence for agnostic sample compression schemes. In particular, we find that the optimal rates of convergence for size- agnostic sample compression schemes are of…
This paper optimizes portfolio compression by reducing excess notional in market contracts.
problem Reduction of excess notional in market contracts to decrease systemic risk.
method Examines and solves portfolio compression problems using graph theory and algorithms.
result Developed a clearing algorithm and method to compute maximum volume conservative compression.
SignSGD analysis quantifies its effects in high dimensions.
problem Understanding signSGD's effects in high-dimensional settings.
method High-dimensional analysis of signSGD, deriving SDE and ODE for risk.
result Quantification of signSGD's effects: effective learning rate, noise compression, diagonal preconditioning, gradient noise reshaping.
The paper compares traditional regression with modern neural network methods for financial hedging and risk compression.
problem Finding optimal hedge ratios and managing portfolio risk using traditional regression methods has limitations.
method The paper introduces regularization techniques and common factor analyses using neural networks to improve upon regression methods.
result Neural network methods provide better performance in hedge ratio estimation and risk compression compared to traditional regression.
New guarantees for asymmetric sketching in compressive learning.
problem Statistical guarantees for compressive learning with asymmetric feature maps.
method Proves existing guarantees carry over to asymmetric scheme with LPD property, applies to quantized sketches.
result Existing statistical guarantees for compressive learning extend to asymmetric schemes with controlled error.
Neural networks have been shown to be vulnerable against minor adversarial perturbations of their inputs, especially for high dimensional data under attacks. To combat this problem, techniques like adversarial training have been employed to obtain models which are robust on the training set. However, the …
New method compresses non-Gaussian distributions exponentially.
problem Efficiently representing and computing non-Gaussian probability distributions.
method Tensor-Network Fourier Methods using QTT representation.
result Exponential compression of non-Gaussian distributions.
The paper identifies and critiques problems with risk matrices using ordinal scales.
problem Problems with risk matrices using ordinal scales.
method Overview of risk assessment process, explanation of fallacies, and suggestions for improvement.
result The paper proposes avoiding risk matrices and using fully quantitative methods instead.
Investigates principles of generalization in list learning, refutes sample compression conjecture.
problem Determining applicability of classical principles in list PAC learning.
method Examines uniform convergence and sample compression in list PAC learning.
result Sample compression fails in list PAC learning, refutes conjecture.
Paper proposes robust compressed sensing using generative models.
problem Estimating high-dimensional vectors from noisy linear equations with heavy-tailed or outlier data.
method Inspired by Median-of-Means (MOM), proposes an algorithm for robust recovery.
result Guarantees recovery for heavy-tailed data, even in the presence of outliers.
Paper proposes Nyström sketches for better adaptive compressive learning.
problem Improving adaptability of sketching for compressive learning.
method Data-dependent Nyström approximation for mean embedding.
result Excess risk can be controlled with geometric assumption.
Efficient algorithms for monophonic halfspaces in graphs simplify learning and compression.
problem Learning and compressing monophonic halfspaces in graphs.
method 2-satisfiability based decomposition theorem, efficient algorithms for various learning problems.
result Achieved efficient and nearly optimal algorithms for various learning problems.
Screening Data Points in Empirical Risk Minimization via Ellipsoidal Regions and Safe Loss Functionscs.LG
We design simple screening tests to automatically discard data samples in empirical risk minimization without losing optimization guarantees. We derive loss functions that produce dual objectives with a sparse solution. We also show how to regularize convex losses to ensure such a dual sparsity-inducing property, and p…
This paper analyzes the training dynamics of binary neural networks using information bottleneck.
problem Training binary neural networks is challenging due to discontinuity in activation functions.
method The approach uses the Information Bottleneck principle to analyze BNN training dynamics.
result Training dynamics of BNNs are different from DNNs, with both phases occurring simultaneously.
We give improved constants for data dependent and variance sensitive confidence bounds, called empirical Bernstein bounds, and extend these inequalities to hold uniformly over classes of functionswhose growth function is polynomial in the sample size n. The bounds lead us to consider sample variance penalization, a nov…
Trustworthy Summarization via Uncertainty Quantification and Risk Awareness in Large Language Modelscs.CL
Study improves summarization reliability in risky scenarios.
problem Reliability of automatic summarization in high-risk contexts.
method Conditional generation with Bayesian inference and entropy regularization.
result Significant improvement in robustness and reliability of summarization.
The soaring demand for intelligent mobile applications calls for deploying powerful deep neural networks (DNNs) on mobile devices. However, the outstanding performance of DNNs notoriously relies on increasingly complex models, which in turn is associated with an increase in computational expense far surpassing mobile d…
C2G-Net improves image classification of similar objects like cells.
problem Classifying images with many similar objects efficiently and interpretably.
method Combines image compression and a CNN with reduced parameters.
result C2G-Net achieves similar accuracy to conventional CNNs but with reduced training time and improved interpretability.
We study the limits and methods of training two-layer autoencoders.
problem Understanding the limits and methods of training two-layer autoencoders.
method Focus on non-linear two-layer autoencoders trained in the proportional regime, using gradient methods.
result Gradient methods achieve the minimizers of the population risk and reveal the structure of the features.
In statistical inference problems, we wish to obtain lower bounds on the minimax risk, that is to bound the performance of any possible estimator. A standard technique to obtain risk lower bounds involves the use of Fano's inequality. In an information-theoretic setting, it is known that Fano's inequality typically doe…
Flexible framework compresses models using LC algorithm.
problem Efficiently compressing neural networks for resource constraints.
method Decouples learning and compression steps with alternating L and C phases.
result Compressed models maintain performance and accuracy.
Reduces multiclass and regression compression schemes to binary ones.
problem Developing efficient learning algorithms for multiclass and regression problems.
method Reduces sample compression schemes for binary classes to multiclass and regression settings.
result Establishes new compression schemes for multiclass and regression problems.
Investing is a compression problem, maximizing growth by minimizing divergence.
problem Maximizing long-term wealth and minimizing risk of ruin in investing.
method Decomposes investing into three terms: money, entropy, and divergence. Uses Kelly Criterion and universal portfolio theory.
result Investing can be seen as a compression problem, with optimal strategies minimizing divergence.
Proposes a link between randomness and compression in deep learning.
problem Improving efficiency in deep learning training.
method Introduces a novel tomographic compression framework called Dual Tomographic Compression (DTC).
result Demonstrates high correlation between learning performance and Gibbs entropy over compression ratios.
Measuring Strategy-Decay Risk: Minimum Regime Performance and the Durability of Systematic Investingq-fin.RM
Measures strategy durability through minimum regime performance, revealing trade-offs between efficiency and resilience.
problem Systematic investing strategies are vulnerable to regime changes, affecting their effectiveness and performance.
method Introduces minimum regime performance (MRP) to quantify the durability of systematic strategies, capturing how performance deteriorates under changing market conditions.
result Higher long-term Sharpe ratios do not always correlate with higher MRP, highlighting a new dimension of portfolio fragility.
Recovering structured signals from noisy measurements with a novel matrix model.
problem Recovering structured signals from noisy linear observations with a non-standard measurement matrix.
method Proves approximate empirical risk minimizer is a robust estimator under certain conditions.
result Effective rank of measurement matrix can serve as a surrogate for number of measurements.
These notes review six lectures given by Prof. Andrea Montanari on the topic of statistical estimation for linear models. The first two lectures cover the principles of signal recovery from linear measurements in terms of minimax risk. Subsequent lectures demonstrate the application of these principles to several pract…
STORM enables edge computing for empirical risk minimization.
problem Training models on edge devices for streaming data.
method Online sketching for empirical risk minimization.
result STORM can estimate least-squares objective accurately.
Paper introduces a new adaptive gradient method with gradient compression for distributed training.
problem Communication overhead in distributed machine learning systems.
method Adaptive gradient method with gradient compression, scalable system BytePS-Compress.
result Convergence rate of for non-convex problems.
Galen algorithm compresses neural networks for specific hardware with reduced latency.
problem Finding optimal compression policies for neural networks on specific hardware.
method Reinforcement learning using pruning and quantization to optimize inference latency.
result Compressed ResNet18 for ARM processor reduced inference latency by 80%.
Study shows physical drift affects put-call parity enforcement, not just option payoffs.
problem Inconsistency between quoted put-call parity and actual market behavior.
method Examined SPX and RUT index options, used drift-preserving GBM term to improve fit.
result Physical drift enters the enforcement of risk-neutral parity, not just option payoffs.
The Compressibility of Minimal Lattice Knotscond-mat.soft
The (isothermic) compressibility of lattice knots can be examined as a model of the effects of topology and geometry on the compressibility of ring polymers. In this paper, the compressibility of minimal length lattice knots in the simple cubic, face centered cubic and body centered cubic lattices are determined. Our r…
HiLLoC compresses large images losslessly using VAEs.
problem Lossless compression of large color photographs.
method Fully convolutional VAE models trained on ImageNet are applied to lossless compression.
result Achieves state-of-the-art compression for full-size ImageNet images.
Neural Normalized Compression Distance and the Disconnect Between Compression and Classificationcs.LG
Neural NCD reveals LLMs don't compress well for classification.
problem The disconnect between compression and classification in neural networks.
method Developed Neural NCD to compare LLMs to classic algorithms, finding classification accuracy not correlated with compression rate.
result Classification accuracy is not predictable by compression rate alone, challenging current understanding.
Better Methods and Theory for Federated Learning: Compression, Client Selection and Heterogeneitycs.LG
This thesis tackles FL challenges with new methods and algorithms.
problem Privacy-preserving machine learning with decentralized data.
method Compression, client selection, and heterogeneity handling.
result Practical FL solutions with mathematically rigorous guarantees.
BDC compresses both sample size and dimensionality of large datasets.
problem Large datasets in both sample size and dimensionality.
method Two-stage framework using Decoded MMD, Reconstruction MMD, and Encoded MMD.
result BDC achieves comparable or superior performance with lower cost and higher compression rates.
Anomaly Detection on Financial Time Series by Principal Component Analysis and Neural Networksq-fin.ST
Proposes a method to detect anomalies in financial time series using PCA and neural networks.
problem Anomalies in financial time series lead to miscalibrated risk models.
method Extract features using PCA, define anomaly score with neural network, calibrate cutoff value.
result The proposed PCA NN approach outperforms other anomaly detection methods.
DoCoFL compresses model updates for cross-device federated learning.
problem Downlink compression for cross-device federated learning where clients may appear only once.
method Proposes DoCoFL framework for downlink compression in cross-device federated learning.
result Significant bi-directional bandwidth reduction with competitive accuracy.
Compressing neural nets is an active research problem, given the large size of state-of-the-art nets for tasks such as object recognition, and the computational limits imposed by mobile devices. We give a general formulation of model compression as constrained optimization. This includes many types of compression: quan…
New taxonomy and evaluation of neural network compression methods.
problem Efficiency of deep neural networks in real-world applications.
method Categorization and evaluation of tensor factorization and probabilistic compression methods.
result SVD and probabilistic compression methods are complementary and give the best results.