A new method improves communication efficiency in distributed learning.
problem Reducing communication overhead in distributed machine learning.
method Transforming contractive compressors into induced unbiased compressors.
result Significant improvements in memory requirements and communication complexity.
We consider lossless compression based on statistical data modeling followed by prediction-based encoding, where an accurate statistical model for the input data leads to substantial improvements in compression. We propose DZip, a general-purpose compressor for sequential data that exploits the well-known modeling capa…
Safe-EF improves federated learning for non-smooth, constrained optimization.
problem Federated learning's communication bottlenecks with high-dimensional model updates.
method Error feedback (EF) for non-smooth convex optimization with safety constraints.
result Safe-EF matches lower complexity bounds and ensures safety constraints.
PowerGossip compresses model differences for decentralized deep learning with low-rank linear compressors.
problem Communication bottleneck in decentralized deep learning models.
method Low-rank linear compressors applied on model differences using power iteration steps.
result Asymptotically independent of network and compression, faster convergence, and comparable performance to tuned compression algorithms.
Paper designs a lossy compression method for lossless prediction.
problem Ensuring high performance on predictive tasks with minimal data.
method Characterizes bit-rate requirements for invariant transformations, designs unsupervised objectives for neural compressors.
result Achieves substantial rate savings on ImageNet compared to JPEG without compromising classification performance.
New biased compression methods lead to faster convergence in distributed learning.
problem Improving convergence rates in distributed learning with biased compression.
method Study of three classes of biased compression operators in distributed learning.
result Biased compressors can lead to linear convergence rates in both single node and distributed settings.
New methods improve distributed optimization on non-iid data.
problem Communication bottleneck in distributed machine learning models.
method Two types of distributed gradient compression methods (D-QSGD and D-EF-SGD) analyzed for non-iid data.
result D-EF-SGD performs better than D-QSGD on non-iid data but can still slow down with high data skewness.
EControl improves fast distributed optimization with compression and error control.
problem Stable convergence issues in distributed training with compression.
method Proposes EControl to regulate error compensation and prove fast convergence.
result Proves fast convergence for EControl in various convex settings without additional assumptions.
Estimates rate-distortion function for large datasets using neural networks.
problem Designing lossy data compression schemes and comparing them with theoretical limits.
method Re-formulate rate-distortion objective and solve using neural networks.
result NERD accurately estimates the rate-distortion function for real-world datasets.
Paper improves communication in distributed optimization, reducing worker-to-server data exchanges.
problem Efficiency in server-to-worker communication in distributed optimization.
method MARINA-P, a novel downlink compression method using correlated compressors; M3, combining MARINA-P with uplink compression.
result MARINA-P achieves provably superior server-to-worker communication complexity with increasing number of workers.
SA-PEF improves federated learning efficiency by correcting gradient mismatches.
problem Slow decay of residual error in federated learning under non-IID data.
method Integrates step-ahead correction with partial error feedback.
result Achieves faster convergence and target accuracy compared to standard EF.
New method reduces distributed non-convex optimization rounds and bits.
problem Efficiently optimizing non-convex models in distributed systems.
method Introduces permutation compressors to reduce communication complexity.
result PermK compressors lead to significant communication complexity improvements.
EF21 improves convergence in distributed machine learning models.
problem Improving convergence in distributed machine learning models.
method Proposes EF21, a new error feedback mechanism.
result EF21 achieves a fast O(1/T) convergence rate for smooth nonconvex problems. Paper improves compressed SGD to reach second-order stationary points.
problem Efficiently reaching second-order stationary points in distributed machine learning.
method Gradient compression and RandomK compressor to improve convergence to second-order stationary points.
result Compressed SGD reaches second-order stationary points with improved communication efficiency.
CSER improves SGD efficiency by resetting errors and partial synchronization.
problem Limited scalability of Distributed Stochastic Gradient Descent (SGD) due to communication bottlenecks.
method Introduces 'error reset' technique and partial synchronization for gradients and models.
result Proves convergence for smooth non-convex problems and accelerates distributed training significantly.
We present a new similarity measure based on information theoretic measures which is superior than Normalized Compression Distance for clustering problems and inherits the useful properties of conditional Kolmogorov complexity. We show that Normalized Compression Dictionary Size and Normalized Compression Dictionary En…
In this paper, we present NESTA, a specialized Neural engine that significantly accelerates the computation of convolution layers in a deep convolutional neural network, while reducing the computational energy. NESTA reformats Convolutions into 3×3 batches and uses a hierarchy of Hamming Weight Compressors to …
Paper optimizes industrial refrigeration using adaptive exploration.
problem Challenges in optimizing real-time industrial processes with unknown characteristics and safety constraints.
method Adaptive and explorative real-time optimization framework with Gaussian process uncertainty quantification.
result Approach increases energy efficiency of refrigeration process, approximating complete information solutions.
Communication overhead is a major bottleneck hampering the scalability of distributed machine learning systems. Recently, there has been a surge of interest in using gradient compression to improve the communication efficiency of distributed neural network training. Using 1-bit quantization, signSGD with majority vote …
New methods reduce communication in distributed training for variational inequalities.
problem Reducing communication in distributed training for high-dimensional models.
method Distributed methods with compressed communication for solving variational inequalities.
result Theoretical guarantees and practical algorithms for compressed communication.
We develop a communication-efficient distributed learning algorithm that is robust against Byzantine worker machines. We propose and analyze a distributed gradient-descent algorithm that performs a simple thresholding based on gradient norms to mitigate Byzantine failures. We show the (statistical) error-rate of our al…
A novel deep probabilistic model for dynamic systems forecasting.
problem Probabilistic forecasting in dynamic systems.
method Combining deep generative models and state space models with recurrent neural networks and variational sequence models.
result Outperforms existing models in system identification benchmarks and real-world centrifugal compressor forecasting.
Paper analyzes and compares ELF algorithms for federated learning.
problem Improving efficiency and privacy in federated learning.
method Proposes P-ELF, D-ELF, and B-ELF algorithms with primal, dual, and bidirectional compression.
result Provides non-asymptotic convergence guarantees under Log-Sobolev inequality.
The family of image visibility graphs (IVGs) have been recently introduced as simple algorithms by which scalar fields can be mapped into graphs. Here we explore the usefulness of such operator in the scenario of image processing and image classification. We demonstrate that the link architecture of the image visibilit…
TopoFisher learns topological summaries by maximizing Fisher information, improving parameter efficiency and inference quality.
problem Simulation-based inference misses key information in low-order statistics, especially for non-Gaussian fields.
method TopoFisher uses a differentiable persistent-homology pipeline that learns topological summaries by maximizing local Gaussian Fisher information.
result TopoFisher recovers much of the available information and outperforms fixed topological vectorizations in weak gravitational lensing.
The paper explores coalescent contractions in contractible spaces, providing criteria and examples.
problem Existence and absence of coalescent contractions in contractible spaces.
method Analysis of contractible finite simplicial complexes and criteria for coalescent contractions.
result Criteria for contractible finite simplicial complexes that ensure no coalescent contractions.
We study gradient compression methods to alleviate the communication bottleneck in data-parallel distributed optimization. Despite the significant attention received, current compression schemes either do not scale well or fail to achieve the target test accuracy. We propose a new low-rank gradient compressor based on …
Computable contracts simplify financial transactions and reduce legal costs.
problem Difficulty in querying, executing, and analyzing text-based financial contracts.
method Develop a Contract Definition Language and illustrate use cases.
result Substantial improvements in customer experience and cost reduction.
In an online contract selection problem there is a seller which offers a set of contracts to sequentially arriving buyers whose types are drawn from an unknown distribution. If there exists a profitable contract for the buyer in the offered set, i.e., a contract with payoff higher than the payoff of not accepting any c…
Optimal execution strategy for merger & acquisition contracts with price impact.
problem Optimal execution and pricing of financial derivatives in M&A deals.
method Indifference utility arguments, considering linear and nonlinear contracts.
result Linear contracts are more expensive and vulnerable to manipulation.
This paper develops a method to select a reference contract for multi-contract quoting to minimize execution risk.
problem Minimizing execution risk in multi-contract quoting sequences.
method Develops a diagnostic framework using order-flow Hawkes forecasts and CLF to select a stable reference contract.
result Event-history and LOB-state signals offer complementary views for reference-contract selection.
Proposes a probabilistic framework for smart contract risk quantification.
problem Quantifying financial risk of smart contract cyber attacks and failures.
method Probabilistic graph-theoretical framework using bond percolation models.
result Analytical results and numerical examples for aggregate loss distribution.
We consider a general framework of optimal mechanism design under adverse selection and ambiguity about the type distribution of agents. We prove the existence of optimal mechanisms under minimal assumptions on the contract space and prove that centralized contracting implemented via mechanisms is equivalent to delegat…
Improved security of smart contracts by classifying them into four categories.
problem Detecting and classifying vulnerabilities in smart contracts efficiently.
method Used AWD-LSTM for multi-class classification, addressing class imbalance.
result Achieved a weighted average Fbeta score of 90.0%.
Study on contracting maps and their rigidity under curvature constraints.
problem Rigidity of contracting maps between manifolds with positive curvature.
method Analysis of curvature pinching and contracting conditions involving singular values.
result Established the relation between curvature pinching and contracting conditions.
We study locally compact contractive local groups, that is, locally compact local groups with a contractive pseudo-automorphism. We prove that if such an object is locally connected, then it is locally isomorphic to a Lie group. We also prove a related structure theorem for locally compact contractive local groups whic…
Study shows some contractible complexes can't have certain immersions.
problem Understanding non-positive immersions in contractible complexes.
method Provided counterexamples to a conjecture by Wise.
result Some contractible complexes do not have non-positive immersions.
This paper presents some partial answers to the following question. QUESTION. If a normal space X is the union of an increasing sequence of open sets U(1), U(2), U(3) ... such that each U(n) contracts to a point in X, must X be contractible? The main results of the paper are: THEOREM 1. If a normal space X is the union…
The simplicial volume of non-R^3 contractible 3-manifolds is infinite.
problem Characterizing contractible 3-manifolds based on their simplicial volume.
method Analyzing the simplicial volume of contractible 3-manifolds and open 3-manifolds.
result The Euclidean space is the unique contractible 3-manifold with vanishing minimal volume.
Study on reinsurance decisions using mean-variance criterion with irreversible contracts.
problem Optimizing reinsurance premiums and contracts in a Stackelberg game with irreversible contracts.
method Unified singular control framework applied to both discrete and continuous time reinsurance contracts.
result A single once-for-all reinsurance contract is preferred over multiple contracts, and the signing time is crucial.
Optimal contracts help principals delegate data collection in decentralized ML.
problem Dealing with information asymmetries in decentralized ML.
method Design of optimal and near-optimal contracts addressing uncertainty in model quality and performance.
result Simple linear contracts achieve 1-1/e fraction of optimal utility.
Optimal contracts are found for agents with quadratic effort costs.
problem Finding optimal contracts in principal-agent problems with quadratic effort costs.
method Modeling the problem using Hamilton-Jacobi-Bellman (HJB) equations and proving the existence of classical solutions.
result Existence of optimal contracts for agents with quadratic effort costs is proven.
Fair insurance contracts are designed to handle default risk using cooperative game theory.
problem Designing fair insurance contracts in the presence of default risk.
method Cooperative game theory to specify premiums and participation in benefit.
result Fair benefit participation emerges as a game outcome involving residual risks.
Flora uses random projections to achieve high-rank updates with low memory usage.
problem Excessive memory usage in large neural networks during training.
method Flora approximates LoRA using random projections to enable high-rank updates with sublinear space complexity.
result Flora achieves high-rank updates with significantly reduced memory usage compared to LoRA.
One can define what it means for a compact manifold with corners to be a "contractible manifold with contractible faces." Two combinatorially equivalent, contractible manifolds with contractible faces are diffeomorphic if and only if their 4-dimensional faces are diffeomorphic. It follows that two simple convex polytop…
This paper investigates Pareto optimal (PO, for short) insurance contracts in a behavioral finance framework, in which the insured evaluates contracts by the rank-dependent utility (RDU) theory and the insurer by the expected value premium principle. The incentive compatibility constraint is taken into account, so the …
New mortgage contracts reduce underwater default by adjusting loan balances, but must balance prepayment incentives.
problem Underwater default incentives in mortgages.
method Analyzes automatic balance adjustment and prepayment penalties in mortgage contracts.
result Automatic balance adjustments are preferable to traditional contracts at certain spreads, reducing underwater default.
We define a new notion of contracting element of a group and we show that contracting elements coincide with hyperbolic elements in relatively hyperbolic groups, pseudo-Anosovs in mapping class groups, rank one isometries in groups acting properly on proper CAT(0) spaces, elements acting hyperbolically on the Bass-Serr…