Why does Deep Learning work? What representations does it capture? How do higher-order representations emerge? We study these questions from the perspective of group theory, thereby opening a new approach towards a theory of Deep learning. One factor behind the recent resurgence of the subject is a key algorithmic step…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,742 papers · 148 categories
Trend · papers per month
48 results for “Layer Repeatability”
Why does Deep Learning work? What representations does it capture? How do higher-order representations emerge? We study these questions from the perspective of group theory, thereby opening a new approach towards a theory of Deep learning. One factor behind the recent resurgence of the subject is a key algorithmic step…
A new neural network initialization method is proposed for faster and more accurate training.
problem Efficient initialization for training multi-layer feedforward neural networks.
method Initialization based on Stein's identity, using eigenvectors of cross-moment matrix.
result The SteinGLM method is faster and more accurate than other initialization methods.
We present a general framework for training deep neural networks without backpropagation. This substantially decreases training time and also allows for construction of deep networks with many sorts of learners, including networks whose layers are defined by functions that are not easily differentiated, like decision t…
A deep convolutional fuzzy system (DCFS) on a high-dimensional input space is a multi-layer connection of many low-dimensional fuzzy systems, where the input variables to the low-dimensional fuzzy systems are selected through a moving window across the input spaces of the layers. To design the DCFS based on input-outpu…
Extreme learning machine (ELM) is a new single hidden layer feedback neural network. The weights of the input layer and the biases of neurons in hidden layer are randomly generated, the weights of the output layer can be analytically determined. ELM has been achieved good results for a large number of classification ta…
The success of deep neural networks has inspired many to wonder whether other learners could benefit from deep, layered architectures. We present a general framework called forward thinking for deep learning that generalizes the architectural flexibility and sophistication of deep neural networks while also allowing fo…
We propose a neural architecture search (NAS) algorithm, Petridish, to iteratively add shortcut connections to existing network layers. The added shortcut connections effectively perform gradient boosting on the augmented layers. The proposed algorithm is motivated by the feature selection algorithm forward stage-wise …
Differentiable cutting-plane layers solve parametric mixed-integer linear optimization problems.
problem Solving parametric mixed-integer linear optimization problems with changing data.
method Introducing cutting-plane layers (CPLs) for differentiable cutting-plane generation.
result The algorithm computes solutions with low integrality gaps and generalizes to unseen instances.
DLBricks automates DL benchmarking on CPUs, reducing effort and time.
problem Lack of representative and up-to-date DL benchmarks on CPUs.
method Decomposes DL models into runnable networks, leveraging layer repetition and auto-generation.
result Accurately estimates DL model performance and speeds up benchmarking time.
In this paper, we made an extension to the convergence analysis of the dynamics of two-layered bias-free networks with one output. We took into consideration two popular regularization terms: the and norm of the parameter vector , and added it to the square loss function with coefficient $λ/…
This paper finds a new way to compress CNN weights, improving on pruning and quantization.
problem Improving performance and storage efficiency of CNNs.
method Identifying and exploiting repeated patterns in CNN weight tensors, using Huffman coding and block sparse matrix formats.
result Achieved compaction ratios of 1.4x to 3.1x in addition to pruning and quantization.
The performance of graph neural nets (GNNs) is known to gradually decrease with increasing number of layers. This decay is partly attributed to oversmoothing, where repeated graph convolutions eventually make node embeddings indistinguishable. We take a closer look at two different interpretations, aiming to quantify o…
A usual reinsurance policy for insurance companies admits one or two layers of the payment deductions. Under optimal criterion of minimizing the conditional tail expectation (CTE) risk measure of the insurer's total risk, this article generalized an optimal stop-loss reinsurance policy to an optimal multi-layer reinsur…
FSNet improves online time series forecasting by balancing fast adaptation and old knowledge.
problem Online time series forecasting challenges in handling abrupt and recurring patterns.
method Inspired by CLS theory, FSNet uses a dynamic balance between fast adaptation and old knowledge retrieval.
result FSNet achieves robustness to both new and recurring patterns through dynamic balancing and associative memory.
LIDS assesses LLM summaries with interpretable key words.
problem Challenges in evaluating the quality of LLM summaries.
method BERT-SVD-based direction metric and SOFARI for key word extraction.
result LIDS provides interpretable key words for layered themes.
This study analyzes why attention layers in neural networks can cause signal loss and proposes a solution.
problem Pathological behavior of attention layers in neural networks, leading to signal loss.
method Spectral analysis using Random Matrix Theory to identify and mitigate rank collapse in width.
result A novel solution to mitigate rank collapse in width by removing outlier eigenvalues.
Optimal strategies are found for a repeated betting game using diffusion approximation.
problem Finding optimal strategies for a repeated betting game with i.i.d. outcomes.
method Constructing a diffusion approximation of the repeated game and analyzing the wealth share process.
result Necessary and sufficient conditions for the wealth share process to be transient or recurrent are derived.
Convolutional networks struggle with repeating patterns in ECGs.
problem Modeling repeating patterns in electrocardiogram signals.
method Demonstrated through ECG examples, highlighting systemic issues in deep learning.
result Counterintuitive effects on generalization in deep networks.
Study explores algorithmic collusion in repeated games using various learning dynamics.
problem Understanding algorithmic collusion in repeated games with different learning dynamics.
method Examines -learning, gradient learning, and other dynamics in a general repeated game setting.
result Characterizes the set of payoff vectors achievable by these dynamics, revealing possibilities for collusion.
We study two systems of tangle equations that arise when modeling the action of the Integrase family of proteins on DNA. These two systems--direct and inverted repeats--correspond to two different possibilities for the initial DNA sequence. We present one new class of solutions to the tangle equations. In the case of i…
Scorio.jl ranks systems from repeated tasks using various methods.
problem Evaluating and ranking systems from repeated responses to shared tasks.
method Common tensor-based interface for multiple ranking methods.
result Pilot experiments show stability and runtime scaling.
We investigate the question of when distinct branched surfaces in the complement of a 2-bridge knot support essential surfaces with identical boundary slopes. We determine all instances in which this occurs and identify an infinite family of knots for which no boundary slopes are repeated.
LAFF algorithm balances adaptability and non-exploitability in repeated games.
problem Low regret in repeated games against unknown opponent classes.
method LAFF algorithm searches within sub-algorithms optimal for each opponent class and uses a punishment policy for exploitation.
result LAFF guarantees sublinear regret uniformly over possible opponents, except exploitative ones, for which it guarantees linear regret.
Link concordance and Whitney towers linked to Milnor invariants.
problem Link concordance and Whitney towers classification.
method Clasper surgeries, Whitney towers, and Milnor invariants.
result Link concordance and Whitney towers classified in terms of Milnor invariants.
Risk is part of the fabric of every business; surprisingly, there is little work on establishing best practices for systematic, repeatable risk identification, arguably the first step of any risk management process. In this paper, we present a proposal that constitutes a more holistic risk management approach, a method…
It has long been known that a Milnor invariant with no repeated index is an invariant of link homotopy. We show that Milnor's invariants with repeated indices are invariants not only of isotopy, but also of self C_k-moves. A self C_k-move is a natural generalization of link homotopy based on certain degree k clasper su…
RSO uses random weight perturbations to train deep networks without gradients.
problem Training deep neural networks efficiently and without gradient information.
method RSO is a gradient-free Markov Chain Monte Carlo approach that updates weights based on mini-batch loss reduction.
result RSO achieves high accuracy (99.1% on MNIST) with significantly fewer updates than traditional methods.
We investigate some geometric properties of the real algebraic variety of symmetric matrices with repeated eigenvalues. We explicitly compute the volume of its intersection with the sphere and prove a Eckart-Young-Mirsky-type theorem for the distance function from a generic matrix to points in . We exhibit conne…
This paper tackles the reduction of redundant repeating generation that is often observed in RNN-based encoder-decoder models. Our basic idea is to jointly estimate the upper-bound frequency of each target vocabulary in the encoder and control the output words based on the estimation in the decoder. Our method shows si…
Machine learning models for repeated measurements are limited. Using topological data analysis (TDA), we present a classifier for repeated measurements which samples from the data space and builds a network graph based on the data topology. When applying this to two case studies, accuracy exceeds alternative models wit…
Study optimizes product assortment for retailers with repeated exposures and patience costs.
problem Optimizing product assortment for online retailers with repeated exposures and varying consumer patience.
method Developed a cascade multinomial logit model to capture repeated exposures and patience costs.
result Proposed an approximation solution to the assortment optimization problem.
Graph Neural Networks (GNNs) are based on repeated aggregations of information across nodes' neighbors in a graph. However, because common neighbors are shared between different nodes, this leads to repeated and inefficient computations. We propose Hierarchically Aggregated computation Graphs (HAGs), a new GNN graph re…
Generative compression technique reduces neural network size and improves performance on microcontrollers.
problem Memory constraints on microcontrollers limit the size of neural networks, especially for 1x1 pointwise (PW) mixers.
method HYPER-TINYPW uses a shared micro-MLP to generate PW kernels from tiny per-layer codes, reducing memory usage.
result HYPER-TINYPW achieves comparable performance to larger models while being significantly smaller (225 kB vs 1.4 MB).
We present Sequential Attend, Infer, Repeat (SQAIR), an interpretable deep generative model for videos of moving objects. It can reliably discover and track objects throughout the sequence of frames, and can also generate future frames conditioning on the current frame, thereby simulating expected motion of objects. Th…
Study analyzes broker's gain from trade in repeated context-based trading.
problem Maximizing traders' net utility in repeated context-based brokerage.
method Proposes algorithms achieving tight regret bounds in full and limited feedback settings.
result Achieves tight 1/2-approximation result for gain from trade.
A new method combines machine learning with mixed-effects models for better repeated measurement analysis.
problem Inference of linear coefficients in partially linear mixed-effects models with complex interactions and high-dimensional variables.
method Double machine learning approach to estimate nonparametrically nonlinear variables, then use standard linear mixed-effects techniques to estimate the linear coefficient.
result The estimated fixed effects coefficient converges at the parametric rate and is semiparametrically efficient.
We propose a random convolutional neural network to generate a feature space in which we study image classification and retrieval performance. Put briefly we apply random convolutional blocks followed by global average pooling to generate a new feature, and we repeat this k times to produce a k-dimensional feature spac…
New neural network processes 3D volumes with improved equivariance.
problem Improving neural network performance on 3D volumes with symmetries.
method Equivariant neural network using moving frames approach.
result Trained model outperforms benchmarks in medical volume classification.
This paper examines transitions in sniping behavior among algorithmic traders, finding new profitable strategies.
problem Understanding transitions from sure to probabilistic sniping in competitive algorithmic trading environments.
method Reinterpretation and extension of Menkveld and Zoican's stylized game, analysis of repeated games, sequential statistical testing.
result Probabilistic sniping can be profitable in certain conditions, resembling the prisoner's dilemma.
Repeated self-distillation improves model performance significantly.
problem How much gain is possible by applying multiple steps of self-distillation?
method Investigated linear regression tasks, applied multiple steps of self-distillation, analyzed excess risk reduction.
result Multi-step self-distillation reduces excess risk by a factor as large as , where is the input dimension.
Comparing Deep Learning Models for the Task of Volatility Prediction Using Multivariate Dataq-fin.ST
Study compares deep learning models for volatility prediction using multivariate data.
problem Predicting volatility using multivariate data.
method Evaluated multiple deep learning models including MLP, RNN, TCN, and Temporal Fusion Transformer.
result Temporal Fusion Transformer and TCN variants outperform classical models and shallow networks.
XSPNs combine SPNs and MEVMs for efficient inference in data with repeated parts.
problem Efficient inference in data with repeated interchangeable parts.
method Introducing Exchangeability-Aware Sum-Product Networks (XSPNs) that combine SPNs and MEVMs.
result XSPNs can be more accurate than conventional SPNs when data contains repeated parts.
Ribbon: Scalable Approximation and Robust Uncertainty Quantification
problem Reliably quantifying predictive uncertainty for complex models
method Ribbon, a scalable approximation to Dirichlet-reweighted bootstrap uncertainty
result Asymptotically equivalent to a flat-prior Laplace approximation under correct likelihood specification, recovers robust sandwich covariance under misspecification
Serverless cloud computing speeds up double machine learning model estimation.
problem Efficiently estimating double machine learning models with minimal cloud resource management.
method Serverless computing with AWS Lambda for repeated cross-fitting.
result Demonstrates significant reduction in estimation times and costs.
The paper uses machine learning to optimize rework policies in semiconductor manufacturing.
problem Optimizing rework steps to increase yield without increasing costs.
method Applied double/debiased machine learning (DML) to estimate treatment effects.
result Derived optimal rework policies and estimated their value empirically.
Origami uses SGX enclaves and blinding to protect deep neural network inference privacy.
problem Protecting deep neural network inference privacy in machine learning services.
method Combines enclave execution, cryptographic blinding, and accelerator-based computation.
result Demonstrates improved privacy-preserving inference performance compared to prior work.
This research extends the Pareto/NBD model using neural networks for better out-of-sample predictions.
problem The limitations of the Pareto/NBD model in predicting out-of-sample data.
method A neural network-based extension of the Pareto/NBD model.
result The proposed method shows extraordinary predictability on repeat purchases at individual and aggregate levels.