A Power Transformcs.LG
Novel power transform unifies various mathematical functions.
problem Normalizing and standardizing datasets.
method Presented a novel power transform.
result Unified various mathematical functions.
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,695 papers · 148 categories
Trend · papers per month
48 results for “power transformations”
Looped transformers with LN converge to power method for principal component prediction.
problem Understanding how transformers learn algorithmic procedures.
method Study of principal component prediction with looped linear transformers and layer normalization.
result Gradient descent trains looped transformers with LN to implement the power method for principal component prediction.
Study reveals Transformer's expressive power and mechanisms.
problem Understanding the approximation properties of Transformer for sequence modeling.
method Systematic study of Transformer's components and their combined effects, establishing approximation rates.
result Reveals roles of critical parameters in Transformer, such as number of layers and attention heads.
Power Transformer Fault Diagnosis with Intrinsic Time-scale Decomposition and XGBoost Classifiercs.LG
A method uses ITD and XGBoost for precise power transformer fault diagnosis.
problem Fault diagnosis of power transformers using DGA data.
method Ranking DGA parameters by skewness, extracting ITD features, and using an XGBoost classifier.
result The method achieves over 95% accuracy in classification.
Transformers with CoT don't enhance reasoning power across all tasks.
problem Does CoT enhance the reasoning power of transformers?
method Examined the memorization capabilities of fixed-precision transformers with and without CoT.
result Transformers with CoT cannot memorize all reasoning tasks, leading to a negative answer.
The paper examines how nonlinear transformations affect ridge sets in manifold learning.
problem Understanding the impact of nonlinear transformations on ridge sets in manifold learning.
method Examined the effects of nonlinear transformations on ridge sets using mathematical proofs and numerical experiments.
result The inclusion relationship $\cR(f\circ p)\subseteq \cR(p)$ holds for strictly increasing and concave transformations, and the Hausdorff distance between transformed and non-transformed ridge sets is smaller.
Researchers prove Transformers are Turing-complete and analyze their components.
problem Understanding the computational power and limitations of Transformers.
method Analyzed Turing-completeness of vanilla and modified Transformers, and necessity of components.
result Transformers with positional masking and positional encodings are Turing-complete.
Transformers model contextual relations using probabilistic measures, revealing their expressive power.
problem Lack of clear understanding of Transformer's ability to model contextual relations.
method Introduced a measure-theoretic framework connecting softmax attention and entropy-regularized optimal transport.
result Transformer architectures can approximate arbitrary contextual relations, and the choice of normalization affects how these relations are represented.
Beyond Black-Box Predictions: Identifying Marginal Feature Effects in Tabular Transformer Networkscs.LG
Proposes a tabular transformer model to maintain feature effect intelligibility.
problem Losing marginal feature effects in deep tabular transformer networks.
method Adapts tabular transformer networks to identify marginal feature effects.
result The model accurately identifies marginal feature effects, matching black-box performance while maintaining intelligibility.
The paper explores transformations between power law problems and geodesics on cones.
problem Solving power law problems and understanding their geometric properties.
method Geometric transformations and cone metrics.
result Derivation of Maclaurin duality and Jacobi-Maupertuis metric reformulation.
New theory explains signal propagation in normalization-free transformers.
problem Understanding signal propagation in normalization-free transformers.
method Deriving recurrence relations for activation statistics and APJNs across layers.
result Transformers with elementwise tanh-like nonlinearities exhibit subcritical signal propagation.
New analysis shows RPE-based Transformers can't approximate all functions.
problem Understanding the limitations of RPE-based Transformers in approximating continuous functions.
method Mathematical analysis and development of a novel attention module (URPE) to overcome limitations.
result RPE-based Transformers can't approximate all continuous sequence-to-sequence functions, even with depth and width.
Training only BatchNorm parameters achieves surprisingly high performance in deep networks.
problem Understanding the role and expressive power of affine parameters in BatchNorm.
method Investigating performance when training only BatchNorm parameters and freezing all other weights.
result BatchNorm achieves high performance by learning to disable around a third of random features.
The study explains transformer scaling laws using statistical and approximation theories.
problem Understanding why transformer scaling laws exist for large models trained on low-dimensional data.
method Established statistical estimation and mathematical approximation theories for transformers on low-dimensional manifolds.
result Predicted a power law between generalization error and model and data sizes, with power depending on intrinsic data dimension.
A novel hybrid data-driven approach is developed for forecasting power system parameters with the goal of increasing the efficiency of short-term forecasting studies for non-stationary time-series. The proposed approach is based on mode decomposition and a feature analysis of initial retrospective data using the Hilber…
CaTs use DAGs with transformers to enforce causal constraints, improving neural network robustness.
problem Neural networks lack inherent causal structure respect, leading to reliability issues.
method Introducing Causal Transformers (CaTs) that operate under predefined causal constraints specified by DAGs.
result CaTs improve robustness and interpretability of neural networks under causal constraints.
We simplify neural networks to 3D to study their topological changes.
problem Understanding how neural network layers affect low-dimensional topological invariants.
method Limiting each layer to a width of 3D space, tracking changes in linking numbers.
result ResNets and transformers are equally powerful in changing linking numbers.
Characterizes bi-Perron numbers with specific Galois conjugates.
problem Understanding bi-Perron numbers with real or unimodular conjugates.
method Characterization through power properties and spectral radii of transformations.
result Bi-Perron numbers with real or unimodular conjugates admit a power as stretch factors or spectral radii.
Classifies conformal transformations in spacetimes without observer horizons.
problem Understanding conformal transformations in spacetimes without observer horizons.
method Proves classification of conformal transformations into two types: escaping and non-escaping.
result Conformal transformations of Einstein's static universe are classified.
We reduce boundary determination of an unknown function and its normal derivatives from the (possibly weighted and attenuated) broken ray data to the injectivity of certain geodesic ray transforms on the boundary. For determination of the values of the function itself we obtain the usual geodesic ray transform, but for…
It is now widely accepted that volatility models have to incorporate the so-called leverage effect in order to to model the dynamics of daily financial returns.We suggest a new class of multivariate power transformed asymmetric models. It includes several functional forms of multivariate GARCH models which are of great…
Long and short memory in economic processes is usually described by the so-called discrete fractional differencing and fractional integration. We prove that the discrete fractional differencing and integration are the Grunwald-Letnikov fractional differences of non-integer order d. Equations of ARIMA(p,d,q) and ARFIMA(…
A simple log-transform fixes heavy-tailed data for generative models.
problem Standard generative models struggle with heavy-tailed data.
method Apply the soft-log transform to data before training and exponentiate samples after generation.
result Log-FM outperforms specialized baselines on multivariate benchmarks.
Variational autoencoder is a powerful deep generative model with variational inference. The practice of modeling latent variables in the VAE's original formulation as normal distributions with a diagonal covariance matrix limits the flexibility to match the true posterior distribution. We propose a new transformation, …
Develops European power option pricing under correlated interest rate and asset processes.
problem Pricing European power options under correlated interest rate and asset processes.
method Martingale method and Girsannov transform.
result Derives European power option pricing formulae under two market assumptions.
A new graph signature invariant to graph automorphisms.
problem Graph symmetry and feature generation.
method Power spectrum signature derived from squared graph Fourier transform.
result Power spectrum signature is stable under graph perturbations.
In this paper, we propose an improvement of Adversarial Transformation Networks(ATN) to generate adversarial examples, which can fool white-box models and black-box models with a state of the art performance and won the 2rd place in the non-target task in CAAD 2018.
Probabilistic thermal stability prediction through sparsity promoting transformer representationstat.ML
Paper improves Tm prediction of protein fragments using sparsity and probabilistic models.
problem Improving accuracy of melting temperature prediction for protein fragments.
method Promoting sparsity in pre-trained transformer models and adopting probabilistic frameworks.
result Mean absolute error of 0.23C for predicting melting temperature.
Fine-tuning normalization layers can reconstruct smaller networks.
problem Understanding the expressive power of fine-tuning normalization layers.
method Random ReLU networks and sparsified networks were fine-tuned to reconstruct target networks.
result Fine-tuning normalization layers can reconstruct networks that are times smaller.
The paper proposes a method to identify power system oscillation modes using blind source separation.
problem Accurately identifying oscillation modes in power systems with renewable energy sources.
method A high-order blind source identification (HOBI) algorithm based on copula statistic combined with Hilbert transform and iteration procedure.
result The method can identify all oscillation modes and model order from a single channel of observation signals, outperforming state-of-the-art methods.
GradPower speeds up language model training with minimal code changes.
problem Slower training of large language models.
method Elementwise sign-power transformation applied to gradients.
result Consistently lower terminal loss across various models and datasets.
Study reveals neural scaling laws in random graphs and natural language models.
problem Understanding the origin of neural scaling laws in complex systems.
method Examined scaling laws in transformers trained on random walks and simplified natural language models.
result Neural scaling laws emerge in the absence of power law structure in data correlations.
GCWSNet improves neural network training speed and accuracy with power transformation.
problem Training deep neural networks efficiently and accurately.
method Developed GCWS for hashing powered-GMM kernel, enabling power transformation on data.
result GCWSNet often improves classification accuracy and converges faster with one epoch.
ViLT is a faster vision-and-language model without convolution or region supervision.
problem Efficiency and expressive power limitations in current VLP models.
method A convolution-free Vision-and-Language Transformer (ViLT) that processes visual inputs similarly to textual inputs.
result ViLT is up to tens of times faster with competitive or better performance.
This work shows how transformers use multi-concept word semantics for efficient in-context learning.
problem Understanding the connection between transformer-based LLMs' multi-concept semantic representation and their innovative in-context learning abilities.
method A concept-based low-noise sparse coding prompt model, leveraging advanced techniques to analyze the exponential convergence of 0-1 loss over non-convex training dynamics.
result Transformers leverage multi-concept word semantics to enable powerful and excellent out-of-distribution in-context learning.
Structured CNN designed using the prior information of problems potentially improves efficiency over conventional CNNs in various tasks in solving PDEs and inverse problems in signal processing. This paper introduces BNet2, a simplified Butterfly-Net and inline with the conventional CNN. Moreover, a Fourier transform i…
Novel privatization framework for high-dimensional variable selection with differential privacy.
problem High-dimensional controlled variable selection with rigorous FDR control under differential privacy constraints.
method Gaussian Johnson-Lindenstrauss Transformation for privatizing the knockoff matrix.
result The proposed private variable selection procedure maintains statistical power even under strict privacy budgets.
Transformers can learn noisy linear systems with depth and IID data.
problem Learning noisy linear dynamical systems with transformers.
method Theoretical analysis of multi-layer and single-layer transformers with respect to -testing loss.
result Single-layer transformers have a non-diminishing lower bound on approximation error, suggesting depth separation.
Transforms any test into anytime-valid with sample savings.
problem Sequential data invalidates classical test guarantees.
method Predicts test outcomes to create anytime-valid stopping rules.
result Ensures Type-I error control and near-optimal power.
A new algorithm reduces communication in distributed SVD by factors.
problem Efficiently compute SVD in distributed systems.
method LocalPower algorithm with weighted aggregation and periodic decay of iterations.
result Reduces communication cost by a factor of .
Our aim is to prove that two formal power series of importance to quantum topology are Gevrey. These series are the Kashaev invariant of a knot (reformulated by Huynh and the second author) and the Gromov norm of the LMO of an integral homology 3-sphere. It follows that the power series associated to a simple Lie algeb…
Sequence models assign probabilities to variable-length sequences such as natural language texts. The ability of sequence models to capture temporal dependence can be characterized by the temporal scaling of correlation and mutual information. In this paper, we study the mutual information of recurrent neural networks …
Transformers can interpolate between arbitrary measures.
problem Understanding the expressive power of Transformers as measure-to-measure maps.
method Provided an explicit choice of parameters for a single Transformer to match N arbitrary input measures to N arbitrary target measures.
result A single Transformer can interpolate between arbitrary measures.
The complex wave representation (CWR) converts unsigned 2D distance transforms into their corresponding wave functions. Here, the distance transform S(X) appears as the phase of the wave function φ(X)---specifically, φ(X)=exp(iS(X)/τwhere τis a free parameter. In this work, we prove a novel result using the higher-orde…
XR-Transformer accelerates XMC by recursively fine-tuning on multi-resolution objectives.
problem Efficiently classifying texts with large label sets.
method Recursive multi-resolution fine-tuning of transformers.
result XR-Transformer achieves 20x faster training time and 54% Precision@1 on Amazon-3M.
New mesh network preserves symmetries in deep learning.
problem No existing mesh processing architecture is equivariant to all symmetries.
method Equivariant attention-based mesh network using relative tangential features.
result The network achieves improved performance and is equivariant to various transformations.
LoRA enhances model adaptability without increasing parameters.
problem Theoretical understanding of LoRA's adaptability.
method Theoretical analysis of LoRA's expressive power.
result LoRA can adapt any model to a smaller target model with a specific rank.
Proves resurgence properties for Habiro elements from radial limits of theta series.
problem Proving resurgence properties for Habiro elements.
method Using strange identities and Borel transform of formal power series.
result Proves Costin and Garoufalidis conjecture for two families of torus knots.