The Interaction-Transformation (IT) is a new representation for Symbolic Regression that restricts the search space into simpler, but expressive, function forms. This representation has the advantage of creating a smoother search space unlike the space generated by Expression Trees, the common representation used in Ge…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,742 papers · 148 categories
Trend · papers per month
48 results for “expressive capability”
Recently, deep learning has achieved huge successes in many important applications. In our previous studies, we proposed quadratic/second-order neurons and deep quadratic neural networks. In a quadratic neuron, the inner product of a vector of data and the corresponding weights in a conventional neuron is replaced with…
New pruning method retains model expressiveness for NLP tasks.
problem Pruning large pretrained transformer models for real-world deployment.
method Mixture Gaussian Prior Pruning (MGPP) algorithm.
result MGPP outperforms existing pruning methods in high sparsity settings.
SimCD simultaneously clusters cells and identifies differential gene expression in scRNA-seq data.
problem Separate clustering and differential expression analysis for scRNA-seq data leads to suboptimal results.
method Develops SimCD, a unified hierarchical gamma-negative binomial model for simultaneous cell clustering and differential expression analysis.
result SimCD outperforms existing methods in discovering cell clusters and capturing dynamic expression changes.
A new tensor ring mixture model improves density estimation efficiency.
problem Efficient probability density estimation in statistical machine learning.
method Tensor ring decomposition with mixture model for adaptive weights.
result Enhanced expressive capability and flexibility in density estimation.
We propose a Text-to-Speech method to create an unseen expressive style using one utterance of expressive speech of around one second. Specifically, we enhance the disentanglement capabilities of a state-of-the-art sequence-to-sequence based system with a Variational AutoEncoder (VAE) and a Householder Flow. The propos…
Transformers struggle to approximate smooth functions, relying on piecewise constant approximations.
problem Understanding the expressivity of Transformers for function approximation.
method Theoretical analysis and experimental validation of Transformer's ability to approximate smooth functions.
result Transformers cannot reliably approximate smooth functions, relying on piecewise constant approximations.
This work explores the relationship between expressivity and generalization in GNNs.
problem Understanding the trade-off between expressivity and generalization in GNNs.
method Introducing a novel framework that connects GNN generalization to the variance in graph structures they can capture.
result Theoretical findings align with empirical results, offering a deeper understanding of how expressivity enhances GNN generalization.
The paper proposes a new probability distribution for rooted trees.
problem Overfitting in tree selection for statistical models.
method Bayesian approach with a generalized probability distribution for rooted trees.
result Recursive methods to evaluate the probability distribution without approximations.
HDNNs can approximate any continuous function, proving their expressivity.
problem Lack of a comprehensive study on the expressivity of HDNNs.
method Discretization of Hamiltonian Neural Ordinary Differential Equations (HNN-ODEs).
result HDNNs can approximate any continuous function over a compact domain.
GNNs with random node initialization are shown to be universally expressive.
problem Limitations of standard GNNs in distinguishing graphs.
method Random node initialization (RNI) to enhance GNNs' expressive power.
result GNNs with RNI are proven to be universally expressive.
The need to efficiently calculate first- and higher-order derivatives of increasingly complex models expressed in Python has stressed or exceeded the capabilities of available tools. In this work, we explore techniques from the field of automatic differentiation (AD) that can give researchers expressive power, performa…
GFN-SR uses deep learning to generate diverse mathematical expressions.
problem Symbolic regression to find best mathematical expressions.
method Traversing a DAG to generate expression trees sequentially with GFlowNet.
result GFN-SR outperforms other SR algorithms in noisy data.
Analytic expressions for deep neural network output under stochastic training.
problem Understanding the impact of noise and hyperparameters on deep neural network performance.
method Taylor expansion of network output to derive analytical expressions for weights and output.
result Noise in training affects generalization by preventing the output from fully converging on train data, but does not provide explicit regularization.
Enhances graph neural networks with spectral and topological information.
problem Improving graph neural networks' expressivity beyond Weisfeiler-Leman hierarchy.
method Integrates spectral information into Persistent Homology diagrams.
result SpectRe is strictly more expressive than PH and spectral information alone.
Kernelised Normalising Flowsstat.ML
Kernelised flows improve density estimation and generation with fewer parameters.
problem Limited expressiveness of flow-based models due to invertibility constraints.
method Integrates kernels into normalising flows to enhance expressiveness and efficiency.
result Kernelised flows outperform neural network-based flows in parameter efficiency and low-data scenarios.
Graph Neural Networks (GNNs) have achieved much success on graph-structured data. In light of this, there have been increasing interests in studying their expressive power. One line of work studies the capability of GNNs to approximate permutation-invariant functions on graphs, and another focuses on the their power as…
Quantum machine learning models can approximate any continuous function.
problem Theoretical understanding of quantum feature maps in machine learning.
method Proving universal approximation property of quantum machine learning models in quantum-enhanced feature spaces.
result Quantum machine learning models are universal approximators of continuous functions.
Simplified NAS for GNN architectures improves efficiency and expressiveness.
problem Efficient and effective discovery of optimal GNN architectures.
method SNAG framework with a novel search space and reinforcement learning.
result SNAG framework outperforms human-designed and existing NAS methods.
Differential Expression Analysis of Dynamical Sequencing Count Data with a Gamma Markov Chainstat.AP
Next-generation sequencing (NGS) to profile temporal changes in living systems is gaining more attention for deriving better insights into the underlying biological mechanisms compared to traditional static sequencing experiments. Nonetheless, the majority of existing statistical tools for analyzing NGS data lack the c…
Deep neural networks can approximate complex functions through repeated compositions of a fixed-size ReLU network.
problem Understanding the expressive power of deep neural networks through function compositions.
method Demonstrated the surprising expressive power of repeated compositions of a single fixed-size ReLU network.
result Repeated compositions of a single fixed-size ReLU network can approximate 1-Lipschitz continuous functions on with an error .
Inspired by the immense success of deep learning, graph neural networks (GNNs) are widely used to learn powerful node representations and have demonstrated promising performance on different graph learning tasks. However, most real-world graphs often come with high-dimensional and sparse node features, rendering the le…
In this work we present a modified neural network model which is capable to simulate Markov Chains. We show how to express and train such a network, how to ensure given statistical properties reflected in the training data and we demonstrate several applications where the network produces non-deterministic outcomes. On…
Paper proposes dp-VAE for preserving spatial context in gene expression data.
problem Inaccessibility of spatial context in single-cell gene expression data.
method Generic representation learning and transfer learning framework with a distance-preserving regularizer.
result dp-VAE effectively reconstructs and imputes spatial context from gene expression data.
This work analyzes how deep neural networks' expressiveness increases with depth and width.
problem Understanding the expressiveness of deep neural networks (DNNs) based on their Lipschitz constants.
method Leveraging random matrix theory, the study characterizes the expressiveness of DNNs by their Lipschitz constant, showing exponential and polynomial increases with depth and width, respectively.
result The expressiveness of DNNs increases exponentially with depth and polynomially with width, consistent with function approximation benefits.
TensorGuide improves LoRA efficiency and expressivity through joint tensor-train optimization.
problem Limited expressivity and generalization of standard LoRA.
method TensorGuide uses a unified tensor-train structure with controlled Gaussian noise to generate correlated low-rank matrices.
result TensorGuide achieves superior accuracy and scalability with fewer parameters compared to standard LoRA and TT-LoRA.
eXponential FAmily Dynamical Systems (XFADS): Large-scale nonlinear Gaussian state-space modelingstat.ML
A new method learns complex dynamical systems from data efficiently.
problem Learning complex dynamical systems from large-scale data efficiently.
method Low-rank structured variational autoencoding framework for nonlinear Gaussian state-space models.
result Consistently demonstrates better predictive capabilities compared to other models.
In this paper we introduce a novel online time series forecasting model we refer to as the pM-GP filter. We show that our model is equivalent to Gaussian process regression, with the advantage that both online forecasting and online learning of the hyper-parameters have a constant (rather than cubic) time complexity an…
PLN-Nets with two linear layers and parallel LN achieve universal approximation.
problem Limitations of standard neural network architectures in universal approximation.
method Introduced PLN-Nets combining two linear layers with parallel LN.
result PLN-Nets achieve universal approximation, while standard LN has limited power.
The scaled complex Wishart distribution is a widely used model for multilook full polarimetric SAR data whose adequacy has been attested in the literature. Classification, segmentation, and image analysis techniques which depend on this model have been devised, and many of them employ some type of dissimilarity measure…
Quantized neural networks can represent all fixed-point functions under certain conditions.
problem Expressive power of quantized neural networks under fixed-point arithmetic.
method Analyzing necessary and sufficient conditions for quantized networks to represent all fixed-point functions.
result Various popular activation functions satisfy the sufficient condition for representing all fixed-point functions.
New neural networks with variable time constants for better time-series prediction.
problem Improving neural network performance in time-series prediction.
method Constructing networks of linear dynamical systems modulated by nonlinear gates, using numerical differential equation solvers.
result Liquid Time-Constant Networks (LTCs) yield superior performance on time-series prediction tasks.
SYNTHONY: A Stress-Aware, Intent-Conditioned Agent for Deep Tabular Generative Models Selectioncs.LG
SYNTHONY selects tabular synthesizers based on stress profiling and user intent.
problem Non-uniform performance of tabular generative models across datasets.
method Stress profiling and intent-conditioned tabular synthesis selection.
result Meta-features predict synthesizer performance, improving selection accuracy.
RNNs struggle with in-context retrieval, while Transformers excel.
problem In-context retrieval capability of RNNs.
method Theoretical analysis and experimental techniques (CoT, RAG, Transformer layer).
result Enhancing RNNs with techniques improves their in-context retrieval capability, closing the representation gap with Transformers.
In biological research machine learning algorithms are part of nearly every analytical process. They are used to identify new insights into biological phenomena, interpret data, provide molecular diagnosis for diseases and develop personalized medicine that will enable future treatments of diseases. In this paper we (1…
BFNs use Bayesian inference and neural networks for generative modeling.
problem Learning from non-stationary data in continual learning.
method Bayesian Flow Networks (BFNs) combining neural network expressiveness and Bayesian inference.
result BFNs effectively model non-stationary data.
To deepen our understanding of graph neural networks, we investigate the representation power of Graph Convolutional Networks (GCN) through the looking glass of graph moments, a key property of graph topology encoding path of various lengths. We find that GCNs are rather restrictive in learning graph moments. Without c…
Toward a Next Generation Particle Precipitation Model: Mesoscale Prediction Through Machine Learning (a Case Study and Framework for Progress)physics.space-ph
New model predicts particle precipitation from magnetosphere to ionosphere.
problem Improving prediction of electron particle precipitation from magnetosphere to ionosphere.
method Compilation of new database, use of machine learning (ML) tools, neural network (PrecipNet).
result PrecipNet achieves >50% reduction in errors and better captures dynamic changes.
A new method STMF improves missing value prediction using tropical semiring.
problem Limited capability of linear models to model complex relations.
method Sparse Tropical Matrix Factorization (STMF) using tropical semiring.
result STMF outperforms NMF on real data, especially in handling extreme values.
This work explores test-time scaling strategies for LLMs, improving sample efficiency and expressiveness.
problem Understanding the sample efficiency and expressiveness of test-time scaling strategies for LLMs.
method Established separation and expressiveness results for self-consistency, best-of-, and self-correction strategies.
result Self-correction enables Transformers to simulate online learning over multiple tasks without prior knowledge.
A Novel Efficient Approach with Data-Adaptive Capability for OMP-based Sparse Subspace Clusteringcs.LG
Orthogonal Matching Pursuit (OMP) plays an important role in data science and its applications such as sparse subspace clustering and image processing. However, the existing OMP-based approaches lack of data adaptiveness so that the data cannot be represented well enough and may lose the accuracy. This paper proposes a…
RGCF improves collaborative filtering by refining graph convolution embeddings.
problem GCN-based recommendation models introduce noise and redundancy, limiting high-order connectivity capture.
method Developed RGCF, a new GCN-based Collaborative Filtering model with redesigned embeddings.
result RGCF significantly outperforms state-of-the-art models on public datasets.
Conditional DGP learns effective kernels from low-fidelity data.
problem Learning effective kernels for multi-fidelity regression.
method Conditional DGP with moment matching for implicit kernel approximation.
result Effective kernels are learned from lower-fidelity data, improving multi-fidelity regression.
Casting neural networks in generative frameworks is a highly sought-after endeavor these days. Contemporary methods, such as Generative Adversarial Networks, capture some of the generative capabilities, but not all. In particular, they lack the ability of tractable marginalization, and thus are not suitable for many ta…
Transformers can approximate posterior predictive distributions through in-context learning.
problem Bayesian prediction tasks, especially beyond point predictions.
method Gradient descent algorithm targeting posterior predictive mean and variance, followed by nonlinear mappings.
result Transformers can implement algorithms to approximate posterior predictive distributions.
Generalizes Hasimoto transformation to arbitrary flows on space curves.
problem Modeling and analyzing wave motions on space curves.
method Develops a mapping between curve evolution and scalar equations, transforming general vector fields in the Frenet frame.
result Binormal flows are generally length preserving but bending energy is fragile.
Survey interprets foundation models' inner workings.
problem Limited explainability of black-box FMs.
method Interpretable methods to reveal FMs' mechanisms.
result Identifies next research directions for FMs.
Tractable yet expressive density estimators are a key building block of probabilistic machine learning. While sum-product networks (SPNs) offer attractive inference capabilities, obtaining structures large enough to fit complex, high-dimensional data has proven challenging. In this paper, we present random sum-product …