The process of designing neural architectures requires expert knowledge and extensive trial and error. While automated architecture search may simplify these requirements, the recurrent neural network (RNN) architectures generated by existing methods are limited in both flexibility and components. We propose a domain-s…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,742 papers · 148 categories
Trend · papers per month
48 results for “flexible neural architectures”
AutoShrink optimizes neural architectures by shrinking cell structures.
problem Resource constraints in deploying DNNs on mobile devices.
method Topology-aware node-based Neural Architecture Search (NAS).
result AutoShrink achieves up to 48% parameter reduction and 34% MACs savings.
The use of automatic methods, often referred to as Neural Architecture Search (NAS), in designing neural network architectures has recently drawn considerable attention. In this work, we present an efficient NAS approach, named HM- NAS, that generalizes existing weight sharing based NAS approaches. Existing weight shar…
DMAE uses neural networks to cluster data with flexible dissimilarity functions.
problem Clustering data with complex dissimilarity functions.
method Integrates a dissimilarity mixture model into deep learning architectures.
result DMAE achieves competitive clustering accuracy compared to other methods.
Smooth neural TPPs using B-splines for better efficiency and accuracy.
problem Efficiently modeling sequences of events in continuous time with neural networks.
method Directly parametrize the CIF as a non-negative combination of B-spline basis functions, predicting coefficients with a neural network.
result Improved computational efficiency and predictive accuracy compared to existing methods.
Labels distilled from images improve model training efficiency and flexibility.
problem Creating synthetic labels for a small set of real images to train models effectively.
method Introduce a more robust and flexible meta-learning algorithm for distillation and an effective first-order strategy based on convex optimization layers.
result Label distillation leads to improved results and greater flexibility in neural architectures.
A theoretical performance analysis of the graph neural network (GNN) is presented. For classification tasks, the neural network approach has the advantage in terms of flexibility that it can be employed in a data-driven manner, whereas Bayesian inference requires the assumption of a specific model. A fundamental questi…
Novel neural GP kernels learn stable, flexible covariance structures.
problem Scalable and flexible covariance kernels for Gaussian processes.
method Directly learn kriging coefficients and conditional standard deviations using deep neural architectures exploiting permutation-equivariant structure.
result Improved training stability and data efficiency with expressive, non-stationary kernels.
DNAS disentangles neural architecture search for better interpretability and performance.
problem Lack of interpretability in existing neural architecture search methods.
method DNAS disentangles the hidden representation of the controller into semantically meaningful concepts.
result DNAS achieves state-of-the-art performance and competitive architectures.
Neural architecture search (NAS) is gaining more and more attention in recent years due to its flexibility and remarkable capability to reduce the burden of neural network design. To achieve better performance, however, the searching process usually costs massive computations that might not be affordable for researcher…
LEMs extend transformer-based architectures for complex execution problems.
problem Handling flexible time boundaries and multiple execution constraints in deep learning.
method Decouples market information processing from execution allocation decisions using TKANs, VSNs, and multi-head attention mechanisms.
result LEMs achieve superior execution performance compared to traditional benchmarks.
Neural networks have proven to be extremely powerful tools for modern artificial intelligence applications, but computational and storage complexity remain limiting factors. This paper presents two compatible contributions towards reducing the time, energy, computational, and storage complexities associated with multil…
LassoFlexNet improves deep learning performance on tabular data.
problem Deep learning underperforms tree-based models on tabular data.
method Incorporates five inductive biases and uses Tied Group Lasso for variable selection.
result LassoFlexNet matches or outperforms leading tree-based models on 52 datasets.
Neural networks fit fewer samples than their parameters suggest in practice.
problem Understanding the practical limitations of neural network flexibility.
method Examination of neural network optimization, parameter efficiency, and loss surfaces.
result Neural networks can only fit training sets with significantly fewer samples than their parameters suggest.
TyXe enables flexible Bayesian neural networks in Pytorch.
problem Uncertainty estimation in neural networks.
method Separates architecture, prior, inference, and likelihood specification; modular choices for priors, guides, and inference techniques.
result Minimal modifications to existing code for Bayesian neural networks.
New neural networks combine additive regression with traditional architectures.
problem Performance limitations and high parameter requirements of traditional neural networks.
method Introduce hybrid deep additive neural networks with simpler activation and basis functions.
result Hybrid neural networks achieve better performance with fewer parameters.
Develops approximately equivariant neural processes for better data modeling.
problem Real-world data often breaks exact equivariance; how to model this?
method General approach to creating approximately equivariant architectures, applicable to any model and symmetry group.
result Approximately equivariant neural processes outperform non-equivariant and strictly equivariant models in regression tasks.
Flexible deep learning models for dynamic accuracy and speed trade-offs.
problem Dynamic accuracy and speed trade-offs in real-world applications.
method Training deep neural networks with a new method allowing flexible numerical precision during inference.
result Achieved comparable accuracy to dedicated models trained at the same precision with dynamic precision settings.
Proposes continuous convolution layers for flexible feature map resizing.
problem Fixed stride limitations in discrete convolution layers.
method Introduces Continuous Convolution (CC) layers that use learned continuous functions.
result Dynamic and consistent resizing of feature maps at any scale, non-integer and axis-dependent.
einspace expands NAS search space to include diverse neural architectures.
problem NAS results are often limited to existing structures; new designs are rare.
method einspace uses a probabilistic context-free grammar to create a versatile search space.
result einspace discovers novel and improved architectures, including convolutions and attention.
Graph neural nets improve discrete choice modeling with network effects.
problem Modeling network effects in discrete choice problems.
method Graph Convolutional Neural Network (GCNN) architecture.
result Higher predictive performance than standard models with interpretability.
Graph convolutional networks adapt the architecture of convolutional neural networks to learn rich representations of data supported on arbitrary graphs by replacing the convolution operations of convolutional neural networks with graph-dependent linear operations. However, these graph-dependent linear operations are d…
Learning workable representations of dynamical systems is becoming an increasingly important problem in a number of application areas. By leveraging recent work connecting deep neural networks to systems of differential equations, we propose \emph{variational integrator networks}, a class of neural network architecture…
Rational neural networks approximate functions more efficiently with less depth.
problem Choosing optimal nonlinear activation functions in neural networks.
method Rational activation functions with optimal bounds and efficiency proofs.
result Rational neural networks approximate smooth functions more efficiently than ReLU networks with exponentially smaller depth.
Adapts deep learning with kernel methods for efficient learning.
problem Combining kernel methods and deep learning for efficient learning.
method Nyström approximation of kernel functions in neural networks.
result Performance comparable to standard architectures on datasets like SVHN and CIFAR100.
Information in neural networks is represented as weighted connections, or synapses, between neurons. This poses a problem as the primary computational bottleneck for neural networks is the vector-matrix multiply when inputs are multiplied by the neural network weights. Conventional processing architectures are not well…
No methods currently exist for making arbitrary neural networks fair. In this work we introduce GRAD, a new and simplified method to producing fair neural networks that can be used for auto-encoding fair representations or directly with predictive networks. It is easy to implement and add to existing architectures, has…
Consider the multivariate nonparametric regression model. It is shown that estimators based on sparsely connected deep neural networks with ReLU activation function and properly chosen network architecture achieve the minimax rates of convergence (up to -factors) under a general composition assumption on the re…
This study compares various superlearner and deep learning architectures (machine-learning-based and neural-network-based) for classification problems across several simulated and industrial datasets to assess performance and computational efficiency, as both methods have nice theoretical convergence properties. Superl…
Convolutional neural networks (CNNs) are effective at solving difficult problems like visual recognition, speech recognition and natural language processing. However, performance gain comes at the cost of laborious trial-and-error in designing deeper CNN architectures. In this paper, a genetic programming (GP) framewor…
Although various linear log-distance path loss models have been developed, advanced models are requiring to more accurately and flexibly represent the path loss for complex environments such as the urban area. This letter proposes an artificial neural network (ANN) based multi-dimensional regression framework for path …
TAAN model learns optimal network architecture for MTL tasks.
problem Improving generalization performance of MTL by finding flexible and accurate shared architecture.
method TAAN model with flexible activation functions and functional regularization.
result TAAN and regularization methods improve MTL performance.
Neural Local Wasserstein Regression models distribution-on-distribution regression with flexible, localized transport maps.
problem Estimating distribution-on-distribution regression with global optimal transport maps or linearization limitations.
method Proposes Neural Local Wasserstein Regression, a flexible nonparametric framework using locally defined transport maps in Wasserstein space.
result Demonstrates effective capture of nonlinear and high-dimensional distributional relationships.
PSI models and infers feature attributions efficiently and accurately.
problem Modeling and inferring feature attributions in flexible predictive models.
method Probabilistic Shapley inference (PSI) framework using latent random variables and a masking-based neural network architecture.
result PSI learns feature attribution distributions centered at Shapley values, revealing meaningful uncertainty.
A neural network model tackles high-dimensional data with latent structures.
problem Modeling high-dimensional data with latent low-dimensional structures.
method Integrates PCA and Soft PCA layers into neural network architecture for factor modeling and non-linear transformations.
result Demonstrates improved performance in forecasting and nowcasting with real-world data.
New LSTM scheme incorporates prior knowledge and measurement uncertainties.
problem Overfitting and insufficient data for accurate time-dependent solutions.
method Sparse Bayesian training algorithm for automatic connection determination.
result Less prone to overfitting, smaller data set required for satisfying accuracy.
Specialized Deep Learning (DL) acceleration stacks, designed for a specific set of frameworks, model architectures, operators, and data types, offer the allure of high performance while sacrificing flexibility. Changes in algorithms, models, operators, or numerical systems threaten the viability of specialized hardware…
To understand the fundamental trade-offs between training stability, temporal dynamics and architectural complexity of recurrent neural networks~(RNNs), we directly analyze RNN architectures using numerical methods of ordinary differential equations~(ODEs). We define a general family of RNNs--the ODERNNs--by relating t…
Proposes a new method to optimize graph neural network architectures on heterogeneous information networks.
problem Weaknesses in instability and inflexibility of existing graph neural architecture search methods.
method Partial Message Meta Multigraph search (PMMM) using a differentiable framework to search for a meaningful meta multigraph.
result Significantly more stable and effective than state-of-the-art heterogeneous GNNs.
Boosts share routing for multi-task learning with flexible sparse connections.
problem Designing suitable sharing mechanisms among multiple tasks in multi-task learning.
method Proposes MTNAS framework to modularize sharing into sub-networks with sparse connections and gating.
result Demonstrates consistent improvement over single-task and typical multi-task methods while maintaining efficiency.
This paper uses graph convolutional networks to improve the accuracy of neural architecture search.
problem Improving the precision of sampled sub-networks in weight-sharing NAS.
method Training a graph convolutional network to fit the performance of sampled sub-networks.
result Achieved higher rank correlation coefficient and better final architecture performance.
Recent dialogue approaches operate by reading each word in a conversation history, and aggregating accrued dialogue information into a single state. This fixed-size vector is not expandable and must maintain a consistent format over time. Other recent approaches exploit an attention mechanism to extract useful informat…
This paper compares deeper and wider neural networks for optimal generalization error in Sobolev losses.
problem The dilemma of choosing between deeper or wider neural networks for optimal generalization error.
method Analytical investigations into the influence of sample points, parameters, and loss function regularity on neural network architecture.
result A higher number of parameters favors wider neural networks, while more sample points and greater loss function regularity favor deeper neural networks.
SDQL uses modular deep Q networks to efficiently learn multi-stage optimal control tasks.
problem Training complex deep reinforcement learning models for multi-stage control tasks is inefficient and unstable.
method Stacked Deep Q Learning (SDQL) with modular Q networks and backward training.
result SDQL efficiently learns optimal control policies for multi-stage tasks with high-dimensional state and action spaces.
Study models weather index insurance pricing by insurers and farmers, finding flexible pricing kernels boost profits.
problem Monopoly pricing of weather index insurance with risk and flexibility considerations.
method Bowley-type sequential game with insurer and farmer, using neural networks for farmer's payoff.
result Flexible pricing kernels increase insurer profits closer to indemnity insurance levels.
Many engineers wish to deploy modern neural networks in memory-limited settings; but the development of flexible methods for reducing memory use is in its infancy, and there is little knowledge of the resulting cost-benefit. We propose structural model distillation for memory reduction using a strategy that produces a …
Neural network learns its size and structure during training.
problem Adapting neural network architecture to specific datasets.
method Flexible setup allowing neural network to learn size and topology during training.
result Trained networks achieve virtually identical performance and have learned optimal structure.
Enhances speech emotion recognition by adapting to varying time scales.
problem Robust emotion recognition from speech audio with temporal variations.
method Introduces multi-time-scale (MTS) convolutional layers to CNNs.
result MTS layers improve generalization, especially on smaller datasets.