Novel approach for forecasting financial time series using neural bag-of-features.
problem Forecasting financial time series with high-dimensionality, velocity, and variety.
method Temporal Logistic Neural Bag-of-Features combining deep neural networks.
result Demonstrated effectiveness on a large-scale financial time series dataset.
Feature learning and deep learning have drawn great attention in recent years as a way of transforming input data into more effective representations using learning algorithms. Such interest has grown in the area of music information retrieval (MIR) as well, particularly in music audio classification tasks such as auto…
BagNet classifies images using local features, making decisions clearer and more understandable.
problem Understanding how deep neural networks make decisions is difficult.
method BagNet is a variant of ResNet-50 that classifies images based on local features without spatial ordering.
result BagNet achieves high accuracy on ImageNet (87.6% top-5 for 33 x 33 px features).
Automatically classifying the tissues types of Region of Interest (ROI) in medical imaging has been an important application in Computer-Aided Diagnosis (CAD), such as classification of breast parenchymal tissue in the mammogram, classify lung disease patterns in High-Resolution Computed Tomography (HRCT) etc. Recently…
New kernels from neural networks show better performance than traditional methods.
problem Improving neural network performance on small datasets.
method Developed algebraic operations to create compositional kernels from neural network architectures.
result Compositional kernels achieve higher accuracy than neural tangent kernels and neural networks on small datasets.
Many objects in the real world are difficult to describe by a single numerical vector of a fixed length, whereas describing them by a set of vectors is more natural. Therefore, Multiple instance learning (MIL) techniques have been constantly gaining on importance throughout last years. MIL formalism represents each obj…
This paper presents a spermwhale' localization architecture using jointly a bag-of-features (BoF) approach and machine learning framework. BoF methods are known, especially in computer vision, to produce from a collection of local features a global representation invariant to principal signal transformations. Our idea …
In recent scene recognition research images or large image regions are often represented as disorganized "bags" of features which can then be analyzed using models originally developed to capture co-variation of word counts in text. However, image feature counts are likely to be constrained in different ways than word …
In multiple instance learning, objects are sets (bags) of feature vectors (instances) rather than individual feature vectors. In this paper we address the problem of how these bags can best be represented. Two standard approaches are to use (dis)similarities between bags and prototype bags, or between bags and prototyp…
Transformer learns hidden structure of EHR data for better prediction.
problem Lack of complete structure information in EHR data.
method Graph Convolutional Transformer using data statistics to learn structure.
result Consistently outperforms previous approaches on various prediction tasks.
In this paper, we deal with two challenges for measuring the similarity of the subject identities in practical video-based face recognition - the variation of the head pose in uncontrolled environments and the computational expense of processing videos. Since the frame-wise feature mean is unable to characterize the po…
New method preserves privacy by aggregating feature-vectors with weighted sums, ensuring label differential privacy.
problem Ensuring privacy in training data aggregation for sensitive labels.
method Learning from bag aggregates (LBA) with weighted Gaussian sums, preserving label differential privacy (label-DP).
result Weighted LBA using iid Gaussian weights with m m m randomly sampled disjoint k k k -sized bags provides ( ε , δ ) (\varepsilon, δ) ( ε , δ ) -label-DP. This work efficiently learns linear threshold functions from label proportions using Gaussian distributions.
problem Efficiently learning linear threshold functions from label proportions.
method Using Gaussian distributions, the algorithm estimates means and covariance matrices, and identifies a low error hypothesis LTF.
result It is possible to efficiently learn LTFs using LTFs when given access to random bags of label proportions.
Boosting weak learners to strong ones from aggregate labels is possible for LLP but not for MIL.
problem Boosting weak learners to strong ones from aggregate labels in learning from label proportions (LLP).
method Using a weak learner on large enough bags to obtain a strong learner for small bags in polynomial time.
result Boosting is possible for LLP but not for MIL.
Paper introduces method to make neural networks symmetrical.
problem Creating symmetrical neural networks for data with inherent symmetries.
method Introduces a method for modifying neural networks to enforce equivariance.
result Group convolutional neural networks are a special case of the introduced framework.
Proposes Neural SDE for better model robustness and generalization.
problem Missing regularization mechanisms in Neural ODE networks.
method Integrates various regularization mechanisms via stochastic noise injection.
result Improves robustness and generalization compared to Neural ODE.
Graphs of neural networks are represented to preserve symmetry, improving performance across various tasks.
problem Lack of equivariance in neural network representations of other neural networks.
method Represent neural networks as computational graphs and use graph neural networks to preserve permutation symmetry.
result Single model encodes diverse neural architectures, outperforming state-of-the-art methods.
Convolutional Neural Processes improve data efficiency in neural processes.
problem Improving data efficiency in neural processes for small datasets.
method Convolutional Neural Processes (ConvNPs) improve data efficiency by leveraging translation equivariance and convolutional neural networks.
result ConvNPs enhance the performance of neural processes in small-data problems.
Neural circuits integrate continuous dynamics efficiently.
problem Efficiently integrating continuous neural dynamics for simulation and learning.
method Compact neural circuits for Runge-Kutta and Adams-Bashforth-Moulton methods.
result Equivalence of neural and numerical integration for polynomial systems.
GNPs use graph neural networks to predict target points with uncertainty quantification.
problem Predicting points on graphs with uncertainty.
method Graph Neural Processes (GNP) that operate on graph data, taking context features and outputting a target point distribution.
result GNPs can quantify uncertainty in graph data predictions.
Investigates how neural network graph structure impacts predictive performance.
problem Lack of understanding between neural network graph structure and predictive performance.
method Developed relational graph representation to analyze neural networks, identifying a 'sweet spot' for improved performance.
result Identified a 'sweet spot' in relational graph structure that significantly improves neural network predictive performance.
Neural Tangents simplifies infinite-width neural networks for research.
problem Training and studying infinite-width neural networks.
method High-level API for specifying complex architectures, analytical or gradient-based training, and automatic distribution.
result Analytical training of infinite-width networks and automatic parallelization.
Novel framework explains generalization in deep neural networks.
problem Understanding and improving generalization in deep neural networks.
method Topological Quantum Neural Networks as the semi-classical limit of Deep Neural Networks.
result Demonstrates that the perceptron, viewed as the semi-classical limit, achieves similar results to standard neural networks without training.
This paper shows overparameterized deep neural networks are convex and learn useful features.
problem Analyzing fully trained overparameterized deep neural networks.
method Generalized neural feature repopulation technique.
result Overparameterized deep neural networks are inherently convex and learn useful features.
Augmented Neural ODEs improve stability and expressiveness of ODEs.
problem Limitations of Neural ODEs in representing complex functions.
method Introducing Augmented Neural ODEs that are more expressive and stable.
result Augmented Neural ODEs outperform Neural ODEs in stability, generalization, and computational efficiency.
Investigates neural codes and their embeddings, proving conjectures and introducing new code types.
problem Analyzing neural codes and their embedding dimensions.
method Combinatorial, topological, and algebraic analysis; proving conjectures; introducing new neural code types.
result Proves conjectures about neural codes and their embeddings, introduces new code types.
Neural networks can approximate functions uniformly across various measures.
problem Universal approximation of functions across different probability measures.
method Proving neural networks are dense in Orlicz spaces, extending classical theorems.
result Neural networks uniformly approximate functions for weakly compact families of measures.
Graph Metanetworks process diverse neural architectures efficiently.
problem Processing diverse neural architectures efficiently.
method Builds metanetworks using graph neural networks to process graphs representing input neural networks.
result Proves GMNs are expressive and equivariant to parameter permutation symmetries.
Quadratic models explain neural network behavior during training.
problem Understanding neural network dynamics during training with large learning rates.
method Developed and tested Neural Quadratic Models.
result Neural Quadratic Models exhibit the 'catapult phase' similar to neural networks.
Optimal rates for shallow ReLU networks in nonparametric regression.
problem Approximating smooth and non-smooth functions with shallow ReLU networks.
method Analysis of shallow ReLU k ^k k neural networks, using variation norms and deep learning theory. result Optimal approximation rates for shallow ReLU networks in nonparametric regression.
New metric compares noisy neural trajectories using optimal transport.
problem Existing metrics fail to capture differences in noisy, dynamic neural responses.
method Proposed an optimal transport distance metric for Gaussian processes.
result Metric effectively compares neural dynamics in different systems.
Paper analyzes Q-learning with neural networks, proving a fast convergence rate.
problem Analyzing the convergence rate of neural Q-learning.
method Finite-time analysis of neural Q-learning with a deep ReLU network.
result Neural Q-learning converges to optimal policy with O ( 1 / T ) O(1/\sqrt{T}) O ( 1/ T ) rate. Neural model guides PBE problem solving in programming.
problem Synthesizing programs from example inputs/outputs.
method Uses a neural model to guide miniKanren's constraint logic programming system.
result Synthesizes programs faster and generalizes to larger problems.
The neural tangent kernel equivalence theorem fails in practice.
problem Does the neural tangent kernel (NTK) equivalence theorem hold in practical neural network training?
method Rigorously derived NTK and conducted numerical experiments to evaluate the equivalence theorem.
result Adding a layer to a neural network and the corresponding updated NTK do not yield matching changes in predictor error.
The FAIRnets Ontology makes neural networks findable, accessible, interoperable, and reusable.
problem The resource-intensive training of neural networks and the lack of training data availability.
method Development of FAIRnets Ontology to model neural networks on a meta-level and creation of a knowledge graph (FAIRnets) of over 18,400 neural networks.
result The FAIRnets Ontology and knowledge graph enable the reuse and recommendation of neural networks to data scientists.
Equivariant neural networks use symmetry to interpret complex data.
problem Interpreting and understanding the behavior of equivariant neural networks.
method Decompose layers into simple representations and analyze nonlinear activation functions.
result Equivariant neural networks can be interpreted using a filtration generalizing Fourier series.
Two new criteria help understand the advantage of deep neural networks.
problem Understanding the advantage of deepening neural networks.
method Proposed two new criteria to evaluate the expressivity of functions computable by deep neural networks.
result Increasing layers is more effective than increasing units in improving the expressivity of deep neural networks.
Use simplified layerwise linear models to understand neural dynamics.
problem Complex neural network dynamics are hard to grasp.
method Apply simplified layerwise linear models to explain neural phenomena.
result Simplified models explain neural collapse, emergence, etc.
Paper finds the best way to estimate neural net distance from samples.
problem Estimating the neural net distance from samples.
method Developed minimax lower and upper bounds for the neural net distance.
result Lower and upper bounds match, validating the empirical neural net distance.
The paper proves consistency of neural networks with regularization.
problem Overfitting in neural networks with large scale data.
method Theoretical framework of neural networks with regularization, sieves method, and minimal neural networks theory.
result The estimated neural network converges to the true underlying function as sample size increases.
FGNN generalizes graph neural networks to capture higher-order dependencies.
problem Capturing higher-order dependencies in graph-structured data.
method Introducing a factor graph neural network (FGNN) that can represent Max-Product Belief Propagation.
result FGNN effectively represents Max-Product Belief Propagation and performs well on both synthetic and real datasets.
Neural networks explained through geometric projections.
problem Understanding the geometric and mathematical underpinnings of neural networks.
method Exploiting connections between integration, Radon transforms, and neural networks.
result Distribution of neural network outputs can be interpreted as nonlinear projections along hypersurfaces.
Study of deep neural networks' NTK evolution during training.
problem Understanding the performance gap between deep neural networks and kernel regression.
method Derive an infinite hierarchy of ordinary differential equations (NTH) to capture gradient descent dynamics of deep neural networks.
result Truncated NTH approximates the dynamic of the NTK up to arbitrary precision under certain conditions.
This study shows why training Neural ODEs is hard and proposes a new method.
problem Training Neural ODEs is challenging, especially in practice.
method Proposed a new stabilization method and provided an analytical convergence analysis.
result Insights and techniques for researchers starting work on Neural ODEs.
Paper benchmarks quantum neural networks against classical ones for binary classification tasks.
problem Comparing quantum neural networks with classical ones for binary classification.
method Evaluated with two toy examples, focusing on model complexity and training data size.
result EQNN and QNN outperform ENN and DNN for smaller parameter sets and training data samples.
Neural persistence measures network complexity using topology.
problem Lack of measures for characterizing and monitoring structural properties of neural networks.
method Topological data analysis on weighted stratified graphs.
result Neural persistence reflects best practices and can shorten training time.
Affine spiking neural networks learn efficiently and generalize well.
problem Learning with spiking neural networks, especially with positive weights.
method Affine encoders and decoders, continuous parameter dependence, gradient-based training.
result Affine spiking neural networks can approximate shallow ReLU networks and generalize well.
Deep neural network detects heart murmur with high accuracy.
problem Detecting heart murmur from heart sound recordings.
method Parallel combination of RNN-BiLSTM and CNN.
result 96-100% sensitivity and specificity, 98% F1 score.