Torch-Points3D simplifies 3D deep learning research and reproducibility.
problem Lack of transparency and reproducibility in 3D deep learning research.
method Modular framework with quality-of-life features, standardized protocols, and open-source implementation.
result Facilitates fair and rigorous evaluation of 3D deep learning methods.
Torch-Struct simplifies structured prediction for deep learning.
problem Difficulty in integrating structured prediction algorithms with deep learning frameworks.
method Develops a library (Torch-Struct) that integrates structured prediction with vectorized, auto-differentiation-based frameworks.
result Significant performance gains over fast baselines and cross-algorithm efficiency.
ξ-torch simplifies physics-informed learning by providing differentiable functionals.
problem Training physics-informed deep neural networks requires differentiable physical simulations.
method ξ-torch offers a library of differentiable functionals for scientific simulations.
result Improves numerical stability and reduces memory requirements for higher order derivatives.
mlr3torch simplifies deep learning in R for tabular and image data.
problem Simplifying deep learning workflows in R for various data types.
method Built on torch, integrates mlr3 ecosystem, supports custom architectures.
result Extensible framework for tabular and image data, including preprocessing and model definition.
R package innsight interprets deep neural networks predictions.
problem Interpreting predictions of deep neural networks.
method Unified and user-friendly framework implementing feature attribution methods for neural networks, independent of deep learning library.
result Offers a variety of visualization tools for tabular, signal, image data or a combination.
This report provides an introduction to some Machine Learning tools within the most common development environments. It mainly focuses on practical problems, skipping any theoretical introduction. It is oriented to both students trying to approach Machine Learning and experts looking for new frameworks.
After the torch of Anders Kock [Taylor series calculus for ring objects of line type, Journal of Pure and Applied Algebra, 12 (1978), 271-293], we will establish the Baker-Campbell-Hausdorff formula as well as the Zassenhaus formula in the theory of Lie groups.
New algorithms reduce communication in GNN training.
problem Higher communication costs in GNNs due to sparse connectivity.
method Parallel algorithms for sparse-dense matrix multiplication.
result Asymptotic reduction in communication compared to previous methods.
PyTorch adds tools for pruning neural networks.
problem Model size and resource constraints in machine learning.
method Pruning techniques to reduce model size and capacity.
result Facilitates adoption of pruning in PyTorch.
In recent years, many publications showed that convolutional neural network based features can have a superior performance to engineered features. However, not much effort was taken so far to extract local features efficiently for a whole image. In this paper, we present an approach to compute patch-based local feature…
Packed-Ensembles improve uncertainty estimation in constrained hardware.
problem Hardware limitations restrict the size of ensembles and network capacity, degrading performance.
method Packed-Ensembles (PE) design and train lightweight structured ensembles by modulating encoding space and parallelizing into a single backbone.
result PE accurately preserves diversity and maintains performance on key metrics like accuracy, calibration, and out-of-distribution detection.
Fast-vollib offers high-performance option pricing and IV computation.
problem Efficiently pricing and computing implied volatility for financial models.
method Open-source Python library with PyTorch, JAX, and CUDA backends, implementing Halley and LBR algorithms.
result High-performance option pricing and IV computation with vectorized implementations.
Development systems for deep learning (DL), such as Theano, Torch, TensorFlow, or MXNet, are easy-to-use tools for creating complex neural network models. Since gradient computations are automatically baked in, and execution is mapped to high performance hardware, these models can be trained end-to-end on large amounts…
DA-GNN improves robustness of GNNs by modeling noise dependencies.
problem Real-world graph node features often contain noise, leading to performance degradation in GNNs.
method DA-GNN captures noise dependencies using variational inference and new benchmark datasets.
result DA-GNN consistently outperforms existing baselines across various noise scenarios.
Most distributed machine learning systems nowadays, including TensorFlow and CNTK, are built in a centralized fashion. One bottleneck of centralized algorithms lies on high communication cost on the central node. Motivated by this, we ask, can decentralized algorithms be faster than its centralized counterpart? Althoug…
nn2poly converts neural networks into interpretable polynomial models.
problem Interpreting complex neural networks.
method NN2Poly method for converting neural networks into polynomial models.
result Captures variable interactions and provides interpretable coefficients.
ACA method improves gradient estimation for neural ODEs, reducing error and training time.
problem Inaccurate gradient estimation methods hinder the performance of neural ODEs on benchmark tasks.
method Adaptive Checkpoint Adjoint (ACA) method that applies trajectory checkpointing, deletes redundant components, and supports adaptive solvers.
result ACA reduces error rate by half and training time by half compared to adjoint and naive methods on image classification tasks.
Study compares metric learning loss functions for speaker verification.
problem Comparing metric learning loss functions for end-to-end speaker verification.
method Cross entropy loss, cosine loss, angular margin loss, center loss, contrastive loss, triplet loss.
result Additive angular margin loss outperforms other loss functions.
Probabilistic programs with dynamic computation graphs can define measures over sample spaces with unbounded dimensionality, which constitute programmatic analogues to Bayesian nonparametrics. Owing to the generality of this model class, inference relies on `black-box' Monte Carlo methods that are often not able to tak…