We present differentiable particle filters (DPFs): a differentiable implementation of the particle filter algorithm with learnable motion and measurement models. Since DPFs are end-to-end differentiable, we can efficiently train their models by optimizing end-to-end state estimation performance, rather than proxy objec…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,742 papers · 148 categories
Trend · papers per month
48 results for “end-to-end differentiable”
We provide an end-to-end differentially private spectral algorithm for learning LDA, based on matrix/tensor decompositions, and establish theoretical guarantees on utility/consistency of the estimated model parameters. The spectral algorithm consists of multiple algorithmic steps, named as "{edges}", to which noise cou…
BPQP improves efficiency of differentiable optimization layers for deep learning.
problem Efficiency in differentiating optimization problems for deep learning models.
method Reformulates optimization problems as quadratic programming, enabling efficient gradient calculation.
result Significant improvement in efficiency (order of magnitude faster) compared to other differentiable optimization layers.
Embedding layers are commonly used to map discrete symbols into continuous embedding vectors that reflect their semantic meanings. Despite their effectiveness, the number of parameters in an embedding layer increases linearly with the number of symbols and poses a critical challenge on memory and storage constraints. I…
Differentiable clustering method using perturbed spanning forests.
problem Efficient clustering in trainable pipelines with noisy data.
method Stochastic perturbations of minimum-weight spanning forests.
result Method performs well even in challenging settings.
This paper introduces the Differentiable Algorithm Network (DAN), a composable architecture for robot learning systems. A DAN is composed of neural network modules, each encoding a differentiable robot algorithm and an associated model; and it is trained end-to-end from data. DAN combines the strengths of model-driven …
Differentiable perturbed optimizers enable end-to-end learning of discrete decisions.
problem Discrete decisions in machine learning pipelines break back-propagation.
method Transform optimizers into differentiable operations using stochastically perturbed optimizers.
result Smoothness of derivatives can be tuned via noise amplitude.
Differentiable resampling improves particle filter performance.
problem Non-differentiability of traditional resampling in particle filters.
method Introduced a neural network resampler (particle transformer) trained with a likelihood-based loss function.
result Learned resampling outperforms traditional methods on synthetic and real-world tasks.
Perceive, Predict, and Plan: Safe Motion Planning Through Interpretable Semantic Representationscs.RO
End-to-end learnable network for safer self-driving with interpretable intermediate representations.
problem Safe motion planning for self-driving vehicles.
method Differentiable semantic occupancy representation for cost calculation in motion planning.
result Significantly outperforms state-of-the-art planners in imitating human behaviors and producing safer trajectories.
Deep NLP models benefit from underlying structures in the data---e.g., parse trees---typically extracted using off-the-shelf parsers. Recent attempts to jointly learn the latent structure encounter a tradeoff: either make factorization assumptions that limit expressiveness, or sacrifice end-to-end differentiability. Us…
Paper proposes an end-to-end learning method for state estimation in robotics.
problem Lack of annotated data for optimising dynamic and measurement models in particle filters.
method End-to-end learning objective based on maximising a pseudo-likelihood function.
result Improves state estimation when large portions of true states are unknown.
End-to-end learning refers to training a possibly complex learning system by applying gradient-based learning to the system as a whole. End-to-end learning system is specifically designed so that all modules are differentiable. In effect, not only a central learning machine, but also all "peripheral" modules like repre…
We present foundations for using Model Predictive Control (MPC) as a differentiable policy class for reinforcement learning in continuous state and action spaces. This provides one way of leveraging and combining the advantages of model-free and model-based approaches. Specifically, we differentiate through MPC by usin…
Deep neural networks work well at approximating complicated functions when provided with data and trained by gradient descent methods. At the same time, there is a vast amount of existing functions that programmatically solve different tasks in a precise manner eliminating the need for training. In many cases, it is po…
The idea of end-to-end learning of communication systems through neural network-based autoencoders has the shortcoming that it requires a differentiable channel model. We present in this paper a novel learning algorithm which alleviates this problem. The algorithm enables training of communication systems with an unkno…
Alternating Direction Method of Multipliers (ADMM) is a widely used tool for machine learning in distributed settings, where a machine learning model is trained over distributed data sources through an interactive process of local computation and message passing. Such an iterative process could cause privacy concerns o…
Although recent work in AI has made great progress in solving large, zero-sum, extensive-form games, the underlying assumption in most past work is that the parameters of the game itself are known to the agents. This paper deals with the relatively under-explored but equally important "inverse" setting, where the param…
Due to the rapid growth of machine learning tools and specifically deep networks in various computer vision and image processing areas, application of Convolutional Neural Networks for watermarking have recently emerged. In this paper, we propose a deep end-to-end diffusion watermarking framework (ReDMark) which can be…
End-to-end trainable graph matching using improved combinatorial solvers.
problem Graph matching in deep learning.
method Combining deep learning with optimized combinatorial solvers.
result Advances state-of-the-art on deep graph matching benchmarks.
Machine reading using differentiable reasoning models has recently shown remarkable progress. In this context, End-to-End trainable Memory Networks, MemN2N, have demonstrated promising performance on simple natural language based reasoning tasks such as factual reasoning and basic deduction. However, other tasks, namel…
This paper presents KeypointNet, an end-to-end geometric reasoning framework to learn an optimal set of category-specific 3D keypoints, along with their detectors. Given a single image, KeypointNet extracts 3D keypoints that are optimized for a downstream task. We demonstrate this framework on 3D pose estimation by pro…
A new differentiable resampling method for Monte Carlo simulations.
problem Improving the efficiency and differentiability of resampling in Monte Carlo simulations.
method Proposes a diffusion model surrogate for resampling, proving consistency and outperforming existing methods.
result The proposed method outperforms state-of-the-art differentiable resampling methods on various benchmarks.
Connectionist temporal classification (CTC) is widely used for maximum likelihood learning in end-to-end speech recognition models. However, there is usually a disparity between the negative maximum likelihood and the performance metric used in speech recognition, e.g., word error rate (WER). This results in a mismatch…
NOVAS: Non-convex Optimization via Adaptive Stochastic Search for End-to-End Learning and Controlcs.LG
NOVAS uses adaptive stochastic search for non-convex optimization in deep networks.
problem Non-convex optimization challenges in deep neural networks.
method Adaptive stochastic search for non-convex optimization.
result NOVAS outperforms existing alternatives in a structured prediction task.
Paper optimizes KWS models using NAS and quantization for limited resources.
problem Developing efficient keyword spotting models in resource-constrained environments.
method Neural Architecture Search (NAS) for model structure optimization and quantization of weights and activations.
result Achieved high accuracy (95.55%) with minimal parameters and operations using NAS and quantization.
Private learning needs more data or better features.
problem Improving differentially private machine learning performance.
method Demonstrates the need for either more private data or better features.
result Private learning requires either more data or better features.
Development systems for deep learning (DL), such as Theano, Torch, TensorFlow, or MXNet, are easy-to-use tools for creating complex neural network models. Since gradient computations are automatically baked in, and execution is mapped to high performance hardware, these models can be trained end-to-end on large amounts…
Unified framework for differentiable graph partitioning with probabilistic cuts.
problem Lack of general guarantees and principled gradients in prior probabilistic relaxations of graph cuts.
method Unified probabilistic framework covering a wide class of cuts, including Normalized Cut, with tight analytic upper bounds.
result Rigorous, numerically stable foundation for scalable, differentiable graph partitioning.
DPLTM uses private winners to improve privacy and utility.
problem Balancing privacy and utility in machine learning models.
method DPLTM applies differential privacy to the lottery ticket method.
result DPLTM converges faster and uses less privacy budget.
Real-world applications often involve domain-specific and task-based performance objectives that are not captured by the standard machine learning losses, but are critical for decision making. A key challenge for direct integration of more meaningful domain and task-based evaluation criteria into an end-to-end gradient…
Deep equilibrium models estimate latent variables from data.
problem Estimating latent variables from data.
method Generalized exponential family models, deep equilibrium networks.
result Deep equilibrium models solve MAP estimates for latent and transformation parameters.
TomOpt: Differential optimisation for task- and constraint-aware design of particle detectors in the context of muon tomographyphysics.ins-det
TomOpt optimizes muon detector designs using differentiable programming.
problem Designing efficient particle detectors for muon tomography.
method Differentiable programming for muon interaction modeling, inference, and optimisation.
result Demonstrated end-to-end differentiable and inference-aware optimisation of particle physics instruments.
Improves regression accuracy by using multiple discrete representations.
problem Improving regression accuracy using deep learning.
method Proposes using multiple discrete representations simultaneously for regression problems.
result Reduces prediction error compared to a baseline RvC approach.
The idea of end-to-end learning of communications systems through neural network -based autoencoders has the shortcoming that it requires a differentiable channel model. We present in this paper a novel learning algorithm which alleviates this problem. The algorithm iterates between supervised training of the receiver …
We study the cross-entropy method (CEM) for the non-convex optimization of a continuous and parameterized objective function and introduce a differentiable variant that enables us to differentiate the output of CEM with respect to the objective function's parameters. In the machine learning setting this brings CEM insi…
Worldsheet wraps a 3D mesh sheet onto a single image to synthesize novel views.
problem Synthesizing novel views from a single image with large viewpoint changes.
method Shrink-wrapping a planar mesh sheet onto the input image, consistent with learned depth.
result Worldsheet consistently outperforms prior methods on single-image view synthesis.
We present DiffTaichi, a new differentiable programming language tailored for building high-performance differentiable physical simulators. Based on an imperative programming language, DiffTaichi generates gradients of simulation steps using source code transformations that preserve arithmetic intensity and parallelism…
Cilia are hairlike structures protruding from nearly every cell in the body. Diseases known as ciliopathies, where cilia function is disrupted, can result in a wide spectrum of disorders. However, most techniques for assessing ciliary motion rely on manual identification and tracking of cilia; this process is laborious…
Automated multi-task learning algorithm that optimizes network topology.
problem Over-sharing in multi-task learning leads to over-generalization and suboptimal performance.
method Tree-structured design space with gumbel-softmax sampling for differentiable network splitting.
result End-to-end trainable algorithm that optimizes network topology for multiple objectives across tasks.
Real-world applications often combine learning and optimization problems on graphs. For instance, our objective may be to cluster the graph in order to detect meaningful communities (or solve other common graph optimization problems such as facility location, maxcut, and so on). However, graphs or related attributes ar…
Differentiable pipeline replaces non-differentiable CAE components for shape optimization.
problem Gradient-based optimization is limited by non-differentiable components in CAE workflows.
method Surrogate models replace non-differentiable pipeline components, enabling gradient-based optimization.
result Gradient-based shape optimization possible without differentiable solvers.
DiPS learns to optimize sketching policies for better recommendation quality.
problem Optimizing sketching policies for long-term user interest prediction in recommender systems.
method Differentiable policy for sketching that learns from training data.
result DiPS requires up to 50% fewer sketch items to achieve the same recommendation quality.
Myriad offers a testbed for integrating machine learning and trajectory optimization.
problem Real-world trajectory optimization challenges.
method JAX-based testbed with machine learning and trajectory optimization integration.
result End-to-end learning and planning with neural ODEs.
dynoNet learns dynamical systems using linear operators.
problem Learning complex dynamical systems.
method dynoNet uses linear dynamical operators for sequence modeling and system identification.
result dynoNet effectively identifies systems on benchmarks.
Hybrid model for online nonlinear prediction using LSTM and soft GBDT.
problem Online nonlinear prediction with manual feature selection and model selection issues.
method End-to-end architecture with LSTM for feature extraction and soft GBDT for regression, jointly optimized.
result Significant performance improvements over conventional methods on real datasets.
Differentially-Private Logistic Regression for Detecting Multiple-SNP Association in GWAS Databasesstat.ML
Following the publication of an attack on genome-wide association studies (GWAS) data proposed by Homer et al., considerable attention has been given to developing methods for releasing GWAS data in a privacy-preserving way. Here, we develop an end-to-end differentially private method for solving regression problems wi…
HardNet adds hard constraints to neural networks without sacrificing performance.
problem Ensuring adherence to input-dependent constraints in neural networks.
method Appends a differentiable enforcement layer to neural networks for end-to-end training with hard constraint guarantees.
result HardNet retains neural networks' universal approximation capabilities and enables efficient optimization.
We introduce the value iteration network (VIN): a fully differentiable neural network with a `planning module' embedded within. VINs can learn to plan, and are suitable for predicting outcomes that involve planning-based reasoning, such as policies for reinforcement learning. Key to our approach is a novel differentiab…