Paper demystifies AD techniques for efficient deep learning.
problem Generalizing backpropagation for complex neural networks.
method Uncovering reverse-mode AD and its connection to delimited continuations, implementing it via operator overloading.
result Efficient reverse-mode AD without auxiliary data structures, combining deep learning and pure library approaches.
Tangent automates derivatives in Python, improving expressiveness and performance.
problem Efficiently calculating derivatives for complex models in Python.
method Source-code transformation for dynamically typed array programming.
result Demonstrates improved expressiveness and performance in automatic differentiation.
Geometric AD framework simplifies derivative computation in JAX.
problem Efficient and accurate automatic differentiation.
method Jet functors and Weil algebras for geometric analysis.
result Unified view of derivative propagation with algebraic exactness.
Study shows AD for neural nets with machine-representable numbers can be incorrect.
problem Correctness of AD for neural nets with machine-representable numbers.
method Analyzed two sets of parameters: incorrect and non-differentiable. Proved bounds and conditions for AD correctness.
result AD can be incorrect for machine-representable numbers, but provides a Clarke subderivative on non-differentiable set.
Teaches matrix calculus for machine learning and optimization.
problem Computing derivatives of functions involving matrices.
method Extends differential calculus to vector spaces, focusing on practical applications in machine learning.
result Introduction of adjoint and reverse-mode differentiation for efficient computation.
Deep learning methods optimize complex models with large datasets.
problem Optimizing large and complex models with deep learning.
method Various gradient optimization techniques for practical machine learning.
result Effective optimization of wide, deep models with many parameters.
RAD estimates gradients with less memory, faster than small batch sizes.
problem Training deep models with stochastic gradient descent requires exact gradients, but they are not needed.
method Developed a framework for randomized automatic differentiation (RAD) to compute unbiased gradient estimates with reduced memory.
result RAD converges in fewer iterations than using a small batch size for feedforward networks and similar number for recurrent networks.
Two gradient-based methods for hyperparameter optimization are introduced, with applications in machine learning.
problem Optimizing hyperparameters for machine learning models.
method Forward and reverse-mode procedures for computing gradients of validation error with respect to hyperparameters.
result Forward-mode procedure suitable for real-time hyperparameter updates, potentially speeding up optimization on large datasets.
ACA method improves gradient estimation for neural ODEs, reducing error and training time.
problem Inaccurate gradient estimation methods hinder the performance of neural ODEs on benchmark tasks.
method Adaptive Checkpoint Adjoint (ACA) method that applies trajectory checkpointing, deletes redundant components, and supports adaptive solvers.
result ACA reduces error rate by half and training time by half compared to adjoint and naive methods on image classification tasks.
A new method for optimizing language-based agentic systems using semantic backpropagation.
problem Lack of proper feedback assignment in optimizing agentic systems.
method Formalization of semantic backpropagation with semantic gradients and semantic gradient descent.
result Our method outperforms existing state-of-the-art methods for solving GASO problems.
Automatic differentiation---the mechanical transformation of numeric computer programs to calculate derivatives efficiently and accurately---dates to the origin of the computer age. Reverse mode automatic differentiation both antedates and generalizes the method of backwards propagation of errors used in machine learni…
A JAX toolbox solves optimal transport problems for point clouds and histograms.
problem Optimal transport problems between point clouds and histograms.
method Automatic and custom reverse mode differentiation, vectorization, just-in-time compilation, and accelerators support.
result Solves a wide range of optimal transport problems including regularized OT, barycenters, Gromov-Wasserstein, and low-rank solvers.
The paper makes inference methods available for Gaussian models with banded precision.
problem Efficient inference for Gaussian models with banded precision.
method Develops linear algebra operators for banded matrices within automatic differentiation frameworks.
result The operators enable efficient variational inference and gradient-based sampling for Gaussian models with banded precision.
A new method computes gradients without backpropagation.
problem Optimization of machine learning models.
method Forward mode automatic differentiation to compute gradients.
result Forward gradient is an unbiased estimate of the gradient, eliminating the need for backpropagation.
AutoBayes simplifies variational inference by composing models and optimizing them.
problem Generalized variational inference complexities and optimization challenges.
method Compositional framework exploiting chain rules for automatic differentiation.
result Optimized models and parameterized statistical games can be locally optimized.
A new method improves convergence of gradient-based optimizers without manual tuning.
problem Improving the convergence rate of gradient-based optimizers.
method Dynamic learning rate adaptation using hypergradient descent.
result Significantly reduces the need for manual tuning of initial learning rates.
The paper generalizes optimization algorithms using category theory.
problem Optimizing functions in a category-theoretic setting.
method Using the Cartesian reverse derivative to generalize gradient descent and Newton's method.
result Properties of optimization algorithms are preserved in the generalized setting, including invariances and convergence.
A new method for estimating complex models and high-dimensional data.
problem Difficulty in computing Hessian of log-density functions for complex models and high-dimensional data.
method Sliced score matching, which projects scores onto random vectors before comparison.
result Sliced score matching can learn deep energy-based models and produce accurate score estimates.
Study compares methods for computing hypergradients in machine learning problems.
problem Computing exact hypergradients in machine learning is difficult.
method Investigates reverse mode iterative differentiation and approximate implicit differentiation methods.
result Unified analysis provides iteration complexity bounds and hierarchy of methods.
Perceptual ad-blocking is vulnerable to attacks, creating new security risks.
problem Vulnerability of perceptual ad-blocking to attacks and new security risks.
method Analysis and creation of adversarial examples to bypass perceptual ad-blocking.
result Perceptual ad-blocking can be bypassed using adversarial examples, introducing new security risks.
A new framework for personalized ad retrieval in e-commerce search.
problem Difficulty in measuring ads retrieved using multiple signals (e.g. user profiles, clicks).
method Employing historical click data to initialize a hierarchical network representing signals, keys, and ads. Training a model to learn weights of edges and selecting the best edges.
result Framework achieves good performance, improving RPM/CTR.
Study proposes an ad creative selection algorithm considering user fatigue.
problem Ad creative effectiveness varies with user exposure.
method Proposes an algorithm that considers user fatigue in ad selection.
result The proposed algorithm outperforms baseline algorithms in real-world testing.
The paper proposes a method to predict audio ad quality using acoustic features.
problem Improving user experience in online music streaming services by ensuring high quality audio advertisements.
method The paper proposes predicting audio ad quality using acoustic features and a proxy metric called Long Click Rate (LCR). A deep learning model is also introduced.
result The proposed deep learning model outperforms other models trained on hand-crafted features for audio ad quality prediction.
Detects accidental clicks on mobile ads to reduce advertiser costs and improve revenue.
problem Accidental clicks on mobile ads lead to wasted revenue for advertisers and ad networks.
method Collect and analyze dwell time data to identify accidental clicks and estimate thresholds.
result Our method reduces advertiser costs and improves ad click-through rates and revenue.
In markets for online advertising, some advertisers pay only when users respond to ads. So publishers estimate ad response rates and multiply by advertiser bids to estimate expected revenue for showing ads. Since these estimates may be inaccurate, the publisher risks not selecting the ad for each ad call that would max…
Improves CTR prediction by considering spatial and temporal auxiliary ads.
problem Improving CTR prediction in online advertising systems.
method Deep Spatio-Temporal Neural Networks (DSTNs) for CTR prediction.
result DSTNs outperform state-of-the-art methods in CTR prediction.
Sharp Minkowski inequality found for AdS-Melvin spacetime surfaces.
problem Proving a Minkowski-type inequality for surfaces in the AdS-Melvin space.
method Used weighted normal flow to prove inequality for general surfaces.
result Sharp Minkowski inequality holds for all surfaces in AdS-Melvin space.
EENMF improves e-commerce sponsored search efficiency and effectiveness.
problem Improving efficiency and effectiveness of e-commerce sponsored search.
method End-to-end neural matching framework (EENMF) for vector-based ad retrieval and neural pre-ranking.
result Significantly outperforms baseline in real e-commerce traffic.
Study neural architectures on learned latent graphs using Schrödinger dynamics.
problem Understanding neural architectures on learned latent graphs.
method Optimizes over stratified moduli space of weighted graphs with Kähler-Hessian metric.
result Multilayer stationary networks are equivalent to global stationary problems on supra-graphs.
A framework combines unsupervised and semi-supervised AD using synthetic anomalies.
problem Improving anomaly detection in both unsupervised and semi-supervised settings.
method Proposes a new framework that uses both known and synthetic anomalies for training.
result Synthetic anomalies improve anomaly modeling in low-density regions and provide optimal convergence guarantees.
Special geometries found in near horizon spacetimes.
problem Understanding the near horizon geometry of extremal Reissner-Nordstrom solutions.
method Analyzing asymptotically AdS2imesS2 spacetimes under null energy conditions. result Asymptotically AdS2imesS2 spacetimes must have special geometries similar to AdS2imesS2. Percival uses deep learning to block ads in real-time, minimizing performance impact.
problem Real-time ad blocking in browsers with minimal performance overhead.
method Deep learning for image classification integrated into browser's rendering pipeline.
result Percival achieves 96.76% accuracy in blocking ads, comparable to block lists.
Enhanced ad event prediction boosts performance.
problem Improving ad performance metrics like CTR and CVR.
method New feature engineering approach for ad event prediction.
result Significantly outperforms alternative prediction algorithms.
Landmark AD improves AD's efficiency without sacrificing performance.
problem Computational burden in diffusion-based sensor fusion.
method Inspired by landmark diffusion, proposes Landmark AD.
result Landmark AD offers superior computational efficiency.
New examples of Lie algebras with ad-invariant metrics found.
problem Finding ad-invariant metrics on nonnice nilpotent Lie algebras.
method Introducing single extension method to construct Lie algebras with ad-invariant metrics.
result Explicit examples of nonnice nilpotent Lie algebras with ad-invariant metrics for dimensions > 10 and steps > 2.
Article shows AdS-quasi-Fuchsian groups' limit sets are not smooth.
problem Smoothness of limit sets of AdS quasi-Fuchsian groups.
method Analyzes limit sets of AdS quasi-Fuchsian groups in PO(n,2).
result Limit sets are never C^1, except for Fuchsian groups.
This study proves energy bounds in specific AdS spacetimes.
problem Proving positive energy theorems in asymptotically locally AdS spacetimes.
method Derived positive energy theorem for spacetimes with compact, Einstein cross-sections.
result First complete proofs of BPS inequalities in AdS and locally AdS spacetimes.
New framework uses EEG to detect brain atrophy in AD, validated on large AD trial.
problem Diagnosis of Alzheimer's disease relies on subjective clinical interpretations.
method Combines Riemannian tangent space mapping and elastic net regression.
result Developed brain atrophy markers validated on large AD trial.
We formulate the variational problem for AdS gravity with Dirichlet boundary conditions and demonstrate that the covariant counterterms are necessary to make the variational problem well-posed. The holographic charges associated with asymptotic symmetries are then rederived via Noether's theorem and `covariant phase sp…
Study on uniqueness of ad-invariant metrics in Lie algebras.
problem Uniqueness of ad-invariant metrics in Lie algebras up to automorphisms.
method Analysis of Lie algebras, cotangent Lie algebras, and specific conditions for uniqueness.
result Uniqueness of ad-invariant metric on T∗g implies solvability of g, but not conversely. Study identifies five AD subtypes using graph diffusion and similarity learning.
problem Identifying homogeneous AD subtypes to improve diagnosis and treatment.
method Unsupervised clustering with graph diffusion and similarity learning.
result Five distinct AD subtypes identified with significant differences in biomarkers and clinical features.
This paper examines how ads on LinkedIn affect user behavior over time.
problem Understanding long-term impact of ads on user engagement and revenue.
method Conducted experiments with randomized member buckets to measure short and long-term effects of ads density.
result Long-term impact of ads is much smaller than short-term impact, and different user cohorts react differently over time.
AdS uniqueness and black hole energy bounds proven.
problem Proving uniqueness of Anti-de Sitter spacetime and energy bounds for AdS black holes.
method Adapted Wang's proof to static asymptotically locally hyperbolic vacuum metrics and higher-genus horizons.
result Negativity of free energy E−TS for AdS black holes with higher-genus horizons. Optimizes bidding strategies for LinkedIn ads across multiple platforms.
problem Optimizing automated bidding agents for dynamic online marketplaces.
method Developed a general optimization framework for buyer's interest, agnostic to auction mechanisms.
result Automatically guarantees the optimality of budget allocation across ad units and platforms.
Paper proposes bypassing implicit assumption in GM-based AD methods.
problem Lack of anomalous data and implicit assumption in GM-based AD methods.
method Integrating Discriminative idea to GMM for AD tasks (DiGMM).
result Establishes a connection between generative and discriminative models for AD.
Geometries and dual field theories linked by AdS/CFT.
problem Understanding the AdS/CFT correspondence.
method Geometric extremization principles informed by physical considerations.
result Key role of Sasaki-Einstein and GK geometry.
The study constructs AdS manifolds from Gromov-Thurston manifolds.
problem Creating hyperbolic and anti-de Sitter structures from Gromov-Thurston manifolds.
method Explicit correspondence between quasifuchsian AdS manifolds and compact quotients of Ø(2d,2)/U(d,1).
result Existence of quasifuchsian AdS manifolds and hyperbolic ends with specified boundary.
Crooked planes are piecewise linear surfaces that were introduced by Drumm in the early 1990s to construct fundamental domains for properly discontinuous actions of free groups on Minkowski 3-space. In a previous paper, we introduced analogues of these surfaces, called AdS crooked planes, in the 3-dimensional anti-de S…