A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
A new Kolmogorov-Arnold network improves function approximation and optimization.
Neural networks provide a rich class of high-dimensional, non-convex optimization problems. Despite their non-convexity, gradient-descent methods often successfully optimize these models. This has motivated a recent spur in research attempting to characterize properties of their loss surface that may explain such succe…
Data assimilation for parameter and state estimation in subsurface transport problems remains a significant challenge due to the sparsity of measurements, the heterogeneity of porous media, and the high computational cost of forward numerical models. We present a physics-informed deep neural networks (DNNs) machine lea…
We present novel empirical observations regarding how stochastic gradient descent (SGD) navigates the loss landscape of over-parametrized deep neural networks (DNNs). These observations expose the qualitatively different roles of learning rate and batch-size in DNN optimization and generalization. Specifically we study…
DIN framework directly models hydraulic conductivity and uncertainty.
Study compares semi-supervised learning methods for anomaly detection in hydraulic systems.
A new pruning method reduces neural network computation without retraining.
We identify a class of over-parameterized deep neural networks with standard activation functions and cross-entropy loss which provably have no bad local valley, in the sense that from any point in parameter space there exists a continuous path on which the cross-entropy loss is non-increasing and gets arbitrarily clos…
Looped transformers outperform standard transformers in complex reasoning tasks due to a specific loss landscape geometry.
Despite the non-convex nature of their loss functions, deep neural networks are known to generalize well when optimized with stochastic gradient descent (SGD). Recent work conjectures that SGD with proper configuration is able to find wide and flat local minima, which have been proposed to be associated with good gener…
LoRA-Curve connects independent LoRA optima through continuous low-loss valleys, improving Bayesian model averaging.
A common difficulty in applications of machine learning is the lack of any general principle for guiding the choices of key parameters of the underlying neural network. Focusing on a class of recurrent neural networks - reservoir computing systems that have recently been exploited for model-free prediction of nonlinear…
Optimal design portfolios improve energy efficiency and reduce risk in uncertain reservoirs.
This paper explores loss landscapes of sparse neural networks, finding unique characteristics compared to dense networks.
In this paper, we present a data-driven model for forecasting the production increase after hydraulic fracturing (HF). We use data from fracturing jobs performed at one of the Siberian oilfields. The data includes features, characterizing the jobs, and geological information. To predict an oil rate after the fracturing…
Deep learning upscales geologic models efficiently.
This paper shows that every sublevel set of the loss function of a class of deep over-parameterized neural nets with piecewise linear activation functions is connected and unbounded. This implies that the loss has no bad local valleys and all of its global minima are connected within a unique and potentially very large…
WSD schedule improves model training efficiency by adapting learning rates dynamically.
A D-Wave quantum annealer (QA) having a 2048 qubit lattice, with no missing qubits and couplings, allowed embedding of a complete graph of a Restricted Boltzmann Machine (RBM). A handwritten digit OptDigits data set having 8x7 pixels of visible units was used to train the RBM using a classical Contrastive Divergence. E…
CNN model predicts fluvial floods quickly and accurately.
Adaptor 'E' extends gradient-based optimizers to explore loss landscapes, improving generalization.
This paper examines SVB's failure and its impact on bank stocks.
Exploiting capacity of sewer system using decentralized control is a cost effective mean of minimizing the overflow. Given the size of the real sewer system, exploiting all the installed control structures in the sewer pipes can be challenging. This paper presents a divide and conquer solution to implement decentralize…
This paper presents a novel Inter Catchment Wastewater Transfer (ICWT) method for mitigating sewer overflow. The ICWT aims at balancing the spatial mismatch of sewer flow and treatment capacity of Wastewater Treatment Plant (WWTP), through collaborative operation of sewer system facilities. Using a hydraulic model, the…
Proposes a method to partition univariate data into unimodal subsets.
This paper proposes a new optimization algorithm called Entropy-SGD for training deep neural networks that is motivated by the local geometry of the energy landscape. Local extrema with low generalization error have a large proportion of almost-zero eigenvalues in the Hessian with very few positive or negative eigenval…
Designs chiral photonic structures using machine learning for efficient optical properties.
Current system thermal-hydraulic codes have limited credibility in simulating real plant conditions, especially when the geometry and boundary conditions are extrapolated beyond the range of test facilities. This paper proposes a data-driven approach, Feature Similarity Measurement FFSM), to establish a technical basis…
New framework reveals thermodynamic principles for LLM training.
SALR improves deep learning generalization by dynamically adjusting learning rates.
HyPV-LEAD detects cryptocurrency anomalies proactively, improving financial security.
A new pruning method finds sparse minimizers in flat regions of deep neural networks.
Study uses TDA to improve OEE forecasting in manufacturing.
A new AI optimization method uses energy-conserving dynamics inspired by Born-Infeld theory.
Quantization-aware training can recover accuracy lost by post-training quantization.
Stochastic gradient descent (SGD) forms the core optimization method for deep neural networks. While some theoretical progress has been made, it still remains unclear why SGD leads the learning dynamics in overparameterized networks to solutions that generalize well. Here we show that for overparameterized networks wit…
Solves local minima problems on smooth manifolds.
Model shows how banks' hidden-to-maturity accounting can mask run risk and lead to financial instability.
Unbalanced data arises in many learning tasks such as clustering of multi-class data, hierarchical divisive clustering and semisupervised learning. Graph-based approaches are popular tools for these problems. Graph construction is an important aspect of graph-based learning. We show that graph-based algorithms can fail…
Inverse modeling for the estimation of non-Gaussian hydraulic conductivity fields in subsurface flow and solute transport models remains a challenging problem. This is mainly due to the non-Gaussian property, the non-linear physics, and the fact that many repeated evaluations of the forward model are often required. In…
Tilting loss functions improves machine learning performance.
VED framework learns low-dimensional latent representations of physical systems.
Stablecoin liquidity was affected by the SVB collapse, with USDC's transparency leading to market reactions.
New algorithm ATENT improves adversarial robustness in neural networks.
A multiple instance dictionary learning approach, Dictionary Learning using Functions of Multiple Instances (DL-FUMI), is used to perform beat-to-beat heart rate estimation and to characterize heartbeat signatures from ballistocardiogram (BCG) signals collected with a hydraulic bed sensor. DL-FUMI estimates a "heartbea…
Randomized algorithms that base iteration-level decisions on samples from some pool are ubiquitous in machine learning and optimization. Examples include stochastic gradient descent and randomized coordinate descent. This paper makes progress at theoretically evaluating the difference in performance between sampling wi…
Learning to optimize - the idea that we can learn from data algorithms that optimize a numerical criterion - has recently been at the heart of a growing number of research efforts. One of the most challenging issues within this approach is to learn a policy that is able to optimize over classes of functions that are fa…
Large SGD step sizes lead to sparse feature learning in neural networks.