Improved neural networks by averaging late-stage weights.
problem Improving the performance of neural networks.
method Ensemble late-stage weights and average them.
result Augmenting standard models with late-phase weights improves generalization.
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,657 papers · 148 categories
Trend · papers per month
48 results for “late phase”
SAM selects flatter minima late in training, improving generalization.
problem Improving neural network generalization under various settings.
method Sharpness-Aware Minimization (SAM) applied late in training.
result SAM efficiently selects flatter minima late in training, improving generalization.
Study shows reverberant phase is not essential for weakly-supervised dereverberation.
problem Evaluating the role of reverberant phase in weakly-supervised dereverberation.
method Statistical Wave Field Theory and recent weak supervision framework.
result Wet phase carries limited useful information and is not essential for weakly supervised dereverberation.
The study examines the dynamic behavior of RMSprop and Adam algorithms.
problem Understanding the training loss curve patterns of adaptive gradient algorithms.
method Careful numerical experiments and theoretical explanations using the signGD flow.
result Adam converges smoother and faster when momentum factors are close to each other.
Novel anti-grokking phase discovered in neural networks, revealed by HTSR layer quality metric.
problem Understanding and detecting overfitting in neural networks.
method 3-layer MLP, weight decay, HTSR layer quality metric , correlation traps, Kolmogorov–Smirnov tests.
result Anti-grokking phase detected late in training, revealed by and correlation traps.
We present here an introduction to Brainstorming approach, that was recently proposed as a consensus meta-learning technique, and used in several practical applications in bioinformatics and chemoinformatics. The consensus learning denotes heterogeneous theoretical classification method, where one trains an ensemble of…
Large learning rates improve neural network generalization, study shows.
problem Understanding why large learning rates lead to better neural network generalization.
method Visual analysis of training and testing loss landscapes, introduction of a nonlinear model.
result Extended phase with large learning rates leads to near-optimal generalization.
This research examines how the error rate of nearest neighbor classifiers varies with dataset size.
problem The scaling of classification error rates with dataset size is not uniform.
method Theoretical analysis of nearest neighbor classifiers, focusing on early and late phases of dataset size.
result The error rate of nearest neighbor classifiers can have fine-grained rates depending on the dataset size and data distribution.
Shared Keyboard: An improved Bayesian design for phase I clinical trials via Beta kernel processstat.AP
Shared Keyboard design improves phase I clinical trials by borrowing information across doses.
problem Interim decisions based on current dose data may overlook signals from neighboring doses.
method Bayesian model-assisted design using Beta kernel process with kernel-weighted pseudo-counts.
result Significant improvements in identifying maximum tolerated dose and safety.
Gradient descent dynamics in neural networks show quenching and activation phases.
problem Understanding training dynamics in neural networks.
method Numerical and phenomenological study of gradient descent algorithm for two-layer neural networks.
result Gradient descent dynamics exhibit quenching and activation phases in under-parametrized networks.
Dynamic Influence Tracker measures changing sample importance during model training.
problem Static influence measurements during training overlook how sample importance varies over time.
method Dynamic Influence Tracker (DIT) captures time-varying sample influence across arbitrary time windows.
result DIT reveals distinct learning phases with shifting priorities and detects corrupted samples more efficiently.
New method finds precise late-time behavior of wave equations.
problem Analyzing late-time behavior of wave equations with inverse-square potentials.
method Physical-space-based method for deriving late-time asymptotics.
result Sharp, uniform decay estimates in time for asymptotic late-time tails.
Study assesses drought and late-frost risks in Bavaria using vine copulas.
problem Assessing risks of late-frost and drought in Bavaria due to climate change.
method Used vine copula models for non-Gaussian and asymmetric dependencies, with univariate and bivariate regression analyses.
result Identified 'at-risk' regions for forest adaptation.
Second part of series studying charged scalar fields on Reissner--Nordström spacetimes.
problem Analyzing late-time behavior and stability of charged scalar fields on black hole backgrounds.
method Purely physical-space based methods, energy estimates, inverse-power laws.
result First pointwise decay estimates for charged scalar fields on black hole backgrounds.
New framework optimizes deep learning training by deferring large batch sizes to late stages.
problem Optimizing batch size scheduling for deep learning training efficiency.
method Introduced the functional scaling law (FSL) framework to analyze and optimize batch size scheduling.
result Large batch sizes can be deferred to late training stages without sacrificing performance.
LITE models improve query-document relevance with learnable late interactions.
problem Improving query-document relevance with lower latency and storage.
method Proposes learnable late-interaction models (LITE) that use factorized query and document embeddings followed by a learnable scorer.
result Empirically, LITE outperforms previous late-interaction models in re-ranking tasks.
New method learns optimal variance schedule for diffusion models.
problem Diffusion models' sensitivity to variance schedule.
method Probabilistic conditioning, learning schedule during training.
result Comparable or superior results in super-resolution microscopy and quantitative phase imaging.
Manifold Markov chain Monte Carlo algorithms have been introduced to sample more effectively from challenging target densities exhibiting multiple modes or strong correlations. Such algorithms exploit the local geometry of the parameter space, thus enabling chains to achieve a faster convergence rate when measured in n…
High frequency trading has led to widespread efforts to reduce information propagation delays between physically distant exchanges. Using relativistically correct millisecond-resolution tick data, we document a 3-millisecond decrease in one-way communication time between the Chicago and New York areas that has occurred…
A lot of attention has been devoted to multimedia indexing over the past few years. In the literature, we often consider two kinds of fusion schemes: The early fusion and the late fusion. In this paper we focus on late classifier fusion, where one combines the scores of each modality at the decision level. To tackle th…
Warm Up Cold-start Advertisements: Improving CTR Predictions via Learning to Learn ID Embeddingscs.LG
Click-through rate (CTR) prediction has been one of the most central problems in computational advertising. Lately, embedding techniques that produce low-dimensional representations of ad IDs drastically improve CTR prediction accuracies. However, such learning techniques are data demanding and work poorly on new ads w…
Late fusion of clinical notes and physiological data improves ICU mortality prediction.
problem Improving ICU mortality prediction using multimodal data.
method Late fusion of clinical notes and physiological time series data with a deep learning architecture.
result Late fusion approach provides statistically significant improvement in mortality prediction performance.
Estimates LATE using combined datasets, overcoming data limitations.
problem Estimating LATE when compliance is incomplete and data is split.
method Combines separately observed datasets to estimate LATE using direct and weighted least squares methods.
result Proposes a stable and practical estimator for LATE.
In the past few years, a lot of attention has been devoted to multimedia indexing by fusing multimodal informations. Two kinds of fusion schemes are generally considered: The early fusion and the late fusion. We focus on late classifier fusion, where one combines the scores of each modality at the decision level. To ta…
Background: Overweight and obesity are an increasing phenomenon worldwide. Predicting future overweight or obesity early in the childhood reliably could enable a successful intervention by experts. While a lot of research has been done using explanatory modeling methods, capability of machine learning, and predictive m…
Restricted Boltzmann Machines (RBM) have attracted a lot of attention of late, as one the principle building blocks of deep networks. Training RBMs remains problematic however, because of the intractibility of their partition function. The maximum likelihood gradient requires a very robust sampler which can accurately …
Do Pension Benefits Accelerate Cognitive Decline in Late Adulthood? Evidence from Rural Chinaecon.GN
Pension benefits in rural China lead to cognitive decline among the elderly.
problem Cognitive decline in late adulthood among rural Chinese elderly.
method Examined the effects of a new pension scheme on cognitive performance.
result Pension benefits negatively impact cognitive functioning, particularly delayed recall.
Why are Megaprojects, Including Nuclear Power Plants, Delivered Overbudget and Late? Reasons and Remediesphysics.soc-ph
In the first section, this report analyses Nuclear Power Plants (NPPs) in the context of megaprojects, explaining why they are often delivered over budget and late. In the second section, the report discusses how Small Modular Reactors (SMRs) might address these issues. Megaprojects are extremely risky and often implem…
Deep Convolutional Neural Networks (CNN) enforces supervised information only at the output layer, and hidden layers are trained by back propagating the prediction error from the output layer without explicit supervision. We propose a supervised feature learning approach, Label Consistent Neural Network, which enforces…
Dynamical Isometry is Achieved in Residual Networks in a Universal Way for any Activation Functionstat.ML
We demonstrate that in residual neural networks (ResNets) dynamical isometry is achievable irrespectively of the activation function used. We do that by deriving, with the help of Free Probability and Random Matrix Theories, a universal formula for the spectral density of the input-output Jacobian at initialization, in…
Multipath is among the major sources of errors in precise positioning using GPS and continues to be extensively studied. Two Fast Fourier Transform (FFT)-based detectors are presented in this paper as GPS multipath detection techniques. The detectors are formulated as binary hypothesis tests under the assumption that t…
The late-stage training dynamics of (stochastic) subgradient descent on homogeneous neural networkscs.LG
SGD converges to critical points of normalized margin in late-stage training for homogeneous neural networks.
problem Analyzing the implicit bias of SGD on homogeneous neural networks.
method Interpreting SGD dynamics as an Euler-like discretization of a conservative field flow associated with the normalized classification margin.
result Normalized SGD iterates converge to the set of critical points of the normalized margin at late-stage training.
The goal of eXtreme Multi-label Learning (XML) is to automatically annotate a given data point with the most relevant subset of labels from an extremely large vocabulary of labels (e.g., a million labels). Lately, many attempts have been made to address this problem that achieve reasonable performance on benchmark data…
We prove the existence of a perimeter-minimizing partition of R^n into regions of unit volume. We conclude with a short tribute to the late Manuel A. Fortes.
Study fusion methods for financial image views to improve robustness against attacks.
problem Improving robustness of financial image views for next-day direction prediction.
method Same-source multi-view learning with early fusion and late fusion, using OHLCV and technical-indicator views, and evaluating pixel-space L-infinity attacks.
result Early fusion can suffer negative transfer under noisy settings, while late fusion is more reliable once labels stabilize.
A study on a surprising phase transition in model generalization error as parameters approach sample size.
problem Understanding the generalization error of overparametrized ridge models.
method Finite sample analysis using continuous Newton method and -norm solution performance.
result The generalization error decreases after the threshold for ridge models.
Enhances the Bishop-Gromov theorem for curved spaces, especially at late times.
problem The Bishop-Gromov theorem's volume growth upperbound is often too loose, especially at late times.
method Identified and quantified the effect of shear, using higher curvature invariants to improve the upperbound.
result Tighter upper bounds on late-time growth rates of geodesic balls in homogeneous spaces with non-positive sectional curvature.
New method uses SBI to infer magnetorotational properties of isolated pulsars.
problem Constrain magnetorotational properties of isolated Galactic radio pulsars.
method Combines population synthesis with SBI to model neutron star birth and evolution.
result Inferred , for lognormal distributions.
Log-periodic self-similarity: an emerging financial law?cond-mat.stat-mech
A hypothesis that the financial log-periodicity, cascading self-similarity through various time scales, carries signatures of a law is pursued. It is shown that the most significant historical financial events can be classified amazingly well using a single and unique value of the preferred scaling factor lambda=2, whi…
Framework for multi-task learning with semiparametric models and nuisance parameters.
problem Improving parameter estimation from diverse, heterogeneous datasets.
method Late fusion multi-task learning framework with two-step process: individual task learning followed by adaptive aggregation.
result The method achieves faster convergence rates compared to individual task learning when tasks share similar parametric components.
One of the core tasks in multi-view learning is to capture relations among views. For sequential data, the relations not only span across views, but also extend throughout the view length to form long-term intra-view and inter-view interactions. In this paper, we present a new memory augmented neural network model that…
This is a review article for Encyclopedia of Complexity and System Science, to be published by Springer http://refworks.springer.com/complexity/. The paper reviews statistical models for money, wealth, and income distributions developed in the econophysics literature since late 1990s.
In this study, we investigated multi-modal approaches using images, descriptions, and titles to categorize e-commerce products on Amazon. Specifically, we examined late fusion models, where the modalities are fused at the decision level. Products were each assigned multiple labels, and the hierarchy in the labels were …
The Giroux correspondence and the notion of a near force-free magnetic field are used to topologically characterize near force-free magnetic fields which describe a variety of physical processes, including plasma equilibrium. As a byproduct, the topological characterization of force-free magnetic fields associated with…
We present RAPID (Real-time Automated Photometric IDentification), a novel time-series classification tool capable of automatically identifying transients from within a day of the initial alert, to the full lifetime of a light curve. Using a deep recurrent neural network with Gated Recurrent Units (GRUs), we present th…
We give examples of closed orientable graph 3-manifolds with fundamental group which is not a subgroup of GL(4,k) for any field k. This answers a question in the Kirby problem list from 1977 which is credited to the late William Thurston.
Generative AI models improve clinical trial data by generating survival outcomes.
problem Generating valid survival outcomes for clinical trials with synthetic data.
method A variational autoencoder (VAE) that jointly generates mixed-type covariates and survival outcomes.
result The method outperforms GAN baselines on fidelity, utility, and privacy metrics.
Human Activity Recognition (HAR) based on motion sensors has drawn a lot of attention over the last few years, since perceiving the human status enables context-aware applications to adapt their services on users' needs. However, motion sensor fusion and feature extraction have not reached their full potentials, remain…