Data analysis in high-dimensional spaces aims at obtaining a synthetic description of a data set, revealing its main structure and its salient features. We here introduce an approach providing this description in the form of a topography of the data, namely a human-readable chart of the probability density from which t…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,695 papers · 148 categories
Trend · papers per month
48 results for “density peaks”
A new clustering algorithm reduces density peaks clustering's computational complexity.
problem High computational complexity of density peaks clustering.
method Sparse distance matrix, sparse search, K-d tree, second-order difference method.
result Reduced computational complexity from to .
As one type of efficient unsupervised learning methods, clustering algorithms have been widely used in data mining and knowledge discovery with noticeable advantages. However, clustering algorithms based on density peak have limited clustering effect on data with varying density distribution (VDD), equilibrium distribu…
Hierarchical nucleation patterns emerge in deep neural network layers.
problem Understanding the generation of meaningful representations in deep neural networks.
method Analysis of the probability density of ImageNet dataset across hidden layers.
result Density peaks in subsequent layers mirror the semantic hierarchy of concepts, resembling nucleation process.
Most density-based clustering methods largely rely on how well the underlying density is estimated. However, density estimation itself is also a challenging problem, especially the determination of the kernel bandwidth. A large bandwidth could lead to the over-smoothed density estimation in which the number of density …
This paper focuses on density-based clustering, particularly the Density Peak (DP) algorithm and the one based on density-connectivity DBSCAN; and proposes a new method which takes advantage of the individual strengths of these two methods to yield a density-based hierarchical clustering algorithm. Our investigation be…
New clustering algorithm for mixed data improves applicability and efficiency.
problem Clustering large, mixed data with improved accuracy and efficiency.
method Developed a new clustering algorithm using peak-finding technique, reducing computational complexity.
result Algorithm detects outliers, clusters of lower density, and determines correct number of clusters.
We propose a Fourier-based approach for optimization of several clustering algorithms. Mathematically, clusters data can be described by a density function represented by the Dirac mixture distribution. The density function can be smoothed by applying the Fourier transform and a Gaussian filter. The determination of th…
Simultaneous Estimation of Noise Variance and Number of Peaks in Bayesian Spectral Deconvolutionphysics.data-an
The heuristic identification of peaks from noisy complex spectra often leads to misunderstanding of the physical and chemical properties of matter. In this paper, we propose a framework based on Bayesian inference, which enables us to separate multipeak spectra into single peaks statistically and consists of two steps.…
Equity auctions show linear price impact up to a large volume, then non-linear.
problem Understanding price impact in equity auctions.
method High-quality data analysis of Paris stock exchange auctions.
result Price impact is linear up to a large volume, then becomes non-linear.
New clustering method using point-set kernel measures similarity.
problem Measuring similarity between objects for clustering.
method Point-set kernel for similarity computation; clustering procedure uses this measure.
result Proposed method is more effective and faster than existing algorithms.
Characteristic time scales of tick quotes on foreign currency markets: an empirical study and agent-based modelphysics.data-an
Power spectrum densities for the number of tick quotes per minute (market activity) on three currency markets (USD/JPY, EUR/USD, and JPY/EUR) for periods from January 1999 to December 2000 are analyzed. We find some peaks on the power spectrum densities at a few minutes. We develop the double-threshold agent model and …
Study examines local extrema and crossing statistics in financial markets.
problem Understanding local extrema and crossing statistics in financial markets.
method Excursion set theory, numerical computation, theoretical prediction, clustering of geometrical measures, cross-correlation, Singular Value Decomposition.
result Excursion sets reveal statistical coherency and sensitivity to crises in financial markets.
Paper proposes bypassing implicit assumption in GM-based AD methods.
problem Lack of anomalous data and implicit assumption in GM-based AD methods.
method Integrating Discriminative idea to GMM for AD tasks (DiGMM).
result Establishes a connection between generative and discriminative models for AD.
I report a new statistical distribution formulated to confront the infamous, long-standing, computational/modeling challenge presented by highly skewed and/or leptokurtic ("fat- or heavy-tailed") data. The distribution is straightforward, flexible and effective. Even when working with far fewer data points than are rou…
A recent proposal of data dependent similarity called Isolation Kernel/Similarity has enabled SVM to produce better classification accuracy. We identify shortcomings of using a tree method to implement Isolation Similarity; and propose a nearest neighbour method instead. We formally prove the characteristic of Isolatio…
This study investigates that a characteristic time scale on an exchange rate market (USD/JPY) is examined for the period of 1998 to 2000. Calculating power spectrum densities for the number of tick quotes per minute and averaging them over the year yield that the mean power spectrum density has a peak at high frequenci…
A study of the personal income distribution in Australiaphysics.soc-ph
We analyze the data on personal income distribution from the Australian Bureau of Statistics. We compare fits of the data to the exponential, log-normal, and gamma distributions. The exponential function gives a good (albeit not perfect) description of 98% of the population in the lower part of the distribution. The lo…
Robust, Nonparametric, Efficient Decomposition of Spectral Peaks under Distortion and Interferenceeess.SP
A robust method for decomposing spectral peaks robust to distortion and interference.
problem Decomposing spectral peaks in the presence of distortion and interference.
method Optimizing a nonparametric approach using pseudo-symmetric functions with nonincreasing behavior.
result Decomposed spectral peaks show pseudo-orthogonal behavior and power preserving equality.
SPADE improves demand forecasting accuracy by 4.5% for post-promotion periods.
problem Overreacting to peak events in demand forecasting leads to biased forecasts.
method SPADE splits forecasting into two tasks: one for peak events and another for post-peak events, using masked convolution filters and a specialized Peak Attention module.
result Overall PPE improvement of 4.5%, 30% improvement for most affected forecasts after promotions and holidays, and 3.9% improvement in PE accuracy.
The sharp peak-flat trough pattern and critical speculationcond-mat.stat-mech
We find empirically a characteristic sharp peak-flat trough pattern in a large set of commodity prices. We argue that the sharp peak structure reflects an endogenous inter-market organization, and that peaks may be seen as local ``singularities'' resulting from imitation and herding. These findings impose a novel strin…
New matrix ensembles better match deep neural network spectral densities.
problem Theoretical spectral density models for deep networks do not match empirical observations.
method Introduced new matrix ensemble classes to better fit observed spectral densities.
result Theoretical models for deep networks are significantly flawed.
New method for estimating lead-lag times between non-synchronously observed point processes.
problem Estimating lead-lag relationships between non-synchronously observed point processes.
method Formulate lead-lag estimation as CPCF shape estimation; propose kernel density estimation-based lead-lag time estimator.
result Proposed method delivers superior numerical performance and effective lead-lag time estimation.
PeakSegJoint: fast supervised peak detection via joint segmentation of multiple count data samplesstat.ML
Joint peak detection is a central problem when comparing samples in genomic data analysis, but current algorithms for this task are unsupervised and limited to at most 2 sample types. We propose PeakSegJoint, a new constrained maximum likelihood segmentation model for any number of sample types. To select the number of…
Mass spectrometry (MS) is an important technique for chemical profiling which calculates for a sample a high dimensional histogram-like spectrum. A crucial step of MS data processing is the peak picking which selects peaks containing information about molecules with high concentrations which are of interest in an MS in…
On-chain Peak Shavingecon.EM
Study on-chain peak shaving to reduce Ethereum transaction costs.
problem Reducing transaction costs in blockchain networks, especially during congested periods.
method Analyzing transaction-level data from multiple firms across various industries to understand scheduling responses and cost management strategies.
result Firms' scheduling responses to congestion vary, leading to different fee savings and residual costs.
The paper explains two distinct peaks in generalization error for neural networks and simpler models, each governed by different factors.
problem Understanding the peaks in generalization error for neural networks and simpler models.
method Analysis of random feature models and comparison with numerical experiments involving deep neural networks.
result The peaks at and are distinct and governed by different factors (noise sensitivity vs. initialization noise).
We study the statistical regularities of opening call auction using the ultra-high-frequency data of 22 liquid stocks traded on the Shenzhen Stock Exchange in 2003. The distribution of the relative price, defined as the relative difference between the order price in opening call auction and the closing price of last tr…
In this letter, we propose a method for period estimation in light curves from periodic variable stars using correntropy. Light curves are astronomical time series of stellar brightness over time, and are characterized as being noisy and unevenly sampled. We propose to use slotted time lags in order to estimate corrent…
Integrating Bayesian Spectral Deconvolution and Expert Scientific Reasoning for Robust Peak Estimationphysics.data-an
Bayesian framework integrates spectral deconvolution with expert reasoning for robust peak estimation.
problem Challenges in extracting meaningful peaks from noisy or complex spectra.
method Bayesian spectral deconvolution coupled with a physical-property regression layer.
result Recovery of weak peaks in poly(lactic acid) IR spectra related to degradation rates.
FLOPART solves peak detection by creating accurate train and test set predictions.
problem Correctly detecting peaks in sequential data.
method Dynamic programming changepoint algorithm with zero train label errors.
result FLOPART provides highly accurate predictions on both train and test sets.
The paper shows how the generalization curve can have multiple peaks, influenced by data and learning algorithm biases.
problem Understanding the generalization behavior of linear regression models under varying parameterizations.
method Analyzes generalization loss in linear regression models with varying parameterizations, both under- and over-parameterized.
result The generalization curve can have an arbitrary number of peaks, and their locations can be controlled.
PEAKS selects key training examples incrementally based on prediction error and kernel similarity.
problem Dynamic data selection in deep learning models.
method Prediction Error Anchored by Kernel Similarity (PEAKS) for incremental data selection.
result PEAKS outperforms existing selection strategies and yields better performance returns as training data size grows.
PEAK tests means of multiple data streams with sequential betting.
problem Testing means of multiple data streams with nonparametric methods.
method Sequential, nonparametric testing using a betting scheme.
result PEAK provides up to 85% reduction in samples for stopping.
LLMs learn peaked distributions slowly due to power-law losses.
problem Slow convergence of loss in training large language models.
method Systematic analysis of toy models and empirical evaluation of LLMs.
result Power-law time scaling with an exponent of 1/3 for learning peaked distributions.
Finite-Time Queue Peak Laws in Stochastic Networks: Logarithmic Scaling After Geometric Thresholdsmath.PR
Finite-time queue peaks in stochastic networks have logarithmic scaling after geometric thresholds.
problem Queue peak laws in stochastic networks with geometric thresholds.
method Self-normalization mechanism
result Logarithmic scaling of queue peaks after geometric thresholds.
Bayesian Quadrature improves ensembling for neural networks with dispersed likelihood peaks.
problem Ensembling neural networks struggles with dispersed, narrow peaks in likelihood surfaces.
method Uses Bayesian Quadrature to construct weighted ensembles of architectures.
result Empirically outperforms state-of-the-art baselines in test likelihood, accuracy, and expected calibration error.
Determining bottom price-levels after a speculative peakcond-mat.stat-mech
During a stock market peak the price of a given stock () jumps from an initial level to a peak level before falling back to a bottom level . The ratios and are referred to as the peak- and bottom-amplitude respectively. The paper show…
In this paper, the fractional order curvature equation in is considered. Assuming has two critical points satisfying certain local conditions, we prove the existence of two-peak solutions.
Nonparametric Extrema Analysis in Time Series for Envelope Extraction, Peak Detection and Clusteringcs.LG
A nonparametric method for time series analysis extracts envelopes, detects peaks, and clusters data.
problem Extracting envelopes, detecting peaks, and clustering in time series data.
method Iterative procedure that minimizes drift to create upper and lower bounding signals, using Viterbi-like path tracking and optimal elimination rules.
result Efficiently calculated solution with near-linear time complexities for various applications.
Populations of species in ecosystems are often constrained by availability of resources within their environment. In effect this means that a growth of one population, needs to be balanced by comparable reduction in populations of others. In neutral models of biodiversity all populations are assumed to change increment…
Paper proposes a network framework for prosumers to manage peak loads in Iran.
problem Balancing renewable prosumers' self-sufficiency with grid integration under uncertainty.
method Distributed contextual stochastic optimization (DCSO) framework with consensus-based sharing.
result Integration of prediction and optimization reduces peak loads and costs.
Traditionally in regression one minimizes the number of fitting parameters or uses smoothing/regularization to trade training (TE) and generalization error (GE). Driving TE to zero by increasing fitting degrees of freedom (dof) is expected to increase GE. However modern big-data approaches, including deep nets, seem to…
The paper analyzes Kernel Density Estimation in high dimensions with varying data and dimensionality.
problem High-dimensional Kernel Density Estimation with growing data and dimensionality.
method Examines the behavior of Kernel Density Estimators in the regime where both data points and dimensionality grow with a fixed ratio.
result Three distinct statistical regimes are identified for Kernel-based density estimates, each with different statistical properties.
New algorithm tackles constrained Markov decision processes with peak constraints.
problem Optimizing dynamic systems with peak constraints.
method Model-free algorithm converting PCMDP to unconstrained problem, applying Q-learning.
result Algorithm achieves -PAC policy under certain conditions.
We win EVA2025 by estimating extreme precipitation events using Peaks Over Thresholds and martingale testing.
problem Estimating the probability of extreme precipitation events with limited data.
method Modeling Peaks Over Thresholds with an exponential distribution and using martingale testing for evaluation.
result Our method outperforms other approaches in estimating extreme precipitation events.
This paper explains why double descent sometimes occurs weakly or not at all from an optimization perspective.
problem Understanding the role of optimization in the phenomenon of double descent.
method Investigates model-wise double descent from an optimization perspective, proposing a unified explanation for its occurrence.
result Model-wise double descent is observed if and only if the optimizer can find a sufficiently low-loss minimum.
Develops a new cluster validity index to find multiple optimal cluster numbers.
problem Finding the optimal number of clusters in real-world data with varying densities, sizes, and shapes.
method A new correlation-based cluster validity index that yields multiple local peaks.
result The new index finds multiple optimal cluster numbers in various scenarios.