The paper analyzes convergence in SGD with momentum and proposes a diagnostic test.
problem Detecting convergence in stochastic gradient descent with momentum.
method Analyzes the transient and stationary phases of SGD with momentum, constructs a statistical diagnostic test.
result The proposed diagnostic test effectively detects convergence in the stationary phase of SGD with momentum.
Triadic-OCD detects changes in data streams robustly and optimally, even in asynchronous settings.
problem Online change detection in data streams with practical constraints.
method Triadic-OCD framework for asynchronous online change detection with provable robustness, optimality, and convergence.
result The proposed triadic-OCD algorithm achieves optimal performance and convergence in asynchronous settings.
Paper presents neural network-based change-point detection methods.
problem Detecting change points in time series data.
method Online neural networks for change-point detection.
result Proposed methods outperform existing algorithms.
Paper proposes a new method to detect convergence in SGD.
problem Detecting the transition from fast progress to oscillation in SGD.
method Analyzes Pflug's test and proposes a novel statistical procedure.
result The novel procedure accurately detects stationarity in SGD.
This paper investigates the grant-free random access with massive IoT devices. By embedding the data symbols in the signature sequences, joint device activity detection and data decoding can be achieved, which, however, significantly increases the computational complexity. Coordinate descent algorithms that enjoy a low…
The paper provides theoretical guarantees for neural network-based anomaly detection.
problem Theoretical guarantees for unsupervised neural network-based anomaly detection.
method Casting anomaly detection as a binary classification problem, establishing non-asymptotic upper bounds and convergence rates.
result The convergence rate on the excess risk matches the minimax optimal rate.
Paper uses optimal transport-based statistics for change point detection.
problem Change point detection in multivariate data.
method Soft rank energy and entropically regularized optimal transport.
result Soft rank energy performs better in real datasets with strong continuity and convergence properties.
Anomaly detection aims to detect abnormal events by a model of normality. It plays an important role in many domains such as network intrusion detection, criminal activity identity and so on. With the rapidly growing size of accessible training data and high computation capacities, deep learning based anomaly detection…
Robustly detects jumps in high-frequency CIR and CKLS models.
problem Jump detection in high-frequency jump-diffusion processes.
method MDPDE-based robust estimators for drift and diffusion coefficients.
result Maximum of normalized residuals converges to Gumbel distribution.
Many iterative procedures in stochastic optimization exhibit a transient phase followed by a stationary phase. During the transient phase the procedure converges towards a region of interest, and during the stationary phase the procedure oscillates in that region, commonly around a single point. In this paper, we devel…
Approximate Markov chain Monte Carlo (MCMC) offers the promise of more rapid sampling at the cost of more biased inference. Since standard MCMC diagnostics fail to detect these biases, researchers have developed computable Stein discrepancy measures that provably determine the convergence of a sample to its target dist…
Detects changes in classifier scores to identify shifts in class priors.
problem Label shift changes in classification data.
method Sequential changepoint detection of classifier scores.
result Outperforms other detection procedures in label shift settings.
Improved defect detection in layered materials using signal separation methods.
problem Challenging defect detection due to strong clutter in layered structures.
method Joint rank and sparsity minimization with an iteratively reweighted nuclear and ℓ1−norm approach, combined with deep learning for parameter optimization. result The proposed approach outperforms conventional methods in terms of accuracy and speed of convergence.
NECO detects out-of-distribution data using neural collapse properties.
problem Detecting out-of-distribution data in machine learning models.
method NECO leverages neural collapse geometric properties to identify OOD data.
result NECO achieves state-of-the-art results on OOD detection tasks.
Gradient boosting can be seen as Gaussian process inference.
problem Improving uncertainty estimates in out-of-domain detection.
method Gradient boosting reformulated as a kernel method converging to Gaussian process inference.
result Gradient boosting can provide better uncertainty estimates through Monte-Carlo estimation of posterior variance.
Improved YOLOv5 model detects mask-wearing with enhanced accuracy.
problem Detecting mask-wearing in high traffic public places.
method Improved YOLOv5l with Multi-Head Attentional Self-Convolution, Swin Transformer Block, I-CBAM module, and enhanced feature fusion.
result 1.1% improvement in mAP(0.5) and 1.3% improvement in mAP(0.5:0.95) compared to YOLOv5l.
Traditional error detection approaches require user-defined parameters and rules. Thus, the user has to know both the error detection system and the data. However, we can also formulate error detection as a semi-supervised classification problem that only requires domain expertise. The challenges for such an approach a…
The Infinite Relational Model (IRM) is a probabilistic model for relational data clustering that partitions objects into clusters based on observed relationships. This paper presents Averaged CVB (ACVB) solutions for IRM, convergence-guaranteed and practically useful fast Collapsed Variational Bayes (CVB) inferences. W…
New method improves community detection for large networks.
problem Inefficient community detection for large sparse networks.
method Decouples row and column labels in likelihood function for fast alternating maximization.
result Strongly consistent estimates of communities with provable convergence guarantee.
Model detects market anomalies using a Hawkes process with hidden Markov chain.
problem Detecting high-frequency market manipulation in cryptocurrency trades.
method Developed a Markov-modulated Hawkes process with piecewise constant excitation kernels.
result Demonstrated the model's effectiveness in detecting suspicious trading activities.
Feature selection places an important role in improving the performance of outlier detection, especially for noisy data. Existing methods usually perform feature selection and outlier scoring separately, which would select feature subsets that may not optimally serve for outlier detection, leading to unsatisfying perfo…
KRCD detects unobserved confounders in nonlinear observational data.
problem Detecting unobserved confounders in nonlinear observational studies.
method Kernel Regression Confounder Detection (KRCD) using reproducing kernel Hilbert spaces.
result KRCD outperforms existing methods and achieves superior computational efficiency.
P-OCS detects OOD samples in a low-dimensional subspace, outperforming existing methods.
problem Efficient OOD detection for deep learning models in open-world environments.
method P-OCS operates in the orthogonal complement of the principal subspace, applying a single projected perturbation.
result P-OCS achieves state-of-the-art OOD detection with negligible computational cost and without requiring model retraining.
We propose a new unsupervised and non-parametric method to detect change points in intricate quasi-periodic signals. The detection relies on optimal transport theory combined with topological analysis and the bootstrap procedure. The algorithm is designed to detect changes in virtually any harmonic or a partially harmo…
GBHT uses gradient boosting for density estimation with theoretical guarantees.
problem Density estimation for unsupervised learning.
method Gradient Boosting Histogram Transform (GBHT) with Negative Log Likelihood loss.
result GBHT achieves faster convergence rates and better performance than base learners in density estimation.
PAD offers a principled approach to malware detection against evasion attacks.
problem Machine Learning techniques for malware detection are vulnerable to evasion attacks.
method PAD proposes a new adversarial training framework with convergence guarantees for robust optimization.
result PAD significantly outperforms state-of-the-art defenses and can harden ML-based malware detection against 27 evasion attacks.
BayPrAnoMeta tackles few-shot industrial image anomaly detection with Bayesian methods.
problem Challenges in industrial image anomaly detection, especially class imbalance and scarcity of labeled samples.
method Bayesian Proto-MAML approach with probabilistic normality models and Bayesian posterior predictive likelihood.
result Consistent and significant AUROC improvements over existing methods in few-shot anomaly detection.
Unified piecewise-linear Ricci flows improve community detection.
problem Improving community detection in graph neural networks.
method Proposed piecewise-linear Ricci curvature flows with surgeries.
result Flow consistently outperforms baseline models on real-world datasets.
New method detects information leakage using approximate Bayes predictor.
problem Unintentional exposure of sensitive information via observable data.
method Statistical learning theory and information theory framework, approximating Bayes predictor's log-loss and accuracy.
result MI can be accurately estimated to detect ILs, outperforming state-of-the-art baselines.
Symmetric nonnegative matrix factorization (SymNMF) has important applications in data analytics problems such as document clustering, community detection and image segmentation. In this paper, we propose a novel nonconvex variable splitting method for solving SymNMF. The proposed algorithm is guaranteed to converge to…
Under Markovian assumptions, we leverage a Central Limit Theorem (CLT) for the empirical measure in the test statistic of the composite hypothesis Hoeffding test so as to establish weak convergence results for the test statistic, and, thereby, derive a new estimator for the threshold needed by the test. We first show t…
Study detects signals in spiked Wigner models using log likelihood ratio.
problem Detecting signals in rank-one spiked Wigner models with non-Gaussian noise.
method Proved asymptotic normality of log likelihood ratio and computed error thresholds.
result Optimal signal-to-noise ratio threshold for reliable detection.
In this paper, we explore techniques centered around periodic sampling of model weights that provide convergence improvements on gradient update methods (vanilla \acs{SGD}, Momentum, Adam) for a variety of vision problems (classification, detection, segmentation). Importantly, our algorithms provide better, faster and …
Unified AI detection framework for various artifacts.
problem Effective oversight and regulation of AI deployment.
method Unified detection framework based on Mahalanobis distance scores (MDS).
result Efficient and robust estimation of covariance matrix for positive samples.
In emerging Internet-of-Nano-Thing (IoNT), information will be embedded and conveyed in the form of molecules through complex and diffusive medias. One main challenge lies in the long-tail nature of the channel response causing inter-symbol-interference (ISI), which deteriorates the detection performance. If the channe…
Framework detects changes in causal dependence between variables.
problem Detecting changes in causal dependence between variables in the presence of confounders.
method Non-parametric approach using kernel mean embeddings and copulas.
result Proposed statistic accurately detects changes in causal dependence.
Data-driven methods such as convolutional neural networks (CNNs) are known to deliver state-of-the-art performance on image recognition tasks when the training data are abundant. However, in some instances, such as change detection in remote sensing images, annotated data cannot be obtained in sufficient quantities. In…
We propose a general formalism of iterated random functions with semigroup property, under which exact and approximate Bayesian posterior updates can be viewed as specific instances. A convergence theory for iterated random functions is presented. As an application of the general theory we analyze convergence behaviors…
New method predicts positive samples with missing labels.
problem Missing labels due to response-dependent factors.
method P(U)U-O-Mixture algorithm for joint estimation.
result Non-convex algorithm leads to optimal statistical error.
Phoneme boundary detection plays an essential first step for a variety of speech processing applications such as speaker diarization, speech science, keyword spotting, etc. In this work, we propose a neural architecture coupled with a parameterized structured loss function to learn segmental representations for the tas…
Dynamic Influence Tracker measures changing sample importance during model training.
problem Static influence measurements during training overlook how sample importance varies over time.
method Dynamic Influence Tracker (DIT) captures time-varying sample influence across arbitrary time windows.
result DIT reveals distinct learning phases with shifting priorities and detects corrupted samples more efficiently.
New algorithms improve community detection in network data with strong consistency.
problem Challenges in effectively adapting spectral clustering techniques and achieving strong consistency in label recovery.
method Proposed Thresholded Cosine Spectral Clustering (TCSC) and one-step Refined TCSC algorithms, with strong consistency proofs.
result One-step Refined TCSC achieves strong consistency in community detection under PABM, correctly recovering all labels with high probability.
BRDAD uses bagging and regularization to improve anomaly detection without labeled data.
problem Anomaly detection in unlabeled data with sensitivity to k-nearest neighbors. method Bagged regularized k-distances (BRDAD) for anomaly detection, converting to convex optimization. result BRDAD addresses sensitivity to hyperparameter choice and improves performance on large datasets.
New method detects change-points in population genetics.
problem Identifying homozygosity islands in a population.
method Penalized maximum likelihood approach with dynamic programming and greedy algorithms.
result Consistent detection of homozygosity islands in population genetics.
This paper introduces a new notion of dimensionality of probabilistic models from an information-theoretic view point. We call it the "descriptive dimension"(Ddim). We show that Ddim coincides with the number of independent parameters for the parametric class, and can further be extended to real-valued dimensionality w…
Graph signal processing detects hallucinations in large language models.
problem Detecting factual reasoning from hallucinations in large language models.
method Modeling transformer layers as dynamic graphs, using spectral analysis to define diagnostics.
result Spectral signatures can distinguish different types of hallucinations and achieve high accuracy.
TULiP estimates uncertainty for deep learning models safely.
problem Reliable uncertainty estimation for deep learning models in the open world.
method TULiP considers a hypothetical perturbation, bounds its effect, and computes uncertainty from sampled predictions.
result TULiP achieves state-of-the-art performance in OOD detection benchmarks.
In this paper we analyze the asymptotic properties of l1 penalized maximum likelihood estimation of signals with piece-wise constant mean values and/or variances. The focus is on segmentation of a non-stationary time series with respect to changes in these model parameters. This change point detection and estimation pr…