Enhanced DFO using adaptive batch-based FD estimates.
problem Derivative-free optimization with imprecise gradient estimates.
method Adaptive batch-based finite difference estimation and dynamic sampling strategy.
result Algorithm achieves convergence rate similar to KW and SPSA methods.
Optimal algorithm for minimizing regret in heavy-tailed bandits.
problem Minimizing regret in stochastic multi-armed bandits with heavy-tailed distributions.
method Proposes an optimal algorithm under the assumption of uniformly bounded moments of order (1+ε).
result Matches the lower bound exactly in the first-order term and provides a finite-time bound on its regret.
In designing personalized ranking algorithms, it is desirable to encourage a high precision at the top of the ranked list. Existing methods either seek a smooth convex surrogate for a non-smooth ranking metric or directly modify updating procedures to encourage top accuracy. In this work we point out that these methods…
In this paper, we apply a mini-batch based negative sampling method to efficiently train a latent factor autoencoder model on large scale and sparse data for implicit feedback collaborative filtering. We compare our work against a state-of-the-art baseline model on different experimental datasets and show that this met…
SpecNet2 improves spectral embedding without orthogonalization, achieving better performance and efficiency.
problem Improving spectral embedding methods for better performance and efficiency.
method Optimizes an equivalent objective of the eigen-problem without orthogonalization, allowing separate row and column sampling.
result Local and global convergence of the new objective using batch-based gradient descent is proven, and improved performance and efficiency are demonstrated on simulated and image datasets.
Paper addresses OPE for dependent bandit samples using MDS and batch updates.
problem Evaluating policies from non-i.i.d. historical data in contextual bandits.
method Constructs an MDS-based estimator for dependent samples, solves batch update and deficient support issues.
result Derives an asymptotically normal estimator for evaluation policy value.
Object detection in streaming images is a major step in different detection-based applications, such as object tracking, action recognition, robot navigation, and visual surveillance applications. In mostcases, image quality is noisy and biased, and as a result, the data distributions are disturbed and imbalanced. Most…
New method improves traffic data recovery for streaming data.
problem Improve data quality in traffic data for ITS.
method Online robust tensor recovery algorithm leveraging spatio-temporal correlations and local consistency.
result Significantly improved computational efficiency and high recovery accuracy.
New algorithm reduces overfitting in neural networks.
problem Overfitting in neural networks.
method Integrates SMC with SGHMC for mini-batch sampling.
result SMCSGHMC outperforms SGD and deep ensembles.
A new learning scheme improves model efficiency and performance.
problem Characterizing correlation between batch-level and global data distributions.
method Epoch-evolving Gaussian Process Guided Learning (GPGL) scheme with context labels and triangle consistency loss.
result Significantly outperforms existing models on mainstream datasets.
Self-paced learning (SPL) mimics the cognitive process of humans, who generally learn from easy samples to hard ones. One key issue in SPL is the training process required for each instance weight depends on the other samples and thus cannot easily be run in a distributed manner in a large-scale dataset. In this paper,…
EnSF improves accuracy in tracking high-dimensional nonlinear systems.
problem Low accuracy in high-dimensional, nonlinear filtering problems.
method Score-based diffusion model, mini-batch Monte Carlo estimator.
result EnSF outperforms state-of-the-art methods in tracking high-dimensional systems.
Can one parallelize complex exploration exploitation tradeoffs? As an example, consider the problem of optimal high-throughput experimental design, where we wish to sequentially design batches of experiments in order to simultaneously learn a surrogate function mapping stimulus to response and identify the maximum of t…
HydaLearn dynamically adjusts task weights for better MTL performance.
problem Constant loss weights in MTL lead to poor results due to drifting relevance and varying mini-batch composition.
method HydaLearn uses mini-batch gradients to dynamically adjust task weights.
result HydaLearn improves performance on synthetic and real-world data.
Sparse GPs improved with nearest neighbor inducing variables.
problem Sparse GPs struggle with large numbers of inducing variables.
method Introduced a hierarchical prior for inducing variables and used nearest neighbor information for sparsity.
result Significant computational gains compared to standard sparse GPs.
NVIDIA cuDNN is a low-level library that provides GPU kernels frequently used in deep learning. Specifically, cuDNN implements several equivalent convolution algorithms, whose performance and memory footprint may vary considerably, depending on the layer dimensions. When an algorithm is automatically selected by cuDNN,…
Paper extends 2D ZSAD to 3D MRI without training, achieving robust anomaly detection.
problem Challenges in extending zero-shot anomaly detection to 3D medical images.
method Constructs localized volumetric tokens by aggregating 2D slices processed by 2D foundation models.
result Training-free, batch-based ZSAD effectively extends from 2D encoders to full 3D MRI volumes.
New auditors assess f-DP privacy with adaptive sampling, avoiding large sample sizes.
problem Empirical auditing of f-DP privacy with adaptive sampling. method Shift focus to f-DP, develop adaptive auditors for whitebox and blackbox settings. result Adaptive auditors detect f-DP violations across the privacy spectrum with statistical guarantees. We consider an online version of the robust Principle Component Analysis (PCA), which arises naturally in time-varying source separations such as video foreground-background separation. This paper proposes a compressive online robust PCA with prior information for recursively separating a sequences of frames into spars…
New method detects bearing faults using multivariate statistical process control.
problem Early detection of bearing faults in rotating machinery.
method Multivariate statistical process control charts applied to Fourier transform features of fixed-time batches.
result Effectiveness in detecting bearing faults across different conditions.
FAWMF adapts weights for implicit feedback recommendation efficiently.
problem Challenges in treating unobserved data as negative in implicit feedback recommendation.
method FAWMF uses a variational auto-encoder with a parameterized neural network to adaptively assign personalized data confidence weights, and fBGD for efficient learning.
result FAWMF and fBGD outperform existing methods in real-world datasets.
CDPL-Net integrates CNN and DL for better image representation.
problem Improving image representation learning by combining CNN and DL.
method CDPL-Net architecture combining CNN and DPL layers, using l1-norm for sparse representation, and efficient stochastic gradient descent.
result Enhanced performance compared to state-of-the-art methods.
Dissertation tackles zero-shot anomaly detection, focusing on consistent anomalies and proposing CoDeGraph framework.
problem Consistent anomalies bias distance-based zero-shot anomaly detection methods.
method Formalized consistent anomalies, identified similarity scaling and neighbor-burnout phenomena, introduced CoDeGraph framework.
result CoDeGraph effectively suppresses consistent anomalies in zero-shot anomaly detection.
New method tackles dynamic data labeling issues with limited labels.
problem Dynamic data labeling with scarce labeled instances.
method Instance exploitation technique for aggressive model adaptation.
result Aggressive model adaptation leads to better performance than standard methods.
LibAUC optimizes X-risks for AI tasks like CID, LTR, and CLR.
problem Optimizing risk functions in AI for tasks like classification, ranking, and representation learning.
method Developed a new mini-batch pipeline for deep X-risk optimization (DXO) algorithms.
result Achieved great success in solving CID, LTR, and CLR tasks with faster convergence and scalable performance.
Paper proposes a new method for efficient second-order neural network training.
problem Infeasibility of Hessian calculation and noisy second-order information in deep learning.
method Adopting complex-step directional derivative (CSFD) for accurate Hessian computation and designing an effective Newton Krylov procedure.
result Our method outperforms existing methods and often converges one-order faster.
New estimators outperform maximum likelihood without hyper-parameter estimation.
problem Improving system identification performance without hyper-parameter estimation.
method Developed generalized Bayes and closed-form biased estimators using excess MSE.
result New estimators have comparable performance to empirical-Bayes-based regularized estimator.
New estimator reduces kernel mean estimation error.
problem Kernel mean estimation in reproducing kernel Hilbert spaces.
method Corrupt data with known distributions and estimate kernel mean under the corrupted distribution.
result The marginalized kernel mean estimator achieves lower estimation error.
Dual Bayesian Affine Estimators for Wiener-type state-space models
problem Estimating parameters in Wiener-type state-space models
method Fixed-point architecture combining two affine estimators
result Dual basis-parameter estimator achieves comparable parameter MSE to purely affine estimator
Enhances gradient estimates for Hermitian Monge-Ampère equations.
problem Improving estimates for Hermitian Monge-Ampère equations.
method Improves gradient estimates using Evans-Krylov and third derivatives estimates.
result Enhanced estimates for second and third order derivatives.
Paper proposes robust estimators for GANs under Wasserstein contamination.
problem Robust estimation of distributions under contamination.
method Wasserstein GAN-based estimators for location, covariance, and regression.
result Proposed estimators are minimax optimal in many scenarios.
New framework converts offline to online estimation using black-box offline estimators.
problem Convert offline estimation algorithms to online estimation algorithms.
method Oracle-Efficient Online Estimation (OEOE) framework.
result Achieves near-optimal online estimation error via black-box offline estimators.
Proposes variational autoencoder for efficient MMSE estimation.
problem Efficient parameterized MMSE estimation for noisy observations.
method Variational autoencoder models data distribution, approximates MMSE.
result Proposed estimator performs well compared to state-of-the-art.
Paper improves Fisher information estimation methods.
problem Estimating Fisher information for location parameters.
method Revisits and improves Bhattacharya estimator, introduces clipped estimator.
result Clipped estimator shows superior convergence rates in Gaussian noise.
Proposes a robust estimator for RD designs.
problem Estimating treatment effects in RD designs.
method Doubly robust estimator combining two estimators.
result Enhances robustness of treatment effect estimators.
New estimator reduces variance in discrete random variables.
problem Estimating gradients for discrete random variables with reduced variance.
method Sampling without replacement and Rao-Blackwellization.
result Our estimator is the most consistent gradient estimator across different entropy settings.
SCOPE estimator improves covariance and precision matrix estimation.
problem Estimating covariance and precision matrices accurately.
method Distributionally robust optimization with convex spectral divergence.
result SCOPE estimator reduces spectral bias and improves condition number.
We present a multi-task learning approach to jointly estimate the means of multiple independent data sets. The proposed multi-task averaging (MTA) algorithm results in a convex combination of the single-task maximum likelihood estimates. We derive the optimal minimum risk estimator and the minimax estimator, and show t…
Obtaining more accurate equity value estimates is the starting point for stock selection, value-based indexing in a noisy market, and beating benchmark indices through tactical style rotation. Unfortunately, discounted cash flow, method of comparables, and fundamental analysis typically yield discrepant valuation estim…
The maximum mean discrepancy (MMD) is a kernel-based distance between probability distributions useful in many applications (Gretton et al. 2012), bearing a simple estimator with pleasing computational and statistical properties. Being able to efficiently estimate the variance of this estimator is very helpful to vario…
Stochastic volatility modelling of financial processes has become increasingly popular. The proposed models usually contain a stationary volatility process. We will motivate and review several nonparametric methods for estimation of the density of the volatility process. Both models based on discretely sampled continuo…
A new copula estimation method using classification.
problem Estimating copula density from joint and marginal distributions.
method Train a classifier to distinguish joint density from product of marginals.
result Empirically outperforms existing copula estimators.
This paper reviews SDR methods for multivariate response regression.
problem Handling sufficient dimension reduction for multivariate response regression.
method Characterizes SDR estimators as inverse or forward regression methods.
result Pooled marginal, projective resampling, distance-based, ordinary least squares, partial least squares, and semiparametric SDR estimators are discussed.
Density ratio estimation is a vital tool in both machine learning and statistical community. However, due to the unbounded nature of density ratio, the estimation procedure can be vulnerable to corrupted data points, which often pushes the estimated ratio toward infinity. In this paper, we present a robust estimator wh…
TAKDE optimizes kernel density estimation for real-time dynamic processes.
problem Real-time density estimation in applications like computer vision and signal processing.
method Derives asymptotic mean integrated squared error (AMISE) upper bound for 'sliding window' kernel density estimator and proposes TAKDE as a novel, theoretically optimal estimator.
result TAKDE outperforms other dynamic density estimators in terms of test log-likelihood and runtime.
We introduce two new estimators of the bivariate Hurst exponent in the power-law cross-correlations setting -- the cross-periodogram and local X-Whittle estimators -- as generalizations of their univariate counterparts. As the spectrum-based estimators are dependent on a part of the spectrum taken into consideration …
Paper bridges score estimation to parameter and density estimation in DDPMs.
problem Efficiently estimating scores for generative models.
method Introduces a framework linking score estimation to parameter and density estimation.
result Denoising score-matching in DDPMs is asymptotically efficient for parameter estimation.
New method for fast volatility estimation robust to change points.
problem Robust high-frequency volatility estimation with change points.
method ℓ1-regularized power variation estimators using LARS for sparse estimation and dynamic programming for change point refinement.
result Minimax rates achieved for volatility estimators, providing accurate and smooth forecasts.