In this paper, we study the problem of estimating the covariance matrix under differential privacy, where the underlying covariance matrix is assumed to be sparse and of high dimensions. We propose a new method, called DP-Thresholding, to achieve a non-trivial -norm based error bound, which is significantly bet…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,742 papers · 148 categories
Trend · papers per month
48 results for “norm-based thresholding”
This paper is concerned about sparse, continuous frequency estimation in line spectral estimation, and focused on developing gridless sparse methods which overcome grid mismatches and correspond to limiting scenarios of existing grid-based approaches, e.g., optimization and SPICE, with an infinitely dense grid…
COMRADE is a communication-efficient, Byzantine-resilient second-order optimization algorithm.
problem Byzantine failures in distributed optimization.
method COMRADE is a communication-efficient, second-order optimization algorithm that uses a simple norm-based thresholding rule to filter out Byzantine workers.
result COMRADE achieves linear-quadratic convergence and is robust against Byzantine workers.
The aim of this paper is to train an RBF neural network and select centers under concurrent faults. It is well known that fault tolerance is a very attractive property for neural networks. And center selection is an important procedure during the training process of an RBF neural network. In this paper, we devise two n…
We investigate the capacity, convexity and characterization of a general family of norm-constrained feed-forward networks.
Two new regularization methods improve neural network performance and complexity control.
problem Improving neural network performance and complexity control with correlated or high-dimensional features.
method Two regularization strategies: covariance-aware ridge and covariance-aware lasso.
result Improves predictive performance and complexity control over standard penalties.
Paper introduces new risk norms based on ES with flexible distortion functions.
problem Risk quantification and anomaly detection in financial data.
method Developed generalized Expected-Shortfall (ES) norms using distortion risk measures and duality theory.
result Unified analytical framework for risk quantification and practical applications.
Network anomaly detection is still a vibrant research area. As the fast growth of network bandwidth and the tremendous traffic on the network, there arises an extremely challengeable question: How to efficiently and accurately detect the anomaly on multiple traffic? In multi-task learning, the traffic consisting of flo…
The paper offers generalization bounds for Transformers that ignore sequence length.
problem Developing generalization bounds for Transformers that are independent of sequence length.
method Covering number approach to upper bound Rademacher complexity of bounded linear transformations.
result Theoretical bounds for Transformer generalization are independent of sequence length.
Unified PAC-Bayesian framework for deep learning generalization.
problem Limitations of existing PAC-Bayesian norm-based bounds for deep neural networks.
method Unified framework using anisotropic Gaussian posteriors and sensitivity matrix.
result Comparable or tighter generalization bounds compared to state-of-the-art approaches.
Paper addresses fault-tolerance in distributed machine learning with stochastic gradient descent.
problem Fault-tolerance in distributed stochastic gradient descent (D-SGD) for machine learning.
method Proposes norm-based comparative gradient elimination (CGE) to robustify D-SGD against Byzantine faulty agents.
result CGE guarantees fault-tolerance against a bounded fraction of Byzantine agents under standard stochastic assumptions.
Extends importance sampling to nonlinear models using adjoint operators.
problem Lack of tools for identifying important data points in nonlinear models.
method Introduces adjoint operator for nonlinear maps, generalizes norm and leverage scores.
result Generalized scores provide approximation guarantees for nonlinear mappings.
Study analyzes adversarial training dynamics without data distribution assumptions.
problem Understanding training dynamics of adversarial training without data distribution assumptions.
method Mean field theory approach to analyze adversarial training in random deep neural networks.
result Upper bounds of adversarial loss derived empirically and theoretically.
A toolkit for path-norms enhances neural network generalization bounds.
problem Establishing generalization bounds for modern neural networks.
method Introducing a comprehensive toolkit for path-norms in ReLU networks with various operations.
result Established generalization bounds for modern neural networks that are the most widely applicable and recover/beat the sharpest known bounds.
Advancements in parallel processing have lead to a surge in multilayer perceptrons' (MLP) applications and deep learning in the past decades. Recurrent Neural Networks (RNNs) give additional representational power to feedforward MLPs by providing a way to treat sequential data. However, RNNs are hard to train using con…
This study extends verifiable learning to boosted tree ensembles, enabling efficient security verification.
problem Efficiently verifying the robustness of boosted tree ensembles against norm-based attackers.
method Formal verification of robustness for large-spread boosted tree ensembles, considering -norm and pseudo-polynomial time for -norm verification.
result Polynomial time verification for -norm attackers, NP-hard for other norms, and pseudo-polynomial time for -norm verification.
New study shows how model complexity affects test risk, challenging classical theory.
problem Understanding how test risk scales with model complexity for large over-parametrized deep networks.
method Developed norm-based capacity measures for random features based estimators, providing precise characterization of estimator's norm concentration and test error.
result Predicted learning curve shows a phase transition from under- to over-parameterization, confirming classical U-shaped behavior with appropriate capacity measures.
We study the effectiveness of non-uniform randomized feature selection in decision tree classification. We experimentally evaluate two feature selection methodologies, based on information extracted from the provided dataset: \emph{leverage scores-based} and \emph{norm-based} feature selection. Experimenta…
Improved robustness for high-dimensional Kalman filtering.
problem Convergence issues in sequential variational inference filter (VIF).
method Variational Kalman Filtering with Hinf-based correction.
result Improved feasibility and robustness in high-dimensional systems.
New bounds adaptively control spectral complexity of trained Transformers.
problem Understanding why Transformers generalize well in machine learning.
method Spectrum-adaptive post hoc generalization bounds for multi-layer Transformers.
result Bounds adaptively trade off spectral complexity against dimension and depth factors.
Decision-calibrated prediction sets improve power system operations by reducing unnecessary costs.
problem Balancing operating costs and reliability in power systems with renewable uncertainty.
method Learn conditional prediction sets as sub-level sets of norm-based score functions, calibrate uncertainty sets based on reliability of downstream decisions.
result Decision-calibrated sets lead to more efficient operations with smaller uncertainty sets and lower costs compared to standard coverage-based calibration.
SWRLDA improves LDA for multi-class classification with edge classes.
problem LDA's vulnerability to edge classes causing biased mean and large distances.
method Self-weighted robust LDA with l21-norm distance criterion.
result SWRLDA outperforms other methods on synthetic and real-world datasets.
New methods predict neural network quality without access to training data.
problem Predicting neural network quality without access to training or testing data.
method Meta-analysis of pretrained models using norm and power law based metrics.
result Power law based metrics can better distinguish well-trained from poorly-trained models.
Semantify-NN verifies neural network robustness against semantic perturbations.
problem Verifying robustness of neural networks against semantic adversarial attacks.
method Inserting semantic perturbation layers (SP-layers) into neural networks to verify robustness.
result Semantify-NN significantly improves robustness verification performance over -norm-based methods.
In this paper we consider the low-rank matrix completion problem with specific application to forecasting in time series analysis. Briefly, the low-rank matrix completion problem is the problem of imputing missing values of a matrix under a rank constraint. We consider a matrix completion problem for Hankel matrices an…
Approximate dynamic programming is a popular method for solving large Markov decision processes. This paper describes a new class of approximate dynamic programming (ADP) methods- distributionally robust ADP-that address the curse of dimensionality by minimizing a pessimistic bound on the policy loss. This approach tur…
Iterative thresholding algorithms seek to optimize a differentiable objective function over a sparsity or rank constraint by alternating between gradient steps that reduce the objective, and thresholding steps that enforce the constraint. This work examines the choice of the thresholding operator, and asks whether it i…
Developed a new thresholding method that connects soft and hard thresholding.
problem Connecting soft and hard thresholding methods in data analysis.
method Scaled soft thresholding method with empirical scaling values.
result Found two sources of over-fitting in the scaled soft thresholding method.
A new algorithm solves the metric nearness problem efficiently.
problem Finding the nearest distance matrix that satisfies triangle inequalities.
method Delayed constraint generation with semismooth Newton based proximal augmented Lagrangian method (PALM).
result Solves problems with up to 10^8 variables and 10^13 constraints efficiently.
In this paper, we propose an unifying view of several recently proposed structured sparsity-inducing norms. We consider the situation of a model simultaneously (a) penalized by a set- function de ned on the support of the unknown parameter vector which represents prior knowledge on supports, and (b) regularized in Lp-n…
Paper proposes a new tensor imputation method for spatiotemporal traffic data with missing patterns.
problem Imputation of corrupted or incomplete traffic data.
method Truncated tensor Schatten p-norm (TSpN) for spatiotemporal traffic data imputation.
result The proposed method outperforms other state-of-the-art tensor-based imputation models in various missing cases.
Joint sparsity offers powerful structural cues for feature selection, especially for variables that are expected to demonstrate a "grouped" behavior. Such behavior is commonly modeled via group-lasso, multitask lasso, and related methods where feature selection is effected via mixed-norms. Several mixed-norm based spar…
We study the relationship between geometry and capacity measures for deep neural networks from an invariance viewpoint. We introduce a new notion of capacity --- the Fisher-Rao norm --- that possesses desirable invariance properties and is motivated by Information Geometry. We discover an analytical characterization of…
New algorithm for reinforcement learning in uncertain environments with unknown thresholds.
problem Safety in reinforcement learning in unknown and uncertain environments.
method Growing-Window estimator sampling and Stochastic Pessimistic-Optimistic Thresholding (SPOT) algorithm.
result Achieves sublinear regret and constraint violation of .
A new SSL method uses instance-dependent thresholds to improve accuracy.
problem Improving semi-supervised learning by better selecting confident unlabeled instances.
method Proposes instance-dependent thresholds that vary based on the ambiguity and error rates of pseudo-labels for each unlabeled instance.
result Demonstrates that instance-dependent thresholds provide a probabilistic guarantee for correct pseudo-labels.
Study stability thresholds of big line bundles, proving bounds and generalizing results.
problem Stability thresholds of big line bundles and their asymptotic behavior.
method Explicit bounds on error terms, using quasi-monomial valuations to compute stability thresholds.
result Proves Jin--Rubinstein--Tian's questions affirmatively.
Adaptive algorithm for outlier detection by balancing arm exploration and threshold estimation.
problem Identifying outliers in a set of rewards where the threshold is a function of all rewards.
method Adaptively updated confidence interval for the threshold based on previous rounds' estimates, balancing exploration of individual arms and the outlier threshold.
result Efficient algorithm with reduced sample complexity for outlier detection.
The Identification of Thresholds and Time Delay in Self-Exciting Threshold AR Model by Waveletmath-ph
In this paper we studied about the wavelet identification of the thresholds and time delay for more general case without the constraint that the time delay is smaller than the order of the model. Here we composed an empirical wavelet from the SETAR (Self-Exciting Threshold Autoregressive) model and identified the thres…
The article examines different thresholding methods for improving PAM algorithm in cancer classification.
problem High-dimensional classification with too many features selected by PAM.
method Extends PAM with hard and order thresholding methods and a deep search algorithm.
result Improved cancer status prediction accuracy and smaller number of features.
In the work of Ammann, Dahl and Humbert it has turned out that the Yamabe invariant on closed manifolds is a bordism invariant below a certain threshold constant. A similar result holds for a spinorial analogon. These threshold constants are characterized through Yamabe-type equations on products of spheres with rescal…
Although the threshold network is one of the most used tools to characterize the underlying structure of a stock market, the identification of the optimal threshold to construct a reliable stock network remains challenging. In this paper, the concept of dynamic consistence between the threshold network and the stock ma…
This paper presents a general framework for norm-based capacity control for weight normalized deep neural networks. We establish the upper bound on the Rademacher complexities of this family. With an normalization where , and , we discuss properties of a width-independent ca…
Conservative Likelihood Ratio Estimator for Infrequent Data Slightly above a Frequency Thresholdstat.ML
Proposes a conservative LR estimator for infrequent data near a frequency threshold.
problem Overestimation of likelihood ratios for infrequent data near a frequency threshold.
method Conservative likelihood ratio estimator for frequencies slightly above a threshold.
result Improves prediction accuracy in named entity context prediction.
New method trains neural networks with threshold activation functions efficiently.
problem Training neural networks with threshold activation functions is challenging due to zero gradients.
method We study weight decay regularized training problems of deep neural networks with threshold activations, showing they can be formulated as convex optimization problems.
result Regularized deep threshold network training problems can be formulated as standard convex optimization problems, paralleling the LASSO method.
Noise makes learning linear thresholds hard, but algorithms can still learn near-optimal thresholds.
problem Learning linear thresholds in noisy data.
method Exploiting natural assumptions on data-generating process.
result Efficient learning of near-optimal linear thresholds is still possible with small data even in the presence of noise.
Bayesian framework proves thresholds for multi-graph alignment feasibility.
problem Determining when multi-graph alignment is statistically possible.
method Developed a Bayesian estimation framework over metric spaces.
result Identified thresholds for Gaussian and sparse Erdős-Rényi models.
Polynomial neural networks explore thresholds for maximum expressiveness.
problem Understanding the limits of polynomial neural networks' expressiveness.
method Introducing activation degree threshold to measure network expressiveness and proving its existence and upper bounds.
result Polynomial neural networks with equi-width architectures achieve the maximum expressiveness.
Paper introduces threshold invariant fairness to ensure equitable predictions across different groups.
problem Machine learning models can be unfair to certain groups based on sensitive attributes.
method Proposes threshold invariant fairness and uses two approximation methods to equalize risk distributions.
result Demonstrates effectiveness in alleviating threshold sensitivity in fairness models.