Optimal iterative thresholding algorithms improve upon hard and soft thresholding.
problem Optimizing sparsity or rank constraints in optimization problems.
method Developed the notion of relative concavity for thresholding operators, finding a new class of operators that are optimal.
result A new class of thresholding operators, including ℓ q \ell_q ℓ q thresholding and reciprocal thresholding, achieves the strongest convergence guarantee. Improved bounds on combining hypothesis classes for binary functions.
problem Understanding how to combine hypothesis classes for binary functions.
method Established upper bounds on Littlestone and threshold dimensions for combined classes.
result Upper bounds are nearly tight and give exponential improvements.
SpaRCe optimizes reservoir computing by learning neuron thresholds to improve performance and prevent forgetting.
problem Improving performance and preventing forgetting in reservoir computing networks.
method Integrates neuron-specific learnable thresholds to optimize sparsity without altering dynamics, learning read-out weights and thresholds via gradient rule.
result Threshold learning improves performance and alleviates catastrophic forgetting.
Flexible per-class regularization improves binary classifiers.
problem Improving binary classifiers by addressing outliers and class imbalance.
method Graph-based adaptive regularization with flexible per-class thresholds.
result Flexible thresholds improve classifier performance and address class imbalance.
Proposes a conservative LR estimator for infrequent data near a frequency threshold.
problem Overestimation of likelihood ratios for infrequent data near a frequency threshold.
method Conservative likelihood ratio estimator for frequencies slightly above a threshold.
result Improves prediction accuracy in named entity context prediction.
A new SSL method uses instance-dependent thresholds to improve accuracy.
problem Improving semi-supervised learning by better selecting confident unlabeled instances.
method Proposes instance-dependent thresholds that vary based on the ambiguity and error rates of pseudo-labels for each unlabeled instance.
result Demonstrates that instance-dependent thresholds provide a probabilistic guarantee for correct pseudo-labels.
The article examines different thresholding methods for improving PAM algorithm in cancer classification.
problem High-dimensional classification with too many features selected by PAM.
method Extends PAM with hard and order thresholding methods and a deep search algorithm.
result Improved cancer status prediction accuracy and smaller number of features.
A new adaptive binarization technique using fuzzy integrals improves image quality.
problem Improving image thresholding quality.
method FLAT (Fuzzy Local Adaptive Thresholding) based on fuzzy integrals.
result The proposed FLAT method produces better image quality than traditional algorithms and neural networks.
Paper discusses new stochastic algorithms for sparse signal recovery.
problem Sparse signal recovery in medical imaging and remote sensing.
method Proposes and analyzes stochastic natural thresholding algorithms.
result Demonstrates improved performance of StoNT algorithms.
Improved learning bounds for corrupted data using thresholded gradient descent.
problem Learning from corrupted data with adversarial noise.
method Thresholded gradient descent for sigmoidal, leaky-ReLU, and ReLU activations.
result Improved approximation bounds for various activation functions.
Improves graph recovery in Gaussian graphical modeling.
problem Calibrating regularization parameters for graph recovery.
method Thresholded adaptive validation applied to graphical lasso.
result Thresholding pipeline improves graph recovery.
A new algorithm improves sample complexity for thresholding in Monte Carlo Tree Search.
problem Determining if the root node value of a tree is at least a given threshold.
method Developed a δ-correct sequential sampling algorithm based on the Track-and-Stop strategy.
result Ratio-based modification of D-Tracking strategy reduces sample complexity and computational cost.
Sparse reconstruction approaches using the re-weighted l1-penalty have been shown, both empirically and theoretically, to provide a significant improvement in recovering sparse signals in comparison to the l1-relaxation. However, numerical optimization of such penalties involves solving problems with l1-norms in the ob…
This work interprets GELU and related activations via a first-order loss function.
problem Understanding and optimizing activation functions in neural networks.
method Complementary interpretation using the Gaussian first-order loss function.
result Calibrated or learned uniform-threshold gates are competitive and often outperform GELU, ReLU, and SiLU/Swish.
Study membership inference under skewed priors and adaptive thresholds, improving attack accuracy.
problem Membership inference in imbalanced settings with selective thresholding.
method Developed PPV metric for skewed priors, threshold selection procedure, and a new inference attack.
result Improved inference attack accuracy in imbalanced settings.
We improve deep threshold networks' memorization capacity exponentially.
problem Memorizing datasets with randomized labels using deep neural networks.
method Using Gaussian random weights in the first layer and binary or integer weights in subsequent layers, we prove a new dependence on minimum distance.
result We show that O ~ ( 1 δ + n ) \widetilde{\mathcal{O}}(\frac{1}{\delta} + \sqrt{n}) O ( δ 1 + n ) neurons and O ~ ( d δ + n ) \widetilde{\mathcal{O}}(\frac{d}{\delta} + n) O ( δ d + n ) weights are sufficient. Synthetic data augmentation can improve imbalanced classification metrics.
problem Improving imbalanced classification metrics
method Developing a framework for analyzing the effects of synthetic data augmentation on score-based classification
result Augmentation can improve AUROC, AUPRC, balanced accuracy, and F1 score
New algorithm identifies good arms with fewer samples when thresholds are close.
problem Good arm identification in bandit problems with small threshold gaps.
method Proposes lil'HDoC algorithm to improve GAI under small threshold gaps.
result Sample complexity of first λ output arm is nearly identical to HDoC algorithm when thresholds are close.
Improves interpretability of anomaly scores in GBRBM-based detection.
problem Difficulty in setting a proper threshold for anomaly scores.
method Proposes a measure based on cumulative distribution and uses simulated annealing for evaluation.
result Established a guideline for setting the threshold using the interpretable measure.
Paper improves gas species identification in complex mixtures using neural networks.
problem Identifying gas species in multi-gas mixtures with high accuracy.
method Multi-label neural networks with optimal thresholding for IR spectroscopy.
result Optimal thresholding improves classification performance over conventional methods.
This paper explains a mechanism called phase collapse that improves image classification accuracy.
problem Understanding the role of non-linearities and convolutional filters in image classification.
method Demonstrates phase collapse as a mechanism that eliminates spatial variability and linearly separates classes.
result Phase collapse improves classification accuracy, while thresholding operators degrade performance.
Improved iterative hard thresholding for faster, sparser solutions.
problem Finding sparser solutions without sacrificing runtime.
method Adaptive regularization framework applied to iterative hard thresholding.
result Returns solutions with sparsity O ( s κ ) O(sκ) O ( s κ ) , improving over existing methods. This paper considers the multi-armed thresholding bandit problem -- identifying all arms whose expected rewards are above a predefined threshold via as few pulls (or rounds) as possible -- proposed by Locatelli et al. [2016] recently. Although the proposed algorithm in Locatelli et al. [2016] achieves the optimal round…
Optimal method detects jumps in jump-diffusion processes.
problem Detecting jumps in jump-diffusion processes with improved finite-sample performance.
method Iterative threshold-kernel method to optimally select threshold parameter.
result Approximate optimal threshold depends on spot volatility, jump intensity, and jump density.
Heuristic tool estimates lactate threshold for easier training decisions.
problem Improving lactate threshold estimation for recreational runners.
method Formalized lactate threshold principles, iterative methodology, heuristic approach.
result Heuristic %60 of 'endurance running speed reserve' is reliable and accessible.
We propose a streaming submodular maximization algorithm "stream clipper" that performs as well as the offline greedy algorithm on document/video summarization in practice. It adds elements from a stream either to a solution set S S S or to an extra buffer B B B based on two adaptive thresholds, and improves S S S by a final…
We define the information threshold in Bayesian decision-making.
problem Understanding the optimal amount of information for reliable classification.
method Defining the information threshold as the point of maximum curvature in the prior vs. posterior curve.
result At the information threshold, additional evidence does not significantly improve posterior probability.
Finite-time queue peaks in stochastic networks have logarithmic scaling after geometric thresholds.
problem Queue peak laws in stochastic networks with geometric thresholds.
method Self-normalization mechanism
result Logarithmic scaling of queue peaks after geometric thresholds.
AIHT improves online high-dimensional quantile regression by separating support discovery and refinement.
problem Online high-dimensional quantile regression with structural sparsity.
method Adaptive Iterative Hard Thresholding (AIHT) alternates stochastic updates with adaptive hard-thresholding steps.
result AIHT achieves logarithmic regret for the sliding-window objective in high-dimensional settings.
New RL algorithm learns optimal policies for MDPs using known structure.
problem Overcoming curse of dimensionality and modeling in MDPs.
method Structure-aware online learning algorithm exploiting known threshold policy.
result Proposed algorithm converges to optimal policy with significant speed improvements.
A RL algorithm learns optimal multi-threshold policies for MDPs.
problem Overcoming the curse of dimensionality in MDPs.
method Structure-aware RL algorithm exploiting multi-threshold optimal policies.
result The algorithm converges to the optimal policy asymptotically.
A simple model explains phase transition in large language models.
problem Understanding the emergence of abilities in large language models.
method Modeling LLM as a sequence-to-sequence random function and using a list decoder.
result A critical threshold exists where the expected number of erroneous sequences grows exponentially.
Improves policies with high certainty, even in small samples.
problem Ensuring new policies are better than the baseline with high probability.
method Leverages powerful safety tests and multiple testing for threshold policies.
result Controls the rate of adopting a worse policy to pre-specified error level.
DSSP improves deep learning training speed by dynamically adjusting staleness thresholds.
problem Time-consuming deep learning training on large datasets.
method Dynamic Stale Synchronous Parallel (DSSP) framework that adapts staleness threshold at runtime.
result DSSP converges faster and achieves higher accuracy than other paradigms.
New technique reduces gender discrimination in credit lending models.
problem Bias and unfairness in credit lending predictions.
method Subgroup Threshold Optimizer (STO) technique.
result Reduces gender discrimination by over 90%.
Square percolation determines threshold for group divergence in random graphs.
problem Threshold for quadratic divergence in random right-angled Coxeter groups.
method Square-graph analysis of random graphs to determine connectivity and divergence.
result Threshold probability for quadratic divergence is \( p_c(n) = \sqrt{\sqrt{6}-2}/\sqrt{n} \).
Lasso-Zero improves support recovery in high-dimensional linear models.
problem Support recovery in high-dimensional linear models with sparse β 0 β^0 β 0 . method Lasso-Zero uses an 'overfit, then threshold' approach with noise dictionaries.
result Lasso-Zero outperforms competitors in support recovery and trade-off between true positives and false discoveries.
Robust learning mixtures of linear regressions improve robustness.
problem Improving robustness in learning mixtures of linear regressions.
method Connecting mixtures of linear regressions and mixtures of Gaussians with thresholding for a quasi-polynomial time algorithm.
result The algorithm has significantly better robustness than previous results.
SoftAD improves classification accuracy with less fine-tuning and fewer computational costs.
problem Improving classification accuracy with less fine-tuning and fewer computational costs.
method SoftAD is a softened, pointwise mechanism that downweights borderline points and limits the effects of outliers.
result SoftAD achieves classification accuracy competitive with flooding and SAM, with a smaller loss generalization gap and model norm.
Paper proposes a method to improve variational inference for sparse networks.
problem Variational inference struggles with sparse networks, leading to inaccurate community detection.
method The method involves hard thresholding the posterior of community assignment after each iteration.
result The proposed method accurately recovers true community labels in sparse networks.
A simple thresholding technique improves graph selection in neural connectivity studies.
problem Graphical model selection for functional neural connectivity in the presence of latent variables.
method Apply a hard thresholding operator to graphical Lasso, neighborhood selection, or CLIME estimators.
result Thresholded estimators outperform existing methods in graph selection consistency and empirical results.
Multi-task feature learning aims to identity the shared features among tasks to improve generalization. It has been shown that by minimizing non-convex learning models, a better solution than the convex alternatives can be obtained. Therefore, a non-convex model based on the capped- ℓ 1 , ℓ 1 \ell_{1},\ell_{1} ℓ 1 , ℓ 1 regularization wa…
RPSF improves shapelet-based random forest accuracy and speed.
problem Limitations of random shapelet forest, including high training cost and limited information per node.
method Combines pairs of shapelets, omits threshold searching, and uses DMDI for interpretability.
result Improves accuracy and training speed of shapelet-based random forest.
Paper analyzes robust matrix completion with efficient nonconvex method and leave-one-out analysis.
problem Robust matrix completion with sparse noise.
method Alternates between projected gradient step for low-rank and thresholding step for sparse noise.
result Achieves linear convergence for general thresholding functions.
This paper calibrates Gaussian process predictive distributions for Bayesian optimization to improve sampling decisions.
problem Lower-tail miscalibration in GP predictive distributions affects BO sampling decisions.
method Introduces goal-oriented calibration for GP predictive distributions below a threshold t t t . result Post-hoc method tcGP improves lower-tail calibration and BO performance.
Monotone neural networks can approximate and interpolate functions efficiently.
problem Understanding the efficiency and expressiveness of monotone neural networks.
method Solving the monotone interpolation problem using depth-4 networks and comparing size bounds with arbitrary networks.
result Monotone neural networks can approximate and interpolate functions efficiently, but may require exponential size in high dimensions.
Developed a new thresholding method that connects soft and hard thresholding.
problem Connecting soft and hard thresholding methods in data analysis.
method Scaled soft thresholding method with empirical scaling values.
result Found two sources of over-fitting in the scaled soft thresholding method.
IHT improves sparse distribution learning.
problem Learning sparse discrete distributions.
method Iterative hard thresholding as a solution, with a greedy approximate projection.
result IHT achieves state of the art results for sparse distribution learning.