The paper reconciles two conflicting fairness criteria in algorithmic risk scores.
problem How to reconcile calibration and equal error rates in algorithmic risk scores.
method Derive necessary and sufficient conditions for existence of calibrated scores achieving equal error rates, then present an algorithm to find the most accurate score subject to both criteria.
result The method can eliminate error disparities while maintaining calibration and improve profit in credit lending.
Study combines expert advice to avoid discrimination without violating equalized error rates.
problem Combining expert advice to avoid discrimination without violating equalized error rates.
method Running separate instances of the classical multiplicative weights algorithm for each group.
result Even for equalized error rates, algorithms with stronger performance guarantees than multiplicative weights cannot preserve non-discrimination.
We show how to compute the Bayes error-rate for speaker verifiers.
problem How many errors does a speaker verifier make in a hundred trials?
method We compute the Bayes error-rate using calibrated likelihood ratios and user-supplied prior probabilities.
result The Bayes error-rate is upper bounded by the minimum of EER, P, and 1-P.
Develops a fair classifier for deep learning models.
problem Ensuring fairness in classification models across different sub-populations.
method Applies Rawlsian principles to minimize error rate on the worst-off sub-population.
result Introduces a practical method to adapt any black-box deep learning model to be fair.
Estimates for neural network risk nearly match Monte Carlo error rates.
problem Understanding the performance of two-layer neural networks.
method Established a priori estimates for the population risk of two-layer neural networks.
result The new estimates are nearly optimal and depend only on function norms, not model parameters.
Neural networks can approximate rectifiable measures with small error.
problem Approximating complex rectifiable measures using neural networks.
method Using ReLU neural networks to approximate (countably) m-rectifiable measures as push-forwards of the Lebesgue measure.
result The approximation error in terms of Wasserstein distance can be made arbitrarily small.
We revisit resampling procedures for error estimation in binary classification in terms of U-statistics. In particular, we exploit the fact that the error rate estimator involving all learning-testing splits is a U-statistic. Thus, it has minimal variance among all unbiased estimators and is asymptotically normally dis…
Adversarial ASV improves speaker verification robustness.
problem Mismatches in training, enrollment, and test conditions degrade deep speaker embeddings.
method Adversarial multi-task training to learn condition-invariant embeddings.
result 8.8% and 14.5% relative EER improvements for known and unknown conditions.
Algorithm learns fair representations without sacrificing accuracy across groups.
problem Mitigating disparity among different demographic subgroups in classification.
method Balanced error rate and conditional alignment of representations.
result Improves utility-fairness trade-off on balanced datasets.
In his seminal work, Schapire (1990) proved that weak classifiers could be improved to achieve arbitrarily high accuracy, but he never implied that a simple majority-vote mechanism could always do the trick. By comparing the asymptotic misclassification error of the majority-vote classifier with the average individual …
For fast and energy-efficient deployment of trained deep neural networks on resource-constrained embedded hardware, each learned weight parameter should ideally be represented and stored using a single bit. Error-rates usually increase when this requirement is imposed. Here, we report large improvements in error rates …
Improved RA detection with SNN outperforming baseline by 26.8% EER.
problem Improving RA detection systems' generalizability and discriminability.
method Multi-task learning with Siamese Neural Networks (SNN) and additional reconstruction loss.
result SNN outperforms baseline by 26.8% EER, and further improvement by 13.8% with reconstruction loss.
Double Q-learning has the same mean-squared error as Q-learning under certain conditions.
problem Comparing the mean-squared error of Double Q-learning and Q-learning.
method Theoretical analysis based on Lyapunov equations for both tabular and linear function approximation settings.
result The asymptotic mean-squared error of Double Q-learning is exactly equal to that of Q-learning under specific conditions.
A new method sparsifies neural networks by reducing sensitive parameters to zero.
problem Challenges of memory-limited applications due to large number of neural network parameters.
method Quantifies output sensitivity, introduces a regularization term to gradually reduce sensitive parameters.
result Surpasses most recent techniques in sparsity and error rates, achieving twice the sparsity at equal error rates in some cases.
New method uses VAEs for blind channel equalization and decoding.
problem Blind channel equalization and decoding without pilot symbols.
method Variational autoencoders (VAEs) for blind channel equalization and decoding.
result Significant improvement in error rate compared to existing methods.
PANDA improves linear discriminant analysis in high dimensions with minimal tuning.
problem Linear discriminant analysis in high-dimensional settings.
method PANDA: a tuning-insensitive method for linear discriminant analysis.
result PANDA achieves optimal convergence rates in estimation error and misclassification rate.
Bayesian method recovers causal structure in SEMs with equal error variances.
problem Recovering causal structure in SEMs with equal error variances.
method Bayesian DAG selection method using g-priors and the key property of minimum expected squared errors.
result The method consistently recovers the true graph without additional distributional assumptions.
In this paper, we prove that some Gaussian structural equation models with dependent errors having equal variances are identifiable from their corresponding Gaussian distributions. Specifically, we prove identifiability for the Gaussian structural equation models that can be represented as Andersson-Madigan-Perlman cha…
Machine learning models predict crash rates on narrow lanes.
problem Impact of narrow lanes on arterial road vehicle crashes.
method Applied random forest and least squares boosting machine learning algorithms to crash data.
result Random forest model identified as best for studying narrow lanes' safety impact.
The machine learning community has become increasingly concerned with the potential for bias and discrimination in predictive models. This has motivated a growing line of work on what it means for a classification procedure to be "fair." In this paper, we investigate the tension between minimizing error disparity acros…
Positive weights improve kernel quadrature's accuracy.
problem Improving kernel quadrature weights to be positive and stable.
method Using convex geometry to approximate the kernel mean embedding with positive weights.
result Positive weights lead to improved kernel quadrature bounds with Monte-Carlo-beating rates.
Proposes a method to learn speaker embeddings for variable duration utterances.
problem Mismatch between training and testing utterance durations degrades speaker verification performance.
method Sliding window segmentation, LSTM, attentive pooling, segment-level and utterance-level embeddings, similarity loss.
result Significant improvement in robustness for duration variant utterances.
The paper introduces a model to measure ASR fairness, addressing key issues.
problem Measuring fairness in ASR systems for different subgroups.
method Mixed-effects Poisson regression to control nuisance factors and handle unobserved heterogeneity.
result The method effectively addresses WER gaps among subgroups and is flexible for practical analyses.
A method for fair binary classification using both labeled and unlabeled data.
problem Achieving fair binary classification with equal true positive rates across sensitive groups.
method Constructive expression for a group-dependent threshold, plug-in classification procedure using labeled and unlabeled data.
result Plug-in classification procedure is statistically consistent and often superior or competitive with state-of-the-art methods.
In this paper, we implement an optical fiber communication system as an end-to-end deep neural network, including the complete chain of transmitter, channel model, and receiver. This approach enables the optimization of the transceiver in a single end-to-end process. We illustrate the benefits of this method by applyin…
Growing interest in automatic speaker verification (ASV)systems has lead to significant quality improvement of spoofing attackson them. Many research works confirm that despite the low equal er-ror rate (EER) ASV systems are still vulnerable to spoofing attacks. Inthis work we overview different acoustic feature spaces…
A new portfolio model DEWSP improves Sharpe ratio by 0.24% to 5.15%.
problem High sensitivity of optimized portfolios to estimation errors.
method Deep learning algorithms predict returns for top-N ranked assets, then equally weight them.
result DEWSPs provide an improvement rate of 0.24% to 5.15% in terms of monthly Sharpe ratio compared to HEWSPs.
In this paper, we present a reverberation removal approach for speaker verification, utilizing dual-label deep neural networks (DNNs). The networks perform feature mapping between the spectral features of reverberant and clean speech. Long short term memory recurrent neural networks (LSTMs) are trained to map corrupted…
Machine learning identifies skill levels in neurosurgery residents.
problem Assessing neurosurgical skill levels in residents.
method Machine learning applied to a VR tumor resection task with 23 skilled and 92 novice participants.
result Fuzzy K-Nearest Neighbors classifier achieved 8.3% equal error rate with 15 selected features.
The problem of f-divergence estimation is important in the fields of machine learning, information theory, and statistics. While several nonparametric divergence estimators exist, relatively few have known convergence properties. In particular, even for those estimators whose MSE convergence rates are known, the asympt…
This work compares and evaluates various sampling methods for neural language models.
problem Lack of systematic comparison and myths about sampling methods.
method Monte Carlo sampling, importance sampling, compensated partial summation, noise contrastive estimation.
result All sampling methods can perform equally well if posterior probabilities are corrected.
Study infers interaction kernels from multiple particle trajectories.
problem Inferring interaction kernels from multiple particle trajectories in stochastic systems.
method Nonparametric inference approach based on regularized maximum likelihood estimator.
result Consistent estimator with near-optimal learning rate independent of state space dimension.
New TTP framework fuses control arms while controlling Type-I error.
problem Bias in borrowing control data from previous trials.
method Kernel two-sample testing via MMD and equivalence testing.
result Higher power than standard TTP methods while maintaining error control.
Optimizer memory affects learning rate sensitivity in shuffle order, impacting fine-tuning noise.
problem Optimizer memory affects the learning rate sensitivity in shuffle order, leading to fine-tuning noise.
method Isolated the mechanism of fixed-clock optimizer memory affecting the learning rate sensitivity in shuffle order, deriving a fit-free way to size the noise.
result Fixed-clock optimizers like AdamW produce a larger first-order noise channel compared to memoryless optimizers, affecting fine-tuning comparisons.
A new feature screening method using projection correlation and knockoffs controls FDR in high-dimensional data.
problem Feature selection in ultra-high dimensional datasets with heavy-tailed errors and multivariate responses.
method Projection correlation for dependence measurement, knockoffs for FDR control, two-step approach.
result The method controls FDR and ensures sure screening under weak assumptions.
This paper tackles worst-class error rate in classification tasks.
problem Minimizing worst-class error rate in classification tasks, especially in medical image classification.
method Designing a boosting approach to bound the worst-class error rate using Deep Neural Networks (DNNs).
result The proposed boosting approach lowers worst-class test error rates while avoiding overfitting.
New research shows unlabeled data is equally valuable as labeled data in certain semi-supervised learning scenarios.
problem Improving learning performance with limited labeled data.
method Statistical models with continuous parameters, showing equal utility of unlabeled data under specific conditions.
result The learning rate of semi-supervised learning scales similarly to supervised learning when unlabeled data is abundant.
Study controls error rates of binary classifiers using hypothesis testing.
problem Traditional binary classifiers have uncontrolled error rates.
method Combines binary classification with statistical hypothesis testing.
result Trained classifiers can be made to meet target error rate thresholds.
This research examines how the error rate of nearest neighbor classifiers varies with dataset size.
problem The scaling of classification error rates with dataset size is not uniform.
method Theoretical analysis of nearest neighbor classifiers, focusing on early and late phases of dataset size.
result The error rate of nearest neighbor classifiers can have fine-grained rates depending on the dataset size and data distribution.
This chapter reviews ML resampling methods for cybersecurity.
problem Estimating ML performance in cybersecurity.
method Resampling techniques for error rate and AUC estimation.
result Established a theoretical framework for ML resampling methods.
We propose the application of a semi-supervised learning method to improve the performance of acoustic modelling for automatic speech recognition based on deep neural net- works. As opposed to unsupervised initialisation followed by supervised fine tuning, our method takes advantage of both unlabelled and labelled data…
Study improves least squares estimation for heavy-tailed errors.
problem Improving least squares estimation under heteroscedastic and heavy-tailed errors.
method Analyzes the rate of convergence of least squares estimator under bounded conditional variance and finitely many moments of errors.
result Upper bounds on rates of convergence of LSE for heavy-tailed errors are found.
The paper develops a method to accurately estimate the Bayes misclassification error rate.
problem Estimating the best achievable classifier performance without learning a Bayes-optimal classifier.
method Learning to benchmark using an ensemble of ε-ball estimators and Chebyshev approximation.
result The proposed method achieves an optimal mean squared error rate of O(N^(-1)) under a smoothness assumption.
Algorithm ensures fairness in online classification with partial feedback.
problem Fairness in online classification with partial feedback.
method Oracle efficient algorithm that satisfies fairness constraints.
result Upper and lower bounds on the cost of fairness constraints.
This paper reviews methods for constructing confidence intervals for error rates in 1:1 matching tasks.
problem Challenges in assessing uncertainty of error rates in matching algorithms, especially when data are dependent and error rates are low.
method Derives and examines statistical properties of methods for constructing confidence intervals for error rates in 1:1 matching tasks.
result Coverage and interval width vary with sample size, error rates, and data dependence.
Framework for eliciting fairness constraints from stakeholders.
problem Complex, nuanced fairness requirements not captured by simple definitions.
method Algorithm for learning accurate models subject to elicited fairness constraints.
result Provably convergent and oracle efficient algorithm with generalization bounds.
Attention-based models have recently shown great performance on a range of tasks, such as speech recognition, machine translation, and image captioning due to their ability to summarize relevant information that expands through the entire length of an input sequence. In this paper, we analyze the usage of attention mec…
New methods improve evaluation of models under varying class imbalance.
problem Optimistic evaluation metrics lead to incorrect conclusions.
method Methods focusing on evaluation under non-constant class imbalance.
result Order of classifiers can change with class imbalance rate.