This work studies scaling laws for low-precision training in high-dimensional linear regression.
problem Optimizing trade-off between model quality and training costs in high-dimensional linear regression.
method Theoretical study of scaling laws for low-precision training within a high-dimensional sketched linear regression framework, analyzing multiplicative and additive quantization.
result Multiplicative quantization maintains full-precision model size, while additive quantization reduces effective model size.
An algorithm reduces breast cancer detection data complexity using effect sizes.
problem Improving accuracy in breast cancer detection.
method Statistical feature selection and SVM classifier with linear kernel.
result SVM classifier achieved over 90% accuracy.
Two methods estimate effect size for online experiments, improving accuracy and efficiency.
problem Determining the correct effect size for online experiment duration.
method Two approaches: hierarchical models and utility theory.
result Proposed methods outperform baseline approaches in accuracy and efficiency.
MiFID II impacts European stock liquidity and price formation.
problem Impact of MiFID II on European stock liquidity and price formation.
method Analyzed effects of MiFID II on European stock markets, focusing on intraday and closing auction liquidity and tick size changes.
result Closing auction volumes increased and price formation became more efficient after MiFID II.
We consider large-scale studies in which it is of interest to test a very large number of hypotheses, and then to estimate the effect sizes corresponding to the rejected hypotheses. For instance, this setting arises in the analysis of gene expression or DNA sequencing data. However, naive estimates of the effect sizes …
A tick size is the smallest increment of a security price. It is clear that at the shortest time scale on which individual orders are placed the tick size has a major role which affects where limit orders can be placed, the bid-ask spread, etc. This is the realm of market microstructure and there is a vast literature o…
A new method calculates intrinsic effective sample size for manifold-valued data.
problem Challenges in choosing effective sample size for manifold-valued data.
method Proposes an intrinsic effective sample size based on kernel discrepancy.
result Establishes an exact finite-sample risk interpretation and consistency of the estimator.
Study finds optimal vocabulary size for neural machine translation.
problem Imbalanced class distribution in language data affects NMT performance.
method Casts NMT as a classification task, analyzes vocabulary sizes, and tests multiple languages.
result Certain vocabulary sizes outperform others, explaining NMT performance.
Estimates effect sizes and power from a pilot experiment.
problem Estimating the distribution of effect sizes in multiple testing settings.
method Uses an inexpensive pilot experiment to estimate effect sizes and power.
result Simple and computationally efficient estimator guarantees the number of discoveries.
Large batch sizes reduce gradient variance in DP-SGD, improving privacy.
problem Understanding why large batch sizes work in DP-SGD.
method Decomposed total gradient variance into subsampling and noise-induced variances, proving batch size independence in the limit.
result Large batch sizes reduce effective total gradient variance, improving privacy in DP-SGD.
New summary measures reveal geometric structure in weighted measures on manifolds.
problem Lack of geometric information in standard weight-only summaries.
method Heat-kernel entropy profiles, tracking nonuniformity across scales.
result Geometric effective sample size discounts nearby or duplicate particles.
We consider the roughness properties of NYSE (New York Stock Exchange) stock-price fluctuations. The statistical properties of the data are relatively homogeneous within the same day but the large jumps between different days prevent the extension of the analysis to large times. This leads to intrinsic finite size effe…
This paper explores how effective sample size, dimensionality, and model performance are related in covariate shift adaptation.
problem Understanding the relationship between effective sample size, dimensionality, and generalization in covariate shift adaptation.
method Building a unified theory connecting effective sample size, data dimensionality, and generalization in the context of covariate shift adaptation.
result Dimensionality reduction or feature selection can increase effective sample size, supporting the practice of reducing dimensionality before covariate shift adaptation.
Study reveals finite-size effects and sensitivity to random numbers in Levy-Levy-Solomon model.
problem Finite-size effects and sensitivity to random numbers in Levy-Levy-Solomon model.
method Simulations and analysis of Levy-Levy-Solomon model with different random number generators and stopping criteria.
result Low-quality pseudo random number generators significantly impact simulation results.
Proves bounds on spanning two-forests and random cut sizes.
problem Counting spanning two-forests and estimating random cut sizes.
method Uses pairwise effective resistances and potential theory.
result Establishes bounds on the number of spanning two-forests and average cut size.
Privacy affects how much data is needed for CVaR optimization.
problem Privacy constraints impact the effective sample size for CVaR optimization.
method Analyzes the privacy-relevant sample size and decomposes CVaR excess risk.
result The effective private tail sample size is εnτ, affecting CVaR learning rates.
Negative step sizes improve second-order methods for neural networks.
problem Second-order methods discard negative curvature, limiting their effectiveness.
method Introduce negative step sizes in second-order methods combined with Wolfe line search.
result Negative step sizes lead to global convergence and improved performance.
Pruning improves model generalization in over-parameterized models, contradicting traditional theories.
problem Pruning's effect on generalization in over-parameterized models.
method Empirical study on standard pruning algorithms and additional regularization effects.
result Pruning leads to better training and regularization, improving generalization.
Analyzes error sources in global feature effect estimation methods.
problem Unexplored error sources in global feature effect estimation methods.
method Systematic, estimator-level analysis of bias and variance.
result Holdout data is theoretically cleanest, but estimation variance depends on sample size and model characteristics.
We demonstrate that the lowest possible price change (tick-size) has a large impact on the structure of financial return distributions. It induces a microstructure as well as it can alter the tail behavior. On small return intervals, the tick-size can distort the calculation of correlations. This especially occurs on s…
The study reveals a transition in neural network performance from infinite-width to variance-limited behavior as dataset size increases.
problem Understanding the transition from infinite-width to variance-limited behavior in neural networks.
method Empirical study of the transition from infinite-width to variance-limited behavior as a function of sample size and network width.
result The critical sample size \( P^* \) is approximately \( \sqrt{N} \) for polynomial regression with ReLU networks.
We add size factor to CAPM and normalize residuals by Volatility Index.
problem Capturing the size effect in CAPM and making residuals Gaussian.
method Insert size effect, normalize residuals by Volatility Index, and fit model to real-world data.
result The new model shows long-term stability and connects to Stochastic Portfolio Theory.
Adaptive step-size improves optimization in complex geometries.
problem Optimizing functions with non-Euclidean geometries.
method Adaptive step-size strategy for optimization algorithms.
result Guaranteed convergence for Adaptive Conditional Gradient Descent.
In this study, we consider classification problems based on neural networks in data-imbalanced environment. Learning from an imbalanced data set is one of the most important and practical problems in the field of machine learning. A weighted loss function based on cost-sensitive approach is a well-known effective metho…
When assets are correlated, benefits of investment diversification are reduced. To measure the influence of correlations on investment performance, a new quantity - the effective portfolio size - is proposed and investigated in both artificial and real situations. We show that in most cases, the effective portfolio siz…
Many financial variables are found to exhibit multifractal nature, which is usually attributed to the influence of temporal correlations and fat-tailedness in the probability distribution (PDF). Based on the partition function approach of multifractal analysis, we show that there is a marked finite-size effect in the d…
New confidence intervals improve treatment effect estimation in randomized experiments.
problem Improving confidence intervals for treatment effects in randomized experiments.
method Systematic exploitation of negative dependence or variance adaptivity.
result Achieved nonasymptotic confidence intervals with the same effective sample size as asymptotic ones.
New framework optimizes deep learning training by deferring large batch sizes to late stages.
problem Optimizing batch size scheduling for deep learning training efficiency.
method Introduced the functional scaling law (FSL) framework to analyze and optimize batch size scheduling.
result Large batch sizes can be deferred to late training stages without sacrificing performance.
MRI image quality affects statistical and predictive analysis of brain morphology.
problem Impact of MRI image quality on statistical and predictive analysis of brain morphology.
method Systematic testing of image quality on univariate statistics and machine learning classification using three large datasets.
result Low-quality MRI data significantly affects detecting significant sex/gender differences in smaller samples, but not in larger ones.
We look at the effect of the tick size changes on the TOPIX 100 index names made by the Tokyo Stock Exchange on Jan-14-2014 and Jul-22-2104. The intended consequence of the change is price improvement and shorter time to execution. We look at security level metrics that include the spread, trading volume, number of tra…
SVM used for estimating treatment effects without confounding.
problem Estimating average treatment effects in the presence of confounding variables.
method Adapts SVM classifier as a kernel-based weighting procedure to balance covariates and estimate causal effects.
result SVM provides a continuous relaxation of the quadratic integer program for balancing covariates and maximizing effective sample size.
When using active learning, smaller batch sizes are typically more efficient from a learning efficiency perspective. However, in practice due to speed and human annotator considerations, the use of larger batch sizes is necessary. While past work has shown that larger batch sizes decrease learning efficiency from a lea…
We test the price momentum effect in the Korean stock markets under the momentum universe shrinkage to subuniverses of the KOSPI 200. Performance of the momentum strategy is not homogeneous with respect to change of the momentum universe. It is found that some submarkets generate the higher momentum returns than other …
Neural networks learn the support of the target function through SGD's implicit regularization effect.
problem Learning the support of the target function in neural networks.
method Investigation of mini-batch SGD's ability to learn the support in the first layer of a neural network.
result Mini-batch SGD effectively learns the support in the first layer by shrinking irrelevant weights, while vanilla GD requires an explicit regularization term.
The Normalized Mutual Information (NMI) has been widely used to evaluate the accuracy of community detection algorithms. However in this article we show that the NMI is seriously affected by systematic errors due to finite size of networks, and may give a wrong estimate of performance of algorithms in some cases. We gi…
An interbank market lets participants pool the risk arising from the combination of illiquid investments and random withdrawals by depositors. But it also creates the potential for one bank's failure to trigger off avalanches of further failures. We simulate a model of interbank lending to study the interplay of these …
Empirical study on SGD hyperparameters and adversarial robustness.
problem Effect of SGD hyperparameters on adversarial robustness and generalization.
method Empirical observation of learning rate, batch size, and momentum effects on adversarial robustness and generalization.
result Constant learning rate to batch size ratio leads to good generalization and almost constant adversarial robustness.
Using detailed statistical analyses of the size distribution of a universe of equity exchange-traded funds (ETFs), we discover a discrete hierarchy of sizes, which imprints a log-periodic structure on the probability distribution of ETF sizes that dominates the details of the asymptotic tail. This allows us to propose …
Recent hardware developments have dramatically increased the scale of data parallelism available for neural network training. Among the simplest ways to harness next-generation hardware is to increase the batch size in standard mini-batch neural network training algorithms. In this work, we aim to experimentally charac…
Study shows more data improves model explanations, aiding reliable knowledge extraction.
problem Challenges in deriving reliable knowledge from machine learning models due to the Rashōmon effect.
method Examined the influence of sample size on explanations from models in a Rashōmon set using SHAP.
result Explanations from <128 samples are highly variable, but agreement improves with more data.
Algorithm identifies interpretable subgroups with elevated treatment effects.
problem Estimating high-dimensional, uninterpretable CATE results.
method Rule sets summarizing CATE estimates, optimizing subgroup size and effect size.
result Frontier of Pareto optimal rule sets for subgroup identification.
How does missing data affect our ability to learn signal structures? It has been shown that learning signal structure in terms of principal components is dependent on the ratio of sample size and dimensionality and that a critical number of observations is needed before learning starts (Biehl and Mietzner, 1993). Here …
State-of-the-art implementations of boosting, such as XGBoost and LightGBM, can process large training sets extremely fast. However, this performance requires that the memory size is sufficient to hold a 2-3 multiple of the training set size. This paper presents an alternative approach to implementing the boosted trees…
Nonlinear kernel regression models are often used in statistics and machine learning because they are more accurate than linear models. Variable selection for kernel regression models is a challenge partly because, unlike the linear regression setting, there is no clear concept of an effect size for regression coeffici…
New bounds for causal effect identification in time series graphs with latent confounders.
problem Identifying causal effects in time series graphs with latent confounders over unbounded time intervals.
method Applying the Causal Identification algorithm to a constant-size segment of the time series graph.
result A bound on the number of past time steps needed for causal effect identification.
AdamP optimizes momentum-based optimizers for scale-invariant weights, improving model performance.
problem Premature decay of effective step sizes in momentum-based optimizers for scale-invariant weights.
method Proposes SGDP and AdamP to eliminate the radial component at each optimizer step, preserving convergence properties.
result Uniform gains across multiple benchmarks, improving model performance.
Modern longitudinal studies collect feature data at many timepoints, often of the same order of sample size. Such studies are typically affected by {dropout} and positivity violations. We tackle these problems by generalizing effects of recent incremental interventions (which shift propensity scores rather than set tre…
Connectivity studies using resting-state functional magnetic resonance imaging are increasingly pooling data acquired at multiple sites. While this may allow investigators to speed up recruitment or increase sample size, multisite studies also potentially introduce systematic biases in connectivity measures across site…