Research
On-device research index

arXiv research

A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.

168,695 papers · 148 categories

Trend · papers per month

126253379505 · Jun 202019922001200920172026
48 results for feature normalization

Adaptive feature normalization improves model robustness to extraneous variables.

problem Degrading model performance due to extraneous variables in deep learning.
method Adaptive feature normalization using instance normalization instead of batch normalization.
result Adaptive normalization leads to significant performance gains across different datasets and architectures.

The choice of normalization affects the coefficients in regularized regression models.

problem The impact of normalization on the coefficients of regularized regression models.
method Investigated lasso, ridge, and elastic net regression with different normalization methods for binary and mixed features.
result Normalization affects the coefficients of regularized regression models, and specific scaling methods can mitigate this effect.

Neural collapse occurs in normalized features over a Riemannian manifold.

problem Understanding neural collapse in normalized feature models.
method Simplified multi-class classification task to a nonconvex optimization problem over the Riemannian manifold, analyzing the landscape of critical points.
result The only global minimizers are neural collapse solutions, with all other critical points being strict saddles.

Feature normalization prevents collapse in non-contrastive learning dynamics.

problem Non-contrastive learning can collapse into a single point due to lack of repulsive force.
method Extended previous theory based on L2 loss to cosine loss, considering feature normalization.
result Cosine loss induces stable equilibrium, preventing collapse even with insufficient repulsive force.

Layer normalization improves federated learning with skewed labels.

problem Label skewness in federated learning datasets.
method Identified feature normalization as key mechanism; applied to latent features before classifier.
result Normalization accelerates global training and improves convergence under extreme label shift.

FlowSelect uses normalizing flows to control FDR in feature selection.

problem Controlled feature selection with knockoffs often fails to control false discovery rate (FDR).
method FlowSelect uses normalizing flows for accurate feature modeling and a novel MCMC-based p-value calculation to enforce knockoff properties.
result FlowSelect consistently controls FDR and demonstrates greater power compared to competing methods.

Proposes Moment Exchange to use moments in image recognition models, improving generalization.

problem Discarding moments in image recognition models reduces stability and training time.
method Moment Exchange: replaces moments of learned features with another image's moments and interpolates labels.
result Improves generalization of recognition models across multiple datasets.

Batch normalization shifts models to rely more on non-robust features.

problem Understanding the impact of batch normalization on deep neural networks.
method Empirical analysis and a framework for disentangling robustness and usefulness.
result Batch normalization increases reliance on non-robust features, decreasing adversarial robustness.

BN helps learn fragile features, which can improve adversarial robustness.

problem The role of batch normalization in adversarial training and its impact on robustness.
method Investigated the expressiveness of BN in learning robust features compared to random features.
result Adversarially fine-tuning BN layers can result in non-trivial adversarial robustness.

The paper proves asymptotic normality for multinomial logistic regression on null covariates.

problem Classical asymptotic normality results fail in high-dimensional multinomial logistic models.
method Developed asymptotic normality and chi-square results for multinomial logistic MLE on null covariates.
result Validated new methodology to test feature significance in high-dimensional classification problems.

This paper proposes a novel generic one-class feature learning method based on intra-class splitting. In one-class classification, feature learning is challenging, because only samples of one class are available during training. Hence, state-of-the-art methods require reference multi-class datasets to pretrain feature …

2018-12-20abs ↗pdf ↗

This paper presents a normalization mechanism called Instance-Level Meta Normalization (ILM~Norm) to address a learning-to-normalize problem. ILM~Norm learns to predict the normalization parameters via both the feature feed-forward and the gradient back-propagation paths. ILM~Norm provides a meta normalization mechanis…

2019-04-06abs ↗pdf ↗

Normalization methods are a central building block in the deep learning toolbox. They accelerate and stabilize training, while decreasing the dependence on manually tuned learning rate schedules. When learning from multi-modal distributions, the effectiveness of batch normalization (BN), arguably the most prominent nor…

2018-10-12abs ↗pdf ↗

Person Re-identification (re-id) faces two major challenges: the lack of cross-view paired training data and learning discriminative identity-sensitive and view-invariant features in the presence of large pose variations. In this work, we address both problems by proposing a novel deep person image generation model for…

2017-12-06abs ↗pdf ↗

Proposes PSCs for UQ in deep nets without retraining.

problem Estimating uncertainty in deep nets with a single pass.
method Identifies sensitive, smooth intermediate layer, fits probabilistic model.
result PSCs achieve UQ and OOD detection performance matching existing methods.

Study links neural network inductive bias, feature learning, and generalization on Boolean functions.

problem Understanding how neural networks learn and generalize on Boolean data.
method End-to-end analysis of depth-2 discrete fully connected networks and DNF formulas, using Monte Carlo learning.
result Predictable training dynamics and interpretable features emerge, linking inductive bias and generalization.

New method improves feature importance assessment in random forests.

problem Improving feature importance measures for random forests.
method Hypothesis testing via self-normalized feature-residual correlation test (FACT).
result The method provides theoretically justified feature importance tests with controlled type I error and appealing power.

We developed a new approach for the analysis of physiological time series. An iterative convolution filter is used to decompose the time series into various components. Statistics of these components are extracted as features to characterize the mechanisms underlying the time series. Motivated by the studies that show …

2015-04-23abs ↗pdf ↗

We introduce online learning algorithms which are independent of feature scales, proving regret bounds dependent on the ratio of scales existent in the data rather than the absolute scale. This has several useful effects: there is no need to pre-normalize data, the test-time and test-space complexity are reduced, and t…

2014-08-09abs ↗pdf ↗

We introduce online learning algorithms which are independent of feature scales, proving regret bounds dependent on the ratio of scales existent in the data rather than the absolute scale. This has several useful effects: there is no need to pre-normalize data, the test-time and test-space complexity are reduced, and t…

2013-05-28abs ↗pdf ↗

Fine-tuning normalization layers can reconstruct smaller networks.

problem Understanding the expressive power of fine-tuning normalization layers.
method Random ReLU networks and sparsified networks were fine-tuned to reconstruct target networks.
result Fine-tuning normalization layers can reconstruct networks that are O(extwidth)O(\sqrt{ ext{width}}) times smaller.

Deep Neural Networks (DNNs) thrive in recent years in which Batch Normalization (BN) plays an indispensable role. However, it has been observed that BN is costly due to the reduction operations. In this paper, we propose alleviating this problem through sampling only a small fraction of data for normalization at each i…

2018-10-25abs ↗pdf ↗

Nearest Neighbors Algorithm is a Lazy Learning Algorithm, in which the algorithm tries to approximate the predictions with the help of similar existing vectors in the training dataset. The predictions made by the K-Nearest Neighbors algorithm is based on averaging the target values of the spatial neighbors. The selecti…

2018-11-13abs ↗pdf ↗

PSMM method optimizes matrix sufficient dimension reduction.

problem Feature matrices with row- and column-wise interpretations require efficient dimension reduction.
method PSMM method converts matrix problem into classification problems using rank-1 normal matrix.
result PSMM outperforms existing methods and provides strong interpretability.

Study on the geometric Dyson Brownian motion of non-square matrix products.

problem Understanding the spectrum of a product of non-square random matrices.
method Proportional depth-width limit followed by mean-field limit, solving Burgers equation.
result Free log-normal law is obtained in the identity-start case.

Improved road segmentation on low-res LIDAR data for autonomous vehicles.

problem Low-resolution LIDAR data affects road segmentation accuracy in autonomous vehicles.
method Subsampled LIDAR data transformation into feature maps, use local normal vector with spherical coordinates.
result Improves road segmentation accuracy on low-resolution LIDAR data.

Edge features contain important information about graphs. However, current state-of-the-art neural network models designed for graph learning, e.g. graph convolutional networks (GCN) and graph attention networks (GAT), adequately utilize edge features, especially multi-dimensional edge features. In this paper, we build…

2018-09-07abs ↗pdf ↗

Real analytic functions can be extended on manifolds with normal crossings.

problem Extending continuous functions to CωC^ω functions on manifolds with normal crossings.
method Employing Cartan Theorems A and B from real analytic geometry.
result Continuous functions on the union of submanifolds with normal crossings can be extended to CωC^ω functions on the entire manifold.

DeepSeekMoE improves language model efficiency with shared experts and normalized gating.

problem Improving sample efficiency in language model architectures.
method Theoretical and empirical analysis of shared experts and normalized sigmoid gating.
result Theoretical and empirical evidence of improved sample efficiency with shared experts and normalized gating.

Training only BatchNorm parameters achieves surprisingly high performance in deep networks.

problem Understanding the role and expressive power of affine parameters in BatchNorm.
method Investigating performance when training only BatchNorm parameters and freezing all other weights.
result BatchNorm achieves high performance by learning to disable around a third of random features.

ReVol normalizes stock price features to mitigate distribution shifts, improving prediction accuracy.

problem Distribution shifts in stock price data hinder accurate prediction.
method ReVol uses normalization, attention-based estimation, and geometric Brownian motion.
result ReVol achieves an average improvement of more than 0.03 in IC and over 0.7 in SR.

simpcomp is an extension to GAP, the well known system for computational discrete algebra. It allows the user to work with simplicial complexes. In the latest version, support for simplicial blowups and discrete normal surfaces was added, both features unique to simpcomp. Furthermore, new functions for constructing cer…

2011-05-26abs ↗pdf ↗

Hybrid pipeline detects spike-and-wave discharges in long-term EEG recordings.

problem Manual identification of spike-and-wave discharges in long-term EEG recordings is labour-intensive and error-prone.
method A hybrid pipeline that combines analytical features with a shallow ANN for SWD detection.
result The method correctly detected 384 out of 392 annotated SWD events, achieving high sensitivity and specificity.