Preprocessing data is an important step before any data analysis. In this paper, we focus on one particular aspect, namely scaling or normalization. We analyze various scaling methods in common use and study their effects on different statistical learning models. We will propose a new two-stage scaling method. First, w…
A new robust scaling approach improves downstream metabolomics analysis.
problem Challenges in choosing scaling techniques for metabolomics data.
method Introduces a weighted scaling approach robust to outliers.
result The proposed method outperforms traditional scaling techniques in both outlier-free and outlier-present datasets.
New method scales features for better clustering.
problem Irregular features disrupt classification.
method Spectral clustering with modified feature scales.
result Outperforms existing methods in experiments.
A new method for automatically learning metric scaling in metric-based meta-learning.
problem Lack of principled method for learning metric scaling parameter.
method Developed a variational metric scaling framework for automatic metric scaling parameter learning.
result Consistently improves the performance of existing metric-based meta-algorithms.
New method selects diffusion scales for graph wavelets.
problem Choosing optimal diffusion scales for graph wavelets.
method Proposes an unsupervised method using information theory.
result Method selects diffusion scales for graph wavelets.
In this paper, we address the problem of measuring and analysing sensation, the subjective magnitude of one's experience. We do this in the context of the method of triads: the sensation of the stimulus is evaluated via relative judgments of the form: "Is stimulus S_i more similar to stimulus S_j or to stimulus S_k?". …
Scaling feature values is an important step in numerous machine learning tasks. Different features can have different value ranges and some form of a feature scaling is often required in order to learn an accurate classifier. However, feature scaling is conducted as a preprocessing task prior to learning. This is probl…
This paper improves conditional multidimensional scaling for incomplete data.
problem Handling missing data in known features for multidimensional scaling.
method Proposes a method to learn low-dimensional configurations with missing known feature values.
result Can learn low-dimensional configurations and impute missing values.
Three methods for tuning HMC diagonal scale matrices compared.
problem Improving Hamiltonian Monte Carlo efficiency with diagonal scale matrices.
method Three approaches: ISG, median crossing frequency, and estimated marginal standard deviations.
result ISG method leads to more efficient sampling in many cases.
Scale-equivariant CNNs handle scale changes for improved performance.
problem Translation equivariance is not sufficient for handling scale changes in CNNs.
method Developed scale-equivariant convolutional networks with steerable filters.
result Demonstrated state-of-the-art results on MNIST-scale and STL-10 datasets.
Online Platt Scaling adapts to varying data distributions.
problem Adapting Platt scaling to non-i.i.d. settings with distribution drift.
method Combines Platt scaling with online logistic regression and calibeating.
result OPS+calibeating method is guaranteed to be calibrated for adversarial outcomes.
New principles needed for scaling large language models, challenging traditional regularization methods.
problem The shift from generalization to scaling in machine learning requires new guiding principles.
method Examining the effectiveness of traditional regularization methods in the scaling-centric era.
result Traditional principles of regularization may not generalize to larger scales, highlighting new phenomena like scaling law crossover.
A new method, tree-SNE, solves the scale problem in t-SNE.
problem Clustering and visualizing high-dimensional data, especially MNIST digits.
method Revisits t-SNE idea to create a 2+1 dimensional embedding with a scale parameter.
result The optimal embedding depends continuously on the scale parameter for all initial conditions.
Adaptive loss scaling speeds up and improves deep learning training.
problem Numerical underflow in mixed precision training.
method Adaptive loss scaling that automatically computes layer-wise loss scale values during training.
result Adaptive loss scaling leads to shorter convergence time and improved accuracy.
The analysis of temporal networks has a wide area of applications in a world of technological advances. An important aspect of temporal network analysis is the discovery of community structures. Real data networks are often very large and the communities are observed to have a hierarchical structure referred to as mult…
This paper improves generative models by using data scaling and theoretical analysis.
problem Challenges in selecting noise distributions for stable learning in generative models.
method Introduces Scale-GAN, which uses data scaling and variance-based regularization.
result Data scaling controls the bias-variance trade-off and improves stability and accuracy.
A sparse modeling is a major topic in machine learning and statistics. LASSO (Least Absolute Shrinkage and Selection Operator) is a popular sparse modeling method while it has been known to yield unexpected large bias especially at a sparse representation. There have been several studies for improving this problem such…
Developed a new thresholding method that connects soft and hard thresholding.
problem Connecting soft and hard thresholding methods in data analysis.
method Scaled soft thresholding method with empirical scaling values.
result Found two sources of over-fitting in the scaled soft thresholding method.
Proposes a more robust rating scale for banks.
problem Inconsistent rating scale validation leading to higher capital requirements.
method Develops a new rating scale that is statistically distinguishable and robust.
result Reduces the calibration probability of default, saving capital requirements.
A new method for reinforcement learning scales errors without tuning.
problem Error scaling varies across reinforcement learning tasks and stages.
method A simple scaling mechanism for temporal-difference learning.
result The method effectively mitigates interference between learning tasks.
CLAPS improves conformal regression by adaptively scaling interval widths based on last-layer Laplace uncertainty.
problem Lack of adaptive interval width scaling in conformal regression for heterogeneous inputs.
method CLAPS uses heteroscedastic last-layer Laplace uncertainty to adaptively scale interval widths, combining aleatoric and epistemic uncertainties.
result CLAPS provides competitive interval efficiency with nominal-level coverage, reducing to aleatoric scaling as epistemic uncertainty decreases.
One of the difficulties of training deep neural networks is caused by improper scaling between layers. Scaling issues introduce exploding / gradient problems, and have typically been addressed by careful scale-preserving initialization. We investigate the value of preserving scale, or isometry, beyond the initial weigh…
Spectral dimensionality reduction methods enable linear separations of complex data with high-dimensional features in a reduced space. However, these methods do not always give the desired results due to irregularities or uncertainties of the data. Thus, we consider aggressively modifying the scales of the features to …
This paper presents a novel scaling method for unbiased risk estimation.
problem Challenges in risk assessment due to limited data, non-stationarity, and heavy tails.
method Develops a statistical framework for efficient risk scaling, extending beyond the square-root-of-time rule.
result Ensures robust and conservative risk estimation, applicable to small sample settings.
Temperature scaling improves model uncertainty but not diversity in LLMs.
problem Improving the calibration and stochasticity of probabilistic models.
method Investigates theoretical properties of temperature scaling in classification and LLMs.
result Temperature scaling increases model uncertainty but not diversity in LLMs.
Paper proposes PPMM for fast estimation of large-scale OTM.
problem Estimation of large-scale optimal transport maps (OTM) is challenging due to the curse of dimensionality.
method Combines projection pursuit regression and sufficient dimension reduction to adaptively select projection directions.
result PPMM consistently estimates the most informative projection direction and weakly converges to the target OTM.
Nearest Neighbors Algorithm is a Lazy Learning Algorithm, in which the algorithm tries to approximate the predictions with the help of similar existing vectors in the training dataset. The predictions made by the K-Nearest Neighbors algorithm is based on averaging the target values of the spatial neighbors. The selecti…
This work analyzes actor-critic methods for faster convergence.
problem Finite-time analysis and sample complexity of two-time-scale actor-critic methods.
method Non-asymptotic analysis under non-i.i.d. setting, proving convergence to first-order stationary point.
result Actor-critic method finds a first-order stationary point with ildeO(ε−2.5) sample complexity. A new L-BFGS method tackles large-scale optimization with fewer evaluations.
problem Efficiently solving large-scale unconstrained optimization problems.
method Proposes a regularized L-BFGS method with line search techniques.
result Shows global convergence and robust performance in numerical tests.
The paper identifies and critiques problems with risk matrices using ordinal scales.
problem Problems with risk matrices using ordinal scales.
method Overview of risk assessment process, explanation of fallacies, and suggestions for improvement.
result The paper proposes avoiding risk matrices and using fully quantitative methods instead.
New method approximates controllability of large networks from coarse summaries.
problem Controlling large-scale linear dynamical systems with incomplete network information.
method Algorithm using stochastic block model to estimate controllability from coarse summaries.
result Average controllability of fine-scale system can be well approximated by coarse-scale system.
AcceleratedLiNGAM speeds up causal discovery methods for large datasets.
problem Slow causal discovery methods for large-scale datasets.
method Parallelized LiNGAM method with GPU acceleration.
result Up to 32-fold speed-up on benchmark datasets.
We propose a new method of learning a sparse nonnegative-definite target matrix. Our primary example of the target matrix is the inverse of a population covariance or correlation matrix. The algorithm first estimates each column of the target matrix by the scaled Lasso and then adjusts the matrix estimator to be symmet…
New methods improve electricity load forecasting using hierarchical transfer learning.
problem Improving electricity load forecasts at national scale using smart meter data.
method Developed two hierarchical transfer learning methods based on stacking and aggregation of experts.
result Significant improvement in predictions compared to benchmark algorithms.
Nowadays stochastic approximation methods are one of the major research direction to deal with the large-scale machine learning problems. From stochastic first order methods, now the focus is shifting to stochastic second order methods due to their faster convergence and availability of computing resources. In this pap…
A new robust and flexible classification method for non-Gaussian data.
problem Robustness to scale changes and non-Gaussian distributions in classical discriminant analysis.
method FEMDA uses arbitrary Elliptically Symmetrical distributions and scale parameters for each data point.
result FEMDA is robust to scale changes and outperforms other methods.
We propose HAMSI (Hessian Approximated Multiple Subsets Iteration), which is a provably convergent, second order incremental algorithm for solving large-scale partially separable optimization problems. The algorithm is based on a local quadratic approximation, and hence, allows incorporating curvature information to sp…
New adaptive first-order methods improve on quasi-Newton variants.
problem Designing efficient gradient methods for practical applications.
method Online scaled gradient methods (OSGM) with new adaptive methods OSGM-Best.
result OSGM-Best matches quasi-Newton variants but requires less memory and cheaper iterations.
New method estimates bidirectional causal effects in large-scale systems.
problem Estimating bidirectional causal effects in systems with mutual dependence and heteroskedasticity.
method Heteroskedasticity-based identification with online kernel learning and random Fourier features.
result Superior accuracy and stability compared to single equation and polynomial approximations.
Two new scalable K-means initialization methods proposed for large-scale clustering.
problem Efficient initialization for large-scale clustering problems.
method Divide-and-conquer approach and random projection method for multiple lower-dimensional subspaces.
result The proposed methods outperform state-of-the-art in large-scale clustering tasks.
Efficiently trains deep Gaussian processes on large datasets.
problem Large-scale data and multi-scale features in function approximation.
method Combines variational learning with MCMC for efficient and accurate training.
result Highly efficient and accurate deep GP training on large-scale data.
Study shows Direct Feedback Alignment fails to offer more efficient scaling than backpropagation.
problem Understanding and optimizing training methods for neural networks.
method Use of scaling laws to compare Direct Feedback Alignment (DFA) and backpropagation.
result DFA fails to offer more efficient scaling than backpropagation.
Paper introduces multi-scale methods to improve CATE estimation from EO data.
problem Challenges in balancing fine-grained and contextual information in EO-based causal inference.
method Multi-Scale Representation Concatenation, combining Vision Transformer and Causal Forests.
result Multi-scale approach captures effect heterogeneity better than single-scale models.
This thesis tackles large-scale learning with kernel methods and proposes scalable algorithms for lifelong robot learning.
problem The challenge of scaling kernel methods to large datasets.
method We analyze and develop approximate learning algorithms, including Nyström and random features, to improve scalability.
result Our methods enable robots to learn continuously and adapt to changing environments efficiently.
Efficient knockoffs for large-scale feature selection.
problem Large-scale feature selection problems.
method Gaussian model-X knockoffs with efficient methods for solving semidefinite programs.
result Efficient knockoffs can be generated with linear complexity in the dimension.
New method trains deep networks robustly without adaptive methods.
problem Training deep networks with robustness and efficiency.
method Scale invariant architecture + SGD + weight decay + gradient clipping.
result SGD can achieve similar performance to adaptive methods like Adam.
Meta-SAGE improves deep RL scalability for CO tasks by adapting pre-trained models to larger-scale problems.
problem Improving scalability of deep reinforcement learning models for combinatorial optimization tasks.
method Meta-SAGE combines a scale meta-learner and scheduled adaptation with guided exploration to adjust model parameters for larger-scale problems.
result Meta-SAGE outperforms previous methods and significantly improves scalability in CO tasks.
Early fault detection using instrumented sensor data is one of the promising application areas of machine learning in industrial facilities. However, it is difficult to improve the generalization performance of the trained fault-detection model because of the complex system configuration in the target diagnostic system…