Study presents MMC model for better fitting multiple choice data.
problem Improving accuracy of latent trait estimates in IRT models.
method Fit autoencoders to MMC model, demonstrating better fit than nominal response model.
result MMC model outperforms traditional IRT models in fit.
This work extends the scaling law to multiple and kernel regression, challenging traditional machine learning principles.
problem Challenging traditional machine learning wisdom with scaling law in large practical models.
method Demonstrates the scaling law in multiple and kernel regression settings.
result The scaling law extends to multiple and kernel regression, providing deeper insights into LLMs.
Proposes a cross-scale residual network for multiple image restoration tasks.
problem Image restoration tasks (super-resolution, denoising, deblocking) have strong correlations.
method Cross-scale residual network exploiting scale-related features and inter-task correlations.
result Outperforms state-of-the-art methods in multiple image restoration tasks.
Paper presents an efficient algorithm for learning minimax risk classifiers with large-scale data.
problem Efficient learning of minimax risk classifiers for large-scale data with multiple classes.
method Combination of constraint and column generation for efficient learning.
result 10x speedup for general large-scale data and 100x speedup with many classes.
We show that an economic system populated by multiple agents generates an equilibrium distribution in the form of multiple scaling laws of conditional PDFs, which are sufficient for characterizing the probability distribution. The existence of the double scaling law is demonstrated empirically for the sales and the lab…
This paper improves traditional Markowitz optimization by considering variance at multiple time scales.
problem Traditional Markowitz optimization limits to a single time scale, ignoring variance across different frequencies.
method Introduces multifrequency optimization allowing specification of target Hurst exponents across multiple time scales.
result Effective risk management strategy that aligns with investor preferences at various time scales.
Improved SVMs learn from few samples with composition and multiple scales.
problem Learning with small sample sizes.
method Transformation-invariant SVMs with composition and locality at multiple scales.
result Kernels based on maximum similarity are positive definite and yield superior accuracy.
HSSE framework embeds single-cell RNA-seq data at multiple scales.
problem Capturing heterogeneous local structure in single-cell RNA-seq data.
method Hierarchical sheaf spectral embedding (HSSE) framework.
result HSSE achieves competitive or improved performance in single-cell RNA-seq data representation learning.
Proposes a new hyperprior and predictive criterion for weakly informative hyperprior in relevance vector machine.
problem Capturing non-homogeneous data structure with limited kernel functions.
method Uses inverse gamma hyperprior with a shape parameter close to zero and a scale parameter not close to zero. Applies multiple kernel method with different widths. Proposes extended predictive information criterion for scale parameter selection.
result Obtains a multiple kernel relevance vector regression model with good predictive accuracy.
The authors study the method of scaling in the context of the study of automorphism groups of complex domains in multiple dimensions. Various types of scaling techniques are compared and contrasted. Applications are given in a number of areas of complex geometric analysis. Relations with other parts of mathematics are …
Bayesian optimization speeds up bioprocess development across scales.
problem Costly and complex bioprocess development across scales and biocatalyst selection.
method Multi-fidelity batch Bayesian optimization framework integrating Gaussian Processes and mixed-variable optimization.
result Reduction in experimental costs and increased yield in bioprocess optimization.
Framework learns surrogates for molecular dynamics across multiple time-scales.
problem Stable molecular dynamics simulations require small time-steps, but long-time-scale moments need repeated simulations.
method Implicit Transfer Operator Learning with denoising diffusion probabilistic models and SE(3) equivariant architecture.
result Models can generate self-consistent stochastic dynamics across multiple time-scales.
Natural image statistics exhibit hierarchical dependencies across multiple scales. Representing such prior knowledge in non-factorial latent tree models can boost performance of image denoising, inpainting, deconvolution or reconstruction substantially, beyond standard factorial "sparse" methodology. We derive a large …
Many real-world time series, such as in health, have changepoints where the system's structure or parameters change. Since changepoints can indicate critical events such as onset of illness, it is highly important to detect them. However, existing methods for changepoint detection (CPD) often require user-specified mod…
Flexible empirical Bayes for large-scale multiple linear regression.
problem Large-scale multiple linear regression with flexible priors and efficient computation.
method Adaptive shrinkage priors combined with variational approximations for hyperparameter estimation.
result The posterior mean from the empirical Bayes method solves a penalized regression problem.
New algorithm clusters hyperspectral images at multiple scales.
problem Clustering hyperspectral images at various scales.
method M-SRDL algorithm using spectral-spatial diffusion distances.
result More accurate clustering labels achieved with spatial regularization.
This work studies scaling laws for low-precision training in high-dimensional linear regression.
problem Optimizing trade-off between model quality and training costs in high-dimensional linear regression.
method Theoretical study of scaling laws for low-precision training within a high-dimensional sketched linear regression framework, analyzing multiplicative and additive quantization.
result Multiplicative quantization maintains full-precision model size, while additive quantization reduces effective model size.
Calibration without labels in multiple testing
problem Interpretable error probabilities in large-scale hypothesis testing
method Constructing pseudo-labels from spacings of ordered p-values result Finding that q-value can be severely miscalibrated A powerful approach for understanding neural population dynamics is to extract low-dimensional trajectories from population recordings using dimensionality reduction methods. Current approaches for dimensionality reduction on neural data are limited to single population recordings, and can not identify dynamics embedde…
Improved average distance classifier for HDLSS settings with multiple population differences.
problem Poor performance of average distance classifier in HDLSS settings with location and scale differences.
method Proposed transformations to the average distance classifier to handle multiple population differences.
result The proposed classifiers perform well even when populations differ in other aspects than location and scale.
New asymptotic e-values improve inference by eliminating data-dependent scaling inefficiency.
problem Data-dependent scaling inefficiency in existing asymptotic e-values.
method Drawing on Bentkus's near-optimal concentration inequalities, introduce Bentkus-type asymptotic e-values.
result Bentkus-type asymptotic e-values consistently deliver sharper inference than existing alternatives.
Method scales up ML science by measuring multiple molecules at once.
problem Scaling up ML-driven science with wet lab experiments.
method Neural extension of compressed sensing for function space.
result Proves orders-of-magnitude gains in information density.
The paper analyzes learning curves for kernel ridge regression with dot-product kernels.
problem Understanding the learning curves for different scaling regimes of data and model.
method Precise formulas for mean test error, bias, and variance in the mo∞ with m/dr constant regime. result A peak in the learning curve at m≈dr/r! for any integer r. Paper proposes a robust framework for detecting multiple periodic components in time series.
problem Detecting multiple periodic components in time series with interlaced patterns and external noise.
method Applying maximal overlap discrete wavelet transform to isolate periodic components, ranking them by wavelet variance, and detecting single periodicity robustly.
result The proposed algorithm outperforms other methods for both single and multiple periodicity detection.
PC-RNN reconstructs MRI images from undersampled data with more details.
problem Recovering fine details from undersampled MRI data.
method Pyramid Convolutional RNN (PC-RNN) with three ConvRNN modules for multi-scale reconstruction.
result PC-RNN outperforms other methods in recovering more details from MRI images.
Recent advancements in recurrent neural network (RNN) research have demonstrated the superiority of utilizing multiscale structures in learning temporal representations of time series. Currently, most of multiscale RNNs use fixed scales, which do not comply with the nature of dynamical temporal patterns among sequences…
This dissertation advances scalable Gaussian processes using iterative methods and pathwise conditioning.
problem The classical Gaussian process formulation is not scalable for large datasets and modern hardware.
method Combining iterative methods and pathwise conditioning to improve scalability.
result Significantly reduced memory requirements and facilitated application to larger datasets.
Detection of power-law behavior and studies of scaling exponents uncover the characteristics of complexity in many real world phenomena. The complexity of financial markets has always presented challenging issues and provided interesting findings, such as the inverse cubic law in the tails of stock price fluctuation di…
Real time large scale streaming data pose major challenges to forecasting, in particular defying the presence of human experts to perform the corresponding analysis. We present here a class of models and methods used to develop an automated, scalable and versatile system for large scale forecasting oriented towards saf…
Corrupting the input and hidden layers of deep neural networks (DNNs) with multiplicative noise, often drawn from the Bernoulli distribution (or 'dropout'), provides regularization that has significantly contributed to deep learning's success. However, understanding how multiplicative corruptions prevent overfitting ha…
A new method for reinforcement learning scales errors without tuning.
problem Error scaling varies across reinforcement learning tasks and stages.
method A simple scaling mechanism for temporal-difference learning.
result The method effectively mitigates interference between learning tasks.
SMTM improves MCMC sampling in high dimensions with multiple proposals and stereographic integration.
problem Improving MCMC performance in high-dimensional sampling.
method Integrating multiple-try Metropolis with stereographic MCMC framework.
result SMTM outperforms classical MTM and other methods in high-dimensional sampling.
We analyze the eigenvalue distribution of a neural network's kernel under specific scaling.
problem Analyzing the eigenvalue distribution of the Neural Tangent Kernel (NTK) of a neural network.
method Asymptotic analysis of the NTK matrix under given scaling conditions.
result The eigenvalue distribution is described as a free multiplicative convolution of the Marchenko-Pastur distribution and a deterministic distribution.
Identifying recurring patterns in high-dimensional time series data is an important problem in many scientific domains. A popular model to achieve this is convolutive nonnegative matrix factorization (CNMF), which extends classic nonnegative matrix factorization (NMF) to extract short-lived temporal motifs from a long …
SHAKE-GNN scales GNNs for large graphs with multi-scale representations.
problem Scaling Graph Neural Networks (GNNs) to large graphs.
method SHAKE-GNN uses a hierarchy of Kirchhoff Forests for stochastic multi-resolution graph decompositions.
result SHAKE-GNN achieves competitive performance on large-scale graph classification benchmarks.
Several recent papers investigate Active Learning (AL) for mitigating the data dependence of deep learning for natural language processing. However, the applicability of AL to real-world problems remains an open question. While in supervised learning, practitioners can try many different methods, evaluating each agains…
Study learning from multiple thinkers providing step-by-step solutions to problems.
problem Learning from multiple, possibly different, thinkers providing step-by-step solutions to problems.
method Active learning algorithm that uses CoT data from multiple thinkers and end-result data.
result Learning can be hard from CoT supervision provided by two or a few different thinkers, but a generic algorithm can learn efficiently.
Paper develops robust methods for large-scale testing without tuning parameters.
problem Heavy-tailed data in high-dimensional settings.
method Revisits Hodges-Lehmann estimator for robust inference without tuning parameters.
result Develops confidence intervals and controls false discovery proportion.
Physics-informed GANs model subsurface flow at the Hanford Site.
problem Uncertainty quantification for subsurface flow modeling at the Hanford Site.
method Physics-informed GANs, hierarchical domain parallelism, multiple GPUs, efficient communication.
result Highly scalable physics-informed GANs model subsurface flow on Summit supercomputer.
Matrix multiplication is a fundamental building block for large scale computations arising in various applications, including machine learning. There has been significant recent interest in using coding to speed up distributed matrix multiplication, that are robust to stragglers (i.e., machines that may perform slower …
Interbank markets are often characterised in terms of a core-periphery network structure, with a highly interconnected core of banks holding the market together, and a periphery of banks connected mostly to the core but not internally. This paradigm has recently been challenged for short time scales, where interbank ma…
Unintended effects from scaling neural network outputs with adaptive learning rates.
problem Adaptive learning rate optimization's behavior is altered by output scaling, leading to misinterpretation.
method Presented a modified optimization algorithm to mitigate unintended effects.
result Adaptive learning rate's effectiveness is significantly impacted by output scaling, especially for small scaling factors.
Detects singularities in complex data to improve machine learning models.
problem Real-world data often contains non-manifold structures (singularities) that can mislead machine learning models.
method Develops a topological framework to quantify local intrinsic dimension and Euclidicity score for multiple scales.
result Identifies singularities and captures local geometric complexity in image data.
The modern scale of data has brought new challenges to Bayesian inference. In particular, conventional MCMC algorithms are computationally very expensive for large data sets. A promising approach to solve this problem is embarrassingly parallel MCMC (EP-MCMC), which first partitions the data into multiple subsets and r…
We present a set of models of the main stylized facts of market price fluctuations. These models comprise dynamical evolution with threshold dynamics and Langevin price equation with multiplicative noise, percolation models to describe the interaction between traders and hierarchical cascade models to unravel the possi…
Convolutional Neural Network (CNN) has demonstrated impressive ability to represent hyperspectral images and to achieve promising results in hyperspectral image classification. However, traditional CNN models can only operate convolution on regular square image regions with fixed size and weights, so they cannot univer…
We compute an approximate Fréchet mean for sets of sparse graphs.
problem Characterizing the location of a set of graphs in a metric space.
method We use the pseudometric defined by the ℓ₂ norm of eigenvalues of adjacency matrices.
result We describe an algorithm to approximate the Fréchet mean of a set of graphs.
In this paper, we propose several dictionary learning algorithms for sparse representations that also impose specific structures on the learned dictionaries such that they are numerically efficient to use: reduced number of addition/multiplications and even avoiding multiplications altogether. We base our work on facto…