Random weights in GNNs match learned weights in performance.
problem Feature rank collapse in GNNs.
method Replacing learned weights with random weights.
result Random weights achieve comparable performance to learned weights, reducing training time and memory usage.
Improved random forest models enhance machine learning predictions.
problem Equal weights for random forest base decision trees are not optimal.
method Proposes algorithms to modify weighting strategy of regular random forest.
result Numerical results show significant improvements over regular random forest.
Random feature maps improve forecasting with cheaper computation.
problem Improving forecasting accuracy with random feature maps.
method Developed a hit-and-run algorithm to select optimal internal weights.
result Optimal internal weights lead to superior forecasting skill.
Optimal weighted random forests improve prediction accuracy.
problem Unequal prediction performance among random forest trees.
method Proposes 1-step and 2-step optimal weighting algorithms.
result Asymptotically optimal in terms of squared loss and risk.
New method clusters hypergraphs using weighted random walks and Laplacians.
problem Clustering hypergraph data with edge-dependent weights.
method Random walks with edge-dependent vertex weights, constructing hypergraph Laplacians for clustering.
result Proposed methods outperform existing hypergraph clustering algorithms.
We introduce a covariance matrix estimator that both takes into account the heteroskedasticity of financial returns (by using an exponentially weighted moving average) and reduces the effective dimensionality of the estimation (and hence measurement noise) via techniques borrowed from random matrix theory. We calculate…
Sparse random networks reduce communication in federated learning.
problem Large communication cost in federated learning.
method Freeze random weights, train stochastic binary mask to sparsify.
result Improves accuracy, reduces communication, speeds convergence.
A weighted random survival forest is presented in the paper. It can be regarded as a modification of the random forest improving its performance. The main idea underlying the proposed model is to replace the standard procedure of averaging used for estimation of the random survival forest hazard function by weighted av…
Study optimal hedging for claims with random weights in discrete time.
problem Optimal hedging for claims with random weights in discrete time.
method Explicit recursive representation of optimal hedging strategy, without ND condition.
result Obtained explicit optimal hedging strategy in a recursive form.
ARGEW improves node embeddings for weighted homophilous graphs by emphasizing strong edge weights.
problem Lack of accurate node embeddings for weighted homophilous graphs.
method ARGEW (Augmentation of Random walks by Graph Edge Weights) augments random walks by emphasizing nodes with larger edge weights.
result ARGEW produces embeddings where node pairs with strong edge weights have closer embeddings.
Gradient descent with random weights in linear regression analyzed for various noise types.
problem Analyzing the impact of random noise on gradient descent in linear regression.
method Gradient descent with randomly weighted data points, various weighting distributions, geometric moment contraction.
result Characterization of implicit regularization and non-asymptotic convergence bounds.
Hypergraphs are used in machine learning to model higher-order relationships in data. While spectral methods for graphs are well-established, spectral theory for hypergraphs remains an active area of research. In this paper, we use random walks to develop a spectral theory for hypergraphs with edge-dependent vertex wei…
The study examines Fisher information matrices and neural tangent kernels for simple ReLU networks with random weights.
problem Understanding the relationship between Fisher information matrices and neural tangent kernels for 2-layer ReLU networks.
method Analyzes Fisher information matrices and neural tangent kernels for 2-layer ReLU networks with random hidden weights, focusing on spectral decomposition and eigenfunctions.
result Obtained an approximation formula for functions represented by 2-layer neural networks.
New class of heavy-tailed distributions shows weighted averages dominate individual variables.
problem Understanding and comparing risks in heavy-tailed distributions.
method Introducing a new class of heavy-tailed distributions and proving stochastic dominance relations.
result Weighted averages of random variables in this class are stochastically larger than individual variables.
The paper proves a distribution claim for neural network Jacobians.
problem Distribution of singular values in deep neural networks.
method Free probability and random matrix theory techniques.
result Singular value distribution matches for specific cases.
Study evaluates thresholds for removing noise from DNN weights using random matrix theory.
problem Removing noise from deep neural network weights for better approximation.
method Model weights as signal + noise, use random matrix theory to estimate thresholds, evaluate using cosine similarity.
result Proposed threshold estimation method improves approximation quality.
A new method to prune neural networks with iterative randomization improves efficiency.
problem Efficiency in pruning randomly initialized neural networks.
method Iteratively randomizing weight values to reduce parameter requirements.
result The method achieves remarkable performance with fewer parameters.
New method calculates Ricci curvature from distances between weighted volumes.
problem Calculating Ricci curvature for weighted Riemannian manifolds.
method Asymptotic retrieval of generalized Ricci tensor from scaled metric derivatives of Wasserstein 1-distances.
result Limiting coarse curvature of random graphs converges to generalized Ricci tensor.
Neural networks with learned biases can approximate any function.
problem Whether neural networks with only learned biases can approximate any continuous function.
method Theoretical and numerical analysis of random weights and learned biases in neural networks.
result Feedforward and recurrent neural networks with random weights can approximate any continuous function and dynamical systems.
Bandlimited random neural networks may not approximate all functions perfectly.
problem Expressive power of shallow neural networks with bandlimited random weights.
method Ridgelet analysis for deriving approximation error lower bounds.
result Bandlimited random weights can lead to non-zero approximation error.
Enhanced ELM reduces randomness in neural network training.
problem Challenges in ELM architecture design and sensitivity to random weight initialization.
method Introduces Effective Non-Random ELM (ENR-ELM) incorporating signal processing concepts.
result ENR-ELM simplifies architecture design and eliminates random weight selection.
Develops a new random forest method for clustered data with improved prediction and inference.
problem Improving prediction and inference accuracy for clustered data with within-cluster dependence.
method Clustered Random Forests, using weighted least squares estimators for leaf predictions.
result Optimal prediction and inference weights vary under covariate shift, necessitating user-chosen weights.
WildWood improves Random Forest predictions using bootstrap out-of-bag samples.
problem Improving Random Forest predictions for supervised learning.
method Uses bootstrap out-of-bag samples to compute improved predictions by aggregating all possible subtrees with exponential weights.
result WildWood produces faster and more competitive predictions compared to other ensemble methods.
In a recently published paper [1], it is shown that deep neural networks (DNNs) with random Gaussian weights preserve the metric structure of the data, with the property that the distance shrinks more when the angle between the two data points is smaller. We agree that the random projection setup considered in [1] pres…
In this work, we propose the kernel Pitman-Yor process (KPYP) for nonparametric clustering of data with general spatial or temporal interdependencies. The KPYP is constructed by first introducing an infinite sequence of random locations. Then, based on the stick-breaking construction of the Pitman-Yor process, we defin…
Improved kernel ridge regression for large datasets using weighted random binning.
problem Efficiently approximating kernel matrices for large-scale datasets.
method Introduced weighted random binning features for locality sensitive hashing.
result Weighted random binning features generate Gaussian processes of any desired smoothness.
Random matrix analysis reveals that neural network weights are mostly random, with some indicating learned information.
problem Understanding how neural networks store information needed for tasks.
method Random matrix theory (RMT) applied to weight matrices of trained deep neural networks.
result Most singular values and eigenvectors of trained neural networks follow universal RMT predictions, suggesting they are random and do not contain system-specific information.
Generative model uses random weighted support points for interpretable data sampling.
problem Creating diverse and interpretable sample sets from large datasets efficiently.
method Random weighted support points from Dirichlet process and Bayesian bootstrap.
result High-quality and diverse outputs at lower computational cost.
Study Gaussian approximation for deep neural networks with random weights.
problem Understanding the distribution of deep neural networks with random weights.
method Established Gaussian approximation bounds in Wasserstein-1 norm.
result Convergence rates of order n−(1/6)L−1+ε for deep networks with proportional layer widths. Echo state networks with random weights can approximate any continuous system.
problem Approximating continuous dynamical systems using echo state networks.
method Randomly generated internal weights and a sampling procedure for activation functions.
result Echo state networks with random weights can approximate any continuous casual time-invariant operators with high probability.
Study on volumes of random inscribed polytopes in projective geometries.
problem Estimating volumes of random inscribed polytopes in projective geometries.
method Central limit theorems and normal approximation for volumes and dual volumes of random inscribed polytopes.
result Established central limit theorems and normal approximation for volumes and dual volumes of random inscribed polytopes.
Paper proposes a new method combining random forests and Lasso selection.
problem Improving random forest performance by applying Lasso regression.
method Adaptive Lasso weighting applied to random forest predictions.
result Unified framework strictly outperforms other methods in simulations and real-world datasets.
A model of random walk on knot diagrams is used to study the Alexander polynomial and the colored Jones polynomial of knots. In this context, the inverse of the Alexander polynomial of a knot plays the role of an Ihara-Selberg zeta function of a directed weighted graph, counting with weights cycles of random walk on a …
RSO uses random weight perturbations to train deep networks without gradients.
problem Training deep neural networks efficiently and without gradient information.
method RSO is a gradient-free Markov Chain Monte Carlo approach that updates weights based on mini-batch loss reduction.
result RSO achieves high accuracy (99.1% on MNIST) with significantly fewer updates than traditional methods.
Neural networks with random hidden nodes have gained increasing interest from researchers and practical applications. This is due to their unique features such as very fast training and universal approximation property. In these networks the weights and biases of hidden nodes determining the nonlinear feature mapping a…
Enhanced Random Forests outperform XGBoost across binary classification datasets.
problem Improving performance of Random Forests in binary classification.
method Adaptive sample and model weighting, iterative algorithm for sample weights, personalized tree weighting schemes.
result Significantly outperforms XGBoost across 15 binary classification datasets.
SWEEN improves certified robustness via weighted ensembling of smoothed classifiers.
problem Finding optimal base classifiers for randomized smoothing.
method Smoothed WEighted ENsembling (SWEEN) scheme.
result SWEEN achieves optimal certified robustness under mild assumptions.
The paper proposes a method to improve forecast combination accuracy using portfolio theory.
problem Improving forecast accuracy by combining multiple forecasts.
method Generates forecast combinations using a portfolio analogy, allowing negative weights for hedging.
result Demonstrates improved performance in weighted random forest forecasts.
Proposes a new random forest weighted local Fréchet regression method.
problem Complex metric space valued responses and curse of dimensionality in Fréchet regression.
method Locally adaptive kernel generated by random forests for local average and local linear Fréchet regression.
result Significantly improves existing Fréchet regression methods with theoretical guarantees.
ABRF uses attention weights to improve RF performance.
problem Improving the performance of random forest models.
method Attention mechanism applied to random forest with optimization and gradient-based methods.
result The proposed ABRF models outperform standard RF models on various datasets.
Paper introduces new approximations for lognormal sums, matching comonotonicity and moments.
problem Approximating sums of lognormal random variables accurately.
method Introduces new approximations based on weighted distribution theory, emphasizing comonotonicity and moment matching.
result Approximations perform better than classical methods, especially in the right tail of the distribution.
Paper tackles ranking items with a semi-random comparison graph and a monotone adversary.
problem Ranking items based on pairwise comparisons from a semi-random comparison graph with a monotone adversary.
method Developed a weighted maximum likelihood estimator (MLE) and an SDP-based approach to reweight the semi-random graph.
result Achieves near-optimal sample complexity, up to a log^2(n) factor, for identifying the top-K preferred items.
AF improves classification models by adaptively weighting trees.
problem Improving classification model performance.
method AF combines OP2T for input-dependent weights and MIO for dynamic refinement.
result AF consistently outperforms RF, XGBoost, and other weighted RF.
We investigate the problem of sequentially predicting the binary labels on the nodes of an arbitrary weighted graph. We show that, under a suitable parametrization of the problem, the optimal number of prediction mistakes can be characterized (up to logarithmic factors) by the cutsize of a random spanning tree of the g…
A new method for approximating softmax and Gaussian kernels with reduced error.
problem Approximating softmax and Gaussian kernels with low error.
method Simplex Random Features (SimRFs) and SimRFs+.
result SimRFs provide the smallest MSE among weight-independent geometrically-coupled PRF mechanisms.
Random feature models approximate functions in Banach spaces efficiently.
problem Approximating functions in Banach spaces efficiently.
method Randomly initialized feature maps and linear readout training.
result Universal approximation in Bochner spaces for Banach space-valued models.
Not all neural network architectures are created equal, some perform much better than others for certain tasks. But how important are the weight parameters of a neural network compared to its architecture? In this work, we question to what extent neural network architectures alone, without learning any weight parameter…
Bayes-optimal learning of deep random networks with Gaussian weights is studied.
problem Learning a target function corresponding to a deep, extensive-width, non-linear neural network with random Gaussian weights.
method Closed-form expressions for Bayes-optimal test error, ridge regression, kernel and random features regression are computed.
result Optimally regularized ridge regression and kernel regression achieve Bayes-optimal performances, while logistic loss yields a near-optimal test error for classification.