arXiv research

Randomly guessing weights helps analyze RL benchmarks objectively.

Forward gradients improve neural network training without backpropagation issues.

The paper introduces negative controls to evaluate causal discovery algorithms, improving their reliability.

Unhinged loss minimization fails to improve classifier accuracy for simple data.

Next generation deep neural networks for classification hosted on embedded platforms will rely on fast, efficient, and accurate learning algorithms. Initialization of weights in learning networks has a great impact on the classification accuracy. In this paper we focus on deriving good initial weights by modeling the e…

We propose using five data-driven community detection approaches from social networks to partition the label space for the task of multi-label classification as an alternative to random partitioning into equal subsets as performed by RAkELd: modularity-maximizing fastgreedy and leading eigenvector, infomap, walktrap an…

A new algorithm finds minimizers in dueling optimization with a monotone adversary.

Gradient descent benefits from tangent kernel advantages under specific conditions.

GUESS improves surrogate model accuracy with adaptive sampling.

We consider the $M$-ary classification problem via crowdsourcing, where crowd workers respond to simple binary questions and the answers are aggregated via decision fusion. The workers have a reject option to skip answering a question when they do not have the expertise, or when the confidence of answering that questio…

New mechanism protects neural network weights from privacy attacks during self-supervised learning.

Spectral clustering is a celebrated algorithm that partitions objects based on pairwise similarity information. While this approach has been successfully applied to a variety of domains, it comes with limitations. The reason is that there are many other applications in which only \emph{multi}-way similarity measures ar…

Prediction intervals in supervised Machine Learning bound the region where the true outputs of new samples may fall. They are necessary in the task of separating reliable predictions of a trained model from near random guesses, minimizing the rate of False Positives, and other problem-specific tasks in applied Machine …

Counterfactual data augmentations may not ensure OOD robustness if performed by a context-guessing machine.

New model shows weak teachers can help strong students learn even with imperfect labels.

The paper analyzes how good initial guesses affect the amount of data needed for low-rank matrix recovery.

Improved iterative methods for risk parity portfolio weights.

Learning algorithms need bias to generalize and perform better than random guessing. We examine the flexibility (expressivity) of biased algorithms. An expressive algorithm can adapt to changing training data, altering its outcome based on changes in its input. We measure expressivity by using an information-theoretic …

Gradient descent amplifies random features in neural networks to useful ones.

Machine learning models have been widely used in security applications such as intrusion detection, spam filtering, and virus or malware detection. However, it is well-known that adversaries are always trying to adapt their attacks to evade detection. For example, an email spammer may guess what features spam detection…

Anderson acceleration (or Anderson mixing) is an efficient acceleration method for fixed point iterations $x_{t+1}=G(x_t)$, e.g., gradient descent can be viewed as iteratively applying the operation $G(x) \triangleq x-α\nabla f(x)$. It is known that Anderson acceleration is quite efficient in practice and can be viewed…

Driver identification has emerged as a vital research field, where both practitioners and researchers investigate the potential of driver identification to enable a personalized driving experience. Within recent years, a selection of studies have reported that individuals could be perfectly identified based on their dr…

Using a bondholder who seeks to determine when to sell his bond as our motivating example, we revisit one of Larry Shepp's classical theorems on optimal stopping. We offer a novel proof of Theorem 1 from from \cite{Shepp}. Our approach is that of guessing the optimal control function and proving its optimality with mar…

New study shows neural networks can generalize without gradient descent, especially in deep settings.

Inspired by coarse-graining approaches used in physics, we show how similar algorithms can be adapted for data. The resulting algorithms are based on layered tree tensor networks and scale linearly with both the dimension of the input and the training set size. Computing most of the layers with an unsupervised algorith…

Stochastic differential equations are an important modeling class in many disciplines. Consequently, there exist many methods relying on various discretization and numerical integration schemes. In this paper, we propose a novel, probabilistic model for estimating the drift and diffusion given noisy observations of the…

We consider the fundamental problem of solving quadratic systems of equations in $n$ variables, where $y_i = |\langle \boldsymbol{a}_i, \boldsymbol{x} \rangle|^2$, $i = 1, \ldots, m$ and $\boldsymbol{x} \in \mathbb{R}^n$ is unknown. We propose a novel method, which starting with an initial guess computed by means of a …

Untrained neural networks can unfairly assign predictions to the same class.

State-of-the-art password guessing tools, such as HashCat and John the Ripper, enable users to check billions of passwords per second against password hashes. In addition to performing straightforward dictionary attacks, these tools can expand password dictionaries using password generation rules, such as concatenation…

We describe a novel family of models of multi- layer feedforward neural networks in which the activation functions are encoded via penalties in the training problem. Our approach is based on representing a non-decreasing activation function as the argmin of an appropriate convex optimiza- tion problem. The new framewor…

Random weights in GNNs match learned weights in performance.

Improved random forest models enhance machine learning predictions.

Study shows gMPNNs struggle with OOD link prediction in larger test graphs.

Machine-generated interpretations do not improve users' guessing accuracy in image classifiers.

Study finds users mostly use recent market and decision information to guess market direction.

Random feature maps improve forecasting with cheaper computation.

A method to derive Lagrangians from field equations in metric-affine theories of gravity.

A theoretical study is presented for a simple linear classifier called reference distance estimator (RDE), which assigns the weight of each feature j as P(r|j)-P(r), where r is a reference feature relevant to the target class y. The analysis shows that if r performs better than random guess in predicting y and is condi…

We collect a few guesses on possible implications of a lower bound on the scalar curvature of a Riemannian manifold on the size and shape of this manifold.

Optimal weighted random forests improve prediction accuracy.

New method clusters hypergraphs using weighted random walks and Laplacians.

We introduce a covariance matrix estimator that both takes into account the heteroskedasticity of financial returns (by using an exponentially weighted moving average) and reduces the effective dimensionality of the estimation (and hence measurement noise) via techniques borrowed from random matrix theory. We calculate…

Sparse random networks reduce communication in federated learning.

A weighted random survival forest is presented in the paper. It can be regarded as a modification of the random forest improving its performance. The main idea underlying the proposed model is to replace the standard procedure of averaging used for estimation of the random survival forest hazard function by weighted av…

Study optimal hedging for claims with random weights in discrete time.

ARGEW improves node embeddings for weighted homophilous graphs by emphasizing strong edge weights.

Understanding deep neural networks is a major research objective with notable experimental and theoretical attention in recent years. The practical success of excessively large networks underscores the need for better theoretical analyses and justifications. In this paper we focus on layer-wise functional structure and…

Gradient descent with random weights in linear regression analyzed for various noise types.