arXiv research

New compression methods handle biased input sequences for more accurate posterior summaries.

New method identifies parameters of wider shallow neural networks with biases.

Paper shows LIME and SHAP explanations can be fooled by adversarial attacks.

Neural networks with random hidden nodes have gained increasing interest from researchers and practical applications. This is due to their unique features such as very fast training and universal approximation property. In these networks the weights and biases of hidden nodes determining the nonlinear feature mapping a…

Survey on biases in image analysis for industrial safety.

This work addresses encoding biases in neural networks by tailoring models with unsupervised losses.

Improves likelihood-free inference by using a new sampling approach to avoid biased data collection.

This paper introduces hierarchical Gaussian process priors for neural networks to capture weight correlations and inductive biases.

Algorithm improves recommendation subset selection in the presence of biases.

BPN defends against adversarial attacks by generating beneficial perturbations.

Self-attention prefers sparse functions of input sequences, reducing sample complexity.

Interview study reveals considerations for designing semi-automated bias detection tools.

Input-cell attention improves RNN saliency detection over time.

Neural nets learn simple distributions first, then more complex ones.

We prove that the binary classifiers of bit strings generated by random wide deep neural networks with ReLU activation function are biased towards simple functions. The simplicity is captured by the following two properties. For any given input bit string, the average Hamming distance of the closest input bit string wi…

Regularized training of an autoencoder typically results in hidden unit biases that take on large negative values. We show that negative biases are a natural result of using a hidden layer whose responsibility is to both represent the input data and act as a selection mechanism that ensures sparsity of the representati…

Derives equations for deep learning biases and weights, showing data complexity reduction.

Free Random Projection enhances reinforcement learning by naturally incorporating hierarchical structure.

The paper proves a distribution claim for neural network Jacobians.

Algorithm improves binary classification of biased grouped data.

We propose an inference method to estimate sparse interactions and biases according to Boltzmann machine learning. The basis of this method is $L_1$ regularization, which is often used in compressed sensing, a technique for reconstructing sparse input signals from undersampled outputs. $L_1$ regularization impedes the …

Bias and flexibility trade off in learning algorithms.

This paper examines biases in foundation models under long-tailed data and proposes a method to mitigate parameter imbalance.

Theoretical analysis of CNNs' inductive biases and their efficiency in approximating functions.

Periodic activation functions improve neural network reliability and interpretability.

Transformers are less sensitive to input perturbations compared to other models.

We analyze the joint probability distribution on the lengths of the vectors of hidden variables in different layers of a fully connected deep network, when the weights and biases are chosen randomly according to Gaussian distributions, and the input is in $\{ -1, 1\}^N$. We show that, if the activation function $φ$ sat…

Randomly initialized transformers show extreme token preferences.

Random deep neural networks are robust to adversarial examples, scaling with input size and dimension.

FANNet analyzes noise tolerance and training bias in neural networks.

Adversarial training, in which a network is trained on both adversarial and clean examples, is one of the most trusted defense methods against adversarial attacks. However, there are three major practical difficulties in implementing and deploying this method - expensive in terms of extra memory and computation costs; …

In many applications, one has side information, e.g., labels that are provided in a semi-supervised manner, about a specific target region of a large data set, and one wants to perform machine learning and data analysis tasks "nearby" that prespecified target region. For example, one might be interested in the clusteri…

Random projections improve GP regression performance, reducing high-dimensional inputs to 1D.

PGNs dynamically infer and use graph structures to improve model generalization.

ExpBERT uses natural language explanations to improve text interpretation.

DeepMaxent uses neural networks to improve species distribution models.

Deep networks can be biased to learn top eigenfunctions of the kernel outside the training set.

This paper proposes using a linear function approximator, rather than a deep neural network (DNN), to bias a Monte Carlo tree search (MCTS) player for general games. This is unlikely to match the potential raw playing strength of DNNs, but has advantages in terms of generality, interpretability and resources (time and …

Machine learning algorithms are optimized to model statistical properties of the training data. If the input data reflects stereotypes and biases of the broader society, then the output of the learning algorithm also captures these stereotypes. In this paper, we initiate the study of gender stereotypes in {\em word emb…

Conditional restricted Boltzmann machines are undirected stochastic neural networks with a layer of input and output units connected bipartitely to a layer of hidden units. These networks define models of conditional probability distributions on the states of the output units given the states of the input units, parame…

This paper studies bounds for the Lipschitz constant of random neural networks.

New proof links initial class bias to DNN trainability, challenging traditional understanding.

LLMs' explanations are often insufficient and vary with input distribution.

We consider training machine learning models that are fair in the sense that their performance is invariant under certain sensitive perturbations to the inputs. For example, the performance of a resume screening system should be invariant under changes to the gender and/or ethnicity of the applicant. We formalize this …

Paper explores how knowledge distillation transfers inductive biases between models.

Paper provides unbiased spectral moment estimates from finite data.

The standard method of generating random weights and biases in feedforward neural networks with random hidden nodes, selects them both from the uniform distribution over the same fixed interval. In this work, we show the drawbacks of this approach and propose a new method of generating random parameters. This method en…

A new method guides neural networks to focus on specific features of input data.