arXiv research

Adaptive networks improve model robustness through conditional normalization.

Sparse Meta Networks adapt deep neural networks incrementally for fast learning.

GDA-HIN adapts across heterogeneous networks by aligning shared and private node types.

We present an approach to adaptively utilize deep neural networks in order to reduce the evaluation time on new examples without loss of accuracy. Rather than attempting to redesign or approximate existing networks, we propose two schemes that adaptively utilize networks. We first pose an adaptive network evaluation sc…

Algorithm for decentralized competition among adaptive agents.

Adaptive dropout and regularization are shown to be dual in linear networks.

Enhances physics-informed neural networks with adaptive sampling and weighting.

AGCRN forecasts traffic using adaptive graph and recurrent learning.

Adaptive weights improve physics-informed neural networks and deep operator networks.

Unintended effects from scaling neural network outputs with adaptive learning rates.

This paper improves neural network generalization by dynamically learning kernel parameters.

Recent works reveal that network embedding techniques enable many machine learning models to handle diverse downstream tasks on graph structured data. However, as previous methods usually focus on learning embeddings for a single network, they can not learn representations transferable on multiple networks. Hence, it i…

Adaptive gradient methods, which adopt historical gradient information to automatically adjust the learning rate, despite the nice property of fast convergence, have been observed to generalize worse than stochastic gradient descent (SGD) with momentum in training deep neural networks. This leaves how to close the gene…

Efficiently selects seed nodes to maximize content influence in unknown social networks.

FLAP adapts policies quickly to new tasks using shared linear representations.

Firms' collaboration networks can decline but remain resilient.

Consider the problem: given the data pair $(\mathbf{x}, \mathbf{y})$ drawn from a population with $f_*(x) = \mathbf{E}[\mathbf{y} | \mathbf{x} = x]$, specify a neural network model and run gradient flow on the weights over time until reaching any stationarity. How does $f_t$, the function computed by the neural network…

Linearized neural networks provide a fast and interpretable way to adapt models to new settings.

iDAD uses neural networks to quickly adapt experiments without likelihoods.

Animals excel at adapting their intentions, attention, and actions to the environment, making them remarkably efficient at interacting with a rich, unpredictable and ever-changing external world, a property that intelligent machines currently lack. Such an adaptation property relies heavily on cellular neuromodulation,…

Calibrates network confidence for unsupervised domain adaptation.

Adaptive gradient methods like AdaGrad are widely used in optimizing neural networks. Yet, existing convergence guarantees for adaptive gradient methods require either convexity or smoothness, and, in the smooth setting, only guarantee convergence to a stationary point. We propose an adaptive gradient method and show t…

This work bridges two views of feature learning in neural networks.

New algorithm improves Bayesian neural networks using adaptive importance sampling.

Supporting evidence for adaptive feature program across diverse models.

The most widely used activation functions in current deep feed-forward neural networks are rectified linear units (ReLU), and many alternatives have been successfully applied, as well. However, none of the alternatives have managed to consistently outperform the rest and there is no unified theory connecting properties…

Autonomous robots need to interact with unknown, unstructured and changing environments, constantly facing novel challenges. Therefore, continuous online adaptation for lifelong-learning and the need of sample-efficient mechanisms to adapt to changes in the environment, the constraints, the tasks, or the robot itself a…

JADAI optimizes design and inference for parameter estimation.

Deep neural networks have excelled on a wide range of problems, from vision to language and game playing. Neural networks very gradually incorporate information into weights as they process data, requiring very low learning rates. If the training distribution shifts, the network is slow to adapt, and when it does adapt…

Stochastic methods with coordinate-wise adaptive stepsize (such as RMSprop and Adam) have been widely used in training deep neural networks. Despite their fast convergence, they can generalize worse than stochastic gradient descent. In this paper, by revisiting the design of Adagrad, we propose to split the network par…

This paper addresses the robust speech recognition problem as an adaptation task. Specifically, we investigate the cumulative application of adaptation methods. A bidirectional Long Short-Term Memory (BLSTM) based neural network, capable of learning temporal relationships and translation invariant representations, is u…

LoRA enhances model adaptability without increasing parameters.

New algorithm reduces misclassification costs in neural networks.

Adaptive optimization methods bias neural network trajectories towards regions of lower local geometry.

Self-adaptive PINNs improve accuracy in stiff PDEs.

Adaptive stepsizing improves sampling in Bayesian neural networks.

Adaptive momentum method solves non-convex min-max problems.

POLA adapts learning rates for online time series prediction.

learn2mix trains neural nets faster by adjusting class proportions dynamically.

Adaptive Group Lasso selects important features in neural networks.

Efficiently estimates longitudinal networks by merging sparse networks.

New method adapts neural networks without losing prior knowledge.

AdamCB optimizes neural network training by adaptively selecting samples.

HALO learns to prune neural networks by adaptively shrinking weights.

This research improves neural network performance with adaptive activation functions in sparse data settings.

Adaptive kernels from neural networks improve model performance.

Unified framework for adaptive connection sampling in GNNs improves performance and robustness.

This paper examines the convergence of adaptive sampling methods for Bayesian neural networks.