SORSCNs improve nonstationary data modeling by self-organizing and adjusting network parameters.
problem Nonstationary data challenges traditional models in continuous learning.
method SORSCNs autonomously adjust network parameters and structure in real-time using adaptive algorithms.
result SORSCNs outperform other models in generalizing to nonstationary data.
DeepRSCN models nonlinear systems using stochastic configurations.
problem Modeling nonlinear dynamic systems efficiently.
method Incrementally constructed deep reservoir computing framework with random parameters and online weight updates.
result DeepRSCN outperforms single-layer networks in efficiency, learning, and generalization.
Stochastic precision allocation improves deep neural networks.
problem Improving computational efficiency and robustness of deep neural networks.
method Developed a learning scheme that allows deep neural networks to stochastically explore multiple precision configurations.
result Stochastic precision allocation leads to improved generalization in deep neural networks.
RSCNs improve neural network performance with KDE for noisy data.
problem Uncertain data regression problems with noisy or outlier samples.
method RSCNs with weighted least squares, KDE for penalty weights, alternating optimization.
result RSCNs with KDE outperform other robust models in function approximation and real-world applications.
New method generates equilibrium glass configurations efficiently.
problem Sampling equilibrium configurations of amorphous materials is slow and difficult.
method Riemannian stochastic interpolation framework combining Riemannian stochastic interpolant and equivariant flow matching.
result Enforcing geometric and symmetry constraints significantly improves generative performance.
New algorithms improve binary neural network configurations.
problem Training binary neural networks efficiently.
method Stochastic message passing algorithms (BP and SP) for discrete inference.
result Stochastic BP and SP find better BNN configurations.
Enhances RSCNs with hybrid regularization for nonlinear dynamics.
problem Modeling nonlinear dynamic systems with uncertainties.
method Recurrent stochastic configuration networks with hybrid regularization.
result The method outperforms other models in nonlinear system identification and industrial tasks.
2DSCNs improve image data analytics by extending SCN to handle spatial information.
problem Limitation of 1D SCNs in preserving spatial information of images.
method Extend SCN to 2DSCNs by stochastically configuring hidden nodes in a matrix-inputs framework.
result 2DSCNs outperform 1D SCNs in image data analytics tasks.
ARSG method improves neural network training speed and generalization.
problem Training neural networks efficiently and accurately.
method Adaptive remote stochastic gradient method combining adaptive techniques.
result ARSG achieves faster convergence and better generalization compared to popular methods.
This paper tackles resource allocation in multi-user communication networks using a coordinated multi-armed bandit approach.
problem Learning unknown stochastic network characteristics and sharing resources efficiently.
method Combines Multi-Armed Bandit learning with a lightweight signalling-based coordination scheme.
result Ensures convergence to a stable allocation of resources with maximal resource utilization.
DSSCN improves lifelong learning of non-stationary data streams through adaptive network construction.
problem Lifelong learning of non-stationary data streams with efficient and adaptive models.
method Deep stacked stochastic configuration network (DSSCN) with self-constructing deep stacked network structure and adaptive hidden unit parameters.
result DSSCN outperforms existing data stream algorithms in continual learning of non-stationary data streams.
Combines neural networks and Gaussian processes for better uncertainty estimation and generalization.
problem Improving output uncertainty estimation and generalization in deep learning.
method Combines neural networks and Gaussian processes with a scalable stochastic inference procedure.
result Achieves better uncertainty estimation and generalization performance than neural networks and Gaussian processes.
Decentralized algorithms can potentially outperform centralized ones in certain scenarios.
problem High communication cost in centralized algorithms.
method Study of a decentralized parallel stochastic gradient descent (D-PSGD) algorithm and theoretical analysis.
result Decentralized algorithms can outperform centralized ones in specific network configurations.
A new method uses modified Boltzmann weights to infer system configurations from observed data.
problem Inference of system configurations from limited observed data.
method Data-driven approach based on re-weighting observed configurations to achieve a flat distribution probability.
result Accurate inference of system configurations with high-temperature re-weighting of observations.
Theoretical study explains grokking in neural networks.
problem Understanding the abrupt transition from fitting to generalizing in neural networks.
method Characterized a shell-core topological configuration of the solution space induced by Adam's optimization dynamics.
result Derived grokking scaling laws for learning rate, batch size, and regularization coefficient.
Deep Rewiring trains very sparse neural networks efficiently.
problem Limits of connectivity in neuromorphic hardware.
method DEEP R algorithm that trains sparsely connected neural networks directly.
result Trains very sparse feedforward and recurrent neural networks with minimal performance loss.
Neural networks predict EV charging station usage from network layout.
problem Designing optimal EV charging station networks.
method Used neural networks to predict usage from station layout.
result Quickly estimates average usage statistics from proposed station placements.
GANs improve stochastic parameterization of the Lorenz '96 model.
problem Improving stochastic parameterizations for sub-grid processes.
method Developed a GAN-based stochastic parameterization for the Lorenz '96 model.
result GAN configurations outperform a bespoke parameterization in skillful forecasts and climate simulations.
Kernel-based algorithm optimizes cellular network configuration through multi-task learning.
problem Optimizing network configuration based on field experience and minimizing exploration cost.
method Kernel-based multi-BS contextual bandit algorithm leveraging conditional kernel embedding for multi-task learning.
result The proposed algorithm reduces exploration cost and improves network performance.
Bayesian optimization tunes distributed SGD parameters for faster convergence.
problem Finding efficient configurations to balance load in distributed SGD.
method Bayesian optimization with a probabilistic model of distributed SGD.
result Optimizer converges to efficient configurations within ten iterations.
EGO optimizes neural network architectures without manual tuning.
problem Designing optimal neural network architectures is difficult and time-consuming.
method Adapted EGO algorithm for efficient optimization of neural network architectures.
result Automatically optimized neural networks achieve competitive performance compared to hand-crafted ones.
Study rare accessible states and robust ensembles in neural networks to improve learning performance.
problem Understanding how neural networks learn from data and avoid poor performance.
method Define robust ensemble (RE) and a new algorithmic scheme to target dense states.
result Improved performance in various optimization problems through targeted dense states.
Greedy algorithm performs well in online matching despite non-i.i.d. connections.
problem Online matching in sparse random graphs with fixed degree distributions.
method Approximating stochastic processes with partial differential equations.
result GREEDY algorithm can outperform RANKING in certain configurations.
Optimizes glmnet configuration for better accuracy and efficiency.
problem Inappropriate glmnet configuration leads to inaccurate solutions and increased computation time.
method Data-driven framework using neural networks to predict accuracy and computation time from dataset characteristics and configuration.
result Automatic selection of optimal configuration maximizing accuracy under a time constraint.
A new optimization method improves deep learning accuracy without hyper-parameter tuning.
problem Computational demands and convergence behavior in deep learning training.
method Stochastic quasi-Gauss-Newton (SQGN) optimization method combining stochastic quasi-Newton, Gauss-Newton, and variance reduction.
result SQGN provides excellent accuracy without hyper-parameter experimentation, improving convergence and computational performance.
Sparse random networks reduce communication in federated learning.
problem Large communication cost in federated learning.
method Freeze random weights, train stochastic binary mask to sparsify.
result Improves accuracy, reduces communication, speeds convergence.
Geometric Occam's Razor shapes deep learning solutions.
problem Understanding the regularization in over-parameterized neural networks.
method Analyzing the geometric model complexity and Dirichlet energy in neural networks.
result Over-parameterized neural networks are implicitly regularized by geometric model complexity.
Minimal elastic networks minimize energy and length at fixed angles.
problem Finding optimal network configurations under elastic constraints.
method Minimizing a combination of elastic energy and length.
result Existence and regularity of minimizers with prescribed angles.
This work proposes a new method for simultaneous probabilistic identification and control of an observable, fully-actuated mechanical system. Identification is achieved by conditioning stochastic process priors on observations of configurations and noisy estimates of configuration derivatives. In contrast to previous w…
Researchers analyze a new neural network training method.
problem Training robust configurations in discrete weight neural networks.
method Replicated simulated annealing combining physics and classical simulated annealing.
result Explicit criteria for algorithm convergence and successful sampling.
Self-supervised method detects replay spoofing using acoustic configurations.
problem Challenges in collecting large-scale datasets for replay spoofing detection.
method Self-supervised pretraining of acoustic configurations using existing datasets.
result The method outperforms baseline by 30% on ASVspoof 2019 physical access dataset.
D-SPIDER-SFO solves nonconvex optimization problems faster on decentralized networks.
problem Finding a decentralized algorithm with similar convergence rate to SPIDER-SFO.
method Proposed D-SPIDER-SFO, a decentralized variant of SPIDER-SFO.
result Achieves a similar gradient computation cost to centralized SPIDER-SFO.
Improved image classification using centroids and stochastic sampling.
problem Limited accuracy of nearest-neighbor classification.
method Coarse-graining (replacing images by centroids) and stochastic sampling of centroids.
result Stochastic sampling of centroids improves classification accuracy.
Study uses CNNs to upscale wind speed data from 100 km to 3 km, improving subgrid-scale variability.
problem Recovering fine-scale wind speed information from coarse data.
method Convolutional neural networks (CNNs) with different input configurations (coarse wind speed, fine-scale topography, diurnal cycle) were tested.
result CNN models with coarse wind and fine topography inputs perform best in generalizing to unseen regions.
Deep learning enhances Hamiltonian Monte Carlo for sampling gauge field configurations.
problem Sampling from complex gauge field topologies efficiently.
method Stacked neural networks to generalize Hamiltonian Monte Carlo.
result Significantly reduces computational cost for generating gauge field configurations.
Neural network tackles continual learning with neuromodulation and local error signals.
problem Catastrophic forgetting in continuous learning.
method Biologically-inspired neural architecture with local learning and neuromodulation, combined with transfer metalearning.
result Superior performance in continual learning tasks compared to other approaches.
Optimizes tensor program execution time using graph neural networks.
problem Finding optimal configurations for tensor programs is infeasible due to large configuration space.
method Trains a graph convolutional network on an abstract syntax tree to predict execution time.
result Graph-based surrogate model outperforms heuristic-based methods.
SpatialSim benchmarks machine learning in recognizing object spatial configurations.
problem Machine learning in recognizing precise geometrical configurations of groups of objects.
method SpatialSim benchmark with tasks of Identification and Comparison, using Graph Neural Networks (MPGNNs).
result MPGNNs outperform baselines in recognizing spatial configurations, highlighting current limits.
Spectral representations improve CNNs without changing the model.
problem Efficient computation and model flexibility in CNNs.
method Spectral pooling, stochastic regularization, complex-coefficient spectral parameterization.
result Spectral representations lead to faster convergence and competitive performance.
Self-configuring deep neural networks with fast training.
problem Training multi-level neural networks efficiently and automatically.
method Training neural networks layer by layer, using adaptive and self-adjusting parameters.
result Ability to self-configure and automatically build deep neural networks.
Neural networks learn molecule and material representations.
problem Learning efficient representations for molecules and materials.
method Continuous-filter convolutional network SchNet.
result SchNet accurately predicts chemical properties across various datasets.
ReLU activations lead to smoother learning curves compared to sigmoidal activations in neural networks.
problem Comparing the performance of ReLU and sigmoidal activations in neural networks.
method Analytical computation of learning curves in shallow networks with different activation functions.
result ReLU networks exhibit continuous transitions in performance, while sigmoidal networks show discontinuous transitions.
This paper investigates how network width and depth affect adversarially robust DNNs.
problem Understanding architectural configurations for adversarially robust DNNs.
method Comprehensive investigation on the impact of network width and depth on adversarial robustness.
result Optimal architectural configuration for adversarial robustness exists and can improve robustness.
Graph neural networks fail to distinguish certain 3D atom configurations.
problem Graph neural networks (GNN) fail to distinguish certain 3D atom configurations.
method Construction of degenerate 3D atom configurations that are indistinguishable by first-order GNNs.
result First-order GNNs are incomplete for 3D atom configurations.
POCA optimizes hyperparameters with adaptive allocation for faster convergence.
problem Optimizing hyperparameters for machine learning models.
method Adaptive allocation of computational budget using Bayesian sampling.
result POCA finds strong configurations faster than its competitors.
Researchers prove long-time existence for two landmark Brownian motion.
problem Proving long-time existence of Brownian motion on configurations of two landmarks.
method Classification and analysis of long-time existence for configurations of exactly two landmarks, using a radial kernel.
result For configurations of exactly two landmarks, long-time existence is possible for certain kernels, but not for others.
Constructs classifiers for neural networks with specific data configurations.
problem Finding global minima of deep ReLU neural networks on sequentially separable data.
method Explicitly constructs zero loss neural network classifiers using cumulative parameters and truncation maps.
result Global minimizers can be described with a limited number of parameters based on the data structure.
Deep learning shows neural networks can be trained with limited data, revealing a low-dimensional manifold of optimal configurations.
problem How neural networks can be trained with limited data despite having billions of potential configurations.
method Using mutual information between layers of a deep neural network to speed up training and find optimal configurations.
result Adding structure to neural networks that enforces higher mutual information between layers speeds training and leads to more accurate results.