A detector subnetwork improves deep learning robustness against adversarial perturbations.
problem Vulnerability of deep neural networks to adversarial perturbations that are imperceptible to humans.
method Augmenting deep neural networks with a binary classification detector subnetwork trained to distinguish adversarial from genuine data.
result Adversarial perturbations can be detected surprisingly well even though they are imperceptible to humans.
New algorithm finds unbiased subnetworks in biased datasets.
problem Finding unbiased subnetworks in biased neural networks.
method Debiased Contrastive Weight Pruning (DCWP) algorithm.
result DCWP significantly outperforms state-of-the-art debiasing methods.
Extract class-specific subnetworks from neural models for better understanding and improved explanations.
problem Understanding and explaining the complex behavior of deep neural networks.
method For each semantic class, extract a class-specific subnetwork with a compressed structure that maintains comparable performance.
result Extracted subnetworks improve explanation saliency and adversarial example detection.
New method finds unbiased subnetworks in biased models for better OOD performance.
problem How to improve out-of-distribution generalization in deep models.
method Functional modular probing method and Modular Risk Minimization.
result Even in biased models, there are unbiased subnetworks that can achieve better OOD performance.
Bayesian deep learning method using subnetwork inference.
problem Improving deep neural networks' calibration and efficiency.
method Perform inference over a subset of model weights, keeping others as point estimates.
result Subnetwork inference enables accurate predictive posteriors without full network approximations.
Greedy selection finds smaller, more accurate subnetworks.
problem Finding smaller, accurate subnetworks in large neural networks.
method Greedy forward selection starting from an empty network.
result Theoretical guarantee of finding subnetworks with lower loss.
New findings show BERT subnetworks can train independently and transfer to various tasks.
problem Finding smaller subnetworks that can train independently and transfer to other tasks.
method Examined pre-trained BERT models for subnetworks that can train independently and transfer to various downstream tasks.
result Found subnetworks at 40% to 90% sparsity that can train independently and transfer to various tasks.
Single model predicts multiple subnetworks for robustness.
problem Efficiently ensemble neural networks for robustness and uncertainty.
method Multi-input multi-output (MIMO) configuration to train independent subnetworks.
result Significant improvement in negative log-likelihood, accuracy, and calibration error.
New method finds smaller, stable subnetworks that can train to full network accuracy.
problem Pruning neural networks at initialization to find subnetworks that can train to similar accuracy.
method Modified IMP to search for subnetworks that could have been obtained by pruning early in training, focusing on 0.1% to 7% through.
result Pruned subnetworks of deeper networks can complete training to match the accuracy of the original network on challenging tasks.
The study finds a theoretical bound for pre-training iterations needed for pruning to yield good subnetwork performance.
problem Discovering efficient subnetworks within pre-trained dense networks.
method Mathematical analysis of a two-layer, fully-connected network, validating with a multi-layer perceptron trained on MNIST.
result A logarithmically dependent threshold on dataset size for successful pruning.
SGD tends to favor simpler subnetworks, improving generalization.
problem SGD's tendency to favor simpler subnetworks over complex ones.
method Identifying invariant sets and analyzing SGD's behavior around them.
result SGD collapses networks to simpler subnetworks, improving generalization.
Proposes a new method for ensembling neural subnetworks.
problem Computational expense and limited flexibility of traditional deep ensembles.
method Sequential Bayesian neural subnetwork ensembling.
result Outperforms traditional ensembles in various metrics.
Logarithmic pruning simplifies lottery ticket hypothesis.
problem Finding efficient subnetworks in large neural networks.
method Logarithmic pruning approach to identify subnetworks.
result Randomly initialized subnetworks achieve comparable performance.
Improved pruning method finds winning neural network subnetworks.
problem Finding a small subnetwork that performs as well as a full neural network.
method Data-dependent pruning criterion using gradient of training loss.
result Data-dependent pruning improves existing pruning algorithms.
New proof shows random neural networks contain sparse subnetworks.
problem Finding sparse subnetworks in random neural networks.
method Proof of the Strong Lottery Ticket Hypothesis using the Random Fixed-Size Subset Sum Problem.
result Guaranteed sparsity of subnetworks in random neural networks.
Randomly initialized networks contain subnetworks that perform similarly to target networks.
problem Proving the lottery ticket hypothesis for neural networks.
method Pruning over-parameterized neural networks to find subnetworks.
result Randomly initialized networks contain subnetworks with similar performance to target networks without additional training.
New classical algorithm outperforms quantum in neural network subnetwork selection.
problem Selecting sparse subnetworks from large neural networks efficiently.
method Quantum-inspired classical algorithm using ridgelet transform sampling.
result Runs in polynomial time, outperforming naive classical methods.
SupSup model learns thousands of tasks without forgetting, using randomly initialized subnetworks.
problem Sequentially learning many tasks without forgetting.
method Randomly initialized base network with task-specific subnetworks (supermasks).
result Gradient-based optimization can identify the correct subnetwork for new tasks.
Meta-ticket finds optimal sparse subnetworks for few-shot learning in randomly initialized neural networks.
problem Avoiding overfitting in few-shot learning for over-parameterized neural networks.
method Meta-learning approach to find optimal sparse subnetworks.
result Meta-ticket discovers sparse subnetworks that adapt to each task, achieving superior meta-generalization.
This letter improves sparse signal detection from one bit compressed sensing measurements.
problem Sparse signal detection from one bit compressed sensing measurements.
method Extended GLRT detector with optimal quantizer design and a double-detector scheme.
result The double-detector scheme outperforms existing methods in detection performance.
Adversarial perturbations fool deepfake detectors with high accuracy.
problem Improving deepfake detection accuracy against adversarial attacks.
method Used adversarial perturbations and two defenses: Lipschitz regularization and Deep Image Prior (DIP).
result Deepfake detectors achieved 27% accuracy on perturbed images, compared to 95% on unperturbed.
This paper studies concept drift detectors for financial time series.
problem Improving accuracy on financial time series with concept drifts.
method Three simple concept drift detectors tailored to financial time series.
result Two of the detectors are as effective as state-of-the-art detectors.
This work proposes a method to optimize hyperparameters without validation data.
problem Optimizing hyperparameters for neural networks with limited data.
method Neural network partitioning and subnetwork optimization.
result The method optimizes hyperparameters efficiently and without validation data.
Graph networks improve particle reconstruction in irregular detectors.
problem Handling irregular particle-detector geometries in particle reconstruction.
method Introduce distance-weighted graph network architectures (GarNet, GravNet layers) for irregular geometry detectors.
result The proposed graph networks provide equally performing or less resource-demanding solutions compared to existing methods.
Deep neural network improves NILM with attention mechanism.
problem Energy disaggregation of individual appliance power demands from aggregate meter readings.
method Regression and classification subnetworks with attention mechanism.
result Proposed model outperforms state-of-the-art on REDD and UK-DALE datasets.
Bayesian detector improves defect detection on rough surfaces.
problem Defect detection on motion blurred images with rough texture.
method Developed Bayesian detector using reflected non-local prior distributions.
result Significantly enhanced accuracy in identifying defects.
New method makes neural networks transparent, revealing learning modes.
problem Lack of interpretability in neural networks.
method Weight pathway analysis (WPA) to decompose neural networks into subnetworks.
result Neural networks store and utilize information holographically, with linear and nonlinear learning modes.
ATSDLN adapts to time series data for anomaly detection.
problem Challenges in selecting and optimizing anomaly detectors for time series data.
method Adaptive Time Series Detector Learning Network (ATSDLN) that selects and optimizes detectors and parameters.
result ATSDLN outperforms other methods in anomaly detection across various datasets.
Machine learning solves Einstein equations without symmetry assumptions.
problem Solving the Euclidean vacuum Einstein equations with a cosmological constant.
method Semi-supervised machine learning with patch-based architecture and coordinate consistency loss.
result Hints against the existence of Ricci-flat metrics on spheres in dimensions 4 and 5.
New method uses eigenspectrum to optimize object detector architectures.
problem Understanding the effects of ImageNet pre-training on object detectors.
method Analysis of eigenspectrum dynamics of feature maps in object detectors.
result Object detectors trained from scratch and ImageNet pre-trained models behave differently.
New method uses neural networks to estimate parameters without needing detector simulations.
problem Estimating parameters in high-energy physics with detector effects.
method Two-level fitting approach: SRGN (Simulation-level fit based on Reweighting Generator-level events with Neural networks).
result Demonstrated using simulated datasets, SRGN can estimate parameters without detector effects.
TomOpt optimizes muon detector designs using differentiable programming.
problem Designing efficient particle detectors for muon tomography.
method Differentiable programming for muon interaction modeling, inference, and optimisation.
result Demonstrated end-to-end differentiable and inference-aware optimisation of particle physics instruments.
Deep neural networks improve MIMO detection performance.
problem Improving detection accuracy in massive MIMO systems.
method Introduced a neural network architecture based on BP algorithms, optimized with deep learning techniques.
result DNN MIMO detectors achieve lower bit error rates compared to other detectors.
DCSO dynamically selects top-performing base detectors for outlier ensembles.
problem Challenges in selecting and combining outlier scores from different detectors.
method DCSO dynamically selects top-performing base detectors based on local k-nearest neighbors.
result DCSO provides consistent performance improvement over static combination approaches.
Improves relevancy of black-box anomaly detectors with user feedback.
problem Users often ignore many detected anomalies, requiring a method to identify and prioritize relevant ones.
method Uses user feedback to adjust anomaly selection process based on identified anomaly types.
result Significant improvements in precision and recall over various anomaly detectors.
A method to interpret neural networks by isolating active subnetworks.
problem Interpreting the complex computations of deep neural networks.
method Switched linear projections to decompose network activity.
result Patterns in deactivated neurons are meaningful for network understanding.
Variational autoencoders model water Cherenkov detector data.
problem Modeling generative distribution of water Cherenkov detector data.
method Variational autoencoders and normalizing flows.
result Demonstrated capability of variational autoencoders for generative modelling.
Deep RNN detects electricity theft in smart grids.
problem Electricity theft in smart grids.
method Generalized deep recurrent neural network (RNN) with gated recurrent unit (GRU) and random hyper-parameter tuning.
result Superior performance compared to existing detectors.
This research uncovers high-performing subnetworks in deep GNNs without training.
problem Challenges in applying SLTH to deeper GNNs with high memory requirements.
method Introduces Multicoated Supermasks (M-Sup) and Multi-Stage Folding for GNNs.
result Uncovered untrained recurrent networks with performance similar to trained models.
Framework predicts clinical severity from rs-fMRI data using network optimization.
problem Predicting clinical severity from rs-fMRI data.
method Joint network optimization framework combining sparse subnetworks and linear regression.
result Framework outperforms standard methods and identifies clinically relevant ASD networks.
The paper builds precise detectors for malicious web requests using proxy logs.
problem Detecting security threats in network traffic.
method Statistical features from proxy logs trained on malware samples.
result Correct detection of 95% of malicious flows, URLs, and infected hosts.
Novel unsupervised MIG detectors improve signal detection in cluttered environments.
problem Signal detection in nonhomogeneous clutter environments.
method Developed novel discriminative MIG detectors using HPD matrices and geometric measures.
result Improved signal detection performance compared to conventional methods.
Unified detector calibration and simulation using MLE from generative models.
problem Combining detector calibration and simulation using traditional methods.
method Maximum likelihood estimation from conditional generative models.
result Prior-independent and non-Gaussian resolutions possible.
A simple method for learning activation functions in neural networks.
problem Determining the best activation function for neural networks is challenging.
method Adding local subnetworks with a small amount of neurons to the neural network.
result The proposed method leads to better results compared to using a pre-defined activation function.
A new framework selects best outlier detectors locally for improved ensemble performance.
problem Challenges in combining outlier detectors without ground truth.
method Locally Selective Combination in Parallel Outlier Ensembles (LSCP) framework.
result LSCP_AOM variant consistently outperforms other methods on real-world datasets.
Deep learning reduces complexity for MIMO DF relay channel detection.
problem Efficient signal detection in MIMO DF relay channels with varying channels.
method Deep learning-based detection networks (NMLDNs) for signal detection in changing channels.
result Deep learning reduces detection complexity without sacrificing performance.
The abstract proves the existence of universal lottery tickets without needing further training.
problem The existence of universal sparse subnetworks in large neural networks.
method Theoretical proofs and technical innovations in pruning and subset sum results.
result Universal tickets exist and do not require further training.
We discuss two views on extending existing methods for complex network modeling which we dub the communities first and the networks first view, respectively. Inspired by the networks first view that we attribute to White, Boorman, and Breiger (1976)[1], we formulate the multiple-networks stochastic blockmodel (MNSBM), …