Adversarial training improves linear regression solutions, revealing sparsity and abrupt interpolation.
problem Adversarial attacks on linear regression models.
method Formulated as a convex problem, adversarial training is used to find robust solutions that are sparse and interpolate data.
result Adversarial training with small disturbances gives the solution with the minimum-norm that interpolates the training data, revealing abrupt transition into interpolation.
Unified framework for deep neural network training using linear programming.
problem Theoretical understanding of deep neural network training problems.
method Unified framework using linear programming to represent training problems.
result Polyhedral representation of training problems with linear sample-size dependency.
Paper trains multi-layer SNNs using NormAD for spatio-temporal error backpropagation.
problem Training multi-layer SNNs with non-linear integrate-and-fire dynamics.
method Formulates training as optimization, uses NormAD for iterative synaptic weight update.
result Validated on 2- and 3-layer SNNs solving spike-based XOR and generic problems.
This work proposes a new adversarial training method based on L2L framework.
problem Training robust neural networks against adversarial attacks.
method Generic learning-to-learn (L2L) framework to learn an optimizer and a robust classifier.
result L2L outperforms existing adversarial training methods in classification accuracy and computational efficiency.
Efficient algorithms speed up adversarial training for linear models.
problem Adversarial training for linear models is computationally expensive.
method Tailored optimization algorithms for regression and classification.
result Significantly faster convergence rates for large-scale problems.
Machine learning speeds up train shunting feasibility determination.
problem Determining the feasibility of train shunting schedules.
method Deep Graph Convolutional Neural Network (DGCNN) to predict feasibility.
result Improves prediction accuracy and computational efficiency.
The paper develops an algorithm to select a subset of training data for efficient regression models.
problem Designing an efficient algorithm for selecting a subset of training data to train regression models quickly without sacrificing accuracy.
method The paper tackles this problem by formulating it as a minimization of training loss with respect to both trainable parameters and subset of training data, subject to error bounds on the validation set. They use a novel problem formulation and represent it with simplified constraints using the dual of the original training problem. They then develop SELCON, an efficient majorization-minimization algorithm for data subset selection, which admits an approximation guarantee.
result The experiments show that SELCON trades off accuracy and efficiency more effectively than the current state-of-the-art.
Paper proposes DSP to accelerate deep learning model training by addressing locking and straggler problems.
problem Training deep neural networks is slow and inefficient due to locking and straggler issues.
method Layer-wise Staleness and DSP algorithm to handle locking and straggler problems.
result DSP achieves significant training speedup with stronger robustness than compared methods.
Novel method improves training RNNs by accelerating gradient descent.
problem Vanishing and exploding gradient problems in RNNs training.
method Adaptive stochastic Nesterov accelerated quasi-Newton method.
result Improved performance in training RNNs with low per-iteration cost.
This paper explores how train-validation splits help in NAS to prevent overfitting.
problem NAS overfits with train-validation splits and needs better generalization guarantees.
method Established refined properties of validation loss and risk for NAS.
result NAS with train-validation splits can select the most generalizable model.
Linear-time graph optimization using reinforcement learning.
problem Solving combinatorial optimization problems on real-world graphs.
method Graph neural network trained with reinforcement learning.
result Approximate solutions in linear time for various graph problems.
Un-trained neural networks outperform trained methods in MRI reconstruction.
problem Accelerated MRI reconstruction with minimal training data.
method Variation of Deep Decoder without training data.
result Un-trained approach significantly outperforms other methods in reconstruction accuracy.
This paper uses QUBO to train machine learning models on quantum computers.
problem Efficiently training machine learning models on quantum computers.
method Formulated three machine learning models (linear regression, SVM, k-means) as QUBO problems.
result Formulations are more efficient or equivalent in time and space complexity to classical methods.
Optimizes deep learning training by treating it as an optimal control problem.
problem Fragility of deep neural networks to adversarial inputs.
method Formulates adversarial training as a min-max optimization problem and interprets it as an optimal control problem.
result Provides the first convergence analysis of adversarial training algorithm.
Study proves existence of robust classifiers in multiclass adversarial training.
problem Proves existence of robust classifiers in multiclass adversarial training.
method Three models of adversarial training in multiclass classification, proving existence of Borel measurable robust classifiers.
result Proves existence of Borel measurable robust classifiers in each model.
New geometric insights reveal properties of adversarial training problems.
problem Adversarial training in binary classification.
method Equivalence with regularized risk minimization and convex relaxations.
result Existence of minimal and maximal solutions, and regular solutions.
Study stabilizes adversarial training in neural networks over infinite-dimensional spaces.
problem Stability issues in adversarial training of neural networks.
method Functional analysis of minimax optimization over infinite-dimensional spaces of continuous functions and probability measures.
result Convergence property of minimax problems under certain conditions, interpreted as stabilization techniques.
Graph Pointer Networks and hierarchical reinforcement learning solve combinatorial optimization problems like TSP.
problem Traveling Salesman Problem (TSP) with constraints.
method Graph Pointer Networks (GPNs) and hierarchical reinforcement learning.
result GPNs and hierarchical RL find optimal solutions for TSP and TSP with time windows.
New methods reduce communication in distributed training for variational inequalities.
problem Reducing communication in distributed training for high-dimensional models.
method Distributed methods with compressed communication for solving variational inequalities.
result Theoretical guarantees and practical algorithms for compressed communication.
This paper speeds up OCSSVM training using SMO.
problem Training One-Class Slab SVMs is slow.
method Uses updated SMO to divide large problems into smaller, analytically solvable subproblems.
result Training OCSSVMs scales better with large datasets.
New method trains Boltzmann machines without supervision.
problem Training unsupervised learning models.
method Mixed binary quadratic feasibility problem formulation.
result Theory validated on XOR patterns.
Unified framework for lifted training and inversion of neural networks.
problem Challenges in gradient-based training of deep neural networks.
method Unified framework encapsulating various lifted training strategies.
result Unified framework improves training landscape and stability.
The empirical NTK diverges from the NTK in classification problems during overtraining.
problem The divergence of empirical NTK from NTK in classification problems during overtraining.
method Demonstrated strictly positive definiteness of NTKs for FCNs and ResNets. Proved divergence of neural network parameters during training with cross-entropy loss.
result The empirical NTK does not uniformly converge to the NTK across all times on the training samples as the network width increases.
Adaptive caching strategy improves SVM training efficiency.
problem Expensive SVM training for large datasets.
method Proposed EFU and HCST caching strategies for kernel value reuse.
result HCST achieves 20% more reduction in training time.
New techniques improve distributed training with compressed gradients.
problem Gradient mismatch problem in local error feedback.
method Step-ahead error feedback and error averaging techniques.
result Our methods handle gradient mismatch and train faster than full-precision training.
We learn a compact surrogate model for optimization problems to reduce training and inference time.
problem Solving optimization problems with unknown parameters is computationally expensive and may lead to suboptimal solutions.
method We represent the optimization problem in terms of meta-variables and learn a low-dimensional surrogate model end-to-end with the predictive model.
result We achieve a large reduction in training and inference time, and improved performance.
Proposes an adversarial framework for handling imbalanced data.
problem Handling imbalanced data in classifier training.
method Adversarial approach to weight majority class samples.
result Boosts classifier performance on imbalanced data.
Exact posterior score estimation for solving linear inverse problems
problem Solving linear inverse problems
method Derive the exact posterior score and use it as a denoising training objective
result EPS outperforms training-free and training-based baselines on various metrics
Co-training improves sequential decision-making policies from multiple views.
problem Learning policies in settings with multiple state-action representations.
method Inspired by co-training for classification, we present a co-training framework for sequential decision making.
result Our framework improves upon learning from a single view alone.
Paper proposes a new method for training small models on regression problems.
problem Training small models for regression problems with noisy labels.
method Developed a new loss function and a multi-task network approach.
result Improved model accuracy on various datasets, consistent across different levels of annotation errors.
Paper tackles online allocation problems using adversarial training.
problem Online bipartite matching, especially in AdWords.
method Constructs a framework combining game theory, adversarial training, and GANs.
result Designs robust algorithms that perform well under practical and adversarial conditions.
New model for density estimation using tensor trains.
problem Estimation of high-dimensional probability density functions.
method Tensor train-based density estimation (TTDE) with Riemannian optimization.
result TTDE outperforms competitors in training speed and performance.
Collecting large training datasets, annotated with high-quality labels, is costly and time-consuming. This paper proposes a novel framework for training deep convolutional neural networks from noisy labeled datasets that can be obtained cheaply. The problem is formulated using an undirected graphical model that represe…
MD tree diagnoses model failures using loss landscape metrics.
problem Diagnose model failures without knowing training configuration.
method MD tree based on loss landscape metrics.
result MD tree achieves 87.7% accuracy in dataset transfer tasks, outperforming validation-based approaches.
We present an end-to-end framework for solving the Vehicle Routing Problem (VRP) using reinforcement learning. In this approach, we train a single model that finds near-optimal solutions for problem instances sampled from a given distribution, only by observing the reward signals and following feasibility rules. Our mo…
In this paper, we study the generative models of sequential discrete data. To tackle the exposure bias problem inherent in maximum likelihood estimation (MLE), generative adversarial networks (GANs) are introduced to penalize the unrealistic generated samples. To exploit the supervision signal from the discriminator, m…
Approach to verify neural network training integrity.
problem Poisoning attacks during neural network training.
method Use of cryptographic mechanisms to verify training integrity.
result Provable verification of neural network training integrity.
Langevin algorithms enhance training of deep neural networks for stochastic control problems.
problem Training acceleration for deep neural networks in stochastic control problems.
method Application of Langevin algorithms to minimize the loss of deep neural networks in stochastic control problems.
result Langevin algorithms improve training on various stochastic control problems.
Paper uses RL and diffusion models to solve Bayesian inverse problems.
problem Bayesian inverse problems with latent biases.
method Relative Trajectory Balance (RTB) for RL, conditional diffusion models, off-policy backtracking exploration.
result RTB improves diffusion model posteriors for inverse problems.
Prob2Vec embeds problems for adaptive tutoring, achieving high similarity accuracy.
problem Retrieve problems with similar mathematical concepts for adaptive tutoring.
method Hierarchical problem embedding algorithm (Prob2Vec) combining abstraction and embedding steps.
result 96.88% accuracy on problem similarity test, significantly outperforming state-of-the-art sentence embedding methods.
Paper proposes transferring neural network learning trajectories.
problem Training deep neural networks is computationally expensive.
method Formulated and solved the learning transfer problem by matching gradients.
result Transferred parameters achieve non-trivial accuracy before training.
The time complexity of support vector machines (SVMs) prohibits training on huge data sets with millions of data points. Recently, multilevel approaches to train SVMs have been developed to allow for time-efficient training on huge data sets. While regular SVMs perform the entire training in one -- time consuming -- op…
Efficient deep learning on exascale supercomputers solves materials imaging inverse problems.
problem Solving scientific inverse problems in materials imaging using deep learning.
method Novel communication strategies in synchronous distributed deep learning, including decentralized gradient reduction and computational graph-aware grouping.
result Achieved near-linear scaling of distributed training up to 27,600 GPUs on Summit, reaching 2.15(4) EFLOPS16. Ripple Walk Training tackles graph neural network training issues for large and deep graphs.
problem Neighbors explosion, node dependence, and oversmoothing in large and deep GNNs.
method Subgraph-based training framework with Ripple Walk Sampler for high-quality subgraph sampling.
result RWT improves training efficiency and reduces space complexity for deep and large GNNs.
Paper improves adversarial training using a learned optimizer.
problem Improving robustness of deep learning models against adversarial attacks.
method Empirically identified PGD attack's limitations and used a learning-to-learn framework to train an adaptive inner optimizer.
result The proposed framework consistently improves model robustness over traditional adversarial training methods.
We consider the problem of training generative models with a Generative Adversarial Network (GAN). Although GANs can accurately model complex distributions, they are known to be difficult to train due to instabilities caused by a difficult minimax optimization problem. In this paper, we view the problem of training GAN…
Learning to Optimize is a recently proposed framework for learning optimization algorithms using reinforcement learning. In this paper, we explore learning an optimization algorithm for training shallow neural nets. Such high-dimensional stochastic optimization problems present interesting challenges for existing reinf…
A meta-learning framework for dynamic ensemble selection.
problem Improving dynamic ensemble selection for classification tasks.
method Proposes a three-step framework using meta-learning to select competent classifiers.
result Problem-dependent scenario yields the best performance.