Compact learning results across various loss functions.
problem Understanding sample complexity in transductive learning.
method Analyzing finite projections and sample complexities for different loss functions.
result Exact compactness of sample complexity holds broadly across realizable and agnostic learning.
Though Convolutional Neural Networks (CNNs) have surpassed human-level performance on tasks such as object classification and face verification, they can easily be fooled by adversarial attacks. These attacks add a small perturbation to the input image that causes the network to misclassify the sample. In this paper, w…
This work extends PAC-Bayesian learning guarantees to non-compact symmetries and non-invariant data.
problem Lack of theoretical guarantees explaining the benefits of symmetries in machine learning models.
method Adapting and tightening PAC-Bayes bounds for non-compact symmetries and non-invariant data distributions.
result Theoretical evidence that symmetric models are preferable for symmetric data, beyond compact groups and invariant distributions.
Study learns convolution operators on compact Abelian groups using regularization.
problem Learning convolution operators on compact Abelian groups.
method Regularization-based approach with ridge regression estimator.
result Characterizes the accuracy of the estimator in terms of finite sample bounds.
New neural networks for non-commutative data.
problem No existing neural networks suitable for non-commutative data.
method Developed compact matrix quantum group equivariant neural networks.
result Characterized weight matrices for easy compact matrix quantum groups.
A new framework CL embeds features and labels for multi-label classification.
problem Exponential growth of output space in multi-label classification.
method Compact Learning (CL) framework that embeds features and labels simultaneously.
result CMLL maximizes label-feature dependency and minimizes label space loss.
Continual lifelong learning is essential to many applications. In this paper, we propose a simple but effective approach to continual deep learning. Our approach leverages the principles of deep model compression, critical weights selection, and progressive networks expansion. By enforcing their integration in an itera…
GNMC reduces XCSF population size while preserving function approximation and policy accuracy.
problem Population bloat in XCSF compaction.
method Introduced GNMC, a novel compaction algorithm.
result GNMC reduces population size significantly without compromising function approximation or policy accuracy.
IRMAE learns compact latent spaces by minimizing rank.
problem Learning compact latent representations in autoencoders.
method Implicitly minimizes the rank of the covariance matrix through gradient descent in multi-layer linear networks.
result Demonstrates validity on image generation and representation learning tasks.
The paper reformulates regression in infinite dimensions as an inverse problem, showing it's equivalent to compact inverse problems.
problem Learning a linear operator between Hilbert spaces from empirical observations.
method Reformulates regression as an inverse problem, proving equivalence to compact inverse problems under specific conditions.
result The inverse problem is equivalent to compact inverse problems in terms of spectral properties and regularisation theory.
Deep neural network learns compact representations for driving tasks.
problem Improving autonomous driving through better neural network representations.
method Inspired by human brain's hierarchical structure and predictive nature, the paper proposes a deep learning framework that learns compact representations of driving concepts.
result The paper successfully learns compact representations using as few as 16 neural units for car and lane concepts.
DCAE learns compact latent representations for one-class novelty detection.
problem Learning compact latent representations for one-class novelty detection.
method DCAE learns compact and collapse-free latent representations through internal discriminative layers of GANs, reconstructing in-class data finely and exclusively.
result DCAE achieves state-of-the-art performance on novelty and adversarial example detection.
Improves deep learning robustness by enforcing local and global compactness.
problem Deep neural networks' vulnerability to adversarial attacks.
method Proposes Adversary Divergence Reduction Network (ADRN) that enforces local/global compactness and clustering assumption.
result Augmenting adversarial training with ADRN components improves robustness.
Simplicial learning improves classification by generating compact sparse representations.
problem Difficulty in distinguishing classes on the same subspace.
method Evolutionary simplicial learning approach to sparse representations.
result Evolutionary simplicial learning outperforms other methods in multi-class classification.
Bayesian neural network predicts planetary instability.
problem Predicting planetary instability in compact systems.
method Novel Bayesian neural network trained on raw orbital elements.
result Model predicts planetary instability times with high accuracy and robust generalization.
Despite the superior performance of deep learning in many applications, challenges remain in the area of regression on function spaces. In particular, neural networks are unable to encode function inputs compactly as each node encodes just a real value. We propose a novel idea to address this shortcoming: to encode an …
Compact semiconductor device models are essential for efficiently designing and analyzing large circuits. However, traditional compact model development requires a large amount of manual effort and can span many years. Moreover, inclusion of new physics (eg, radiation effects) into an existing compact model is not triv…
Compact models for NOX formation during methane combustion are created using a new algorithm.
problem Creating accurate models for NOX formation during complex combustion processes.
method Adapted Machine Learning Optimization of Chemical Kinetics (MLOCK) algorithm with Latin Square method for virtual reaction network generation.
result Compact models with high fidelity (>75%) in reproducing industry-defined performance targets are generated.
We present a new method of blackbox optimization via gradient approximation with the use of structured random orthogonal matrices, providing more accurate estimators than baselines and with provable theoretical guarantees. We show that this algorithm can be successfully applied to learn better quality compact policies …
The margin-based softmax loss functions greatly enhance intra-class compactness and perform well on the tasks of face recognition and object classification. Outperformance, however, depends on the careful hyperparameter selection. Moreover, the hard angle restriction also increases the risk of overfitting. In this pape…
Compact models learn photocurrent dynamics from radiation-induced excess carrier density.
problem Accurate but computationally expensive physics-based photocurrent models for semiconductor devices.
method Dynamic Mode Decomposition (DMD) for learning reduced order models from internal state data.
result Physics-aware, compact delayed photocurrent models accurately approximate internal excess carrier dynamics.
Deep learning detects arrhythmia from RR-interval ECG data.
problem Diagnosing arrhythmia using ECG data.
method Convolutional neural network (CNN) on time-sliced RR-interval data.
result Compact system achieves accurate arrhythmia detection.
Proper regularization is critical for speeding up training, improving generalization performance, and learning compact models that are cost efficient. We propose and analyze regularized gradient descent algorithms for learning shallow neural networks. Our framework is general and covers weight-sharing (convolutional ne…
We study the learnability of a class of compact operators known as Schatten--von Neumann operators. These operators between infinite-dimensional function spaces play a central role in a variety of applications in learning theory and inverse problems. We address the question of sample complexity of learning Schatten-von…
Learning compact and interpretable representations is a very natural task, which has not been solved satisfactorily even for simple binary datasets. In this paper, we review various ways of composing experts for binary data and argue that competitive forms of interaction are best suited to learn low-dimensional represe…
Automated method creates compact chemical models from detailed ones, reducing complexity and improving accuracy.
problem Creating accurate low-dimensional chemical kinetic models from detailed ones is time-consuming and requires expert knowledge.
method Machine Learned Optimisation of Chemical Kinetics (MLOCK) algorithm systematically perturbs sub-models to find optimal compact models.
result Compact models (15 species) retain ~87% fidelity to detailed models, outperforming previous methods.
Fisher loss improves deep domain adaptation by learning discriminative within-class compact and between-class separable representations.
problem Improving deep domain adaptation performance by learning discriminative representations.
method Proposes a Fisher loss to learn discriminative representations that are within-class compact and between-class separable.
result Noticeable improvements in deep domain adaptation performance, e.g., 6.67% absolute improvement in mean accuracy on the Office-Home dataset.
Machine learning finds a compact fixed point action for SU(3) gauge theory.
problem Finding accurate and compact parametrizations of fixed point actions for SU(3) gauge theory.
method Used machine learning, specifically a gauge equivariant convolutional neural network.
result Obtained a superior parametrization of a fixed point action for SU(3) gauge theory.
Develops Gaussian processes on non-compact Lie groups.
problem Invariance to symmetries in non-Euclidean spaces.
method Constructive techniques for stationary Gaussian processes.
result Makes non-Euclidean Gaussian processes compatible with standard software.
We present a two-stage approach for learning dictionaries for object classification tasks based on the principle of information maximization. The proposed method seeks a dictionary that is compact, discriminative, and generative. In the first stage, dictionary atoms are selected from an initial dictionary by maximizing…
We present a novel cost function for semi-supervised learning of neural networks that encourages compact clustering of the latent space to facilitate separation. The key idea is to dynamically create a graph over embeddings of labeled and unlabeled samples of a training batch to capture underlying structure in feature …
Tactile information is important for gripping, stable grasp, and in-hand manipulation, yet the complexity of tactile data prevents widespread use of such sensors. We make use of an unsupervised learning algorithm that transforms the complex tactile data into a compact, latent representation without the need to record g…
New framework SEU solves lifelong learning's catastrophic forgetting issue.
problem Catastrophic forgetting in lifelong learning.
method Introduces Neural Architecture Search into lifelong learning to dynamically adapt model structures for different tasks.
result Achieves higher accuracy with significantly smaller model size (25-33% of state-of-the-art methods).
This paper reviews methods to create compact neural networks for IoT applications.
problem Complex deep neural networks are costly and slow, hindering real-world deployment.
method Automatic synthesis of compact, accurate DNN/LSTM models.
result Compact neural networks reduce energy consumption, memory, and inference time.
Machine learning uses invariant theory to restrict function classes.
problem Creating function classes that respect physical law constraints.
method Using equivariant machine learning and Malgrance's method to parameterize functions.
result Explicitly parameterizes equivariant functions between linear spaces.
The persistence diagram is an increasingly useful tool from Topological Data Analysis, but its use alongside typical machine learning techniques requires mathematical finesse. The most success to date has come from methods that map persistence diagrams into vector spaces, in a way which maximizes the structure preserve…
Sparse Gaussian processes with compact kernels for faster inference.
problem Efficient Gaussian process inference with high computational complexity.
method Parametric families of compactly-supported kernels for sparse matrix representations.
result Sub-quadratic inference complexity and improved performance on real-world tasks.
Classifies compact radio sources in the Galactic plane using machine learning.
problem Challenges in processing large volumes of radio continuum survey data.
method Produced a curated dataset of ~20,000 images, trained two classifiers: gradient-boosted decision trees and convolutional neural networks.
result High classification accuracy (F1-score>90%) for separating Galactic objects from the extragalactic background.
Metric learning aims at learning a distance which is consistent with the semantic meaning of the samples. The problem is generally solved by learning an embedding for each sample such that the embeddings of samples of the same category are compact while the embeddings of samples of different categories are spread-out i…
Autoregressive networks can achieve promising performance in many sequence modeling tasks with short-range dependence. However, when handling high-dimensional inputs and outputs, the huge amount of parameters in the network lead to expensive computational cost and low learning efficiency. The problem can be alleviated …
Compact neural network for ECG classification reduces resource needs.
problem Current reliance on deep learning for ECG analysis requires extensive resources and large datasets.
method Simple ANN architecture with advanced feature engineering.
result Achieved 97.36% accuracy in classifying 4 types of arrhythmias.
Most deep neural networks use simple, fixed activation functions, such as sigmoids or rectified linear units, regardless of domain or network structure. We introduce differential equation units (DEUs), an improvement to modern neural networks, which enables each neuron to learn a particular nonlinear activation functio…
New local method solves Yamabe problems on compact and non-compact manifolds.
problem Yamabe problems on compact and non-compact manifolds.
method Local method for compact and non-compact manifolds.
result Generalizes Brezis and Nirenberg's nonlinear eigenvalue problem to subsets of manifolds.
Compact curve solution emerges from non-compact curve.
problem Constructing solutions from non-compact curves.
method Slingshot solution to curve shortening flow.
result Compact embedded solution exists for a finite time.
Kernel approximation via nonlinear random feature maps is widely used in speeding up kernel machines. There are two main challenges for the conventional kernel approximation methods. First, before performing kernel approximation, a good kernel has to be chosen. Picking a good kernel is a very challenging problem in its…
Compact metrics on Heisenberg manifolds have a specific condition for being relatively compact.
problem Conditions for relatively compact sets of left invariant metrics on Heisenberg manifolds.
method Necessary and sufficient condition for relatively compact sets of left invariant metrics.
result A condition for a set of left invariant metrics to be relatively compact in the moduli space.
Compact models for methane/air combustion reduce complexity without sacrificing accuracy.
problem Creating accurate, computationally efficient models for methane combustion.
method Data-oriented three-step methodology: 1) Remove non-essential species, 2) Numerically optimize to key species profiles, 3) Machine learning to refine parameters.
result Produced 19 and 15 species compact models that outperform current state-of-the-art models in accuracy and range of conditions.
The report analyzes infinite-dimensional output space regression.
problem Learning theory in vector-valued RKHS regression.
method Integral operator technique with spectral theory for non-compact operators.
result Results with minimal assumptions using Chebyshev's inequality.