Single Class Universum-SVM uses additional data to improve single class learning.
problem Improving single class learning with limited positive data.
method Proposes Single Class Universum-SVM, incorporating additional data with different distribution.
result Empirical comparisons show the utility of the proposed approach.
New method learns multi-class from single-class data with confidences.
problem Learning multi-class from single-class data without additional data.
method Empirical risk minimization framework for multi-class classification.
result Method achieves Bayes-consistency with noisy confidences.
We formulate a new class of conditional generative models based on probability flows. Trained with maximum likelihood, it provides efficient inference and sampling from class-conditionals or the joint distribution, and does not require a priori knowledge of the number of classes or the relationships between classes. Th…
New neural networks learn single-index models efficiently.
problem Learning low-dimensional structure in high-dimensional data.
method Shallow neural networks with frozen biases, studied via gradient flow.
result Generalization guarantees match near-optimal sample complexity.
Characterizes learnability of multioutput functions in various settings.
problem Learning multioutput function classes in batch and online settings.
method Characterizes learnability based on single-output restrictions.
result Complete characterization of learnability in multioutput classification and regression.
Develops a new method for efficient stochastic bilevel optimization.
problem Stochastic bilevel optimization problems in machine learning applications.
method Single-Timescale stochAstic BiLevEl optimization (STABLE) method.
result Achieves the same order of sample complexity as stochastic gradient descent for single-level optimization.
Efficiently learns Single-Index Models with constant factor approximation.
problem Learning Single-Index Models under L22 loss with unknown link functions. method An efficient algorithm using alignment sharpness for optimization.
result Achieves constant factor approximation to optimal loss for various distributions and link functions.
SGD shows distinct phases in learning single-index models, achieving optimal sample complexity and regret.
problem Learning single-index models with SGD in adaptive data settings.
method Stochastic gradient descent (SGD) with an optimal learning rate schedule.
result SGD achieves near-optimal sample complexity and regret guarantees across both burn-in and learning phases.
RL in MFGs is as hard as solving many single-agent RL problems.
problem Learning Nash Equilibrium in Mean-Field Games (MFGs).
method Introduce P-MBED to measure model complexity, develop a novel exploration strategy, and establish polynomial sample complexity results.
result Learning Nash Equilibrium in MFGs is no more statistically challenging than solving a logarithmic number of single-agent RL problems.
A new classifier encodes local neighborhoods for each class using Fly Bloom Filters.
problem Efficiently classify data with single-pass learning.
method Proposes a new classifier that encodes local neighborhoods for each class with per-class Fly Bloom Filters.
result The proposed classifier's performance is competitive with nearest-neighbor classifiers and other single-pass classifiers.
A new method for federated learning with only positive labels.
problem Learning with only positive labels leads to poor classifier performance.
method Federated Averaging with Spreadout (FedAwS) framework.
result FedAwS can almost match the performance of conventional learning with negative labels.
Multi-expert L2D underfits more severely, requiring new methods.
problem Underfitting in multi-expert L2D settings.
method PiCCE (Pick the Confident and Correct Expert), a surrogate-based method.
result PiCCE effectively reduces multi-expert L2D to a single-expert-like problem, resolving underfitting.
Paper proposes methods to learn with multiple incorrect labels per example.
problem Learning with a single incorrect label per example limits potential.
method Proposes a novel problem setting allowing multiple incorrect labels per example and two learning methods.
result Demonstrates improved learning with multiple incorrect labels compared to single incorrect labels.
A new teacher-class network method compresses DNNs by distributing knowledge to multiple student networks.
problem Overwhelming size of Deep Neural Networks (DNNs).
method Single teacher with multiple student networks, transferring knowledge to each student.
result The combined knowledge of the class of students achieves better performance and reduces parameters.
Study SGD dynamics in sequence models, revealing training phases and influence of sequence length.
problem Understanding SGD in sequence models like attention networks.
method Derived closed-form population loss and analyzed SGD dynamics for SSI models.
result Two distinct training phases: escape from uninformative initialization and alignment with target subspace.
New algorithms for multitask learning with long-term memory.
problem Learning from tasks partitioned into unknown segments with associated hypotheses.
method Online multitask learning algorithms exploiting segmentation and hypothesis association.
result Regret bounds and efficient algorithms for various hypothesis classes.
Estimates joint causal effects using single-variable interventions on nonlinear models.
problem Estimating joint causal effects from single-variable interventions.
method Identifiability result and practical estimator for decomposing causal effects.
result Joint effects can be inferred without joint interventional data for nonlinear additive models.
Recent advances in neuroscience have revealed many principles about neural processing. In particular, many biological systems were found to reconfigure/recruit single neurons to generate multiple kinds of decisions. Such findings have the potential to advance our understanding of the design and optimization process of …
Despite their ability to memorize large datasets, deep neural networks often achieve good generalization performance. However, the differences between the learned solutions of networks which generalize and those which do not remain unclear. Additionally, the tuning properties of single directions (defined as the activa…
PTBCC improves accuracy in multi-class annotation aggregation by learning from prototype confusion matrices.
problem Inaccurate and insufficient confusion matrices for annotators in multi-class classification tasks.
method PTBCC (ProtoType learning-driven Bayesian Classifier Combination) uses prototype confusion matrices to capture annotator expertise.
result PTBCC achieves up to 15% accuracy improvement and 3% higher average accuracy compared to existing methods.
Paper tackles offline RL with weak assumptions on both function classes and data coverage.
problem Achieve sample-efficient offline RL with weak assumptions on both factors.
method Simple algorithm based on primal-dual formulation of MDPs, with density-ratio function modeling dual variables.
result Polynomial sample complexity achieved under realizability and single-policy concentrability.
It was recently shown that architectural, regularization and rehearsal strategies can be used to train deep models sequentially on a number of disjoint tasks without forgetting previously acquired knowledge. However, these strategies are still unsatisfactory if the tasks are not disjoint but constitute a single increme…
Generative model generates images with multiple object classes.
problem Generating images with multiple object classes.
method Conditional Deep Convolutional GAN architecture, stabilized against collapse.
result System generates diverse samples with inter-object relationships.
This thesis tackles learning with multi-class abstention and multi-expert deferral, improving model reliability and efficiency.
problem Improving model reliability and efficiency in large language models (LLMs) by leveraging multiple experts.
method Developed new surrogate losses and consistency guarantees for multi-class classification and regression with deferral.
result Strong consistency guarantees for surrogate losses in multi-class classification and regression with deferral.
Universal MLPs with a single hidden layer can learn any function.
problem Learning on various data structures like sequences, images, sets, and graphs.
method Using group theory, the paper proves the universality of a broad class of equivariant MLPs with a single hidden layer.
result Having a hidden layer on which the group acts regularly is sufficient for universal equivariance (invariance).
Multi-task learning (MTL) is a supervised learning paradigm in which the prediction models for several related tasks are learned jointly to achieve better generalization performance. When there are only a few training examples per task, MTL considerably outperforms the traditional Single task learning (STL) in terms of…
Recent work on single-view 3D reconstruction shows impressive results, but has been restricted to a few fixed categories where extensive training data is available. The problem of generalizing these models to new classes with limited training data is largely open. To address this problem, we present a new model archite…
Gen1S learns novel classes with 1-shot data using residual space and generative models.
problem Learning new classes with limited data in a growing dataset.
method Mapping embeddings to a residual space, using generative models to learn multi-modal distribution, and applying it as a structural prior.
result Consistent improvement over state-of-the-art methods in recognizing novel classes.
Transformers learn low-dimensional target functions efficiently in-context.
problem Efficiently learning nonlinear target functions in-context using transformers.
method Nonlinear MLP layer in transformers optimized by gradient descent, focusing on single-index target functions.
result Transformers can learn target functions with low-dimensional structures efficiently in-context.
Noise Sensitivity Exponent controls statistical-computational gaps in learning.
problem Understanding when learning is statistically possible yet computationally hard in high-dimensional statistics.
method Investigating statistical-computational gaps in single- and multi-index models using Noise Sensitivity Exponent.
result Noise Sensitivity Exponent governs statistical-computational gaps in high-dimensional learning.
Graph neural networks improve AMG convergence for sparse systems.
problem Efficiently constructing algebraic multigrid prolongation operators for sparse linear systems.
method Train a graph neural network to learn prolongation operators from matrix classes, using an unsupervised loss function.
result Improved convergence rates compared to classical AMG methods.
PEC improves class-incremental learning by measuring prediction error.
problem Challenges in class-incremental learning, particularly forgetting and class imbalance.
method Prediction Error-based Classification (PEC) measures prediction error of a model trained on data from a class.
result PEC outperforms other methods in class-incremental learning across multiple benchmarks.
This paper investigates multi-task reinforcement learning in non-Markovian decision making, showing benefits in sample efficiency.
problem Investigating multi-task reinforcement learning in non-Markovian decision making processes.
method Developed a joint model class for tasks and used the η-bracketing number to quantify complexity and similarity. result Multi-task reinforcement learning can improve sample efficiency in non-Markovian decision making processes.
Randomly biased data makes complex models as easy to learn as simple ones.
problem Learning complex models like multi-index and sparse Boolean functions.
method Introducing a small random shift in the first moment of the data distribution.
result Randomly biased data makes Gaussian single index models and sparse Boolean functions as easy to learn as linear functions.
We develop a multi-task convolutional neural network (CNN) to classify multiple diagnoses from 12-lead electrocardiograms (ECGs) using a dataset comprised of over 40,000 ECGs, with labels derived from cardiologist clinical interpretations. Since many clinically important classes can occur in low frequencies, approaches…
New framework for learning with class-conditional multi-label noise.
problem Class labels corrupted with conditional probabilities for multiple labels.
method Formalized as CCMN framework, established unbiased estimators, proved consistency with multi-label loss functions, implemented partial multi-label learning method.
result Effectiveness validated on multiple datasets and metrics.
Single-step samplers generate high-quality samples efficiently.
problem Sampling from unnormalized distributions is computationally expensive.
method Developed consistent diffusion samplers that generate samples in a single step.
result Single-step samplers produce high-fidelity samples with less than 1% of traditional samplers' evaluations.
Less-than-one-shot learning tackles few-shot learning with minimal data.
problem Training models on very small datasets while maintaining accuracy.
method Soft-label k-Nearest Neighbors classifier and theoretical lower bounds analysis.
result Achieving learning of multiple classes with fewer than the required samples.
Single-spike neurons can approximate as well as multi-spike neurons.
problem Limitation of single-spike neurons in spiking neural networks.
method Comparison of single-spike and multi-spike neural networks.
result Single-spike and multi-spike neural networks are equivalent in approximation capabilities.
Adversarial meta-learning computes Gamma-minimax estimators for vague prior knowledge.
problem Estimating parameters with vague prior knowledge.
method Adversarial meta-learning algorithms for Gamma-minimax estimators.
result Convergence guarantees and neural network class for selection.
We used convolutional neural networks (CNNs) for automatic sleep stage scoring based on single-channel electroencephalography (EEG) to learn task-specific filters for classification without using prior domain knowledge. We used an openly available dataset from 20 healthy young adults for evaluation and applied 20-fold …
New framework infers multiple classes per image for one-shot learning.
problem Inferring multiple classes per image in one-shot learning.
method Compositional embedding framework with joint training of embedding and composition/query functions.
result Compositional embedding models outperform existing methods on various datasets.
Super-OT combines GANs and optimal transport for lineage tracing.
problem Lineage tracing in single-cell RNA-seq data.
method Supervised learning framework with GANs for optimal transport.
result Super-OT outperforms Waddington-OT in predicting cell differentiation outcomes.
We study the problem of learning representations with controllable connectivity properties. This is beneficial in situations when the imposed structure can be leveraged upstream. In particular, we control the connectivity of an autoencoder's latent space via a novel type of loss, operating on information from persisten…
Paper achieves ε−2 sample complexity for actor-critic methods with minimal assumptions.
problem Achieving ε−2 sample complexity for actor-critic methods under minimal assumptions. method Single-loop, single-timescale implementation; coupled Lyapunov drift framework.
result First ildeO(ε−2) sample complexity guarantee for finding an ε-optimal policy. This work connects hardness of approximation and learning.
problem Hardness of approximation and learnability in machine learning.
method Shows a single hardness property implying both approximation and learning hardness.
result Obtains new results on hardness of approximation and learnability of specific functions.
We propose a new splitting criterion for a meta-learning approach to multiclass classifier design that adaptively merges the classes into a tree-structured hierarchy of increasingly difficult binary classification problems. The classification tree is constructed from empirical estimates of the Henze-Penrose bounds on t…
We unify recent neural approaches to one-shot learning with older ideas of associative memory in a model for metalearning. Our model learns jointly to represent data and to bind class labels to representations in a single shot. It builds representations via slow weights, learned across tasks through SGD, while fast wei…