TempBalance boosts model performance with low data.
problem Low-data training and fine-tuning in model alignment.
method Inspired by HT-SR theory, TempBalance balances training quality across layers.
result TempBalance improves model performance as data decreases.
Adding uninformative labels improves tumor segmentation in low-data mammography.
problem Improving tumor segmentation in mammography with limited data.
method Used seemingly uninformative labels from non-expert annotators to turn a multi-label task into a multi-class problem.
result Performance gains in tumor segmentation are achieved in low-data settings with additional uninformative labels.
Deep ensembles don't necessarily improve calibration in low data regimes.
problem Calibration issues in deep learning models, especially in low data regimes.
method Examination of data-augmentation, ensembling, and post-processing calibration methods.
result Standard ensembling techniques can lead to less calibrated models in low data regimes.
Enhances classification accuracy on low data sets using synthetic data.
problem Low sample size in data augmentation.
method Variational Autoencoder and manifold sampling.
result Significant improvement in classification accuracy (e.g., 88.6% vs 80.7%).
Ensemble pre-trained models for low data transfer learning.
problem Training good models from scratch in low data regime.
method Fine-tune nearest-neighbour ranked pre-trained models to create ensembles.
result Achieves state-of-the-art performance with lower inference budget.
Proposes GPLFR for predicting high-dimensional outputs with few data.
problem Predicting high-dimensional outputs from limited data.
method GPLFR combines Gaussian process and linear-Gaussian decoding for high-dimensional prediction.
result GPLFR outperforms existing methods in predicting high-dimensional outputs.
Stein discrepancy improves UDA performance in low-data scenarios.
problem Improving model performance on unlabeled target domains with limited data.
method Proposes a novel UDA framework using Stein discrepancy, an asymmetric measure that depends on the target distribution through its score function.
result Consistently outperforms prior UDA approaches under limited target data across multiple benchmarks.
This study compares MLPs and KANs in low-data regimes, finding MLPs with personalized activation functions outperform KANs.
problem Comparing MLPs and KANs in low-data regimes.
method Introduced an effective technique for designing MLPs with unique, parameterized activation functions for each neuron.
result MLPs with personalized activation functions achieve significantly higher predictive accuracy with only a modest increase in parameters, especially in low-data regimes.
A new method for anomaly detection adapts to local non-stationarity in low-data regimes.
problem Adapting conformal anomaly detection to handle distribution shifts in real-world data.
method Proposes a continuous inference relaxation using continuous weighted kernel density estimation to decouple local adaptation from tail resolution.
result Restores detection capabilities and statistical power in low-data regimes while maintaining valid error control.
Kernelised flows improve density estimation and generation with fewer parameters.
problem Limited expressiveness of flow-based models due to invertibility constraints.
method Integrates kernels into normalising flows to enhance expressiveness and efficiency.
result Kernelised flows outperform neural network-based flows in parameter efficiency and low-data scenarios.
Improved computed tomography reconstruction with deep learning and deep image prior.
problem Low data efficiency in computed tomography reconstruction.
method Combining learned primal-dual methods with deep image prior for improved quality and generalization.
result Proposed methods outperform state-of-the-art in low data regime.
Stochastic convex optimization problems with expectation constraints (SOECs) are encountered in statistics and machine learning, business, and engineering. In data-rich environments, the SOEC objective and constraints contain expectations defined with respect to large datasets. Therefore, efficient algorithms for solvi…
The goal of few-shot learning is to learn a classifier that generalizes well even when trained with a limited number of training instances per class. The recently introduced meta-learning approaches tackle this problem by learning a generic classifier across a large number of multiclass classification tasks and general…
New framework improves fairness in small data settings.
problem Ensuring fairness in low-data environments.
method Combines posterior sampling exploration with fair classification.
result Framework maximizes accuracy while meeting fairness constraints.
This study improves UAV identification using RF signals with one-shot generative methods.
problem Limited RF environments and signal variability make traditional RF identification ineffective.
method Introduces one-shot generative methods to augment RF signals for UAV identification.
result One-shot generative methods outperform traditional methods in low-data regimes.
Obesity is an important concern in public health, and Body Mass Index is one of the useful (and proliferant) measures. We use Convolutional Neural Networks to determine Body Mass Index from photographs in a study with 161 participants. Low data, a common problem in medicine, is addressed by reducing the information in …
BCD Nets use variational inference to estimate DAGs with uncertainty.
problem Uncertainty in inferring causal graphs from limited data.
method Variational inference framework for Bayesian DAG estimation.
result BCD Nets outperform maximum-likelihood methods in low data regimes.
Meta learning improves with contextualizers that adapt to examples.
problem Few shot classification with limited labeled data.
method Implement contextualizers as generalizable prototypes for gradient-based meta learning.
result Contextualizers significantly boost performance on various few shot learning datasets.
Study compares deep feature methods for anomaly detection in limited data scenarios.
problem Handling limited data in industrial inspection applications.
method Three approaches (KNN, Mahalanobis, PaDiM) using pre-trained deep features with data augmentation.
result Data augmentation significantly improves performance in small data regimes.
The paper proposes a machine learning framework for portfolio optimization with limited data.
problem Low data environments and regime uncertainty in portfolio optimization.
method A teacher-student learning pipeline with CVaR optimizer generating supervisory labels and neural models trained on real and synthetic data.
result Student models can match or outperform the CVaR teacher and achieve improved robustness under regime shifts.
Effective training of neural networks requires much data. In the low-data regime, parameters are underdetermined, and learnt networks generalise poorly. Data Augmentation alleviates this by using existing data more effectively. However standard data augmentation produces only limited plausible alternative data. Given t…
Bayesian autoencoders discover physics from noisy data.
problem Challenges in identifying governing equations and coordinates from noisy, low-data real-world data.
method Bayesian SINDy autoencoders with hierarchical Bayesian sparsifying prior and adaptive empirical Bayesian method.
result Better physics discovery with lower data and fewer training epochs, along with valid uncertainty quantification.
We train a network to generate mappings between training sets and classification policies (a 'classifier generator') by conditioning on the entire training set via an attentional mechanism. The network is directly optimized for test set performance on an training set of related tasks, which is then transferred to unsee…
pFedGP uses Gaussian processes for personalized federated learning with improved kernel learning.
problem Learning personalized models across clients with limited data.
method pFedGP uses Gaussian processes with deep kernel learning, including a shared neural network kernel and inducing points.
result pFedGP achieves well-calibrated predictions and significantly outperforms baseline methods.
We propose a meta-learning algorithm utilizing a linear transformer that carries out null-space projection of neural network outputs. The main idea is to construct an alternative classification space such that the error signals during few-shot learning are quickly zero-forced on that space so that reliable classificati…
3D convolutional neural networks are difficult to train because they are parameter-expensive and data-hungry. To solve these problems we propose a simple technique for learning 3D convolutional kernels efficiently requiring less training data. We achieve this by factorizing the 3D kernel along the temporal dimension, r…
Today, the dominant paradigm for training neural networks involves minimizing task loss on a large dataset. Using world knowledge to inform a model, and yet retain the ability to perform end-to-end training remains an open question. In this paper, we present a novel framework for introducing declarative knowledge to ne…
Active learning improves SR by proposing experiments in data-limited settings.
problem Efficiently gathering data for symbolic regression with physical constraints.
method Query by committee using the Pareto frontier of equations, with physical constraints.
result Reduces data required for SR and achieves state-of-the-art results.
Study examines mean estimation in high dimensions with small data.
problem Efficiently estimating mean in high-dimensional data with limited data size.
method Extensive experimentation of various mean estimation techniques.
result Developed robust methods for mean estimation with low data size.
Method estimates multiple related Gaussian distributions using Laplacian regularization.
problem Jointly estimate multiple related zero-mean Gaussian distributions.
method Laplacian regularized stratified model fitting with hyper-parameters to encourage covariance closeness.
result The method performs well, especially in low data regimes, as demonstrated in finance, radar, and weather.
Foundation models leak sensitive data in synthetic tabular data generation, especially LLaMA 3.3 70B.
problem Privacy leakage in synthetic tabular data generation using foundation models.
method Benchmarked three foundation models (GPT-4o-mini, LLaMA 3.3 70B, TabPFN v2) against four baselines on 35 real-world tables.
result Foundation models, especially LLaMA 3.3 70B, have the highest privacy risk in synthetic tabular data generation.
We propose a new class of probabilistic neural-symbolic models, that have symbolic functional programs as a latent, stochastic variable. Instantiated in the context of visual question answering, our probabilistic formulation offers two key conceptual advantages over prior neural-symbolic models for VQA. Firstly, the pr…
Recent advances in machine learning have made significant contributions to drug discovery. Deep neural networks in particular have been demonstrated to provide significant boosts in predictive power when inferring the properties and activities of small-molecule compounds. However, the applicability of these techniques …
Gradient-based meta-learning techniques are both widely applicable and proficient at solving challenging few-shot learning and fast adaptation problems. However, they have practical difficulties when operating on high-dimensional parameter spaces in extreme low-data regimes. We show that it is possible to bypass these …
The supervised learning paradigm is limited by the cost - and sometimes the impracticality - of data collection and labeling in multiple domains. Self-supervised learning, a paradigm which exploits the structure of unlabeled data to create learning problems that can be solved with standard supervised approaches, has sh…
Geometric interpretation improves VAE performance and robustness.
problem Improving Variational Autoencoder performance and robustness.
method Introducing a geometric perspective on VAEs, sampling from the Riemannian latent space.
result Improved generation and interpolations with competitive or better performance on benchmark datasets.
Deep reinforcement learning requires a heavy price in terms of sample efficiency and overparameterization in the neural networks used for function approximation. In this work, we use tensor factorization in order to learn more compact representation for reinforcement learning policies. We show empirically that in the l…
Manipulating data, such as weighting data examples or augmenting with new instances, has been increasingly used to improve model training. Previous work has studied various rule- or learning-based approaches designed for specific types of data manipulation. In this work, we propose a new method that supports learning d…
Study improves posterior inference in neural processes with limited data.
problem Improving posterior predictive inference in probabilistic models with scarce conditioning data.
method Examined effects of pooling operators and variational families on posterior quality in neural processes.
result Novel neural process architectures lead to superior posterior predictive samples in image completion/in-painting tasks.
CPATTA uses conformal prediction for efficient test-time adaptation.
problem Low data selection efficiency in existing ATTA methods.
method Conformal Prediction, online weight-update algorithm, domain-shift detector, staged update scheme.
result CPATTA consistently outperforms state-of-the-art methods by 5% in accuracy.
This paper studies how neural network architecture affects the speed of training. We introduce a simple concept called gradient confusion to help formally analyze this. When gradient confusion is high, stochastic gradients produced by different data samples may be negatively correlated, slowing down convergence. But wh…
New algorithm improves matrix estimation with one-sided covariates.
problem Estimating matrix means with unobserved row covariates.
method Proposes an algorithm for nonparametric matrix estimation with observed column covariates.
result Achieves minimax optimal nonparametric rate in moderately proportioned matrices.
GIT uses gradient estimators to target interventions for causal discovery.
problem Challenges in inferring causal structure from observational data.
method GIT uses gradient estimators to target interventions for causal discovery.
result GIT performs on par with competitive baselines, surpassing them in low-data regimes.
In drug-discovery-related tasks such as virtual screening, machine learning is emerging as a promising way to predict molecular properties. Conventionally, molecular fingerprints (numerical representations of molecules) are calculated through rule-based algorithms that map molecules to a sparse discrete space. However,…
PILLAR improves SP learning with less private data.
problem Efficiently learning with semi-private data under privacy constraints.
method Uses pre-trained public data features to reduce private data requirements.
result Significantly lower private labelled sample complexity achieved.
The input data features set for many data driven tasks is high-dimensional while the intrinsic dimension of the data is low. Data analysis methods aim to uncover the underlying low dimensional structure imposed by the low dimensional hidden parameters by utilizing distance metrics that consider the set of attributes as…
EHVI outperforms scalarized EI in MOBO for molecule design.
problem Benchmarking MOBO strategies for molecule design.
method Compared EHVI against fixed-weight scalarized EI in MOBO.
result EHVI consistently outperforms scalarized EI in molecular optimization tasks.
Paper tackles robustness in adversarial noise with a meta-optimizer.
problem Sensitivity to adversarial noise hinders machine learning deployment.
method Meta-optimizer learns to robustly optimize models using adversarial examples.
result Meta-optimizer transfers adversarial knowledge to new models without generating new examples.