arXiv research

FiT combines transfer and meta-learning for efficient few-shot image classification.

KM method reduces ConvNet parameters to 9% higher accuracy with minimal additional memory.

CAM-GAN improves GANs for continual learning with efficient feature map transformations.

We compare communication efficiencies of two compelling distributed machine learning approaches of split learning and federated learning. We show useful settings under which each method outperforms the other in terms of communication efficiency. We consider various practical scenarios of distributed learning setup and …

Adaptive multi-domain learning reduces parameter count for efficient deep learning.

Recent work on policy learning from observational data has highlighted the importance of efficient policy evaluation and has proposed reductions to weighted (cost-sensitive) classification. But, efficient policy evaluation need not yield efficient estimation of policy parameters. We consider the estimation problem give…

Efficiently learns deep factor graphs using Gaussian belief propagation.

Efficiently models learning curves using Gaussian processes with latent Kronecker structure.

We propose reinforcement learning on simple networks consisting of random connections of spiking neurons (both recurrent and feed-forward) that can learn complex tasks with very little trainable parameters. Such sparse and randomly interconnected recurrent spiking networks exhibit highly non-linear dynamics that transf…

To keep up with increasing dataset sizes and model complexity, distributed training has become a necessity for large machine learning tasks. Parameter servers ease the implementation of distributed parameter management---a key concern in distributed training---, but can induce severe communication overhead. To reduce c…

A new FL algorithm reduces communication overhead by selectively updating model parameters.

Proposes efficient model for continual learning that grows model over task-specific parameters.

We introduce a new embarrassingly parallel parameter learning algorithm for Markov random fields with untied parameters which is efficient for a large class of practical models. Our algorithm parallelizes naturally over cliques and, for graphs of bounded degree, its complexity is linear in the number of cliques. Unlike…

Efficiently estimate Boolean product distribution parameters from truncated samples.

Investigates optimal parameter allocation in Transformers for efficiency and expressivity.

We introduce a novel method that enables parameter-efficient transfer and multi-task learning with deep neural networks. The basic approach is to learn a model patch - a small set of parameters - that will specialize to each task, instead of fine-tuning the last layer or the entire network. For instance, we show that l…

Efficiently learns exponential family distributions with i.i.d. samples.

Unified Bayesian Optimization framework for model selection balancing effectiveness and training efficiency.

Energy-efficient navigation constitutes an important challenge in electric vehicles, due to their limited battery capacity. We employ a Bayesian approach to model the energy consumption at road segments for efficient navigation. In order to learn the model parameters, we develop an online learning framework and investi…

Transfer learning assumes classifiers of similar tasks share certain parameter structures. Unfortunately, modern classifiers uses sophisticated feature representations with huge parameter spaces which lead to costly transfer. Under the impression that changes from one classifier to another should be ``simple'', an effi…

Efficiently learns neural network parameters from streaming data.

Deep learning predicts neural network parameters efficiently.

Efficient estimators for smooth Hilbert-valued parameters with theoretical guarantees.

Generalizes conformal prediction to multiple learnable parameters for efficient prediction sets.

Transformer learns to estimate negative binomial parameters efficiently.

Meta-learning adjusts TD learning's eligibility trace parameter for more efficient reinforcement learning.

SMRL uses score matching for efficient RL with exponential family models.

This work improves communication efficiency in federated learning over wireless networks by optimizing energy consumption.

PARMESAN learns from memory without parameters for fast, efficient continual learning.

We consider estimating a low-dimensional parameter in an estimating equation involving high-dimensional nuisances that depend on the parameter. A central example is the efficient estimating equation for the (local) quantile treatment effect ((L)QTE) in causal inference, which involves as a nuisance the covariate-condit…

Quaternion Conformer GAN (QC-GAN) is a parameter-efficient speech enhancement framework that combines a Quaternion Conformer generator with MetricGAN-based training.

Online VSMC efficiently learns SSM parameters in streaming data.

SAP learns efficient task-specific parameter subspaces for few-shot learning.

UAVs learn to collect data from IoT sensors efficiently.

Bayesian neural networks improve likelihood-free inference efficiency.

Soft modularization improves sample efficiency and performance in reinforcement learning.

Fine-tuning large language models requires minimal data, making them efficient.

Purpose: Machine learning is broadly used for clinical data analysis. Before training a model, a machine learning algorithm must be selected. Also, the values of one or more model parameters termed hyper-parameters must be set. Selecting algorithms and hyper-parameter values requires advanced machine learning knowledge…

We develop a more efficient NGD method for structured parameters.

While recent continual learning methods largely alleviate the catastrophic problem on toy-sized datasets, some issues remain to be tackled to apply them to real-world problem domains. First, a continual learning model should effectively handle catastrophic forgetting and be efficient to train even with a large number o…

Paper improves Thompson Sampling for linear contextual bandits.

Transformers can efficiently approximate nonparametric regression with minimal parameters and sequences.

PAC-Bayes theory improves ICP efficiency and coverage.

Topological data analysis and its main method, persistent homology, provide a toolkit for computing topological information of high-dimensional and noisy data sets. Kernels for one-parameter persistent homology have been established to connect persistent homology with machine learning techniques. We contribute a kernel…

Machine learning identifies boundaries of real solutions in polynomial systems.

New method uses low-fidelity simulations to efficiently infer parameters of high-fidelity models.

LaLoRA prevents forgetting in LoRA fine-tuning.

DoWG optimizer automatically adapts to convex and nonsmooth problems without tuning.