Survey of spectral, probabilistic, and deep metric learning methods.
problem Developing effective distance metrics for various machine learning tasks.
method Divided into spectral, probabilistic, and deep approaches, covering various techniques and their applications.
result Comprehensive overview of metric learning methods, including new developments and applications.
We present the Variational Adaptive Newton (VAN) method which is a black-box optimization method especially suitable for explorative-learning tasks such as active learning and reinforcement learning. Similar to Bayesian methods, VAN estimates a distribution that can be used for exploration, but requires computations th…
This paper introduces Dex, a reinforcement learning environment toolkit specialized for training and evaluation of continual learning methods as well as general reinforcement learning problems. We also present the novel continual learning method of incremental learning, where a challenging environment is solved using o…
Pretraining reinforcement learning methods with demonstrations has been an important concept in the study of reinforcement learning since a large amount of computing power is spent on online simulations with existing reinforcement learning algorithms. Pretraining reinforcement learning remains a significant challenge i…
Recent years have witnessed growing interest in machine learning-based models and techniques for low-dose X-ray CT (LDCT) imaging tasks. The methods can typically be categorized into supervised learning methods and unsupervised or model-based learning methods. Supervised learning methods have recently shown success in …
This article introduces machine learning methods for solving PDEs.
problem Approximating solutions of partial differential equations.
method Machine learning methods, including physics-informed neural networks and deep operator learning.
result Recent advances in machine learning have made PDE solutions more accessible.
Proposes a new metric learning method for image recognition.
problem Improving image recognition performance using learned distance representations.
method Introduces a Generalized Hybrid Metric Loss (GHM-Loss) to learn hybrid proximity features combining geometric and probabilistic spaces.
result Demonstrates superior performance compared to existing methods on public datasets.
Discusses new probabilistic morphisms and geometric methods in machine and statistical learning.
problem Addressing challenges in statistical, machine, and manifold learning.
method Introduces category of probabilistic morphisms and geometric methods.
result New insights and applications in various learning fields.
Paper analyzes convergence of continual learning with adaptive methods.
problem Preventing catastrophic forgetting in sequential learning tasks.
method Adaptive method for nonconvex continual learning (NCCL) adjusts step sizes of previous and current tasks.
result Proposed adaptive method achieves same convergence rate as SGD when catastrophic forgetting is suppressed.
SALSA automatically adjusts learning rates in stochastic gradient methods.
problem Automatic adjustment of learning rates in stochastic gradient methods.
method SALSA uses a line-search procedure to gradually increase the learning rate, then a statistical test to decrease it.
result SALSA matches the performance of best hand-tuned learning rate schedules in deep learning tasks.
A method learns common bias for multiple low-variance tasks without hyper-parameter tuning.
problem Learning common bias for multiple low-variance tasks without manual tuning.
method Two variants of online learning methods (aggressive and lazy) that update bias after each datapoint or at the end of each task.
result Across-tasks regret bound derived for the method, showing faster rates for aggressive variant and standard rates for lazy variant.
Deep learning improves asset pricing and risk premium measurement.
problem Improving asset pricing and risk premium measurement using deep learning.
method Investigates various deep learning methods for asset pricing, especially for risk premia measurement.
result RNNs with memory mechanism and attention have the best performance in terms of predictivity.
Explosive growth in data and availability of cheap computing resources have sparked increasing interest in Big learning, an emerging subfield that studies scalable machine learning algorithms, systems, and applications with Big Data. Bayesian methods represent one important class of statistic methods for machine learni…
New method simplifies and improves imitation learning without adversarial techniques.
problem Stable optimization and convergence issues in adversarial imitation learning methods.
method Proposes a non-adversarial framework for imitation learning, providing stronger convergence guarantees.
result Shows AIRL as a special case and derives new algorithms for offline imitation learning.
Machine learning develops rapidly, which has made many theoretical breakthroughs and is widely applied in various fields. Optimization, as an important part of machine learning, has attracted much attention of researchers. With the exponential growth of data amount and the increase of model complexity, optimization met…
Survey of graph learning methods for combinatorial optimization problems.
problem Efficient and effective analysis of graphs for combinatorial optimization problems.
method Two-stage framework: Graph representation learning followed by machine learning.
result Recent studies have shown promise in using machine learning to solve graph-based combinatorial optimization problems.
Novel method reduces radiomic data annotation needs.
problem Insufficient labeled radiomic data for disease diagnosis.
method Collaborative self-supervised learning with two pretext tasks.
result Outperforms other self-supervised methods on radiomic data.
PEARL combines multiple representation learning methods to enhance model performance.
problem Different representation learning methods extract distinct data aspects, potentially missing important insights.
method Combines multiple representation learning approaches using surrogate loss functions for efficient weight estimation.
result Asymptotically achieves optimal performance in downstream tasks, assigning nonzero weights to correctly specified models.
New method learns stochastic process representations without exact reconstruction.
problem Learning exact representations of high-dimensional noisy stochastic processes.
method CReSP framework for contrastive learning of stochastic processes.
result Effective for learning representations of various stochastic processes.
This thesis explores optimization methods for high-dimensional machine learning problems.
problem High-dimensional optimization challenges in machine learning.
method Intuition and convergence proofs for stochastic gradient descent and momentum methods.
result Explanation of why common machine learning optimization methods are successful.
Develops a new method to improve performance in multi-objective learning problems.
problem Gradient bias in multi-objective learning leading to degraded performance.
method Stochastic Multi-objective gradient Correction (MoCo) method that guarantees convergence without increasing batch size.
result Demonstrates effectiveness of MoCo method in simulations on multi-task learning.
Local update methods' performance depends on learning rates, affecting convergence rates and alignment with true loss.
problem The performance of local update methods in federated learning and meta-learning is sensitive to learning rates.
method Proved that local update methods perform SGD on a surrogate loss function, characterized the surrogate loss, and derived convergence rates.
result Proper learning rate tuning is crucial for near-optimal behavior in communication-limited settings.
A new method improves semi-supervised learning by handling tasks with different attribute spaces.
problem Existing methods assume tasks share the same attribute space, limiting their applicability.
method Meta-learning approach that embeds labeled and unlabeled data in task-specific spaces using neural networks.
result Improves test performance on tasks with small labeled data using unlabeled and various task data.
New method learns diverse solutions in reinforcement learning without gradient bias.
problem Lack of diverse solutions in reinforcement learning tasks.
method Maximizes state-action-based mutual information directly, using variational lower bound.
result Successfully learns an infinite set of diverse solutions.
Paper tackles robust imitation learning from noisy demonstrations.
problem Learning from noisy demonstrations is challenging.
method Optimizes a classification risk with a symmetric loss, combining pseudo-labeling and co-training.
result Our method is more robust than state-of-the-art methods.
We discuss the relevance of the recent Machine Learning (ML) literature for economics and econometrics. First we discuss the differences in goals, methods and settings between the ML literature and the traditional econometrics and statistics literatures. Then we discuss some specific methods from the machine learning l…
Meta-learning algorithms are studied for their data efficiency in personalized federated learning.
problem Data efficiency in meta-learning algorithms.
method Algorithmic stability techniques to derive bounds on transfer risk and a new evaluation framework.
result Different meta-learning methods require varying amounts of supervision for effective generalization.
Deep Learning has the hierarchical network architecture to represent the complicated features of input patterns. Such architecture is well known to represent higher learning capability compared with some conventional models if the best set of parameters in the optimal network structure is found. We have been developing…
A new metric-based principal curve method learns 1D manifolds from spatial data.
problem Learning 1D manifolds from spatial data.
method Metric-based Principal Curve (MPC) approach.
result The method effectively learns the shape of 1D manifolds from synthetic and real datasets.
Book introduces deep learning methods with math, theory, and applications.
problem Understanding deep learning algorithms and their mathematical foundations.
method Reviews various ANN architectures and optimization methods, covers theoretical aspects.
result Provides a solid mathematical foundation for deep learning.
Survey categorizes methods for learning state representations in reinforcement learning.
problem Addressing challenges in complex observation spaces for sequential decision making.
method Categorizes six main classes of methods for learning state representations.
result Enhances understanding of state representation learning in reinforcement learning.
Numerous learning methods for fuzzy cognitive maps (FCMs), such as the Hebbian-based and the population-based learning methods, have been developed for modeling and simulating dynamic systems. However, these methods are faced with several obvious limitations. Most of these models are extremely time consuming when learn…
AIRL learns robust, generalizable reward functions from demonstrations.
problem Learning robust reward functions from demonstrations for changing environments.
method Adversarial Inverse Reinforcement Learning (AIRL) with hierarchical disentangled rewards.
result Generalizable policies and comparable results to state-of-the-art methods.
Adaptive methods learn from multiple datasets, leveraging similarities and robust to outliers.
problem Simultaneously analyze multiple datasets with possible similarities and differences.
method Adaptive multi-task learning methods that automatically utilize similarities and handle differences.
result Sharp statistical guarantees and robustness against outlier tasks demonstrated.
Learning to infer Bayesian posterior from a few-shot dataset is an important step towards robust meta-learning due to the model uncertainty inherent in the problem. In this paper, we propose a novel Bayesian model-agnostic meta-learning method. The proposed method combines scalable gradient-based meta-learning with non…
Structure learning of Bayesian networks has always been a challenging problem. Nowadays, massive-size networks with thousands or more of nodes but fewer samples frequently appear in many areas. We develop a divide-and-conquer framework, called partition-estimation-fusion (PEF), for structure learning of such big networ…
New method uses kernel methods to approximate ground states of quantum Hamiltonians efficiently.
problem Approximating ground states of quantum Hamiltonians using neural networks is computationally expensive.
method Introduces a statistical learning approach using kernel methods to make optimization trivial.
result Ground state properties of arbitrary gapped quantum Hamiltonians can be reached with polynomial resources.
Random feature method approximates operators with theoretical guarantees and reduced computation.
problem Approximating operators between infinite dimensional Banach spaces using machine learning.
method Random feature operator learning method with theoretical guarantees and error bounds.
result The random feature method can achieve similar or better test errors than kernel-based methods and neural networks with significantly reduced training times.
Due to myriads of classes, designing accurate and efficient classifiers becomes very challenging for multi-class classification. Recent research has shown that class structure learning can greatly facilitate multi-class learning. In this paper, we propose a novel method to learn the class structure for multi-class clas…
Proposes a new method to learn distance metrics for semi-supervised learning.
problem Inconsistency between perturbed input sets and lack of pairwise relationship information.
method Metric Learning by Similarity Network (MLSN) co-training with a classification network to learn distance metrics adaptively.
result Performs better than state-of-the-art methods on empirical tasks.
Paper uses AI methods to forecast Bitcoin prices.
problem Inaccurate Bitcoin price predictions in previous studies.
method Combines EEMD and LSTM for next-day price forecast.
result Improves Bitcoin price prediction accuracy.
This paper introduces a new feature learning technique based on error representation.
problem Learning high-level features for classification from diverse and imbalanced data.
method Inverse feature learning using error representation approach.
result Significantly better performance compared to state-of-the-art techniques.
This work analyzes the generalization properties of learned reconstruction methods for inverse problems.
problem Understanding the reliability and stability of learned reconstruction methods for inverse problems.
method Develops a general framework to interpret learned reconstruction methods in statistical learning context and performs their sample error analysis.
result Estimates the dependence of learned operators on training data, providing insights into their generalization properties.
Machine learning detects phishing websites by identifying common characteristics.
problem Phishing websites deceive users and steal sensitive information.
method Compared multiple machine learning methods for predicting phishing websites.
result Machine learning can identify common characteristics of phishing websites.
New active learning method uses combinatorial coverage to improve data transfer and reduce bias.
problem Inability to transfer sampled data to new models and sampling bias issues.
method Data-centric active learning methods utilizing combinatorial coverage.
result Sampling data with coverage leads to better data transfer and competitive sampling bias.
Paper proposes a new method to learn features from error representations.
problem Learning from error representations in machine learning.
method Inverse feature learning (IFL) based on deep clustering.
result IFL leads to improved performance in classification and clustering.
Deep learning methods improve subsurface flow modeling efficiency.
problem Efficiently modeling subsurface flow with uncertain parameters.
method Two categories of deep-learning based inverse modeling methods: surrogate-based and direct.
result Deep-learning methods significantly accelerate subsurface flow modeling.
Proposes a method to accelerate safe sequential learning using offline data.
problem Limited exploration due to disconnected safe regions and slow task learning.
method Safe transfer sequential learning using Gaussian processes and offline data.
result Enhances global exploration across multiple disjoint safe regions with lower data consumption.