Automatic differentiation---the mechanical transformation of numeric computer programs to calculate derivatives efficiently and accurately---dates to the origin of the computer age. Reverse mode automatic differentiation both antedates and generalizes the method of backwards propagation of errors used in machine learni…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,742 papers · 148 categories
Trend · papers per month
48 results for “Automatic Learning”
AutoSGD automatically adjusts learning rates for SGD.
problem Inefficient manual learning rate selection for SGD.
method AutoSGD automatically adjusts learning rates based on iteration.
result Empirical results show strong performance across various optimization tasks.
This paper presents a novel approach for automatic rule learning applicable to an autonomous driving system using real driving data.
D-Adaptation automatically sets optimal learning rates without manual tuning.
problem Optimizing learning rates for efficient convergence in machine learning.
method D-Adaptation, which asymptotically achieves optimal learning rates without back-tracking or additional evaluations.
result D-Adaptation automatically matches hand-tuned learning rates across diverse problems.
Develops a mathematical model for automatic differentiation in machine learning.
problem Current automatic differentiation lacks a simple mathematical model for machine learning.
method Articulates relationships between program differentiation and nonsmooth functions, provides a class of functions and nonsmooth calculus.
result Shows how nonsmooth calculus applies to stochastic approximation methods and evidence of artificial critical points.
Method uses Seq2Seq learning to automatically generate recovery commands for ICT systems.
problem Manual decision-making for recovery commands is time-consuming and error-prone.
method Seq2Seq neural network model trained on past logs and commands.
result The model can estimate accurate recovery commands from new failures.
Automatically generates a deep RL curriculum for faster and more stable learning.
problem How to automatically generate a curriculum for deep RL agents.
method Interprets curriculum generation as an inference problem, learning task distributions progressively.
result Curricula significantly improve learning performance across various environments and deep RL algorithms.
Purpose: Machine learning is broadly used for clinical data analysis. Before training a model, a machine learning algorithm must be selected. Also, the values of one or more model parameters termed hyper-parameters must be set. Selecting algorithms and hyper-parameter values requires advanced machine learning knowledge…
Learn to automatically plug domain-specific modules into a common network.
problem Learning inflexibility and computational intensiveness in multi-domain learning.
method Neural Architecture Search (NAS) for data-driven adapter plugging and structure design.
result NAS-driven MDL model achieves comparable performance to existing approaches.
AutoGD automatically adjusts learning rates for gradient descent.
problem Optimizing learning rates for gradient descent methods.
method AutoGD automatically adjusts learning rates based on iteration.
result AutoGD can recover the optimal rate of GD for a broad class of functions.
ARDA automatically augments datasets for machine learning models.
problem Improving machine learning model performance through data augmentation.
method ARDA combines data searching, joining, and feature selection to augment datasets.
result Training models on ARDA's augmented datasets leads to better performance.
GOLS-I automatically determines learning rates for various neural network training algorithms.
problem Adapting learning rates in stochastic training algorithms for neural networks.
method Gradient-Only Line Search (GOLS-I) for automatically setting learning rates.
result GOLS-I learning rate schedules are competitive with manually tuned rates across multiple algorithms, architectures, datasets, and loss functions.
Paper presents a Transformer model for automatic domain adaptation.
problem Challenges in selecting or designing domain adaptation algorithms.
method Transformer model approximates and selects domain adaptation algorithms.
result Transformers can approximate and automatically select domain adaptation algorithms.
Automatic, Dynamic, and Nearly Optimal Learning Rate Specification by Local Quadratic Approximationstat.ML
A new method automatically and dynamically sets learning rates in deep learning.
problem Determining the appropriate learning rate in deep learning tasks is challenging and often subjective.
method Local Quadratic Approximation (LQA) to automatically and dynamically set learning rates.
result The proposed method leads to nearly optimal learning rates in a computationally efficient way.
We present a system for the automatic differentiation of a higher-order functional array-processing language. The core functional language underlying this system simultaneously supports both source-to-source automatic differentiation and global optimizations such as loop transformations. Thanks to this feature, we demo…
Automatically learns flexible symmetry constraints in neural networks using gradients.
problem Fixed hard constraints on neural network functions that cannot be adapted.
method Improves parameterisations of soft equivariance and optimizes marginal likelihood using differentiable Laplace approximations.
result Achieves equivalent or improved performance on image classification tasks compared to baselines with hard-coded symmetry.
New deep learning method validated across multiple sleep staging databases.
problem Improving automatic sleep scoring accuracy across different datasets.
method Ensemble of local models using deep learning for automatic sleep staging.
result Good general performance compared to human experts and state-of-the-art methods.
Paper proposes NNAFC for automatic financial factor construction.
problem Manual factor construction is time-consuming and prone to bias.
method NNAFC uses neural networks to automatically construct diversified financial factors.
result NNAFC outperforms GP in constructing more informative and diversified factors.
This paper proposes automatic tuning of Bayesian Optimization's acquisition function.
problem Optimizing black-box functions with noisy, expensive evaluations and hyperparameter tuning.
method Exploring heuristics to automatically tune acquisition functions in Bayesian Optimization.
result Demonstrates effectiveness of heuristics in automatic Bayesian Optimization.
ACL improves DRL by adapting task difficulty to agent's capabilities.
problem Improving sample efficiency and performance in DRL.
method Adapting task difficulty based on agent's capacity.
result Enhanced sample efficiency and asymptotic performance.
Meta-SAC automatically tunes SAC's entropy temperature for better exploration.
problem Exploration-exploitation dilemma in reinforcement learning.
method Meta-SAC uses metagradient and a novel meta objective to automatically adjust SAC's entropy temperature.
result Meta-SAC outperforms SAC-v2 by 10% on the humanoid-v2 task.
SALSA automatically adjusts learning rates in stochastic gradient methods.
problem Automatic adjustment of learning rates in stochastic gradient methods.
method SALSA uses a line-search procedure to gradually increase the learning rate, then a statistical test to decrease it.
result SALSA matches the performance of best hand-tuned learning rate schedules in deep learning tasks.
SAFE automates feature engineering for industrial tasks efficiently and scalably.
problem Efficiency and scalability of automatic feature engineering methods for industrial tasks.
method SAFE (Scalable Automatic Feature Engineering) method, which provides excellent efficiency and scalability.
result SAFE method provides prominent efficiency and competitive effectiveness in industrial tasks.
S4 learns new self-supervision automatically, improving accuracy with less human effort.
problem Lack of direct supervision in machine learning.
method Combines deep learning and probabilistic logic to automatically generate and verify new self-supervision.
result S4 can automatically propose accurate self-supervision, matching supervised methods with less human effort.
A recurring problem when building probabilistic latent variable models is regularization and model selection, for instance, the choice of the dimensionality of the latent space. In the context of belief networks with latent variables, this problem has been adressed with Automatic Relevance Determination (ARD) employing…
AutoFIS automatically selects important feature interactions for CTR prediction models.
problem Manual feature interaction design is inefficient and prone to noise.
method Two-stage algorithm: search stage relaxes feature interactions to continuous parameters, re-train stage refines model performance.
result AutoFIS significantly improves CTR and CVR of FM-based models.
RobustSleepNet automates sleep stage classification for any PSG montage.
problem Manual sleep staging is tedious and expensive; automatic methods are limited by PSG montage and demographic differences.
method RobustSleepNet is a deep learning model that handles arbitrary PSG montages and is robust to demographic changes.
result RobustSleepNet achieves 97% F1 score on unseen data, outperforming specific training datasets by 2%.
We present a model that can automatically learn alignments between high-dimensional data in an unsupervised manner. Our proposed method casts alignment learning in a framework where both alignment and data are modelled simultaneously. Further, we automatically infer groupings of different types of sequences within the …
Ontology learning is a critical task in industry, dealing with identifying and extracting concepts captured in text data such that these concepts can be used in different tasks, e.g. information retrieval. Ontology learning is non-trivial due to several reasons with limited amount of prior research work that automatica…
Automatic differentiation (AD) is an essential primitive for machine learning programming systems. Tangent is a new library that performs AD using source code transformation (SCT) in Python. It takes numeric functions written in a syntactic subset of Python and NumPy as input, and generates new Python functions which c…
agtboost speeds up gradient tree boosting with automatic complexity adjustment.
problem Speeding up and simplifying gradient tree boosting computations.
method Adaptive gradient tree boosting with automatic complexity adjustment and feature importance.
result Significant decrease in computation time and simplification of model complexity.
Paper learns dictionaries for sparse signal recovery using automatic differentiation.
problem Learning dictionaries for sparse signal recovery from noisy data.
method Approximates reconstructions using FB algorithm and learns dictionaries with projected gradient descent.
result Successfully learns 1D TV dictionary from piecewise constant signals.
Achieving advancements in automatic recognition of emotions that music can induce require considering multiplicity and simultaneity of emotions. Comparison of different machine learning algorithms performing multilabel and multiclass classification is the core of our work. The study analyzes the implementation of the G…
Background: Despite recent significant progress in the development of automatic sleep staging methods, building a good model still remains a big challenge for sleep studies with a small cohort due to the data-variability and data-inefficiency issues. This work presents a deep transfer learning approach to overcome thes…
The progress of deep convolutional neural networks has been successfully exploited in various real-time computer vision tasks such as image classification and segmentation. Owing to the development of computational units, availability of digital datasets, and improved performance of deep learning models, fully automati…
Existing automatic music generation approaches that feature deep learning can be broadly classified into two types: raw audio models and symbolic models. Symbolic models, which train and generate at the note level, are currently the more prevalent approach; these models can capture long-range dependencies of melodic st…
genriesz: A Python Package for Automatic Debiased Machine Learning with Generalized Riesz Regressionstat.ML
Python package automates causal parameter estimation using Riesz regression.
problem Efficient estimation of causal and structural parameters.
method Automatic DML and generalized Riesz regression framework.
result Automatic construction of balancing link functions for generalized Riesz regression.
AutoKE automates embedding physical knowledge into neural networks for complex engineering problems.
problem Complex physical equations in engineering problems.
method AutoKE framework using deep neural networks, equation parsing, automatic differentiation, adaptive weights, and NAS.
result Automatically embeds physical knowledge into neural networks for complex equations efficiently.
Deep Reinforcement Learning (DRL) is a trending field of research, showing great promise in many challenging problems such as playing Atari, solving Go and controlling robots. While DRL agents perform well in practice we are still missing the tools to analayze their performance and visualize the temporal abstractions t…
CoLA automates efficient numerical linear algebra for complex matrix structures.
problem Efficiently solving large-scale linear algebra problems with complex matrix structures.
method Combining linear operator abstraction with compositional dispatch rules.
result Automatic and efficient numerical algorithms for various linear algebra operations.
This paper introduces a new Urban Point Cloud Dataset for Automatic Segmentation and Classification acquired by Mobile Laser Scanning (MLS). We describe how the dataset is obtained from acquisition to post-processing and labeling. This dataset can be used to learn classification algorithm, however, given that a great a…
Improves numerical solution of ill-conditioned linear systems for machine learning.
problem Wastefulness and instability in solving ill-conditioned linear systems.
method autonugget combines Richardson extrapolation to determine the solution of the ill-conditioned system, improving accuracy over a single nugget.
result Improves accuracy of numerical solution of ill-conditioned linear systems.
This work presents an approach to automatically induction for non-greedy decision trees constructed from neural network architecture. This construction can be used to transfer weights when growing or pruning a decision tree, allowing non-greedy decision tree algorithms to automatically learn and adapt to the ideal arch…
Implicit Q-learning and SARSA adjust step-sizes automatically, improving stability and performance.
problem Numerical instability and slow progress in Q-learning and SARSA due to step-size calibration.
method Reformulate iterative updates as fixed-point equations, scaling step-sizes inversely with feature norms.
result Implicit methods maintain stability over broader step-size ranges and achieve comparable convergence rates.
Cascading flows improve variational inference in structured programs.
problem Challenges in variational inference for complex probabilistic programs.
method Integrates normalizing flows and ASVI to create cascading flows, which embed the forward-pass of probabilistic programs.
result Cascading flows outperform normalizing flows and ASVI in structured inference problems.
Joint Unsupervised and Supervised Training for Automatic Speech Recognition via Bilevel Optimizationcs.CL
Novel approach trains ASR models with less supervision using bilevel optimization.
problem Training acoustic models for ASR with minimal supervision.
method Bilevel optimization with unsupervised and supervised losses.
result Achieves superior performance compared to existing methods.
A key aspect of automating predictive machine learning entails the capability of properly triggering the update of the trained model. To this aim, suitable automatic solutions to self-assess the prediction quality and the data distribution drift between the original training set and the new data have to be devised. In …
Automatically identifies RRLyrae stars from VVV survey data.
problem Classifying RRLyrae stars from a large dataset of light curves.
method Developed an automatic ML-based procedure to identify RRLs, using features like period and intensity, and pseudo-colors.
result Constructed an ensemble classifier with Recall of 0.48 and Precision of 0.86 over 15 tiles.