Skip connections improve biologically-inspired learning rules.
problem Biologically-inspired learning rules often underperform compared to backpropagation.
method Introduced skip connections between intermediate layers in biologically-motivated learning rules.
result Skip connections can match the performance of backpropagation and are robust to hyper-parameters.
A major goal of unsupervised learning is to discover data representations that are useful for subsequent tasks, without access to supervised labels during training. Typically, this involves minimizing a surrogate objective, such as the negative log likelihood of a generative model, with the hope that representations us…
Biological neurons learn tensor decompositions of higher-order correlations using nonlinear Hebbian plasticity.
problem Learning higher-order correlations in biological neurons.
method Introduce and study generalized nonlinear Hebbian learning rules.
result Neurons can learn tensor eigenvectors of higher-order input correlation tensors.
New learning rules for wide neural networks without backpropagation.
problem Training wide neural networks efficiently and without backpropagation.
method Input-weight alignment driven by gradient descent in the NTK regime.
result Biologically-motivated learning rules equivalent to backpropagation in wide networks.
The backpropagation of error algorithm (BP) is impossible to implement in a real brain. The recent success of deep networks in machine learning and AI, however, has inspired proposals for understanding how the brain might learn across multiple layers, and hence how it might approximate BP. As of yet, none of these prop…
Spiking neural networks (SNNs) with a lattice architecture are introduced in this work, combining several desirable properties of SNNs and self-organized maps (SOMs). Networks are trained with biologically motivated, unsupervised learning rules to obtain a self-organized grid of filters via cooperative and competitive …
Finding biologically plausible alternatives to back-propagation of errors is a fundamentally important challenge in artificial neural network research. In this paper, we propose a learning algorithm called error-driven Local Representation Alignment (LRA-E), which has strong connections to predictive coding, a theory t…
This study benchmarks transcriptomics models for perturbation analysis, finding scVI and PCA superior.
problem Limited evaluation of transcriptomics foundation models for perturbation analysis.
method Developed a novel evaluation framework using diverse public datasets from different sequencing techniques and cell lines.
result scVI and PCA identified as superior models for understanding biological perturbations.
We introduce Delay Pruning, a simple yet powerful technique to regularize dynamic Boltzmann machines (DyBM). The recently introduced DyBM provides a particularly structured Boltzmann machine, as a generative model of a multi-dimensional time-series. This Boltzmann machine can have infinitely many layers of units but al…
Phylogenetic tree reconstruction is traditionally based on multiple sequence alignments (MSAs) and heavily depends on the validity of this information bottleneck. With increasing sequence divergence, the quality of MSAs decays quickly. Alignment-free methods, on the other hand, are based on abstract string comparisons …
The functions of proteins and RNAs are determined by a myriad of interactions between their constituent residues, but most quantitative models of how molecular phenotype depends on genotype must approximate this by simple additive effects. While recent models have relaxed this constraint to also account for pairwise in…
InfoSEM infers gene regulatory networks without GT labels, improving performance.
problem Inferring GRNs from gene expression data with high accuracy and avoiding biases.
method InfoSEM uses deep generative models with informative priors (textual gene embeddings).
result InfoSEM outperforms existing models by 38.5% across four datasets.
Error backpropagation is a highly effective mechanism for learning high-quality hierarchical features in deep networks. Updating the features or weights in one layer, however, requires waiting for the propagation of error signals from higher layers. Learning using delayed and non-local errors makes it hard to reconcile…
New derivation shows how a three-factor learning rule is derived from Oja's rule.
problem Deriving a three-factor learning rule from Oja's rule.
method Using frame theory to systematically derive EGHR-PCA from Oja's rule.
result A principled derivation of a biologically plausible learning rule.
R2N learns interpretable rules and literals from numerical features.
problem Lack of expressive vocabulary in rule-based decision models.
method Relational Rule Network (R2N) learns literals and rules end-to-end.
result Learned literals improve prediction accuracy and rule conciseness.
New framework learns interpretable rule ensembles without sacrificing accuracy.
problem Trade-off between accuracy and interpretability in rule ensembles.
method Introduces local interpretability and a regularizer to promote it, using coordinate descent with local search.
result Learns rule ensembles with fewer rules to explain individual predictions, maintaining comparable accuracy.
NeuRules learns interpretable rule lists from data without pre-discretization.
problem Interpretable rule lists for high-stakes decisions in machine learning.
method Neuro-symbolic continuous optimization with temperature annealing.
result NeuRules outperforms existing methods in learning interpretable rule lists.
DeepCTRL integrates rules into deep learning models, allowing flexible control at inference.
problem Lack of flexibility in incorporating rules into deep learning models.
method Integrates rule representations into deep neural networks, enabling flexible control at inference.
result Improves rule verification ratio and accuracy gains at downstream tasks.
In this article supervised learning problems are solved using soft rule ensembles. We first review the importance sampling learning ensembles (ISLE) approach that is useful for generating hard rules. The soft rules are then obtained with logistic regression from the corresponding hard rules. In order to deal with the p…
New methods prune unpromising rules from KGs, improving scalability and runtime.
problem Scalability issues in walk-based rule learning from KGs.
method Rule Hierarchy Framework (RHF) and Hierarchical Pruning (HPMs).
result Significant reductions in runtime and number of learned rules without compromising predictive performance.
Paper extends transfer learning for decision rules, improving treatment rule estimation.
problem Estimating optimal individualized treatment rules under changing conditions.
method Bayes decision rules and low-dimensional empirical risk minimization.
result Consistent estimators and risk bounds established under mild conditions.
Study explores reinforcement learning in a complex game environment, analyzing rule inference and policy learning.
problem Learning optimal policies in environments with hidden rules.
method Investigated using the Game Of Hidden Rules (GOHR) environment, employing Feature-Centric and Object-Centric state representations with a Transformer-based A2C algorithm.
result Transformer-based A2C models outperform traditional methods in GOHR, demonstrating the effectiveness of representation strategies.
CRL approach improves understanding of heterogeneous treatment effects in complex diseases.
problem Estimating heterogeneous treatment effects in complex diseases.
method Causal rule learning (CRL) workflow consisting of rule discovery, selection, and analysis.
result CRL outperforms other methods in providing interpretable estimates of HTE.
Generative models learn rules at different timescales, revealing a 'innovation window'.
problem Generative models' convergence to empirical training distribution rather than population distribution.
method Rule-valid synthetic tasks, analyzing τrule and τmem across training timescales. result The 'innovation window' widens with increasing dataset size and narrows with rule complexity.
A method for collecting human supervision that combines rules and instance labels.
problem Lack of labeled data and inefficient human supervision.
method Rule-exemplar method with training algorithm for joint denoising and model training.
result Our algorithm is more accurate than existing methods and effectively denoises rules.
A new learning rule consistently reduces error over data samples.
problem Finding a learning rule that consistently reduces error over all data distributions.
method A deterministic, data-dependent partitioning rule that only partitions cyclic intervals with sufficient empirical diversity of labels.
result The expected error is monotone non-increasing with the sample size under every data distribution.
Advances rule-based multi-label classification using conformal prediction.
problem Improving accuracy and decision making in multi-label classification.
method Combines conformal prediction with rule-based learning to provide natural conformity scores and calibrate rule assessments.
result Calibrated conformity scores enhance prediction accuracy and decision making.
Improved convergence speed of principal component analysis through modified learning rules.
problem Slow convergence for covariance matrices with close eigenvalues.
method Introduced an additional term to the objective function to mitigate convergence issues.
result Significantly improved convergence speed confirmed through simulations.
In a physical neural system, where storage and processing are intimately intertwined, the rules for adjusting the synaptic weights can only depend on variables that are available locally, such as the activity of the pre- and post-synaptic neurons, resulting in local learning rules. A systematic framework for studying t…
Throughout music history, theorists have identified and documented interpretable rules that capture the decisions of composers. This paper asks, "Can a machine behave like a music theorist?" It presents MUS-ROVER, a self-learning system for automatically discovering rules from symbolic music. MUS-ROVER performs feature…
A new method learns interpretable decision rules using submodular optimization.
problem Learning interpretable decision rules from data.
method Submodular optimization approach for selecting rules from a large set.
result The method effectively learns interpretable rule sets from real datasets.
Combines neural networks and expert rules for concept-based learning.
problem Extending concept-based learning with machine learning models.
method Form constraints for joint probability distribution and represent feasible set as a convex polytope.
result Neural networks can be trained to satisfy expert rules without violating them.
LI-ITR combines flexible ML with interpretable approximations for personalized treatment rules.
problem Combining flexibility and interpretability in personalized treatment rules.
method Uses variational autoencoders and a mixture of interpretable experts.
result Accurately recovers true local coefficients and optimal treatment strategies.
In the artificial intelligence field, learning often corresponds to changing the parameters of a parameterized function. A learning rule is an algorithm or mathematical expression that specifies precisely how the parameters should be changed. When creating an artificial intelligence system, we must make two decisions: …
Two local learning rules are investigated to avoid weight transport in neural networks.
problem Local learning rules that avoid weight transport are unstable and require tuning.
method Investigated two non-local learning rules and a more robust local rule.
result Non-local learning rules match state-of-the-art performance and operate effectively in noisy updates.
Identifies learning rules from neural network observables.
problem Determine the underlying plasticity rules governing learning in biological systems.
method Simulated idealized neuroscience experiments with artificial neural networks to generate a dataset of learning trajectories. Used linear and non-linear classifiers to identify learning rules from aggregate statistics of weights, activations, and activity changes.
result Different classes of learning rules can be separated solely on the basis of aggregate statistics of the weights, activations, or instantaneous layer-wise activity changes.
Adaptive rule improves kernel-based gradient descent performance.
problem Improving convergence speed of kernel-based gradient descent algorithms.
method Empirical effective dimension for stopping rule, learning theory analysis, integral operator approach.
result Optimal learning rates and iteration bounds for KGD with adaptive stopping rule.
The article reviews scoring rules for estimating and evaluating forecasts.
problem Evaluating probabilistic forecasts and estimating probability distributions.
method Mathematical foundations and characterization of scoring rules.
result Important families of scoring rules and their applications in statistics and machine learning.
We consider the problem of learning a sparse rule model, a prediction model in the form of a sparse linear combination of rules, where a rule is an indicator function defined over a hyper-rectangle in the input space. Since the number of all possible such rules is extremely large, it has been computationally intractabl…
New rule-based method for classification with scalability, interpretability, and fairness.
problem Developing a scalable and fair classification method.
method Column generation for linear programming, decision tree-based heuristic, and rule-based optimization.
result The method returns interpretable rules with optimal weights and addresses fairness constraints.
The paper develops deep learning models for personalized treatment rules in survival analysis.
problem Deriving optimal treatment rules for bivariate survival outcomes in randomized trials.
method Adaptive prediction-powered learning using deep neural networks and stochastic policies.
result Maximizes joint survival probability beyond fixed time points (t1,t2). FedRule uses graph neural networks to recommend rules for smart homes without centralizing data.
problem Manual rule setup for smart devices is inefficient and privacy-compromising.
method FedRule constructs user-specific graphs for rule recommendation, using federated learning to protect privacy.
result FedRule achieves comparable performance to centralized methods and outperforms others.
In this paper, we study the problem of learning probabilistic logical rules for inductive and interpretable link prediction. Despite the importance of inductive link prediction, most previous works focused on transductive link prediction and cannot manage previously unseen entities. Moreover, they are black-box models …
Paper proposes robust methods for estimating optimal treatment rules with censored survival data.
problem Estimating optimal treatment rules for censored survival data.
method Developed two robust criteria and a sampling-based difference-of-convex algorithm for learning optimal treatment rules.
result Proposed methods show improved performance compared to existing methods in simulations and real data.
ASTRA uses unlabeled data and weak rules to train deep models effectively.
problem Learning with weak supervision rules is challenging due to their heuristic and noisy nature.
method ASTRA framework that considers contextualized representations and pseudo-labels for unlabeled data, and a rule attention network to aggregate labels.
result Significant improvements over state-of-the-art baselines on text classification benchmarks.
New learning rules achieve optimal sample complexity for weakly supervised classification.
problem Learning from label proportions in weakly supervised settings.
method Debiased proportional square loss and EasyLLP learning rule.
result Achieves optimal sample complexity in both realizable and agnostic settings.
Multi-label classification (MLC) is a supervised learning problem in which, contrary to standard multiclass classification, an instance can be associated with several class labels simultaneously. In this chapter, we advocate a rule-based approach to multi-label classification. Rule learning algorithms are often employe…
Optimal Volt/VAR control rules designed using deep learning.
problem Designing optimal Volt/VAR control rules for DERs to regulate voltage fluctuations.
method Formulated as a deep learning problem, where a DNN emulates Volt/VAR dynamics and optimizes rule parameters.
result DNN-based optimization outperforms MINLP in efficiency and accuracy.