The paper uses linear function approximators to bias MCTS for general games.
problem Improving MCTS playing strength for general games.
method Using linear function approximators with local features for self-play training.
result Significantly improved playing strength in multiple board games.
Neural networks outperform kernels by learning features better.
problem Current theories of feature learning do not adequately assess feature quality.
method Introduced feature quality metric and examined existing theories empirically.
result Current theories of feature learning do not provide a sufficient foundation for neural network generalization.
We address the problem of automatic generation of features for value function approximation. Bellman Error Basis Functions (BEBFs) have been shown to improve the error of policy evaluation with function approximation, with a convergence rate similar to that of value iteration. We propose a simple, fast and robust algor…
Study analyzes seasonal hydroclimatic features across climates and continents.
problem Lack of seasonal hydroclimatic feature analysis for Koppen-Geiger climates and continents.
method Global-scale analysis of 13,000 time series using 7 features.
result Notable differences in feature magnitudes across Koppen-Geiger climate classes and continental regions.
Deep neural networks and Gaussian processes are shown to be equivalent through activation functions.
problem Understanding the relationship between neural networks and Gaussian processes.
method Developing an equivalence theory based on activation functions and kernels.
result Models can be seen as neural networks with improved uncertainty prediction or deep Gaussian processes with increased accuracy.
Feature Squeezing is a recently proposed defense method which reduces the search space available to an adversary by coalescing samples that correspond to many different feature vectors in the original space into a single sample. It has been shown that feature squeezing defenses can be combined in a joint detection fram…
New framework uses time series features for predicting streamflow in ungauged areas.
problem Predicting streamflow in areas without gauging stations.
method Developed regression-based streamflow regionalization using a wide range of time series features from large datasets.
result Certain time series features, like entropy and autocorrelation, are better predictors of streamflow than traditional catchment attributes.
Paper presents a method to extract and interpret knowledge from a spiking neural classifier.
problem Extracting and interpreting knowledge from a spiking neural classifier with time-varying synaptic weights.
method The method involves encoding real-valued input data into spike patterns, training the classifier, and mapping the weighted postsynaptic potential to feature strength functions (FSFs).
result The FSFs represent the extracted knowledge from the classifier and can be used for classification and interpretation.
A new weighted dissimilarity measure reduces positioning errors in feature-based systems.
problem Reducing errors in feature-based positioning systems, especially in areas with high variability.
method Iterative scheme using location-dependent standard deviations as weights.
result Maximum radial positioning error reduced by 40% using the weighted dissimilarity measure.
Optimal feature learning strength improves generalization in deep networks.
problem Understanding how feature learning strength affects generalization in practical settings.
method Empirical studies and theoretical analysis of gradient flow dynamics in two-layer ReLU nets.
result Optimal feature learning strength yields substantial generalization gains, contrary to the prevailing intuition.
CutMix enhances feature learning in neural networks, improving test accuracy.
problem Understanding and improving feature learning in neural networks using patch-level augmentation.
method Three distinct methods: vanilla training, Cutout training, and CutMix training were studied.
result CutMix training yields the highest test accuracy and learns all features and noise vectors evenly.
SLOE speeds up logistic regression in high dimensions with accurate signal strength estimation.
problem Poor performance of logistic regression in high-dimensional settings.
method SLOE reparameterizes the signal strength for faster and more accurate estimation.
result SLOE provides a fast and accurate method for dimensionality correction in logistic regression.
We introduce a new method of performing high dimensional discriminant analysis, which we call multiDA. We achieve this by constructing a hybrid model that seamlessly integrates a multiclass diagonal discriminant analysis model and feature selection components. Our feature selection component naturally simplifies to wei…
New method quantifies model complexity for better interpretation.
problem Complex models produce misleading interpretation results.
method Functional decomposition to quantify model complexity.
result Post-hoc interpretation of complex models is more reliable and compact.
Most of metric learning approaches are dedicated to be applied on data described by feature vectors, with some notable exceptions such as times series, trees or graphs. The objective of this paper is to propose a metric learning algorithm that specifically considers relational data. The proposed approach can take benef…
A new method detects interactions in machine learning models.
problem Interpreting non-linear and interaction effects in machine learning models.
method Regional effect plots with implicit interaction detection.
result The method quantifies and interprets feature effects reliably, less confounded by interactions.
Even before deep learning architectures became the de facto models for complex computer vision tasks, the softmax function was, given its elegant properties, already used to analyze the predictions of feedforward neural networks. Nowadays, the output of the softmax function is also commonly used to assess the strength …
Motivation: Biomarker discovery from high-dimensional data is a crucial problem with enormous applications in biology and medicine. It is also extremely challenging from a statistical viewpoint, but surprisingly few studies have investigated the relative strengths and weaknesses of the plethora of existing feature sele…
Synaptic strength can be seen as probability to propagate impulse, and according to synaptic plasticity, function could exist from propagation activity to synaptic strength. If the function satisfies constraints such as continuity and monotonicity, neural network under external stimulus will always go to fixed point, a…
Convolutional Neural Networks(CNNs) are both computation and memory intensive which hindered their deployment in mobile devices. Inspired by the relevant concept in neural science literature, we propose Synaptic Pruning: a data-driven method to prune connections between input and output feature maps with a newly propos…
Dimensionality reduction is one of the key issues in the design of effective machine learning methods for automatic induction. In this work, we introduce recursive maxima hunting (RMH) for variable selection in classification problems with functional data. In this context, variable selection techniques are especially a…
Introduces a new feature importance measure using Gram-Schmidt decorrelation.
problem Determining feature influence strength and nature in datasets.
method Gram-Schmidt decorrelation and random forest regression.
result Empirical comparison of new estimators with established methods.
Bayesian model infers strengths from noisy tennis match outcomes.
problem Ranking tennis players from match outcomes.
method Bayesian approach to infer unobserved strengths and mapping function.
result Bayesian approach robust to different model specifications.
A new method selects features efficiently for high-dimensional data.
problem High computational costs and memory requirements in high-dimensional data.
method QuickSelection uses the strength of neurons in sparse autoencoders to select features.
result QuickSelection achieves the best trade-off of accuracy, speed, and memory usage.
This work provides bounds on the performance of prediction models in the predict-then-optimize framework.
problem Generalizing the performance of prediction models in the predict-then-optimize framework with the SPO loss function.
method Deriving generalization bounds using the Natarajan dimension and exploiting the strength property of the feasible region.
result Improved generalization bounds for the SPO loss function, scaling logarithmically in the number of extreme points and linearly in the decision dimension.
Predicting the occurrence of links is a fundamental problem in networks. In the link prediction problem we are given a snapshot of a network and would like to infer which interactions among existing members are likely to occur in the near future or which existing interactions are we missing. Although this problem has b…
Double descent risk in L2-regularized models explained and mitigated.
problem Risk of overparameterized models in machine learning.
method Analysis of L2-regularized models, two-layer neural networks, and CNNs.
result Double descent risk in L2-regularized models can be explained and mitigated by adjusting regularization strengths.
The paper investigates causal relationships in heart failure prediction using machine learning.
problem Understanding the causal relationships between clinical variables and heart failure.
method Proposes a new computational framework for causal structure discovery (CSD) of mixed-type clinical variables for binary disease outcomes.
result Feature importance from nonlinear classifiers strongly correlates with causal strength of variables, but not differentiating cause and effect.
The goal of the present study is to identify autism using machine learning techniques and resting-state brain imaging data, leveraging the temporal variability of the functional connections (FC) as the only information. We estimated and compared the FC variability across brain regions between typical, healthy subjects …
Mapper and Ball Mapper tools for complex data analysis.
problem Exploring and visualizing high-dimensional data and scalar functions.
method Combining Mapper and Ball Mapper, adding new features for encoding structure and symmetries.
result A new hybrid algorithm, Mapper on Ball Mapper, for comparing high-dimensional data descriptors.
New method for interpreting complex ML models.
problem Interpreting complex black-box ML models.
method Functional decomposition of black-box predictions into simpler subfunctions.
result Main effects provide insights into feature contributions and interactions.
Study on fluctuations in neural network kernels and predictions, focusing on finite width effects.
problem Characterizing fluctuations in finite width neural networks.
method Dynamical mean field theory analysis of wide but finite feature learning neural networks.
result Fluctuations in kernels and predictions are dynamically coupled, leading to reduced variance in feature learning regimes.
Sparse feature selection improves batch RL efficiency.
problem High-dimensional batch RL with many features.
method Sparse linear function approximation, Lasso, group Lasso, fitted Q-evaluation, fitted Q-iteration.
result Sparse feature selection makes batch RL more sample efficient.
This paper categorizes and analyzes existing outlying aspect mining methods.
problem Finding unique features in data objects that differ from others.
method Grouping and analyzing existing outlying aspect mining approaches in three categories.
result Comparison of strengths, weaknesses, and time complexities of different techniques.
We provide a unified analysis of the predictive risk of ridge regression and regularized discriminant analysis in a dense random effects model. We work in a high-dimensional asymptotic regime where p,n→∞ and p/n→γ∈(0,∞), and allow for arbitrary covariance among the features. For both metho…
Integrated Computational Materials Engineering (ICME) aims to accelerate optimal design of complex material systems by integrating material science and design automation. For tractable ICME, it is required that (1) a structural feature space be identified to allow reconstruction of new designs, and (2) the reconstructi…
TGNN combines GNN and SMM for better trading network predictions.
problem Predicting asset prices in trading networks with structural impact factors.
method Combines GNN and SMM for asset price prediction.
result TGNN outperforms existing methods in prediction accuracy.
SGD quickly learns a spurious XOR feature before the signal feature, revealing learning dynamics.
problem Over-reliance on spurious correlations in neural networks trained by SGD.
method Theoretical analysis of SGD on two-layer ReLU networks trained on XOR data.
result SGD learns the spurious feature first and exponentially fast, dominating the signal feature.
We study graph matching with correlated Gaussian features and find thresholds for exact recovery.
problem Graph matching with correlated Gaussian features.
method Information-theoretic thresholds and conditions for exact and almost exact recovery.
result Contextual information introduces a richer structure, with thresholds for exact and almost exact recovery no longer coinciding.
DRSVM uses deep learning to rank relative attributes between image pairs.
problem Classifying relative attributes between image pairs.
method Deep Siamese network with rank SVM loss function.
result DRSVM outperforms state-of-the-art methods on multiple datasets.
We seek decision rules for prediction-time cost reduction, where complete data is available for training, but during prediction-time, each feature can only be acquired for an additional cost. We propose a novel random forest algorithm to minimize prediction error for a user-specified {\it average} feature acquisition b…
PNA improves GNNs for graph data with multiple aggregators.
problem Capturing continuous features in graph neural networks.
method Combines multiple aggregators with degree-scalers.
result PNA outperforms existing models on graph theory and real-world tasks.
DIGEN benchmark provides synthetic datasets for ML algorithm evaluation.
problem Understanding and comparing machine learning algorithms' performance.
method Synthetic datasets generated using 40 mathematical functions to evaluate machine learning algorithms.
result DIGEN resource facilitates understanding why algorithms perform poorly and provides ideas for improvement.
The paper introduces a new method for modeling correlations in high-dimensional data.
problem Modeling correlations among multiple dimensions in high-dimensional data.
method Population random measure embedding (PRME) using random functions and neural networks.
result The proposed method can efficiently learn correlations among multiple dimensions.
Paper proposes BTuD for unsupervised feature selection.
problem Feature selection in unsupervised learning.
method Bayesian Tucker decomposition (BTuD) with Gaussian residual.
result Successfully applied to various datasets.
Paper presents a simple method for accurate user localization in urban areas.
problem Poor GPS performance in dense urban environments.
method Localization based on pathloss using base station signal strengths and RadioUNet approximations.
result Accurate user location extraction in urban environments.
Identifies learning rules from neural network observables.
problem Determine the underlying plasticity rules governing learning in biological systems.
method Simulated idealized neuroscience experiments with artificial neural networks to generate a dataset of learning trajectories. Used linear and non-linear classifiers to identify learning rules from aggregate statistics of weights, activations, and activity changes.
result Different classes of learning rules can be separated solely on the basis of aggregate statistics of the weights, activations, or instantaneous layer-wise activity changes.
Models with many signals, high-dimensional models, often impose structures on the signal strengths. The common assumption is that only a few signals are strong and most of the signals are zero or close (collectively) to zero. However, such a requirement might not be valid in many real-life applications. In this article…