Paper identifies how neural networks learn features.
problem Understanding how neural networks automatically select features.
method Deep Neural Feature Ansatz, implementing average gradient outer product.
result Recursive Feature Machines achieve state-of-the-art performance.
RocketStack integrates predictions from multiple base learners using a recursive stacking architecture up to ten levels.
problem Feature redundancy, complexity, and computational burden in deep stacking.
method Level-aware recursive stacking with pruning and compression techniques.
result Increasing accuracy with depth and outperforming standalone ensembles at later levels.
AGOP mechanism explains deep neural collapse in neural networks.
problem Explaining the rigid structure of data representations in deep neural networks.
method Introducing AGOP and Deep RFM to demonstrate DNC.
result AGOP mechanism causes deep neural collapse in neural networks.
Proposes a two-stage sector rotation method using machine learning and deep learning.
problem Identifying sectors with high investment attractiveness based on market conditions.
method Two-stage methodology: 1) Predict ETF prices using market indicators and feature selection, 2) Rank sectors based on predicted returns and select top sectors.
result The proposed methodology outperforms equally weighted portfolios and Echo State Networks show outstanding performance.
Machine learning finds new natural laws from noisy data.
problem Discovery of natural laws relies on human inspiration.
method Recursive-LASSO-based symbolic regression (RLS) method.
result Data-driven formulation of natural laws from noisy data.
RFM reduces feature space for linear models, improving sparse recovery.
problem Sparse linear regression and low-rank matrix recovery.
method Recursive Feature Machines (RFM) that alternates between reweighting feature vectors by AGOP and learning prediction function.
result RFM generalizes IRLS and outperforms deep linear networks.
Tab-TRM uses recursive model for insurance pricing on tabular data.
problem Insurance pricing on tabular data.
method Adapts recursive latent reasoning to insurance modeling using a compact, parameter-efficient network.
result Improves insurance pricing accuracy using iterative refinement of latent tokens.
Study analyzes feedback complexity for sparse feature retrieval in deep networks.
problem Learning sparse superposed features with feedback.
method Analysis of feedback complexity in sparse settings, including triplet comparisons.
result Establishes tight bounds and strong upper bounds for feature recovery.
Since the machine learning techniques are improving rapidly, it has been shown that the image recognition techniques in deep neural networks can be used to detect jet substructure. And it turns out that deep neural networks can match or outperform traditional approach of expert features. However, there are disadvantage…
Recursive Feature Machines show grokking in modular arithmetic without neural networks.
problem Grokking in modular arithmetic tasks.
method Recursive Feature Machines (RFM) with Average Gradient Outer Product (AGOP).
result RFM and neural networks learn block-circulant features to solve modular arithmetic.
We develop an approach for feature elimination in statistical learning with kernel machines, based on recursive elimination of features.We present theoretical properties of this method and show that it is uniformly consistent in finding the correct feature space under certain generalized assumptions.We present four cas…
Breaking symmetry in training data is key for generalization in feature learning kernels.
problem Grokking in algebraic tasks, where models perform well on training but fail on unseen data.
method Used Recursive Feature Machine (RFM) with AGOP to learn task-relevant features, breaking symmetry in training data.
result Generalization occurs only when symmetry in the training set is broken, and RFM generalizes by recovering underlying invariance group action.
A new ML algorithm solves complex economic control problems.
problem Solving high-dimensional, finite-horizon stochastic control problems in economics.
method Deep neural network representation of optimal policy functions with three key features.
result Efficiently solves various economic control problems including recursive utility and growth models.
Implicit deep learning prediction rules generalize the recursive rules of feedforward neural networks. Such rules are based on the solution of a fixed-point equation involving a single vector of hidden features, which is thus only implicitly defined. The implicit framework greatly simplifies the notation of deep learni…
Two new feature selection algorithms improve on RFE.
problem Optimal feature selection for faster and more accurate models.
method Fibonacci and k-Subsecting Recursive Feature Elimination.
result Faster feature selection with comparable predictive performance.
Tangle machines are topologically inspired diagrammatic models. Their novel feature is their natural notion of equivalence. Equivalent tangle machines may differ locally, but globally they are considered to share the same information content. The goal of tangle machine equivalence is to provide a context-independent me…
Recursive neural networks have widely been used by researchers to handle applications with recursively or hierarchically structured data. However, embedded control flow deep learning frameworks such as TensorFlow, Theano, Caffe2, and MXNet fail to efficiently represent and execute such neural networks, due to lack of s…
Dimensionality reduction is one of the key issues in the design of effective machine learning methods for automatic induction. In this work, we introduce recursive maxima hunting (RMH) for variable selection in classification problems with functional data. In this context, variable selection techniques are especially a…
New algorithm reduces rank constrained optimization problems.
problem Rank constrained optimization problems in machine learning and statistics.
method Recursive Importance Sketching (RISRO) algorithm.
result RISRO offers clear advantages over existing algorithms and converges efficiently.
Score-based martingale posteriors improve uncertainty quantification in deep neural networks.
problem Uncertainty quantification in deep neural networks
method Score-based martingale posteriors
result SMPs provide a fast, deterministic way to simulate the limiting random variable.
New random forest algorithms for PU learning minimize risk directly.
problem Learning from positive and unlabeled data.
method Recursive greedy risk minimization for decision trees.
result Efficient PU random forest algorithm with robustness and low hyperparameter tuning.
RAMPART ranks top-k features more accurately than existing methods.
problem Accurate ranking of important features in machine learning.
method Adaptive sequential halving strategy combined with ensembling techniques.
result RAMPART achieves the correct top-k ranking with high probability.
Deep learning solves dynamic programming with recursive utility.
problem Challenges in solving high-dimensional discrete-time dynamic programming problems with recursive utility.
method Certainty Equivalent Learning (CEL) algorithm that learns certainty-equivalent value directly with neural networks.
result Accurate value and policy approximations in high-dimensional problems, comparable to VFI in some cases.
Inspired by a growing interest in analyzing network data, we study the problem of node classification on graphs, focusing on approaches based on kernel machines. Conventionally, kernel machines are linear classifiers in the implicit feature space. We argue that linear classification in the feature space of kernels comm…
A new algorithm SRG-DQN reduces variance in deep Q-learning.
problem Inaccurate estimation of anchor points in SVRG for deep Q-learning.
method Introduces recursive gradient variance reduction for stochastic gradient updates.
result Demonstrates improved efficiency and effectiveness of SRG-DQN on reinforcement learning tasks.
Recursive KalmanNet combines neural networks with Kalman filters for precise state estimation.
problem State estimation in systems with noisy measurements and non-Gaussian noise.
method Recursive KalmanNet uses a recurrent neural network to estimate states with consistent error covariance, optimizing for Gaussian negative log-likelihood.
result Recursive KalmanNet outperforms conventional Kalman filters and deep learning-based estimators in non-Gaussian noise conditions.
GADGET framework decomposes global feature effects using recursive partitioning.
problem Misleading global feature effects when feature interactions are present.
method Generalized additive decomposition of global effects (GADGET) based on recursive partitioning.
result Minimizes interaction-related heterogeneity of local feature effects.
New framework for DNN training guarantees convergence to global minimum.
problem Training deep neural networks to converge to global minimum.
method Reformulated minimization problem with recursive algorithmic framework, using bounded style assumptions.
result Convergence to an ε-(global) minimum with O(1/ε^3) gradient computations.
Advertising and feed ranking are essential to many Internet companies such as Facebook and Sina Weibo. Among many real-world advertising and feed ranking systems, click through rate (CTR) prediction plays a central role. There are many proposed models in this field such as logistic regression, tree based models, factor…
CEFOL uses deep learning for dynamic programming with recursive utility.
problem Challenges in solving dynamic programming problems with recursive utility.
method Introduces a separate neural network for certainty equivalent, uses first-order optimality conditions to learn value and policy functions.
result CEFOL achieves high accuracy in learning value and policy functions, matching VFI benchmarks.
ThriftyNet uses a single convolutional layer recursively to maximize parameter usage.
problem Maximizing the use of parameters in deep convolutional neural networks.
method A single convolutional layer is used recursively, with normalization, non-linearities, downsampling, and shortcuts to maintain model expressivity.
result ThriftyNet achieves competitive performance with significantly fewer parameters.
GRANITE unifies feature-based explanation methods to reduce disagreement.
problem Disagreement among feature-based explanation methods.
method GRANITE partitions feature space into regions minimizing interaction and distribution influences.
result Unified and consistent feature explanations.
Filter banks are a popular tool for the analysis of piecewise smooth signals such as natural images. Motivated by the empirically observed properties of scale and detail coefficients of images in the wavelet domain, we propose a hierarchical deep generative model of piecewise smooth signals that is a recursion across s…
New recursive algorithm estimates conditional kernel mean embeddings in Hilbert space.
problem Estimating conditional distributions in RKHS for supervised learning.
method Recursive algorithm in L2 space for conditional kernel mean map. result Strong L2 consistency of recursive estimator proved. Hybrid model for online nonlinear prediction using LSTM and soft GBDT.
problem Online nonlinear prediction with manual feature selection and model selection issues.
method End-to-end architecture with LSTM for feature extraction and soft GBDT for regression, jointly optimized.
result Significant performance improvements over conventional methods on real datasets.
Deep-RLS uses deep learning to improve PCA for better source separation.
problem Improving PCA for better source separation in nonlinear systems.
method Inspired by RLS, Deep-RLS unfolds RLS iterations into a deep neural network.
result Deep-RLS significantly improves accuracy in recovering source signals.
NeSS combines neural and symbolic approaches for better compositional generalization.
problem Lack of compositional generalization in deep learning models.
method NeSS uses a neural network to generate traces, executed by a symbolic stack machine with sequence manipulation.
result Achieves 100% generalization performance across multiple domains.
Recommendation systems and computing advertisements have gradually entered the field of academic research from the field of commercial applications. Click-through rate prediction is one of the core research issues because the prediction accuracy affects the user experience and the revenue of merchants and platforms. Fe…
Automated service classification plays a crucial role in service discovery, selection, and composition. Machine learning has been widely used for service classification in recent years. However, the performance of conventional machine learning methods highly depends on the quality of manual feature engineering. In this…
xDeepInt learns both vector-wise and bit-wise feature interactions.
problem Learning feature interactions for CTR prediction and recommendation.
method Polynomial Interaction Network (PIN) architecture with subspace-crossing mechanism.
result xDeepInt outperforms state-of-the-art models in CTR prediction and recommendation.
This paper studies recursive ensembles driven by Fibonacci updates, improving learning dynamics.
problem Improving learning dynamics in recursive ensemble learning.
method Develops second-order recursive architectures with Fibonacci-type update flows.
result Establishes global convergence conditions and generalization bounds for recursive ensembles.
Complex problems may require sophisticated, non-linear learning methods such as kernel machines or deep neural networks to achieve state of the art prediction accuracies. However, high prediction accuracies are not the only objective to consider when solving problems using machine learning. Instead, particular scientif…
The paper examines the potential of deep learning to support decisions in financial risk management. We develop a deep learning model for predicting whether individual spread traders secure profits from future trades. This task embodies typical modeling challenges faced in risk and behavior forecasting. Conventional ma…
State-of-the-art machine learning frameworks support a wide variety of design features to enable a flexible machine learning programming interface and to ease the programmability burden on machine learning developers. Identifying and using a performance-optimal setting in feature-rich frameworks, however, involves a no…
Archipelago provides interpretable explanations of feature interactions in machine learning models.
problem Interpreting the impact of feature interactions on predictions in machine learning models.
method Archipelago is a novel framework for extracting and attributing feature interactions in a scalable and interpretable manner.
result Archipelago provides significantly more interpretable explanations of feature interactions than comparable methods.
This paper identifies duplicate questions on Quora using machine and deep learning models.
problem Detecting semantically identical questions on Quora to improve user experience.
method Applied machine learning and deep learning techniques on Quora's dataset.
result Xgboost model with character level term frequency and inverse term frequency achieved 85.82% accuracy.
Graph Neural Networks (GNNs) are a powerful tool for machine learning on graphs.GNNs combine node feature information with the graph structure by recursively passing neural messages along edges of the input graph. However, incorporating both graph structure and feature information leads to complex models, and explainin…
A new feature selection method using random forest and Kolmogorov filter.
problem Ultra-high dimensional data feature selection.
method Fused Kolmogorov filter with random forest based recursive feature elimination.
result Selection and L2 consistency under weak conditions.