VFDS selects dynamic features for efficient HAR tasks, optimizing performance-cost trade-offs.
problem Optimizing feature selection for varying costs and dynamic contexts in machine learning tasks.
method Bayesian learning framework with variational dynamic selection policy.
result VFDS selects different features under changing contexts, saving sensory costs while maintaining HAR accuracy.
A new method for dynamic feature selection outperforms existing approaches.
problem Sequentially selecting features based on current information in machine learning.
method Greedy selection of features based on conditional mutual information, combined with a learning approach for optimization.
result The method outperforms existing feature selection methods in experiments.
Feature normalization prevents collapse in non-contrastive learning dynamics.
problem Non-contrastive learning can collapse into a single point due to lack of repulsive force.
method Extended previous theory based on L2 loss to cosine loss, considering feature normalization.
result Cosine loss induces stable equilibrium, preventing collapse even with insufficient repulsive force.
Study on fluctuations in neural network kernels and predictions, focusing on finite width effects.
problem Characterizing fluctuations in finite width neural networks.
method Dynamical mean field theory analysis of wide but finite feature learning neural networks.
result Fluctuations in kernels and predictions are dynamically coupled, leading to reduced variance in feature learning regimes.
Random feature maps improve forecasting of chaotic dynamical systems.
problem Forecasting chaotic dynamical systems with high accuracy.
method Data-driven random feature maps with tanh activation, skip connections, and localization.
result Effective forecasting skill for dynamical systems with dimensions up to 512.
Realizations of stochastic process are often observed temporal data or functional data. There are growing interests in classification of dynamic or functional data. The basic feature of functional data is that the functional data have infinite dimensions and are highly correlated. An essential issue for classifying dyn…
SGD in DLNs reveals feature learning dynamics.
problem Understanding SGD dynamics in DLNs during saddle-to-saddle training.
method Stochastic Langevin dynamics with anisotropic, state-dependent noise; one-dimensional per-mode SDEs; Boltzmann distribution approximation.
result SGD noise encodes feature learning progression but does not alter saddle-to-saddle dynamics.
In various web applications like targeted advertising and recommender systems, the available categorical features (e.g., product type) are often of great importance but sparse. As a widely adopted solution, models based on Factorization Machines (FMs) are capable of modelling high-order interactions among features for …
Study examines infinite limits of transformer dynamics, identifying key parameterizations.
problem Understanding the training dynamics of transformer models in the feature learning regime.
method Analysis of infinite scaling limits using dynamical mean field theory.
result Identified parameterizations that admit well-defined infinite width and depth limits.
Dynamic acquisition of features improves predictions with limited data.
problem Limited or uncertain data requires additional relevant information for accurate assessments.
method Proposes models that dynamically acquire new features using conditional mutual information and arbitrary conditional flow.
result Demonstrates superior performance over baselines in multiple settings.
A RL framework selects features to balance bias and accuracy dynamically.
problem Bias in automated feature selection when predictors are correlated.
method Multi-component reward function with policy gradient for dynamic regularization and bias mitigation.
result Model balances fairness and accuracy during training.
A novel model uses ODE-based random features to model nonlinear dynamical systems.
problem Modeling highly nonlinear dynamical systems with uncertainty quantification.
method Compositions of physics-informed random features derived from ODEs, combined with deep Gaussian processes and approximate Bayesian inference.
result The model effectively captures nonlinear behavior in real-world multivariate time series data and achieves comparable performance to other models on benchmark tasks.
Two algorithms achieve optimal logarithmic regret in feature-based dynamic pricing.
problem Optimal pricing for products with features based on online learning.
method Developed EMLP and ONSP algorithms for stochastic and adversarial settings, proving O ( d log T ) O(d\log{T}) O ( d log T ) regret bounds. result Achieved optimal O ( d log T ) O(d\log{T}) O ( d log T ) regret, improving over existing bounds. Study reveals conditions for neural networks to forget learned features.
problem Understanding feature unlearning in neural networks.
method Infinite-width limit analysis with stochastic gradient descent, fast-slow dynamics.
result Conditions for feature unlearning are determined by the strength of nonlinear terms and initial weights.
AdaEnsemble learns adaptive feature interactions for CTR prediction.
problem Learning feature interactions for CTR prediction in recommender systems and Ads ranking.
method AdaEnsemble is a Sparsely-Gated Mixture-of-Experts (SparseMoE) architecture that dynamically selects feature interaction depth.
result AdaEnsemble achieves better prediction accuracy and inference efficiency compared to state-of-the-art models.
Gradient Starvation causes neural networks to focus on a subset of features, leading to imbalance.
problem Neural networks focusing on only a subset of features despite having access to more.
method Using Dynamical Systems theory, identified properties of learning dynamics leading to feature imbalance.
result Gradient descent can lead to feature imbalance in over-parameterized networks.
Parameterized state space models in the form of recurrent networks are often used in machine learning to learn from data streams exhibiting temporal dependencies. To break the black box nature of such models it is important to understand the dynamical features of the input driving time series that are formed in the sta…
Numerous social, medical, engineering and biological challenges can be framed as graph-based learning tasks. Here, we propose a new feature based approach to network classification. We show how dynamics on a network can be useful to reveal patterns about the organization of the components of the underlying graph where …
Paper introduces TSSDMN for modeling dynamic multilayer networks.
problem Capturing temporal and cross-layer dynamics in multilayer networks.
method Tensor State Space Model (TSSDMN) using symmetric Tucker decomposition.
result TSSDMN uniquely captures temporal dynamics within and across layers.
StrGNN detects anomalies in dynamic graphs by analyzing subgraphs and temporal features.
problem Detecting anomalies in dynamic graphs with structural changes.
method StrGNN is an end-to-end model that uses structural subgraphs and temporal features for anomaly detection.
result StrGNN effectively detects anomalies in dynamic graphs, as shown by extensive experiments.
Study models deep learning training dynamics using locally elastic SDEs to reveal feature separability.
problem Understanding how deep learning models separate features from different classes during training.
method Modeling deep learning training using locally elastic SDEs with a drift term reflecting backpropagation impact.
result Local elasticity in SDEs leads to linear separability of features, resulting in vanishing training loss.
Study on feature learning dynamics in infinite-depth neural networks, focusing on ResNets.
problem Understanding how features evolve during training in deep neural networks, especially in the large-depth limit.
method Conditional Gaussian representations and SDE system with decoupled backward weights.
result Depth-induced suppression of forward-backward coupling in infinite-depth networks, leading to a decoupled forward-backward SDE system.
Study shows how large neural networks avoid overfitting through decoupling of feature learning and complexity growth.
problem Understanding inductive bias and generalization in large neural networks.
method Dynamical mean field theory applied to large two-layer networks.
result Training dynamics of large networks exhibit a separation of timescales, decoupling feature learning and overfitting.
Scaling feature values is an important step in numerous machine learning tasks. Different features can have different value ranges and some form of a feature scaling is often required in order to learn an accurate classifier. However, feature scaling is conducted as a preprocessing task prior to learning. This is probl…
DIVA clusters dynamic data without needing cluster count, outperforming baselines.
problem Clustering complex, dynamic data without prior knowledge of cluster count.
method Nonparametric Dirichlet Process Mixtures with memoized online variational inference.
result DIVA outperforms state-of-the-art in classifying complex data with changing features.
Ensembles dynamic models using random feature approximations.
problem Online scalable Bayesian learning with dynamic models and ensembling.
method Random feature approximations and dynamic models using random walks.
result Better performance with alternative basis expansions like Hilbert space Gaussian processes.
We introduce a framework for dynamic adversarial discovery of information (DADI), motivated by a scenario where information (a feature set) is used by third parties with unknown objectives. We train a reinforcement learning agent to sequentially acquire a subset of the information while balancing accuracy and fairness …
Dynamic time warping (DTW) can be used to compute the similarity between two sequences of generally differing length. We propose a modification to DTW that performs individual and independent pairwise alignment of feature trajectories. The modified technique, termed feature trajectory dynamic time warping (FTDTW), is a…
Randomized feature models learn interaction kernels from agent paths.
problem Learning interaction kernels from noisy agent paths.
method Randomized feature algorithm and sparse regression.
result Pruned features reduce overfitting and lower simulation cost.
Transformers handle infinite dimensional inputs effectively by feature extraction and dynamic feature selection.
problem Understanding the approximation and estimation ability of Transformers with infinite dimensional inputs.
method Anisotropic smoothness analysis and feature extraction properties of Transformers.
result Transformers avoid the curse of dimensionality and dynamically select important features.
Method learns dynamics from sparse, irregular feature data.
problem Learn system dynamics from sparse, irregularly sampled feature time series.
method Formulates as high-dimensional linear regression using signatures.
result Oracle bound on prediction error with explicit sampling dependencies.
Enhances machine learning for dynamic, interconnected entities.
problem Lack of systematic feature engineering for dynamic, interconnected entities.
method Augments current graph machine learning with comprehensive feature engineering in space and time.
result Improves supervised learning on heterogeneous, attributed entities interacting over time.
Steerable neural ODEs on homogeneous spaces for equivariant feature dynamics.
problem Learning continuous-time equivariant dynamics of vector-valued features on homogeneous spaces.
method Introduces steerable neural ordinary differential equations on homogeneous spaces, interpreting features as sections of associated vector bundles over M M M . result Steerable NODEs are G G G -equivariant when the flow and connection are G G G -invariant, and they incorporate existing models. DVE models dynamic changes in feature embeddings for better sequence-aware applications.
problem Dynamic characterization of feature variations in time-varying data.
method Dynamic Variational Embedding (DVE) using recurrent neural networks.
result DVE models intrinsic nature and temporal variation of nodes effectively.
Study agnostic feature-based dynamic pricing models with linear policies and noisy valuations.
problem Tackles dynamic pricing with unknown noise and no assumptions on data.
method Studies two agnostic models: linear policy and linear noisy valuation, presenting algorithms and regret bounds.
result Demonstrates no-regret learning is possible under weak assumptions, but noisy feedback is not significantly more useful than bandit feedback.
Supporting evidence for adaptive feature program across diverse models.
problem Analyzing feature learning in neural networks.
method Over-parameterized sequence models and feature error measure (FEM).
result FEM is decreasing during training of adaptive feature models.
AFS-BM improves model accuracy by dynamically selecting features.
problem Feature selection challenges in ML, especially scalability and adaptability.
method Joint optimization for feature selection and model training with binary masking.
result AFS-BM achieves significant improvements in model accuracy and computational efficiency.
Model precision in a classification task is highly dependent on the feature space that is used to train the model. Moreover, whether the features are sequential or static will dictate which classification method can be applied as most of the machine learning algorithms are designed to deal with either one or another ty…
Proposes a method to update model weights by dynamically changing features during training.
problem Updating model weights efficiently during training.
method Dynamic feature replacement during training to update model weights.
result Increased model accuracy with same training time and complexity.
Paper tackles dynamic assortment with dual contexts, improving revenue in e-commerce.
problem Maximizing revenue in e-commerce with personalized recommendations from vast catalogs.
method Low-rank dynamic assortment model and upper confidence bound approach.
result Regret bound of i l d e O ( ( d 1 + d 2 ) r T ) ilde{O}((d_1+d_2)r\sqrt{T}) i l d e O (( d 1 + d 2 ) r T ) for dynamic assortment problem. Study on test risk dynamics in learning theory with stochastic gradient flow.
problem Understanding test risk in stochastic gradient flow dynamics.
method Path integral formulation for small learning rates, explicit computation for weak features.
result Explicit corrections due to stochastic term in dynamics, good agreement with simulations.
Large SGD step sizes lead to sparse feature learning in neural networks.
problem Sparse feature learning in neural networks with large step sizes.
method Empirical observations and theoretical analysis of SGD dynamics.
result Large step sizes induce implicit regularization leading to sparse predictors.
SGD quickly learns a spurious XOR feature before the signal feature, revealing learning dynamics.
problem Over-reliance on spurious correlations in neural networks trained by SGD.
method Theoretical analysis of SGD on two-layer ReLU networks trained on XOR data.
result SGD learns the spurious feature first and exponentially fast, dominating the signal feature.
Data augmentation affects feature importance, enhancing learning for neural networks.
problem Understanding the effect of data augmentation on feature importance and learning dynamics.
method Analyzing a two-layer convolutional neural network in a multi-view data model.
result Data augmentation alters feature importance, making certain features more likely to be learned.
Empirical data on real complex systems are becoming increasingly available. Parallel to this is the need for new methods of reconstructing (inferring) the topology of networks from time-resolved observations of their node-dynamics. The methods based on physical insights often rely on strong assumptions about the proper…
Diffusion models learn balanced data representations, unlike classification models.
problem Understanding feature learning in diffusion models.
method Proposed a feature learning framework to analyze diffusion models' training dynamics.
result Diffusion models encourage learning balanced and comprehensive representations.
A new method extracts features and reconstructs moments in dynamical systems using information geometry.
problem Reconstructing moments in dynamical systems efficiently and accurately.
method Information-geometric approach on spaces of probability measures.
result Moments can be expanded in eigenfunctions of a kernel integral operator, enabling nonparametric forecasting.
A new framework for generating predictive features in noisy multivariate time series.
problem Predicting noisy multivariate time series with limited user effort.
method Develops a feature programming framework based on spin-gas dynamical Ising models.
result Validated the method on synthetic and real-world datasets.