QSAR models struggle to predict activity cliffs, but graph isomorphism features improve AC-sensitivity.
problem QSAR models struggle to predict activity cliffs (ACs).
method Nine distinct QSAR models combining molecular representation methods and regression techniques.
result Graph isomorphism features improve AC-sensitivity.
Study compares GNNs and classical molecular featurisations for molecular property and cliff prediction.
problem Comparing GNNs and classical featurisations for molecular property and cliff prediction.
method Systematic exploration and comparison of PDVs, ECFPs, and GNNs; introduction of substructure pooling.
result Sort & Slice outperforms hash-based folding in ECFP vectorization.
Study shows cliff-learning in transfer learning from foundation models.
problem Data-scaling of transfer learning from foundation models in low data regimes.
method Investigation of cliff-learning phenomenon through foundation-model analysis and toy models.
result Cliff-learning reflects compatibility between priors and tasks.
A new method for unsupervised disentanglement using axis-aligned cliffs.
problem Unsupervised disentanglement of latent factors under nonlinear maps.
method Encouraging axis-aligned discontinuities (cliffs) in the estimated density of factors.
result Cliff method outperforms baselines on disentanglement benchmarks.
Rank-based Bayesian Optimization improves molecule selection in chemical systems.
problem Optimizing chemical compounds using traditional regression models.
method Introducing Rank-based Bayesian Optimization (RBO) using ranking models.
result RBO outperforms regression-based BO, especially for rough landscapes and activity cliffs.
We established existence of periodic Reeb orbits for a large class of tight contact structures on closed 3-manifolds, notably the Stein fillable structures, based on a fundamental theorem of Cliff Taubes on symplectic 4-manifolds.
We show the existence of a modified Cliff(1,1) structure compatible with an Osserman 0-model of signature (2,2). We then apply this algebraic result to certain classes of pseudo-Riemannian manifolds of signature (2,2). We obtain a new characterization of the Weyl curvature tensor of an (anti-)self-dual manifold and we …
This paper improves robot traders' market impact sensitivity.
problem Market impact in automated trading systems.
method Critiqued existing methods, introduced MLOFI, and demonstrated new algorithms.
result New imbalance-sensitive trader-agents exhibit market impact effects.
Framework mitigates overfitting in quantitative trading strategies.
problem Overfitting during strategy transition from backtest to live trading.
method Three-stage protocol: IS, WFA, OOS; majority pass, purge gaps, cliff veto, etc.
result Demonstrates how to detect overfitting through performance decay and drawdown behavior.
We present Tanaka's prolongation procedure for filtered structures on manifolds discovered in [Tanaka N., J. Math. Kyoto. Univ. 10 (1970), 1-82] in a spirit of Singer-Sternberg's description of the prolongation of usual G-structures [Singer I.M., Sternberg S., J. Analyse Math. 15 (1965), 1-114; Sternberg S., Prentice-H…
Framework quantifies financial NLP robustness under regime shifts.
problem Semantic and causal drift in financial news narratives.
method Four metrics: FCAS, PCS, TSV, NLICS.
result Transformer models are more affected by semantic drift.
We propose a joint source and channel coding (JSCC) technique for wireless image transmission that does not rely on explicit codes for either compression or error correction; instead, it directly maps the image pixel values to the complex-valued channel input symbols. We parameterize the encoder and decoder functions b…
Learning to optimize - the idea that we can learn from data algorithms that optimize a numerical criterion - has recently been at the heart of a growing number of research efforts. One of the most challenging issues within this approach is to learn a policy that is able to optimize over classes of functions that are fa…
New method identifies critical states to improve RL agent explainability and speed.
problem Challenges in RL agent explainability and action selection timing.
method Identify critical states based on action-based variance in Q-function, prioritize exploitation on these states.
result Identified critical states accelerate RL in grid worlds and deep RL tasks.
Light pillars over rippled water appear parallel due to projection geometry.
problem Light pillars over rippled water appear parallel.
method Developing a geometric optics model
result The phenomenon is explained using the specular reflection rule, projection geometry, and physics of surface slopes.
New framework tackles deep financial reporting bottleneck by improving hallucination and coherence.
problem Statistical smoothing trap in LLMs limits deep financial reporting quality.
method DeepNews Framework integrates information foraging, schema-guided planning, and adversarial prompting.
result DeepNews system achieves 25% acceptance rate in blind test, significantly outperforming SOTA.
Many activation functions have been proposed in the past, but selecting an adequate one requires trial and error. We propose a new methodology of designing activation functions within a neural network at each layer. We call this technique an "activation ensemble" because it allows the use of multiple activation functio…
This paper studies activation sparsity in large language models, finding key trends and implications.
problem Activation sparsity in large language models (LLMs) can be improved for efficiency and interpretability.
method Proposes PPL-p% sparsity, analyzes trends with training data, width-depth ratio, and parameter scale. result ReLU is more efficient for sparsity than SiLU, and deeper architectures can improve sparsity.
Activity recognition from sensor data deals with various challenges, such as overlapping activities, activity labeling, and activity detection. Although each challenge in the field of recognition has great importance, the most important one refers to online activity recognition. The present study tries to use online hi…
For more than a decade Vytelingum's Adaptive-Aggressive (AA) algorithm has been recognized as the best-performing automated auction-market trading-agent strategy currently known in the AI/Agents literature; in this paper, we demonstrate that it is in fact routinely outperformed by another algorithm when exhaustively te…
Many neural network architectures rely on the choice of the activation function for each hidden layer. Given the activation function, the neural network is trained over the bias and the weight parameters. The bias catches the center of the activation, and the weights capture the scale. Here we propose to train the netw…
BinaryDuo improves BNNs by coupling binary activations, outperforming state-of-the-art models.
problem Gradient mismatch in BNNs due to binarizing activations.
method Using gradient of smoothed loss function to estimate gradient mismatch, proposing BinaryDuo scheme with coupled ternary activations.
result BinaryDuo outperforms state-of-the-art BNNs on various benchmarks.
Evolutionary algorithms improve neural network performance by discovering better activation functions.
problem The choice of activation function affects neural network performance, but ReLU remains dominant.
method Defined a tree-based search space of candidate activation functions and used evolutionary algorithms (mutation, crossover, exhaustive search) to explore and discover better functions.
result Replacing ReLU with evolved activation functions statistically significantly increases network accuracy.
Active learning method balances bias and variance under class imbalance.
problem Active learning under label shift when class proportions differ.
method Mediated Active Learning under Label Shift (MALLS) using a 'medial distribution'.
result MALLS reduces asymptotic sample complexity under arbitrary label shift.
New method learns neural network activation functions from data.
problem Learning activation functions for neural networks.
method Model each neuron's activation function as a small neural network.
result Learned activation functions improve network performance.
Study active learning of PTFs with derivative access.
problem Active learning of polynomial threshold functions (PTFs).
method Algorithm for active learning degree-d univariate PTFs with derivative access. result Computational efficient algorithm for active learning degree-d univariate PTFs. Collaborative filtering is a useful technique for exploiting the preference patterns of a group of users to predict the utility of items for the active user. In general, the performance of collaborative filtering depends on the number of rated examples given by the active user. The more the number of rated examples giv…
Bayesian adaptive designs can be biased by active learning, especially with misspecified models.
problem Active learning bias in Bayesian adaptive experimental designs.
method Analysis of linear and preference learning models, empirical testing.
result Model misspecification and noise influence active learning bias in Bayesian designs.
Derives time-averaged active inference from control principles.
problem Finite-horizon or discounted-surprise problems in active inference.
method Derives infinite-horizon, average-surprise active inference from optimal control principles.
result Unified objective functional for sensorimotor control.
RAN model recognizes multiple activities from unlabeled sensor data.
problem Handling weakly labeled multi-activity data from wearable sensors.
method Recurrent Attention Networks (RAN) for sequential multi-activity recognition and localization.
result RAN model can infer multiple activities and determine activity locations from unlabeled data.
A new indicator measures project risk from activity durations.
problem Managing project risks throughout the lifecycle.
method Activity Risk Index (ARI) based on Schedule Risk Baseline.
result Identifies activities contributing most to project uncertainty.
Hidden Markov Models detect hand gestures from wearable sEMG signals.
problem Detecting activity regions in continuous sEMG signals.
method Hidden Markov Models applied to sEMG signals for gesture recognition.
result Average accuracy of 96.25% for activity onsets and 87.5% for activity terminations.
Active testing reduces label costs for efficient model evaluation.
problem Real-world applications require expensive test labels, disconnecting from existing model evaluation methods.
method Derives acquisition strategies to select test points efficiently, addressing label bias and variance.
result Active testing improves model evaluation efficiency without sacrificing accuracy.
Survey of trainable activation functions in neural networks.
problem Improving neural network performance through trainable activation functions.
method Taxonomy and comparison of recent and past models of trainable activation functions.
result Many trainable activation functions are equivalent to adding neuron layers with fixed activation functions and simple constraints.
This paper analyzes and improves active learning techniques for real-world projects.
problem Reducing labelling effort in machine learning models with real-world constraints.
method Systematic study of active learning issues, proposing techniques to address model convergence, annotation error, and dataset imbalance.
result Presentation of two techniques to speed up active learning: partial uncertainty sampling and larger query size.
Study active nematic forces on curved surfaces, revealing new coupling mechanisms.
problem Understanding active nematic forces on curved surfaces.
method Developed a thermodynamically consistent surface model with nematic activity, analyzed topological defects.
result Active defects contribute both tangential and normal forces on curved surfaces.
This paper provides an overview of activation functions in neural networks.
problem Confusion in activation function selection and properties in deep learning.
method Analytic review of popular activation functions.
result Clarification of activation function properties and selection.
Bayesian active learning method improved for censored regression data.
problem Challenges in estimating BALD for censored regression data.
method Derived entropy and mutual information for censored distributions, developed C-BALD objective, proposed novel modelling approach. result Demonstrated C-BALD outperforms other methods in censored regression. DUNs improve active learning by dynamically adjusting model complexity.
problem Active learning datasets grow, leading to model bias or overfitting.
method Depth Uncertainty Networks (DUNs) infer model depth for flexibility and complexity.
result DUNs outperform other BNN variants on active learning tasks, reducing overfitting.
Overfitting frequently occurs in deep learning. In this paper, we propose a novel regularization method called Drop-Activation to reduce overfitting and improve generalization. The key idea is to drop nonlinear activation functions by setting them to be identity functions randomly during training time. During testing, …
Large deviation principle for deep neural networks with ReLU activation.
problem Understanding the behavior of deep neural networks with ReLU activation.
method Proving a large deviation principle for networks with Gaussian weights and ReLU activation functions.
result Simplified expressions and power-series expansions for the ReLU case.
TransFall uses transfer learning to improve activity recognition from mobile sensors.
problem Performance degradation due to platform and user movement differences.
method Two-tier data transformation, label estimation, and model generation layers.
result TransFall enhances activity recognition accuracy for new scenarios.
EmbraceNet fusion model for multi-sensor activity recognition.
problem Human activity recognition using multiple sensors is challenging.
method Independent processing of each sensor, feature fusion with EmbraceNet, post-processing, and additional processes.
result Improved performance in SHL recognition challenge.
New proof shows neural networks can memorize training data with high accuracy.
problem Proving neural networks can memorize training data with high accuracy.
method Probabilistic construction exploiting sparsity.
result Proves neural networks with threshold or mixed threshold-ReLU activations can memorize training data.
We consider active learning of deep neural networks. Most active learning works in this context have focused on studying effective querying mechanisms and assumed that an appropriate network architecture is a priori known for the problem at hand. We challenge this assumption and propose a novel active strategy whereby …
Data-aware activation function customization reduces neural network error.
problem Current neural networks lack consideration for specific activation functions.
method Linear algebraic explanation and Diaconis-Shahshahani Approximation Theorem criteria for activation functions.
result Using an even activation function like seagull can reduce neural network error by orders of magnitude.
In the last few years there has been a growing interest in Human Activity Recognition~(HAR) topic. Sensor-based HAR approaches, in particular, has been gaining more popularity owing to their privacy preserving nature. Furthermore, due to the widespread accessibility of the internet, a broad range of streaming-based app…
Active learning selects inputs for GPSSM to learn latent states.
problem Optimally learn latent states of a GPSSM through active selection of inputs.
method Use mutual information to select informative inputs; approximate mutual information for GPSSM.
result Effective active learning of GPSSM dynamics in physical systems.