Complex-valued neural networks avoid spurious local minima.
problem Finding spurious local minima in neural networks.
method Proved no spurious local minima for shallow complex neural networks with quadratic activations.
result Complex-valued weights eliminate spurious local minima in neural networks.
The paper proposes methods to find a shared active subspace for multivariate vector-valued functions.
problem Minimizing the deviation between function evaluations in the original and reconstructed spaces.
method Manipulating gradients or SPD matrices to identify a shared structure.
result Summing SPD matrices often identifies the best shared active subspace.
Neural networks learn distance-based representations, not just intensity.
problem Understanding how neural networks interpret and learn from internal activations.
method Manipulated ReLU and Absolute Value activations to observe sensitivity to distance and intensity perturbations.
result Neural networks are highly sensitive to small distance-based perturbations, challenging the intensity-based interpretation.
A new Shapley value approach for neural networks interpretable and stable.
problem Neural networks' interpretability and training stability issues.
method Shapley value approximation for ReLU activation, globally continuous Shapley gradient, Shapley Activation function.
result SA consistently outperforms ReLU in training convergence, accuracy, and stability.
Complex-valued neural networks can approximate any continuous function with bounded widths and depths.
problem Approximating continuous functions with complex-valued neural networks of bounded widths and depths.
method Analyzing activation functions and proving universality for complex-valued networks.
result Deep narrow complex-valued networks are universal if and only if their activation function is neither holomorphic, nor antiholomorphic, nor R-affine. S2D selectively decays large singular values to improve quantization of neural activations.
problem Large activation outliers in transformer models cause accuracy drops during quantization.
method Selective Spectral Decay (S2D) that surgically regularizes only the largest singular values. result Significantly reduces activation outliers and produces well-conditioned representations.
Proposes LAMA for principled SVDD hyperparameter estimation.
problem Estimating SVDD hyperparameters is difficult and heuristic methods are unreliable.
method LAMA (Local Active Min-Max Alignment) using active learning and kernel alignment.
result LAMA provides evidence-based estimates for SVDD hyperparameters with quality scores.
Deep gated networks help understand training and generalization in deep learning.
problem Understanding the role of SGD in training and generalization of deep neural networks with ReLU activation.
method Developed deep gated networks (DGNs) as a framework to analyze training and generalization in DNNs with ReLU activation.
result Gate adaptation is key for generalization in deep neural networks.
ADS filters data points for efficient batch active learning.
problem Efficiently selecting data points for annotation in parallel settings.
method Active Data Shapley (ADS) using the Shapley value of data.
result Significantly increases efficiency of active learning by 6x.
CVNNs improve performance in tasks with complex-valued inputs.
problem Improving performance in tasks with complex-valued inputs.
method Analyze the approximation properties of complex-valued neural networks (CVNNs).
result Quantitative approximation bounds for CVNNs, showing error scales as m−k/(2n). Equity activity is an essential topic for financial market studies. To explore its statistical regularities, we comprehensively examine the trading value, a measure of the equity activity, of the 3314 most-traded stocks in the U.S. equity market and find that (i) the trading values follow a log-normal distribution; (ii…
An activation boundary for a neuron refers to a separating hyperplane that determines whether the neuron is activated or deactivated. It has been long considered in neural networks that the activations of neurons, rather than their exact output values, play the most important role in forming classification friendly par…
ARL uses queries to learn rewards, focusing on cost vs. reward value.
problem How to efficiently use queries to learn rewards in reinforcement learning.
method Proposed and evaluated heuristic approaches for ARL in multi-armed bandits and MDPs.
result Challenging aspects of ARL highlighted, including intractability of value computation.
Activation functions are essential for deep learning methods to learn and perform complex tasks such as image classification. Rectified Linear Unit (ReLU) has been widely used and become the default activation function across the deep learning community since 2012. Although ReLU has been popular, however, the hard zero…
This paper investigates the scaling dependencies between measures of "activity" and of "size" for companies included in the FTSE 100. The "size" of companies is measured by the total market capitalization. The "activity" is measured with several quantities related to trades (transaction value per trade, transaction val…
Universal approximation for ODENet and ResNet with a single activation function.
problem Approximating complex dynamical systems with limited vector fields.
method Examined ODENet and ResNet with vector fields composed of a single activation function and affine mapping.
result ODENet and ResNet with restricted vector fields can uniformly approximate those with general vector fields.
We address challenges of active learning under scarce informational resources in non-stationary environments. In real-world settings, data labeled and integrated into a predictive model may become invalid over time. However, the data can become informative again with switches in context and such changes may indicate un…
Paper develops a risk scoring framework for tokenized RWA markets.
problem Tokenized assets may not reflect true risk due to illiquidity and concentration.
method Develops a risk scoring framework based on observable indicators.
result Assets with limited transfer activity and concentrated ownership have high empirical risk.
Agent uses message passing to optimize robot navigation, balancing exploration and exploitation.
problem Optimizing robot navigation in continuous-valued spaces with uncertainty.
method Autoregressive active inference agent using message passing on a factor graph.
result Agent modulates action based on predictive uncertainty, leading to better model of dynamics.
A new PLL method uses class activation values to improve robustness.
problem Weakly supervised learning with noisy data and adversarial perturbations.
method Subjective logic with class activation values for uncertainty representation and label weight re-distribution.
result More robust predictions under high noise levels, out-of-distribution examples, and adversarial perturbations.
Many activation functions have been proposed in the past, but selecting an adequate one requires trial and error. We propose a new methodology of designing activation functions within a neural network at each layer. We call this technique an "activation ensemble" because it allows the use of multiple activation functio…
Active learning selects optimal measurement times for inferring continuous paths from sparse data.
problem Inferring continuous probability paths from sparse snapshots in high-fidelity domains like single-cell biology.
method Extends active experimentation to the space of measures using Linearized Optimal Transport (LOT) for probabilistic surrogate modeling.
result Empirical results show that the proposed strategy outperforms uncertainty-agnostic baselines.
The paper addresses sampling bias in risk-based active learning.
problem Sampling bias in active learning leads to poor decision-making performance.
method The paper uses a semi-supervised Gaussian mixture model with an EM algorithm to counteract sampling bias.
result The EM algorithm effectively incorporates pseudo-labels for unlabelled data, reducing sampling bias.
Mitigates faults in DNNs by clipping activation values, improving their resilience.
problem Fault tolerance of DNNs in safety-critical applications.
method Clipping activation functions to reduce impact of faulty weights.
result Significant improvement in classification accuracy (68.92%) for fault mitigation.
Improved accuracy with fewer labels using MixMatch and active learning.
problem Achieving high accuracy with limited labeled data.
method Combining MixMatch and active learning techniques.
result Significant improvement in accuracy on CIFAR-10, CIFAR-100, and SVHN datasets.
Extends active subspace analysis to infinite dimensions.
problem Dimension reduction in infinite dimensional functionals.
method Defines an operator for Hilbert space, extends Euclidean properties, proposes Monte Carlo procedure.
result Desirable properties extend to infinite dimensional setting.
Complex-valued neural networks can approximate any continuous function.
problem Generalizing the universal approximation theorem to complex-valued networks.
method Characterizing activation functions for complex networks to approximate any continuous function.
result Different activation functions are required for deep vs shallow complex networks to achieve universal approximation.
The paper proposes a method to identify latent factors from sampled and fired graph data.
problem Identifying latent factors from sampled and fired graph data.
method The paper presents a theoretical and practical approach to build an identifier of latent factor activations.
result The method successfully identifies latent factor activations from sampled and fired graph data.
We introduce the active exploration problem in Markov decision processes (MDPs). Each state of the MDP is characterized by a random value and the learner should gather samples to estimate the mean value of each state as accurately as possible. Similarly to active exploration in multi-armed bandit (MAB), states may have…
Unified framework for hybrid learning and optimization via active inference.
problem Sequential decisions in black-box evaluations requiring both task improvement and uncertainty reduction.
method Pragmatic Curiosity (PraC) framework that evaluates queries by balancing information gain and pragmatic value.
result Unified approach reduces decision risk and improves coverage of critical regions without task-specific rules.
Feature missing is a serious problem in many applications, which may lead to low quality of training data and further significantly degrade the learning performance. While feature acquisition usually involves special devices or complex process, it is expensive to acquire all feature values for the whole dataset. On the…
Extraction of missing attribute values is to find values describing an attribute of interest from a free text input. Most past related work on extraction of missing attribute values work with a closed world assumption with the possible set of values known beforehand, or use dictionaries of values and hand-crafted featu…
Bayesian active learning method improved for censored regression data.
problem Challenges in estimating BALD for censored regression data.
method Derived entropy and mutual information for censored distributions, developed C-BALD objective, proposed novel modelling approach. result Demonstrated C-BALD outperforms other methods in censored regression. AIF reformulated as convex MDP for adaptive behavior.
problem Adaptive behavior and policy optimization.
method Formulating AIF as convex MDP, deriving mirror descent algorithm.
result EFE minimization in AIF is equivalent to reward maximization in latent MDP, with epistemic component.
We adapt Shapley values to explain model uncertainty, connecting it to information theory.
problem Explaining uncertainty in model predictions.
method Adapted Shapley value framework to quantify feature contributions to predictive uncertainty.
result Deep connections between Shapley values and information theory quantities.
Wi-Fringe recognizes named gestures using WiFi CSI without training data.
problem Lack of training data for WiFi-based gesture recognition.
method WiFi CSI values paired with activity names for recognition.
result Wi-Fringe detects activities with or without training data.
New comparison shows differences in how value is incorporated in AIF and CAI.
problem Clarifying the relationship between Active Inference and Control-as-Inference.
method Formal comparison of AIF and CAI frameworks.
result Primary difference is how value is incorporated into generative models.
LSQ+ improves quantization of neural nets with Swish activations, achieving state-of-the-art results.
problem Quantization of neural nets with Swish activations, especially negative activations, leads to significant performance loss.
method Introduces learnable scale and offset parameters for asymmetric quantization, and uses MSE-based initialization for quantization parameters.
result Significantly outperforms LSQ for low-bit quantization of neural nets with Swish activations, achieving up to 5.6% gain with W2A2 quantization of EfficientNet-B0.
A new MARL framework for community-based cooperation with transfer and active exploration.
problem Flexible coordination patterns in multi-agent systems with community structures.
method Community-based multi-agent reinforcement learning with transfer and active exploration.
result Provably convergent actor-critic algorithms for structured information sharing and transfer learning.
Active learning improves GP regression on complex, high-dimensional data.
problem Improving Gaussian Process regression in high-dimensional spaces with discontinuous functions.
method Combines manifold learning with active learning to optimize data selection and reduce dimensionality.
result Superior performance over random learning in synthetic data experiments.
We define an activity dependent branching ratio that allows comparison of different time series Xt. The branching ratio bx is defined as bx=E[ξx/x]. The random variable ξx is the value of the next signal given that the previous one is equal to x, so ξx={Xt+1∣Xt=x}. If bx>1, the process is …
A new indicator measures project risk from activity durations.
problem Managing project risks throughout the lifecycle.
method Activity Risk Index (ARI) based on Schedule Risk Baseline.
result Identifies activities contributing most to project uncertainty.
A new KAN variant uses sinusoidal activations to approximate functions.
problem Approximating multivariable functions using neural networks.
method Replacing inner and outer functions in Kolmogorov-Arnold representation with weighted sinusoidal functions.
result The new KAN variant outperforms fixed-frequency Fourier transform and achieves comparable performance to MLPs.
Paper proposes a modified uncertainty sampling method to speed up preference learning from noisy humans.
problem Learning preferences from humans with limited queries and noisy responses.
method Modified uncertainty sampling using expected output value to speed up preference learning.
result The modified method outperforms the baseline uncertainty sampling in preference learning.
Active subspaces on Riemannian manifolds generalize Euclidean principles.
problem Understanding how scalar-valued quantities change over Riemannian manifolds.
method Generalization of active subspaces from Euclidean to Riemannian spaces using parallel transport.
result The method provides a new way to study scalar-valued quantities on manifolds, differing from extrinsic approaches.
New analysis shows capacity of treelike neural networks with various activations.
problem Analyzing the capacity of treelike neural networks with diverse activations.
method Utilized Random Duality Theory and its partially lifted version to handle various activations.
result The capacity of treelike neural networks decreases for large network width but converges to a constant value.
We introduce the stochastic multiplicative point process modelling trading activity of financial markets. Such a model system exhibits power-law spectral density S(f) ~ 1/f**beta, scaled as power of frequency for various values of beta between 0.5 and 2. Furthermore, we analyze the relation between the power-law autoco…
Interesting theoretical associations have been established by recent papers between the fields of active learning and stochastic convex optimization due to the common role of feedback in sequential querying mechanisms. In this paper, we continue this thread in two parts by exploiting these relations for the first time …