Proposes bipolar activation functions to shift layer mean activations towards zero.
problem Training deep neural networks with high layer mean activations.
method Extends ReLU-family activation functions to shift mean activations towards zero.
result Improves language modeling and classification tasks with competitive results.
Proposes PKM for soft K-means clustering.
problem Soft K-means (m=1) unsolved since 1981.
method Probabilistic K-Means (PKM) via nonlinear programming.
result Proposed methods solve PKM efficiently.
Randomly initialized wide neural networks with zero-mean activations are nearly independent, potentially solving AI interpretability limits.
problem Measuring the limits of AI interpretability.
method Randomly initialized neural networks with large width and zero-mean activation functions.
result Neural networks with zero-mean activations are nearly independent, solving the computational no-coincidence conjecture.
New method for optimizing neural networks with quantized weights and activations.
problem Improving resource efficiency of deep neural networks.
method Mean-field theory applied to quantized activation networks.
result Closed-form equation for maximal trainable depth, showing Lmax∝N1.82. Mean representations of VAEs are correlated but still useful for tasks.
problem Correlation between mean and sampled representations of VAEs.
method Selective posterior collapse to identify active and passive variables.
result Passive variables in mean representations are correlated but uncorrelated in sampled ones.
Wide BNNs with odd activations fail to approximate data under mean-field inference.
problem Theoretical limitations of mean-field variational inference in wide, deep Bayesian neural networks.
method Analysis of mean-field variational inference in fully-connected BNNs with odd activation functions and Gaussian likelihood.
result The optimal mean-field variational posterior predictive distribution converges to the prior predictive distribution as network width increases.
LCW reduces activation shift in neural networks, improving training efficiency and generalization.
problem Activation shift in neural networks leading to non-zero mean preactivation values.
method Linearly constrained weights (LCW) to reduce activation shift in fully connected and convolutional layers.
result LCW resolves the vanishing gradient problem and improves generalization of neural networks.
New method selects diverse mini-batches for active learning.
problem Reduce labeled data for deep learning models.
method Sequential selection of diverse mini-batches using K-means clustering.
result Achieves comparable or better performance than previous methods.
The paper tackles matching a desired mean in causal systems through shift interventions.
problem Matching a desired mean in causal systems.
method Defining Markov equivalence classes, proposing active learning strategies, deriving lower bounds.
result Proposed active learning strategies require fewer interventions than previous approaches, especially for certain graph classes.
Efficiently selects nearest neighbors for labeling to speed up active learning.
problem Intractable active learning and search for large-scale unlabeled data.
method Restricts candidate pool to nearest neighbors of labeled set.
result Achieved similar performance to global approach but reduced computational cost by up to 3 orders of magnitude.
A new algorithm tackles active exploration in noisy MDPs.
problem Accurately estimating state mean values in MDPs with varying noise levels.
method Introduces a novel learning algorithm to balance exploration and exploitation.
result Active exploration in MDPs can be more challenging than in MAB.
New collaborative algorithm improves personalized mean estimation in online settings.
problem Online estimation of means from multiple, possibly overlapping distributions.
method Novel collaborative strategy for active querying and mean estimation.
result Algorithm improves mean estimates through communication among agents.
Study on function sensitivity in random DNNs using large deviation theory.
problem Understanding function sensitivity in finite-size deep neural networks.
method Large deviation theory and path integral analysis applied to random DNNs with ReLU and sign activations.
result Random DNNs with ReLU activations are more robust to parameter perturbations.
Deep nets exhibit 'Neural Collapse' during training's final phase, simplifying decision-making.
problem Understanding and optimizing deep learning training phases.
method Direct measurements on three deepnet architectures across seven datasets.
result Deep nets exhibit 'Neural Collapse' during training's final phase, simplifying decision-making.
We propose the point process model as the Poissonian-like stochastic sequence with slowly diffusing mean rate and adjust the parameters of the model to the empirical data of trading activity for 26 stocks traded on NYSE. The proposed scaled stochastic differential equation provides the universal description of the trad…
Abstract: Investigates the role of activation functions in neural networks and their physical basis.
problem Understanding the role of activation functions in neural networks and their physical basis.
method Formalizes the use of activation functions in neural inference by relating them to phase transitions in statistical physics.
result Reveals the physical justification for the performance of typical activation functions in neural networks.
The study connects deep neural networks with statistical mechanics, revealing natural activation functions.
problem Understanding the activation functions in deep neural networks.
method Statistical Mechanics model of deep neural networks, focusing on encoding, validation, and propagation steps.
result A set of natural activations including Sigmoid, tanh, ReLU, and Swish are identified.
New insights on active sequential prediction for mean estimation.
problem Active sequential prediction-powered mean estimation problem.
method Combining uncertainty-based suggestion with a constant probability, analyzing non-asymptotic bounds, and using no-regret learning.
result The optimal query probability is close to the constraint when using no-regret learning.
Study shows sample noise impacts active learning performance.
problem Impact of sample noise on active learning performance.
method Proposed Incremental Weighted K-Means for noisy samples.
result Robust sampler improves synthetic tasks but only marginally in real-life.
In this paper we present a new method for motion tracking of tumors in liver ultrasound image sequences. Our algorithm has two main steps. In the first step, we apply mean shift algorithm with multiple features to estimate the center of the target in each frame. Target in the first frame is defined using an ellipse. Ed…
Gradient descent converges to minimum Bayes risk for two-layer ReLU networks in mean field regime.
problem Training two-layer ReLU networks using gradient descent in the mean field regime.
method Describes a condition for convergence to minimum Bayes risk, extending previous results to ReLU-activated networks.
result The condition for convergence does not depend on initialization and concerns weak convergence of network realization.
A new multi-class active learning method combining informativeness and representativeness.
problem Efficiently labeling large datasets with limited resources.
method A hybrid informative and representative criterion approach for multi-class active learning.
result The proposed method outperforms state-of-the-art methods on multiple UCI datasets.
Bayesian approach predicts molecule bioactivity with uncertainty, aiding active learning.
problem Predicting bioactivity of small molecules accurately and reliably.
method Bayesian semi-supervised graph convolutional neural networks.
result Bayesian approach estimates uncertainty and enables active learning.
The paper studies how neural networks evolve representations, finding a unique fixed point for nonlinear activations.
problem Understanding how neural networks transform input data across layers.
method Theoretical framework for the evolution of the kernel sequence, using mean-field regime and Hermite polynomials.
result For nonlinear activations, the kernel sequence converges globally to a unique fixed point.
Deep belief network improves smartphone activity recognition.
problem Activity recognition on mobile devices.
method Categorization through deep belief network.
result 98.25% correct diagnosis in training data, 93.01% in test data.
Model explains periodic trading in financial markets through game theory.
problem Understanding periodic trading activities in financial markets.
method Mean-field liquidation game with major-minor players.
result Existence and uniqueness of Nash equilibrium established.
Analysis shows exploding and vanishing gradients in neural nets with ReLU activations.
problem Exploding and vanishing gradients in neural nets with ReLU activations.
method Rigorous analysis of gradient behavior in randomly initialized fully connected networks with ReLU activations.
result Empirical variance of gradients is exponential in beta, the sum of reciprocals of hidden layer widths.
This paper compares spike sorting techniques for rat brain neuronal activity.
problem Improving the accuracy of spike sorting for neuronal activity analysis.
method Three-step spike sorting process: detection, feature extraction, and clustering. Various methods are compared.
result Kernel PCA outperforms in feature extraction, leading to better spike sorting results.
Leaky ReLU activations improve the calibration of Bayesian neural networks.
problem Bayesian neural networks struggle with mean-field variational inference for ReLU activations.
method Investigated the effect of activation functions on the calibration of Bayesian neural networks.
result Leaky ReLU activations lead to more Gaussian-like weight posteriors and lower expected calibration error.
Deep active inference learns policies from sensory inputs.
problem Learning policies in partially observable domains.
method Optimizes expected free energy with a variational autoencoder.
result Comparable or better performance than deep Q-learning.
The paper defines successful active management and introduces a framework.
problem The elusive criteria for successful active management in the literature.
method Introducing definitions of key concepts and a logically coherent evaluation framework.
result A strong defense of active management emerges through the defined concepts.
The paper tackles mean-variance analysis in Bayesian optimization under uncertainty.
problem Optimizing decisions in uncertain environments considering trade-offs between average and variance of risk.
method Developed bounds for mean and variance risk measures in Gaussian Process models and proposed AL algorithms for multi-task, multi-objective, and constrained optimization scenarios.
result Proposed AL algorithms effectively address the mean-variance trade-off in uncertain optimization scenarios.
We investigate the variety of a portfolio of stocks in normal and extreme days of market activity. We show that the variety carries information about the market activity which is not present in the single-index model and we observe that the variety time evolution is not time reversal around the crash days. We obtain th…
This article compares and evaluates active learning methods for outlier detection with one-class classifiers.
problem Selecting an appropriate active learning method for outlier detection with one-class classifiers is challenging due to diverse assumptions and objectives.
method The article categorizes various methods, proposes evaluation criteria, and conducts extensive experiments to compare them.
result Formulated guidelines for selecting active learning methods for outlier detection with one-class classifiers.
GAPA method provides efficient uncertainty quantification for pretrained networks.
problem Reliable uncertainty estimates for pretrained models are challenging.
method Post-hoc Gaussian Process Activations (GAPA) method that shifts Bayesian modeling from weights to activations.
result GAPA method provides efficient uncertainty quantification without altering the backbone's predictions.
Paper tackles zero-shot activity recognition using video features and text embeddings.
problem Zero-shot activity recognition with videos.
method Auto-encoder model for multimodal joint embedding, 3D convolutional action recognition for visual features, GloVe word embeddings for textual features.
result Improved zero-shot recognition results with top-n accuracy and mean Nearest Neighbor Overlap.
COBRA efficiently clusters data using pairwise constraints with minimal queries.
problem Clustering datasets with user-defined constraints.
method Over-clusters data with K-means, then merges clusters based on constraints.
result COBRA outperforms state-of-the-art methods in clustering quality and runtime.
Few-shot learning improved with semi-supervised and active methods.
problem Few-shot classification with limited labeled data.
method Prototypical Networks features combined with K-means clustering guided by few labeled examples and user feedback. result Active adaptation through user feedback significantly improves performance.
Deep neural networks improve mean function estimation for functional data.
problem Estimating mean functions of functional data.
method Deep neural networks with ReLU activation, sparsely connected.
result Achieves optimal nonparametric convergence rate in empirical norm.
A new Universal Activation Function improves performance across various machine learning tasks.
problem Achieving near optimal performance in different machine learning tasks.
method Optimization algorithms evolve the UAF's parameters to match the optimal activation function for each task.
result The UAF converges to near optimal performance in classification, quantification, and reinforcement learning tasks.
New activation function BrownianReLU improves LSTM network performance on financial time series.
problem Gradient instability in noisy financial time series data.
method Introduces BrownianReLU, a stochastic activation function based on Brownian motion.
result Significantly improved predictive accuracy and generalization on financial datasets.
Post-hoc explanations improve CNNs by replacing final linear layer with k-means classifier.
problem CNNs lack accurate data representation in their built-in prototypes.
method Introduces k-means-based post-hoc explanations for CNNs, leveraging spatial consistency of convolutional receptive fields.
result Using shallower, less compressed feature activations improves semantic fidelity at the cost of slight predictive performance.
Narrow neural networks have unbounded decision regions.
problem Understanding decision regions of narrow neural networks.
method Analyzing decision regions of neural networks with width ≤ input dimension.
result All connected components of decision regions are unbounded.
Tool detects tax evasion on social media using multi-modal deep learning.
problem Detecting tax evasion on social media platforms.
method Developed a multi-modal deep neural network combining comments, hashtags, and images.
result Multi-modal deep neural network achieved AUC of 0.808 and F1 score of 0.762.
Active learning algorithms propose which unlabeled objects should be queried for their labels to improve a predictive model the most. We study active learners that minimize generalization bounds and uncover relationships between these bounds that lead to an improved approach to active learning. In particular we show th…
We propose an active learning method for discovering low-dimensional structure in high-dimensional Gaussian process (GP) tasks. Such problems are increasingly frequent and important, but have hitherto presented severe practical difficulties. We further introduce a novel technique for approximately marginalizing GP hype…
Active inference uses machine learning to prioritize data labeling for more efficient statistical inference.
problem Efficiently collecting data points for statistical inference with limited labels.
method A machine learning-assisted approach that identifies uncertain data points for labeling.
result Achieves the same level of accuracy with fewer samples, resulting in smaller confidence intervals and more powerful p-values.
DiAL uses Bayesian Dirichlet random fields for active learning with sparse labels.
problem Active learning with limited labeled data.
method Bayesian Dirichlet random field for feature-conditional class probabilities, calibrating with graph Laplacian.
result Competitive performance in low-label rate graph learning tasks.