Analytical solution found for a three-layer network with a specific activation function.
problem Understanding the power of depth in neural networks.
method Found analytical solutions for a three-layer network with a matrix exponential activation function.
result Analytical solutions for equations involving a three-layer network with a matrix exponential activation function.
The paper proves deep neural networks with analytic activation can approximate any function.
problem Approximating functions with neural networks using analytic activation functions.
method Elementary proofs for real and complex networks, Stone-Weierstrass theorem, Mergelyan's theorem.
result Closure of neural network classes equals space of polynomials for analytic activation.
Analytic networks with bounded coefficients can't outperform polynomial approximations.
problem Approximation limits of neural networks with analytic activation functions under coefficient constraints.
method Deterministic analysis using comparison argument and Bernstein-type estimates.
result Networks with analytic activation functions and controlled coefficients cannot outperform classical polynomial approximation rates on non-analytic targets.
We present analytical exploration of novel activation functions as consequence of integration of several ideas leading to implementation and subsequent use in habitability classification of exoplanets. Neural networks, although a powerful engine in supervised methods, often require expensive tuning efforts for optimize…
We explore the efficacy of using a novel activation function in Artificial Neural Networks (ANN) in characterizing exoplanets into different classes. We call this Saha-Bora Activation Function (SBAF) as the motivation is derived from long standing understanding of using advanced calculus in modeling habitability score …
COHORTNEY groups web users based on activity patterns.
problem Lack of academic discussion on cohort analysis for user behavior.
method Unsupervised non-parametric machine learning approach.
result COHORTNEY outperforms traditional methods in cohort analysis.
Recent years have witnessed amazing outcomes from "Big Models" trained by "Big Data". Most popular algorithms for model training are iterative. Due to the surging volumes of data, we can usually afford to process only a fraction of the training data in each iteration. Typically, the data are either uniformly sampled or…
Analytical method finds deeper optima in two-layer ReLU networks.
problem Training two-layer ReLU networks with analytical methods.
method Analytically finding critical points of the loss function for one layer while keeping the other fixed.
result Significantly smaller training loss values on real datasets compared to gradient descent methods.
Deep networks with path norm regularization can approximate analytic functions.
problem Approximating analytic functions with neural networks.
method Path norm regularized deep networks with activation function.
result Deep networks can approximate analytic functions with logarithmic dependence on approximation error.
We propose a novel Shapley value approach to help address neural networks' interpretability and "vanishing gradient" problems. Our method is based on an accurate analytical approximation to the Shapley value of a neuron with ReLU activation. This analytical approximation admits a linear propagation of relevance across …
TransFall uses transfer learning to improve activity recognition from mobile sensors.
problem Performance degradation due to platform and user movement differences.
method Two-tier data transformation, label estimation, and model generation layers.
result TransFall enhances activity recognition accuracy for new scenarios.
This paper provides an overview of activation functions in neural networks.
problem Confusion in activation function selection and properties in deep learning.
method Analytic review of popular activation functions.
result Clarification of activation function properties and selection.
ROI-driven data analytics guides investment in empirical data analysis.
problem Determining the optimal depth and breadth of data analytics.
method Conceptual framework validated through empirical studies focusing on dependency extraction in Mozilla Firefox project.
result ROI-driven data analytics helps avoid over-analyzing empirical data.
Wide networks with polynomial activations have proven asymptotic behavior.
problem Understanding the behavior of neural networks in the large width limit.
method Proving a conjecture for deep networks with polynomial activation functions.
result Tight bounds on the behavior of wide networks during stochastic gradient descent and derivation of their finite-width dynamics.
Study shows big winner stocks significantly impact passive and active investment strategies.
problem Impact of big winner stocks on passive and active investment strategies.
method Numerical and analytical techniques applied to historical stock price data.
result Concentrated portfolios underperform equally weighted indexes due to missing big winner stocks.
A cost-effective approach to label acquisition using active learning markets.
problem Improving model fitting and training for predictive analytics.
method Formalizing market clearing as an optimisation problem, integrating budget constraints and improvement thresholds, using two active learning strategies with distinct pricing mechanisms.
result Superior performance with fewer labels acquired compared to conventional methods.
Detecting patterns in real time streaming data has been an interesting and challenging data analytics problem. With the proliferation of a variety of sensor devices, real-time analytics of data from the Internet of Things (IoT) to learn regular and irregular patterns has become an important machine learning problem to …
Deep neural networks approximate analytic functions in high dimensions with exponential rates.
problem Approximating analytic functions in high-dimensional spaces using neural networks.
method Analyzing convergence rates of ReLU and ReLU^k activations in L2(Rd,γd) for d∈N∪{∞}. result Exponential convergence rates for analytic functions in L2(Rd,γd) for d∈N, and dimension-independent bounds for d=∞. Wide neural networks can learn complex functions like gravitational force law.
problem Learning complex functions like gravitational force law with neural networks.
method Extending theoretical bounds to analytic functions on the sphere using SGD and ReLU networks.
result Wide ReLU networks can learn analytic functions efficiently with proportional number of samples.
We consider active maximum a posteriori (MAP) inference problem for Hidden Markov Models (HMM), where, given an initial MAP estimate of the hidden sequence, we select to label certain states in the sequence to improve the estimation accuracy of the remaining states. We develop an analytical approach to this problem for…
TS-Insight visualizes Thompson Sampling for better debugging and trust.
problem Thompson Sampling's black box nature hinders debugging and trust.
method TS-Insight is a visual analytics tool that traces evolving posteriors and evidence counts.
result Visualizations help in verifying, diagnosing, and explaining Thompson Sampling dynamics.
The study analyzes ETFs' portfolio optimization and tail-risk management.
problem Analyzing the performance of actively managed ETFs in managing risk and diversification.
method Daily Bloomberg data for 30 funds, evaluating various strategies under long-only and long-short constraints.
result Tangency-type portfolios generally outperform buy-and-hold benchmarks, while minimum-variance and CVaR-minimizing portfolios sacrifice upside for downside control.
Dropout schedules can be optimized to significantly reduce model test loss.
problem Improving model performance in neural networks.
method Developed a mean-field theory of dropout at the edge of chaos, proposing front-loaded dropout schedules.
result Front-loaded dropout schedules reduce test loss by 18-35% over constant dropout.
New methods for uncertainty in neural networks with leaky ReLU activations.
problem Uncertainty in feed-forward neural networks with random input perturbations.
method Analytical expressions for PDF and moments of neural network output, linearization of leaky ReLU, Gaussian copula surrogate models.
result Accurate statistical results for large input perturbations, excellent agreement with Monte Carlo simulations.
Combines additivity and active subspaces for high-dimensional Gaussian process modeling.
problem High-dimensional Gaussian process modeling challenges due to the curse of dimensionality.
method Combines additivity and active subspaces with a multi-fidelity strategy.
result Shows advantages through experiments on synthetic functions and datasets.
The increasing accessibility of data provides substantial opportunities for understanding user behaviors. Unearthing anomalies in user behaviors is of particular importance as it helps signal harmful incidents such as network intrusions, terrorist activities, and financial frauds. Many visual analytics methods have bee…
Automatic analysis of teacher and student interactions could be very important to improve the quality of teaching and student engagement. However, despite some recent progress in utilizing multimodal data for teaching and learning analytics, a thorough analysis of a rich multimodal dataset coming for a complex real lea…
Proposes an active RBI framework using Rényi information measures for more informed decision-making.
problem Optimal latent variable estimates in real-time settings with streaming noisy observations.
method Unified inference and query selection steps through Rényi entropy and α-divergence; new objective called Momentum for exploration.
result Analytically demonstrates superior performance compared to conventional methods like mutual information.
Over the last few years, traffic data has been exploding and the transportation discipline has entered the era of big data. It brings out new opportunities for doing data-driven analysis, but it also challenges traditional analytic methods. This paper proposes a new Divide and Combine based approach to do K means clust…
Enhanced network threat detection using KG, LLM, and imbalanced learning.
problem Challenges in network threat detection due to complex attack patterns and limited historical data.
method Integrated framework combining Knowledge Graph, Imbalanced Learning, and Large Language Model.
result Improved threat capture rate by 3%-4% and increased interpretability of risk predictions.
New method for estimating gradients in stochastic binary networks.
problem Challenges in training neural networks with binary activations and weights.
method Combines sampling and analytic approximation steps to estimate gradients accurately.
result Significantly reduced variance at the cost of small bias, leading to practical tradeoffs.
We propose a feed-forward inference method applicable to belief and neural networks. In a belief network, the method estimates an approximate factorized posterior of all hidden units given the input. In neural networks the method propagates uncertainty of the input through all the layers. In neural networks with inject…
Big data analytics improves healthcare through early detection and quality life.
problem Limited access to healthcare data hinders evidence-based decision-making.
method Analysis of healthcare data using various tools and techniques.
result Big data analytics enhances healthcare quality and patient outcomes.
New acquisition functions improve Bernoulli LSE.
problem Efficiently estimating regions where a Bernoulli function is above or below a threshold.
method Developed new look-ahead acquisition functions for Gaussian process classification models.
result Demonstrated clear benefits of new acquisition functions on benchmark and real-world tasks.
The paper analyzes the role of ReLU gates in deep learning networks.
problem Understanding the role of gates in deep learning networks.
method Developed neural path features (NPF) and neural path values (NPV) to characterize the active sub-networks during training.
result The neural path kernel associated with NPFs is a fundamental quantity that characterizes the information stored in the gates of a DNN.
Crowdsourcing and active learning reduce manual annotation in social media event classification.
problem Manual annotation is time-consuming and resource-intensive in social media event classification.
method Crowdsourcing pipeline combined with active learning strategies.
result Active learning strategies help reduce the number of tweets needed for classification.
Tempered sigmoids improve deep learning privacy.
problem Privacy-preserving deep learning with strict differential privacy guarantees.
method Developed tempered sigmoid activation functions for deep learning models.
result Tempered sigmoids outperform ReLU in achieving state-of-the-art accuracy.
A general Boltzmann machine with continuous visible and discrete integer valued hidden states is introduced. Under mild assumptions about the connection matrices, the probability density function of the visible units can be solved for analytically, yielding a novel parametric density function involving a ratio of Riema…
Due to the success of deep learning to solving a variety of challenging machine learning tasks, there is a rising interest in understanding loss functions for training neural networks from a theoretical aspect. Particularly, the properties of critical points and the landscape around them are of importance to determine …
GNIs induce a regulariser that penalizes high-frequency components in neural network activations.
problem Understanding the regularizing effect of Gaussian noise injections on neural network activations.
method Deriving the explicit regularizer by marginalizing out injected noise and analyzing its effect in the Fourier domain.
result GNIs induce a regularizer that produces calibrated classifiers with large margins.
Bayesian active learning reduces data needed for latent variable models.
problem Active learning for latent variable models, especially mixtures of linear regressions and HMMs with GLM observations.
method Maximum-mutual-information input selection for discrete latent variable regression models.
result Active learning can achieve large gains for mixtures of linear-Gaussian models and substantially reduces data needed for GLM-HMM.
This paper explains how batch normalization auto-tunes the regularization parameter based on data statistics.
problem Batch normalization accelerates deep learning training but the exact relationship to regularization is unclear.
method Theoretical analysis and empirical validation of batch normalization's role in auto-tuning the regularization parameter.
result Batch normalization auto-tunes the regularization parameter based on data statistics.
Mobile networks possess information about the users as well as the network. Such information is useful for making the network end-to-end visible and intelligent. Big data analytics can efficiently analyze user and network information, unearth meaningful insights with the help of machine learning tools. Utilizing big da…
Paper proposes a method for efficient online classification using siamese networks and active learning.
problem Online learning challenges with limited, nonstationary, and imbalanced data.
method Synergistic combination of siamese neural networks and active learning with multi-sliding window approach.
result Significantly outperforms baselines and state-of-the-art algorithms in terms of learning speed and performance.
New algorithms optimize neural networks with ReLU activations using sampling.
problem Optimizing trained neural networks with ReLU activations.
method Iterative algorithm and enhanced algorithm using sampling and neighborhood search.
result The methods reduce the initial MIP problem into smaller LP or MIP problems.
Review of Gerber-Shiu function for practical actuarial science.
problem Difficulty in numerical approximation and statistical inference of Gerber-Shiu function.
method Comprehensive review of formulations, surplus processes, numerical methods, and statistical inference.
result Enhanced understanding and practical guide for Gerber-Shiu function.
New method finds basins of attraction without needing system models.
problem Determining basins of attraction (BoA) for nonlinear systems without prior knowledge.
method Hybrid Active Learning (HAL) method combining AST, AL, and DBS.
result Efficiently finds and labels boundary of BoA without model knowledge.
We use neural networks as control variates with geometric integration techniques.
problem Analytic integration of neural network approximations for variance reduction.
method Integration domain subdivision using computational geometry for MLPs with continuous piecewise linear activation functions.
result Neural networks can be used as control variates with geometric integration methods.