Deep ReLU networks have surprisingly few activation patterns at initialization.
problem Limited expressivity of deep ReLU networks despite theoretical potential.
method Analyzed the number of activation patterns in ReLU networks at initialization and during training.
result The average number of activation patterns is bounded by the total number of neurons raised to the input dimension.
For using neural networks in safety critical domains, it is important to know if a decision made by a neural network is supported by prior similarities in training. We propose runtime neuron activation pattern monitoring - after the standard training process, one creates a monitor by feeding the training data to the ne…
Study develops a new model for predicting individual mobility based on activity patterns.
problem Existing mobility prediction methods overlook underlying activity patterns.
method Activity-based hidden Markov model (IOHMM) using transit smart card data.
result Proposed model achieves similar prediction performance to LSTM but also provides behavioral insights.
Probabilistic ESI model improves brain activity pattern analysis.
problem Noise sensitivity and lack of time-varying pattern flexibility in traditional ESI methods.
method Hierarchical graph prior with spanning tree constraint and alternating convex search algorithm.
result Significant improvements in source localization performance, especially at high noise levels.
Detects audio adversarial examples using anomalous pattern detection.
problem Identifies adversarial audio attacks in deep neural networks.
method Applies anomalous pattern detection in activation space of audio models.
result Can detect adversarial examples with up to 0.98 AUC, no degradation on benign samples.
Model predicts individual insurance claim reserves using activation patterns.
problem Accurately predicting individual claim reserves in insurance contracts.
method Multinomial logistic regression to model claim activation and development.
result The model generates accurate predictions of total and per coverage reserves.
Bayesian calibration improves ABMs for predicting travel patterns.
problem Calibrating ABMs for accurate travel pattern predictions.
method Gaussian Process emulator with deep learning dimensionality reduction for high-dimensional, non-stationary data.
result Improved accuracy in predicting travel patterns using traffic flow data.
The paper examines topological features of ReLU networks and their relation to decision boundaries and training loss.
problem Understanding the topological structure of ReLU neural network activation patterns.
method Polytope decomposition of feature space, Fiedler partition of dual graph, homology computation of cellular decomposition.
result The Fiedler partition of the dual graph correlates with decision boundaries in binary classification tasks, and similar patterns in training loss and polyhedral cell-count emerge in regression tasks.
Paper detects abnormalities in brain activity patterns using unsupervised learning.
problem Detecting abnormalities in resting-state brain activity patterns.
method Two strategies: autoencoder approach and next frame prediction.
result Both approaches can learn useful representations of rs-fMRI data for abnormality detection.
Cryptocurrency market activity is decomposed into recurring and noise components, revealing patterns tied to macroeconomic reports.
problem Investigating temporal patterns of cryptocurrency market activity.
method Decomposition of market activity measures into recurring and noise components via correlation matrix formalism.
result Recurring market activity bursts coincide with significant U.S. macroeconomic reports, indicating their influence.
A method to interpret neural networks by isolating active subnetworks.
problem Interpreting the complex computations of deep neural networks.
method Switched linear projections to decompose network activity.
result Patterns in deactivated neurons are meaningful for network understanding.
In the wake of recent advances in experimental methods in neuroscience, the ability to record in-vivo neuronal activity from awake animals has become feasible. The availability of such rich and detailed physiological measurements calls for the development of advanced data analysis tools, as commonly used techniques do …
Extracts patterns from mobile network data for better resource management.
problem Improving network efficiency and resource allocation for mobile users.
method Spatiotemporal analysis of internet activity records (IARs) data.
result Developed a mobile traffic partitioning scheme.
Active learning method for neural population dynamics using optogenetics.
problem Efficiently selecting neurons to stimulate for identifying neural population dynamics.
method Developed active learning procedure for low-rank regression to determine informative photostimulation patterns.
result Demonstrated a two-fold reduction in data required for predictive power using low-rank linear dynamical systems model.
ActiLabel learns activity patterns across diverse sensor devices.
problem Limited adoption of activity recognition models across different domains due to diverse sensor devices.
method Combination of graph model and optimal tiered mapping for learning activity labels.
result Superior performance compared to state-of-the-art methods on public datasets.
New method allows sheets to morph into multiple shapes via spatially varying stimuli.
problem Limitation of current shape-programmed sheets to achieve only one target geometry.
method Patterning the stimulus itself for spatiotemporal control over local deformation magnitudes.
result A single physical sample can be induced to traverse a continuous family of target geometries.
The paper proves neural networks with ReLU and softmax can approximate any function.
problem Approximating functions and class labels in neural networks.
method Extended universal approximator theory to neural networks with ReLU and softmax.
result Neural networks with ReLU and softmax can approximate any function and class labels.
Deep neural networks maximize variation when few nodes change activation.
problem Maximizing variation in deep neural networks.
method Theoretical analysis of ReLU activation function and layer node numbers.
result Maximal variation occurs when few nodes change activation.
Modeling and estimating dynamic graphs from binary pattern sequences.
problem Extracting dominant correlation structures from time-dependent binary patterns.
method State-space model of an Ising-type network composed of multiple undirected graphs, sequential Bayes algorithm.
result The method outperforms traditional methods in uncovering overlapping graphs and estimating dynamics of weights.
Neural networks encode inputs deterministically and categorically, behaving like hash encoders.
problem Understanding the encoding properties of neural networks.
method Analyzed the input space partitioned by ReLU-like activations in neural networks.
result Neural networks can be represented by unique activation patterns, similar to hash encoders.
Decodes neural activity to assess latent states in real-world driving tasks.
problem Understanding latent states during complex tasks in natural settings.
method Domain-generalized models trained on controlled lab paradigms applied to ecologically valid driving tasks.
result Changes in neural activity correlate with changes in behavior and task performance.
New test assesses shared brain activity across different cognitive modalities.
problem Determining if different cognitive modalities use overlapping neural representations.
method Formulated a statistical hypothesis testing approach using permutation testing.
result New test (CMPT) has greater statistical power than cross-modal decoding while maintaining low Type I errors.
Neural networks plateau during training, identified and quantified.
problem Plateau phenomenon in gradient descent training of ReLU networks.
method Identification and quantification of plateau phenomenon; new iterative training method ANLS.
result Plateaux correspond to periods of constant activation patterns; quantification of gradient flow dynamics; characterization of stationary points.
The paper uses action graphs to predict user engagement in Snapchat.
problem Understanding what motivates users to engage with mobile social apps.
method Formalized in-app action transition patterns as action graphs, analyzing their characteristics to predict future engagement.
result Action graphs can characterize user behavior patterns and inform future engagement.
TransFall uses transfer learning to improve activity recognition from mobile sensors.
problem Performance degradation due to platform and user movement differences.
method Two-tier data transformation, label estimation, and model generation layers.
result TransFall enhances activity recognition accuracy for new scenarios.
A model of associative memory is studied, which stores and reliably retrieves many more patterns than the number of neurons in the network. We propose a simple duality between this dense associative memory and neural networks commonly used in deep learning. On the associative memory side of this duality, a family of mo…
A new VAD method uses respiration patterns from video to detect speech.
problem Improving VAD performance in noisy audio recordings.
method Extract respiration patterns from video, use neural models to detect speech.
result Efficacy demonstrated through experiments on real acoustic environments.
Deep ReLU networks with extra parameters have mostly good loss landscapes.
problem Finding good local minima in the loss landscape of deep neural networks.
method Analyzing shallow and deep ReLU networks with extra parameters on a generic dataset.
result Most activation patterns correspond to regions with no bad local minima.
FDive helps analysts create relevant patterns in high-dimensional datasets.
problem Difficult detection of relevant patterns in large, high-dimensional datasets.
method FDive uses a visual active learning system with pattern-based similarity measures to rank and classify data.
result FDive enhances the quality and understanding of relevance models, leading to new insights.
New method clusters travel behavior data from 1990-2017.
problem Challenges in analyzing large-scale travel data.
method Divide and Combine K-means clustering on time series data.
result Activity-travel patterns can be grouped into three clusters.
Commercial activity trackers are set to become an essential tool in health research, due to increasing availability in the general population. The corresponding vast amounts of mostly unlabeled data pose a challenge to statistical modeling approaches. To investigate the feasibility of deep learning approaches for unsup…
Detects anomalous inputs in neural networks using subset scanning.
problem Detecting adversarial noise and out-of-distribution samples in neural networks.
method Subset scanning applied to neural network activations using non-parametric scan statistics.
result Identifies the most anomalous subset of node activations in neural networks.
Taxonomy for ML in simulations, covering patterns and algorithms.
problem Enhancing simulations using machine learning.
method Presentation of eight patterns and three algorithmic areas.
result Catalog of activities and patterns for ML integration in simulations.
People are increasingly relying on the Web and social media to find solutions to their problems in a wide range of domains. In this online setting, closely related problems often lead to the same characteristic learning pattern, in which people sharing these problems visit related pieces of information, perform almost …
Paper predicts crimes using historical data and machine learning.
problem Predicting and preventing crime activities.
method Supervised learning with decision tree, k-nearest neighbor, Random Forest, and Adaboost algorithms.
result The system predicts crimes with better accuracy using historical data.
DoPa detects various physical adversarial attacks on CNNs.
problem Vulnerability of CNNs to physical adversarial attacks.
method Interprets CNN's vulnerability, adds self-verification stage.
result Achieves 90% success rate for image attacks and 92% for audio attacks.
The study reveals distinct patterns in retail investors' holding periods affecting stock returns.
problem Understanding the impact of retail investors' investment horizons on stock returns.
method Using self-reported holding periods from StockTwits, the study categorizes retail investors into long-horizon and short-horizon groups and analyzes their return patterns.
result Long-horizon retail investors exhibit underreaction to earnings announcements, while short-horizon investors show overreaction.
Model detects patterns in noisy binary data, explaining neuron activity in terms of cell assemblies.
problem Detecting structure in noisy or approximate repeats of patterns in sparse binary data.
method Probabilistic binary latent variable model based on Noisy-OR model, inferring sparse activity in latent variables.
result Model successfully extracts and explains latent structure in spiking neural data.
GGP models multivariate time series with latent sub-sequences for diverse behaviors.
problem Modeling multivariate time series with diverse behaviors and patterns.
method Graph Gamma Process (GGP) linear dynamical systems with latent sub-sequences.
result GGP models exhibit good predictive performance and reveal interpretable latent patterns.
Deep learning models can overfit noisy data without losing generalization.
problem Understanding the generalization of deep learning models in noisy data.
method Empirical investigation of epoch-wise double descent in fully connected neural networks trained on CIFAR-10 with 30% label noise.
result The model achieves strong re-generalization on test data after overfitting noisy training data, corresponding to a 'benign overfitting' state.
Deep neural network detects falls from IoT sensor data.
problem Detecting irregular patterns in IoT streaming data for fall detection.
method Deep neural network model using accelerometer data.
result 98.75 percent accuracy in detecting falls.
Study analyzes online student behavior patterns using log data.
problem Understanding and optimizing student learning in online educational systems.
method Non-negative matrix factorization techniques for soft clustering.
result Behavioral changes of individual students and the system over time.
New method detects money laundering in Bitcoin using minimal labels.
problem Detecting money laundering in Bitcoin transactions with scarce labels.
method Active learning approach to anomaly detection.
result 5% of labels are sufficient to match supervised baseline performance.
New method learns routines from inertial data without privacy concerns.
problem Recognizing human activities with limited sets of specific activities.
method Metric learning problem defined for time series, SS2S architecture.
result Clustering recovers daily routines from learned distance.
DNNs are seen as two cooperating classifiers.
problem Lack of a robust framework to predict DNN generalization.
method Tracing activation patterns and viewing DNNs as continuous and discrete systems.
result DNNs can be viewed as two cooperating classifiers.
Collaborative filtering is a useful technique for exploiting the preference patterns of a group of users to predict the utility of items for the active user. In general, the performance of collaborative filtering depends on the number of rated examples given by the active user. The more the number of rated examples giv…
We consider a few quantities that characterize trading on a stock market in a fixed time interval: logarithmic returns, volatility, trading activity (i.e., the number of transactions), and volume traded. We search for the power-law cross-correlations among these quantities aggregated over different time units from 1 mi…
We describe the impact of the intra-day activity pattern on the autocorrelation function estimator. We obtain an exact formula relating estimators of the autocorrelation functions of non-stationary process to its stationary counterpart. Hence, we proved that the day seasonality of inter-transaction times extends the me…