DPPNets approximate DPP sampling with deep learning for efficient subset selection.
problem Efficiently sampling from Determinantal Point Processes (DPPs) with high diversity and quality.
method Developed DPPNets using transformer networks with an inhibitive attention mechanism.
result Samples from DPPNets receive high likelihood under the more expensive DPP alternative, demonstrating efficiency.
Paper presents a method for estimating Hawkes process parameters.
problem Estimating parameters of Hawkes processes with self-excitation or inhibition.
method Maximum likelihood estimation for Hawkes processes with self-excitation or inhibition.
result The proposed estimator provides more accurate estimations in the inhibition context.
Paper tackles selfless sequential learning with neural inhibition to improve future task capacity.
problem Learning tasks in sequence with limited model capacity.
method Study regularization strategies and activation functions, proposing a novel representation sparsity regularizer.
result Representation sparsity regularizer improves performance over alternative regularizers.
We present a framework to learn privacy-preserving encodings of images that inhibit inference of chosen private attributes, while allowing recovery of other desirable information. Rather than simply inhibiting a given fixed pre-trained estimator, our goal is that an estimator be unable to learn to accurately predict th…
GAttNHP predicts future events in temporal knowledge graphs by encoding long-range dependencies and handling mutual excitation.
problem Forecasting future events in temporal knowledge graphs due to long-range dependencies, mutual excitation, and heavy-tailed inter-arrival times.
method GAttNHP uses a self-attention encoder, semantic soft-grouping, and NCQ regression to address these issues.
result GAttNHP improves entity and time prediction on six benchmark TKG datasets compared to state-of-the-art baselines.
Recent studies have shown that information disclosed on social network sites (such as Facebook) can be used to predict personal characteristics with surprisingly high accuracy. In this paper we examine a method to give online users transparency into why certain inferences are made about them by statistical models, and …
LM-SNNs use lattice maps to classify and cluster images.
problem Image classification and clustering.
method Lattice map spiking neural networks with cooperative and competitive interactions, inhibition strategies, and biologically motivated learning rules.
result LM-SNNs effectively classify and cluster images using self-organized filters.
Neuromorphic column performs online unsupervised clustering.
problem Real-time clustering of streaming data.
method Localized, spike timing-dependent plasticity (STDP) neural column.
result Prototype column performs similarly to k-means clustering.
NNNH uses neural networks to model complex event patterns.
problem Analyzing multi-dimensional nonlinear Hawkes processes with mutual excitation and inhibition.
method NNNH employs feedforward neural networks to model individual kernels and base intensity, optimizing parameters via Stochastic Gradient Descent.
result NNNH accurately captures complexities of nonlinear Hawkes processes, as demonstrated by numerical experiments.
Mamba Hawkes Process improves modeling of event sequences with long-term dependencies.
problem Modeling mutual inhibition and nonlinearity in asynchronous event sequences.
method Introduces Mamba Hawkes Process using Mamba state space architecture.
result MHP outperforms existing models across various datasets.
Unsupervised learning by hidden units with biological plausibility.
problem Training neural networks without labeled data.
method Global inhibition in hidden layer to learn feature detectors.
result Learned feature detectors enable supervised training of higher layers.
Neurons and networks in the cerebral cortex must operate reliably despite multiple sources of noise. To evaluate the impact of both input and output noise, we determine the robustness of single-neuron stimulus selective responses, as well as the robustness of attractor states of networks of neurons performing memory ta…
This paper optimizes kernel and acquisition functions for high-dimensional Bayesian Optimization.
problem Bayesian Optimization struggles with high-dimensional problems due to computational inefficiency.
method The paper leverages the additionality of the objective function to map kernel and acquisition functions in lower-dimensional subspaces, improving efficiency.
result Efficient optimization of acquisition function in high-dimensional problems.
Facing a heavy task, any single person can only make a limited contribution and team cooperation is needed. As one enjoys the benefit of the public goods, the potential benefits of the project are not always maximized and may be partly wasted. By incorporating individual ability and project benefit into the original pu…
Investigates consumption and investment strategies with preference for liquid assets.
problem Infinite horizon consumption-portfolio problem with liquid and illiquid risky assets.
method Analyzes properties of value function, categorizes solvency regions, and characterizes optimal policy.
result Liquidity preference leads to higher liquid wealth and lower consumption, potentially negative allocation to illiquid asset.
Deep generative model discovers inhibitors for unknown targets.
problem Discovering novel inhibitor molecules for unknown drug targets.
method Deep generative framework trained on protein sequences, small molecules, and interactions.
result Micromolar-level inhibition observed for two out of four synthesized candidates, including activity against SARS-CoV-2 variants.
In a market system, regulations are designed to prevent or rectify market failures that inhibit fair exchange, such as monopoly or transactions with hidden costs. Because regulations reduce profits to those possessing unfair advantage, these advantaged corporations (whether individuals, companies, or other collective o…
This is part 2 of a 3-part article where we provide an O(n2)-algorithm to produce a surgery presentation of a 3-manifold induced by a gem with a resolution. In this part we produce a sequence of colored simplicial 2-complexes which are inverses and dual to the sequence of gems produced in the first part. The refinem…
DNN is presenting human-level performance for many complex intelligent tasks in real-world applications. However, it also introduces ever-increasing security concerns. For example, the emerging adversarial attacks indicate that even very small and often imperceptible adversarial input perturbations can easily mislead t…
New method adds uncertainty estimation to softmax outputs.
problem Uncertainty estimation in neural networks.
method Extend softmax layer with an additional constant input.
result Performs comparably to more computationally expensive methods.
New GP-VAE model improves scalability and performance.
problem Inability of conventional VAEs to model correlations between data points.
method Principled sparse inference approaches to improve scalability of GP-VAEs.
result New model outperforms existing approaches in runtime and memory usage.
Insect-inspired neural nets boost machine learning accuracy with limited data.
problem Difficulty of machine learning with scarce training data.
method Deployed MothNet, a computational model of insect olfactory network, as a feature generator.
result Cyborgs improved test set accuracy by 6% to 33% compared to baseline ML methods.
NetBiTE predicts drug sensitivity and identifies biomarkers in cancer.
problem Predicting drug sensitivity and identifying biomarkers in cancer.
method NetBiTE combines prior knowledge and gene expression data using a biased tree ensemble approach.
result NetBiTE outperforms RF in predicting IC50 drug sensitivity for drugs targeting membrane receptor pathways.
Study efficient policy value estimation with sublinear samples.
problem Estimating optimal policy value in stochastic disjoint linear bandits.
method Sublinear sample estimation of optimal policy value.
result Achieves near optimal estimation error with sublinear samples.
We report analytical results for the development of the viscous fingering instability in a cylindrical Hele-Shaw cell of radius a and thickness b. We derive a generalized version of Darcy's law in such cylindrical background, and find it recovers the usual Darcy's law for flow in flat, rectangular cells, with correctio…
Adapts Lifelong Learning for Federated Learning to prevent model drift.
problem Federated Learning on non-i.i.d. data leads to model divergence and forgetting.
method Adds a penalty term to the loss function to encourage convergence of local models to a shared optimum.
result The method improves performance in image recognition tasks compared to existing approaches.
New findings allow infinite mean intensity Hawkes processes to be stable.
problem Stability condition for Hawkes processes with infinite mean intensity.
method Analysis of Quadratic Hawkes processes with infinite mean intensity.
result Quadratic Hawkes processes are always stationary with infinite mean intensity when total endogeneity ratio exceeds unity.
Study finds reinforcement learning performance plateaus due to environmental interference.
problem Catastrophic interference hinders sample efficiency in reinforcement learning.
method Empirical study in ALE, controlled experiments, analysis of prediction errors.
result Interference causes performance plateaus and degrades policies used to reach them.
In this work, an ensemble of economic interacting agents is considered. The agents are arranged in a linear array where only local couplings are allowed. The deterministic dynamics of each agent is given by a map. This map is expressed by two factors. The first one is a linear term that models the expansion of the agen…
ED-VAE improves VAEs by explicitly including entropy components in ELBO.
problem Limitations of traditional VAEs with ELBO in generating high-quality samples and interpreting latent spaces.
method Introduces ED-VAE, a re-formulation of ELBO that includes entropy and cross-entropy components.
result Significantly enhances model flexibility and improves interpretability and generative performance.
Many events occur in the world. Some event types are stochastically excited or inhibited---in the sense of having their probabilities elevated or decreased---by patterns in the sequence of previous events. Discovering such patterns can help us predict which type of event will happen next and when. We model streams of d…
Parallel computing has played an important role in speeding up convex optimization methods for big data analytics and large-scale machine learning (ML). However, the scalability of these optimization methods is inhibited by the cost of communicating and synchronizing processors in a parallel setting. Iterative ML metho…
Model-based learning algorithms have been shown to use experience efficiently when learning to solve Markov Decision Processes (MDPs) with finite state and action spaces. However, their high computational cost due to repeatedly solving an internal model inhibits their use in large-scale problems. We propose a method ba…
Estimates effect sizes and power from a pilot experiment.
problem Estimating the distribution of effect sizes in multiple testing settings.
method Uses an inexpensive pilot experiment to estimate effect sizes and power.
result Simple and computationally efficient estimator guarantees the number of discoveries.
Under-parameterization hinders deep RL's efficiency.
problem Implicit under-parameterization impairs data-efficiency in deep RL.
method Characterized and mitigated the rank collapse of value network features.
result Controlling rank collapse improves deep RL performance.
Reconstructing network connectivity from the collective dynamics of a system typically requires access to its complete continuous-time evolution although these are often experimentally inaccessible. Here we propose a theory for revealing physical connectivity of networked systems only from the event time series their i…
Adapts MBDOE for real-time parameter estimation in complex systems.
problem Costly posterior inference and design optimization in nonlinear systems.
method Combines DAD with differentiable mechanistic models for real-time parameter estimation.
result Demonstrated on four systems, including a DC motor.
Proposes a new model for complex multivariate event data.
problem Modeling complex multivariate event data with spatio-temporal dynamics.
method Integrates spatial information into latent state evolution through learned temporal and spatial decay dynamics.
result Successfully recovers sensible temporal and spatial intensity structure in multivariate spatio-temporal point patterns.
Nyström KPCA balances computational efficiency and statistical accuracy.
problem Computational burden in large sample situations for kernel methods.
method Theoretical analysis of Nyström approximate kernel principal component analysis (KPCA).
result Nyström approximate KPCA matches statistical performance of non-approximate KPCA while being computationally beneficial.
Paper proposes an integrated M&D approach for large multistream data.
problem Inability to progress in monitoring and diagnostics due to high-dimensionality and volume of multistream data.
method Adaptive Principal Component monitoring (APC) and Principal Component Signal Recovery (PCSR).
result The integrated M&D approach enables early detection and streamlined SPC.
New interpretation of attention in Transformers and Graph Attention Networks.
problem Understanding and improving attention mechanisms in deep learning models.
method Decomposed attention into a kernel and a normalization term; generalized the kernel function and norm.
result Generalized attention leads to better performance on various tasks.
New approach improves multi-head attention by making heads less similar.
problem Multi-head attention can lead to similar features, reducing model expressiveness.
method Proposes a non-parametric approach using Bayesian techniques to make heads repel each other.
result Improves feature diversity, leading to better representations and performance.
Regularizes attention scores in vision transformers using bootstrapping.
problem Noisy and diffused attention maps in ViT limit interpretability.
method Statistical learning techniques, bootstrapping of attention scores.
result Improves shrinkage and sparsity of attention scores.
Random forests with attention and self-attention improve regression performance.
problem Improving regression model performance on various datasets.
method Proposes new models using attention and self-attention mechanisms to solve regression problems.
result The models improve model performance on many datasets.
Aligns attention distributions for improved accuracy and robustness.
problem Improving the accuracy and robustness of neural networks using attention mechanisms.
method Alignment attention that encourages key and query distributions to match within each head.
result Alignment attention leads to better accuracy, uncertainty estimation, and robustness across various tasks.
A framework for transformer attention layers derived from SVR.
problem Developing principled attention mechanisms for transformers.
method Mapping self-attention to SVR, deriving new attention types.
result Improved transformer performance and efficiency.
Elliptical Attention improves transformer performance by focusing on contextually relevant features.
problem Transformer models suffer from representation collapse and are vulnerable to contaminated samples.
method Uses Mahalanobis distance to define hyper-ellipsoidal neighborhoods for attention weights.
result Elliptical Attention reduces representation collapse and enhances model robustness.
Survey categorizes attention models across various domains.
problem Understanding and improving attention models in neural networks.
method Taxonomy and review of existing techniques.
result Provides a structured overview of attention models.