The paper improves neural network predictions by integrating process knowledge.
problem Improving neural network predictions for process execution data.
method Integrates background process knowledge into neural networks with attention mechanisms.
result Improves prediction accuracy for process execution data.
The paper introduces a knowledge score for GPR predictions to assess their reliability.
problem Uncertainty in probabilistic predictions from Gaussian process regression models.
method A knowledge score quantifying the reduction of uncertainty in GPR predictions.
result The knowledge score improves prediction accuracy in various tasks.
The paper improves Gaussian processes by adding sum constraints, enhancing prediction accuracy.
problem Improving Gaussian process predictions with background knowledge constraints.
method Conditioning the prior distribution on sum constraints to ensure fulfillment of linear and nonlinear constraints.
result The approach fulfills constraints with high precision and improves prediction accuracy.
Machine learning has become pervasive in multiple domains, impacting a wide variety of applications, such as knowledge discovery and data mining, natural language processing, information retrieval, computer vision, social and health informatics, ubiquitous computing, etc. Two essential problems of machine learning are …
Improves transparency and incorporates prior knowledge in Gaussian Process models.
problem Challenges in understanding and expressing prior assumptions in complex Bayesian models.
method Introduces self-explaining variational posterior distributions for Gaussian Processes.
result Allows incorporation of both general and feature-specific prior knowledge.
Two approaches extend knowledge distillation to Gaussian Processes, showing relationships to existing methods.
problem Applying knowledge distillation to Gaussian Processes for regression and classification.
method Data-centric and distribution-centric approaches to extend distillation to GPR and GPC.
result Distribution-centric approach for GPC approximately corresponds to data duplication and scaling.
New framework uses background knowledge to speed up causal discovery.
problem Scalable causal discovery for large datasets.
method Utilizes background knowledge during causal discovery process.
result Background knowledge reduces computational requirements and improves structure quality.
KD2M unifies feature matching in neural networks.
problem Matching neural network activations for knowledge transfer.
method Formalizes distribution matching strategy.
result New theoretical results for KD.
Bayesian approach for solving systems of linear PDEs with boundary conditions.
problem Modeling data efficiently with prior knowledge from systems of linear PDEs.
method Construct multi-output Gaussian process priors using Gröbner bases and pullback parametrizations.
result Gaussian process priors can represent solutions to systems of linear PDEs adhering to boundary conditions.
Proposes using entity embedding vectors to improve Gaussian Process models for knowledge transfer across cell lines.
problem Lack of reuse of experimental data for predicting novel processes.
method Hybrid Gaussian Process models with entity embedding vectors to represent product identity.
result Improved performance in predicting novel processes compared to traditional methods.
Paper proposes AI for stock market forecasting using external knowledge.
problem Forecasting stock prices influenced by external factors.
method Learning from historical data and external temporal knowledge graphs modeled as Hawkes processes.
result Dynamic representations effectively rank stocks based on returns.
Advanced kernels improve Gaussian process accuracy by incorporating domain knowledge.
problem Improving function approximation accuracy in Gaussian processes.
method Advanced kernel designs that enforce specific function properties (symmetry, periodicity) and non-stationarity.
result Advanced kernels significantly enhance function approximation accuracy and relevance.
This paper improves Gaussian process predictions by integrating prior knowledge.
problem Gaussian processes lack predictive power when prior information is ignored.
method Derive mean and covariance functions from previous data using weighted sums of basis functions.
result Integrating prior knowledge significantly increases look-ahead time and accuracy.
It is very useful to integrate human knowledge and experience into traditional neural networks for faster learning speed, fewer training samples and better interpretability. However, due to the obscured and indescribable black box model of neural networks, it is very difficult to design its architecture, interpret its …
In complex transfer learning scenarios new tasks might not be tightly linked to previous tasks. Approaches that transfer information contained only in the final parameters of a source model will therefore struggle. Instead, transfer learning at a higher level of abstraction is needed. We propose Leap, a framework that …
New method combines FMEA and Bayesian Network for root cause analysis in lithium-ion battery production.
problem Complex cause-effect relationships in lithium-ion battery production.
method Combining FMEA with Bayesian Network to detect and resolve inconsistencies.
result Holistic method builds large-scale cross-process Bayesian Failure Network for root cause analysis.
Developing new ideas and algorithms in the fields of graph processing and relational learning requires public datasets. While Wikidata is the largest open source knowledge graph, involving more than fifty million entities, it is larger than needed in many cases and even too large to be processed easily. Still, it is a …
Gryffin optimizes categorical variables in materials design, leveraging expert knowledge.
problem Optimizing categorical variables in complex design choices like molecule selection.
method Bayesian optimization with smooth approximations to categorical distributions, incorporating expert knowledge.
result Gryffin accelerates discovery of promising molecules and materials, highlighting relevant correlations.
ATL learns from many streaming processes without labeled data.
problem Knowledge transfer across many streaming processes with covariate shift and drifts.
method Autonomous transfer learning with generative and discriminative phases, KL divergence optimization, and elastic network structure.
result Improved performance and faster training speed compared to existing methods.
Knowledge graph construction consists of two tasks: extracting information from external resources (knowledge population) and inferring missing information through a statistical analysis on the extracted information (knowledge completion). In many cases, insufficient external resources in the knowledge population hinde…
Since deep learning became a key player in natural language processing (NLP), many deep learning models have been showing remarkable performances in a variety of NLP tasks, and in some cases, they are even outperforming humans. Such high performance can be explained by efficient knowledge representation of deep learnin…
Gradient boosting can be seen as Gaussian process inference.
problem Improving uncertainty estimates in out-of-domain detection.
method Gradient boosting reformulated as a kernel method converging to Gaussian process inference.
result Gradient boosting can provide better uncertainty estimates through Monte-Carlo estimation of posterior variance.
Develops a simulation-based method to translate expert knowledge into prior distributions for Bayesian models.
problem Effective incorporation of expert knowledge into prior distributions for diverse model structures.
method Simulation-based stochastic gradient descent to learn hyperparameters of parametric priors from expert knowledge.
result Method is adaptable to various elicitation techniques and independent of model structure.
Novel framework for learning infinitesimal generator of stochastic processes.
problem Challenges in learning infinitesimal generator due to unbounded nature and state space dimensionality.
method Introduces a novel framework based on energy functional, integrates physical priors, and uses reduced-rank estimator in RKHS.
result Learning bounds independent of state space dimension and non-spurious spectral estimation.
Recent variants improve knowledge distillation performance.
problem Improving the performance of knowledge distillation.
method Introducing additional components or changing the learning process.
result These variants have shown promising results.
It is always demanding to learn robust visual representation for various learning problems; however, this learning and maintenance process usually suffers from noise, incompleteness or knowledge domain mismatch. Thus, robust representation learning by removing noisy features or samples, complementing incomplete data, a…
NLP techniques improve drug discovery by analyzing chemical and protein text.
problem Improving drug discovery through better analysis of chemical and protein text.
method Natural language processing techniques applied to biochemical entities.
result Enhanced prediction of molecular properties and design of novel molecules.
Unified framework detects changes in complex system models.
problem Accurate identification of dynamic changes in simulation models.
method Combines machine learning and process-driven simulation modeling.
result Significantly improves change point detection accuracy.
Recent research in psycholinguistics has provided increasing evidence that humans predict upcoming content. Prediction also affects perception and might be a key to robustness in human language processing. In this paper, we investigate the factors that affect human prediction by building a computational model that can …
New algorithm improves knowledge transfer in dynamic decision-making.
problem Utilizing data from existing ventures to improve decision-making in new ventures.
method Proposes Transferred Fitted Q-Iteration algorithm for estimating optimal action-state function Q∗. result Significantly improved final learning error of Q∗ function. Language models can answer questions without external knowledge.
problem How much knowledge can be stored in language models?
method Fine-tuning pre-trained language models to answer questions without external context.
result Fine-tuned models perform competitively with open-domain systems.
Bayesian neural networks improve with summary information and Dirichlet process.
problem Lack of prior knowledge in BNNs for complex architectures.
method Incorporates external summary information about predicted probabilities using a Dirichlet process.
result Improves model accuracy, uncertainty calibration, and robustness.
Model uses LLMs to process numerical data guided by natural language descriptions.
problem Challenges in integrating prior knowledge into probabilistic models.
method Developed LLM Processes to condition numerical predictive distributions on natural language.
result Improved predictive performance and structured qualitative descriptions.
Enhances model compression with multi-teacher knowledge distillation.
problem Uncertainty evaluation and diverse teacher expertise in model deployment.
method Bayesian inference and teacher-informed prior with entropy-based weighting.
result Improved predictive accuracy and robust uncertainty quantification.
Adaptive learning, also known as adaptive teaching, relies on learning path recommendation, which sequentially recommends personalized learning items (e.g., lectures, exercises) to satisfy the unique needs of each learner. Although it is well known that modeling the cognitive structure including knowledge level of lear…
This paper explores the connection between adversarial and knowledge transferability.
problem Understanding the factors affecting knowledge transferability.
method Theoretical analysis and practical metrics for adversarial transferability.
result Adversarial transferability and knowledge transferability are closely related.
We study the use of knowledge distillation to compress the U-net architecture. We show that, while standard distillation is not sufficient to reliably train a compressed U-net, introducing other regularization methods, such as batch normalization and class re-weighting, in knowledge distillation significantly improves …
PhyDNN uses physics knowledge to improve drag force prediction models.
problem Complex physical processes in fluid dynamics are hard to model accurately.
method Physics-guided structural priors and aggregate supervision for deep learning.
result PhyDNN achieves a significant 8.46% improvement in drag force prediction.
This paper discovers classification models from sequential data without prior knowledge.
problem Lack of prior knowledge in defining kernels for online classification.
method Adapts GP-based time-series structure discovery with SMC to learn new features from sequential data.
result Improves classification accuracy by 10% on real-world data.
Optimizes expensive experiments by incorporating expert knowledge.
problem Expensive experiments require minimizing the number of trials.
method Bayesian optimization with posterior sampling of expert knowledge.
result Demonstrates significant efficiency gains in experiments and hyperparameter tuning.
Proposes a Gaussian process model for constrained dynamics learning.
problem Challenges in identifying constrained dynamics of mechanical systems.
method Combines analytical mechanics with Gaussian process regression.
result Improves data efficiency and constraint integrity in predictions.
Incorporating domain knowledge into the modeling process is an effective way to improve learning accuracy. However, as it is provided by humans, domain knowledge can only be specified with some degree of uncertainty. We propose to explicitly model such uncertainty through probabilistic constraints over the parameter sp…
Improved meta-learning for dynamics using additional structured knowledge.
problem Meta-learning for dynamics with limited raw observations.
method Extended Neural ODE Process model to use privileged information.
result Improved accuracy and calibration on simulated dynamics tasks.
Knowledge bases are employed in a variety of applications from natural language processing to semantic web search; alas, in practice their usefulness is hurt by their incompleteness. Embedding models attain state-of-the-art accuracy in knowledge base completion, but their predictions are notoriously hard to interpret. …
Deep network compression has been achieved notable progress via knowledge distillation, where a teacher-student learning manner is adopted by using predetermined loss. Recently, more focuses have been transferred to employ the adversarial training to minimize the discrepancy between distributions of output from two net…
A model-based approach to forecasting chaotic dynamical systems utilizes knowledge of the physical processes governing the dynamics to build an approximate mathematical model of the system. In contrast, machine learning techniques have demonstrated promising results for forecasting chaotic systems purely from past time…
A new model improves the performance of knowledge distillation in GNNs.
problem Inconsistent performance of existing knowledge distillation techniques in GNNs.
method Proposes a new model, Routing-by-Memory (RbM), a form of Mixture-of-Experts (MoE), to address performance concerns.
result Demonstrates experimentally that RbM achieves considerably more consistent performance across multiple datasets.
GAttNHP predicts future events in temporal knowledge graphs by encoding long-range dependencies and handling mutual excitation.
problem Forecasting future events in temporal knowledge graphs due to long-range dependencies, mutual excitation, and heavy-tailed inter-arrival times.
method GAttNHP uses a self-attention encoder, semantic soft-grouping, and NCQ regression to address these issues.
result GAttNHP improves entity and time prediction on six benchmark TKG datasets compared to state-of-the-art baselines.