Constructing Dynamic Master Logic Models as Knowledge Graphs for Complex System Diagnostics Using Retrieval-Augmented Large Language Models

Overview§

This paper presents a framework for automatically constructing Dynamic Master Logic (DML) models as Knowledge Graphs (KG-DML) from unstructured technical documentation. It leverages Retrieval-Augmented Generation (RAG) and Large Language Models (LLMs) to extract hierarchical functional and structural relationships, scaling from small systems to a decommissioned Boiling Water Reactor's Low-Pressure Coolant Injection (LPCI) system. The resulting KG enables upward failure propagation, downward dependency tracing, and diagnostic reasoning.

Methodology§

RAG-based Hierarchical Extraction§

The construction process iteratively builds the DML hierarchy top-down. For each functional objective node, the system uses a retriever (e.g., dense passage retrieval) to fetch relevant document chunk–targeted queries. The LLM then extracts child nodes and dependency links, along with logical gate types (AND/OR). The prompt instructs the LLM to output structured JSON, which is parsed and inserted into a graph database (e.g., Neo4j). To ensure consistency, the framework performs multiple RAG passes with overlapping queries.

Logical Gate Representation§

DML models capture how lower-level system elements combine to satisfy higher-level functions. Each parent-child relationship is associated with a gate:

• AND gate: all children must be satisfied.
• OR gate: at least one child suffices.

In the knowledge graph, these are represented as edge attributes or intermediate gate nodes. The LLM is prompted to explicitly assign gate logic, and a consistency check rejects contradictions (e.g., a node with both AND and OR to the same child).

Validation Strategy§

The authors propose a multi-level validation:

• Layer-specific precision/recall: comparing extracted nodes and edges against a manually constructed ground truth DML.

$$\text{Precision} = \frac{|\text{Correct Extracted}|}{|\text{All Extracted}|}, \quad \text{Recall} = \frac{|\text{Correct Extracted}|}{|\text{Ground Truth}|}$$

• Gate consistency: verifying that logical operators match the descriptions in the source documents.
• Structural integrity: checking that the graph is acyclic and every leaf is linked to a valid physical component.

Implementation Sketch§

The following PyTorch-like pseudocode illustrates the RAG-LLM extraction step using a transformer-based retriever and a generative model (e.g., GPT-4). It shows how contextual retrieval and structured output parsing are integrated.

import torch
from transformers import AutoTokenizer, AutoModel

class RAGDMLBuilder:
    def __init__(self, retriever_model, llm):
        self.retriever_tokenizer = AutoTokenizer.from_pretrained(retriever_model)
        self.retriever = AutoModel.from_pretrained(retriever_model)
        self.llm = llm  # e.g., OpenAI GPT-4 wrapper

    def retrieve(self, query, docs, top_k=5):
        # Encode query and docs
        q_emb = self.retriever(**self.retriever_tokenizer(query, return_tensors="pt")).last_hidden_state.mean(dim=1)
        doc_embs = torch.stack([
            self.retriever(**self.retriever_tokenizer(d, return_tensors="pt")).last_hidden_state.mean(dim=1)
            for d in docs
        ])
        scores = torch.nn.functional.cosine_similarity(q_emb, doc_embs)
        top_indices = scores.topk(top_k).indices
        return [docs[i] for i in top_indices]

    def extract_children(self, parent_node, docs):
        contexts = self.retrieve(parent_node, docs)
        prompt = f"From the context, list child elements for {parent_node}. Return JSON with 'children' and 'gate' (AND/OR)."
        response = self.llm(prompt, contexts)
        return response["children"], response["gate"]

# Usage
builder = RAGDMLBuilder("sentence-transformers/all-MiniLM-L6-v2", llm)
children, gate = builder.extract_children("Coolant Injection", tech_docs)

Results and Impact§

The LPCI system application showed consistent KG construction across repeated runs with high precision/recall and logical gate accuracy. This demonstrates that LLM+RAG pipelines can transform technical manual text into executable functional models for reliability analysis, potentially reducing manual modeling effort in safety-critical domains.