dev-rcfa-memory/BE_IRCFA.md

be-ircfa Backend Architecture

This document explains the backend of the memory-enabled i-RCFA application in beginner-friendly terms.

The backend's shape

be-ircfa
│
├── run.py                  Local launcher; starts src.main:app
├── pyproject.toml          Python/Poetry project configuration
├── requirements/           Dependency groups
├── Dockerfile              Backend container definition
├── docker-compose.yaml     Local/container orchestration
├── docs/                   Design notes and reports
├── scratch/                Experiments and diagnostic scripts
├── tests/                  Automated and manual tests
├── workspace/              Workspace/tool files
└── src/                    Actual Python application package
    ├── main.py             Creates FastAPI and middleware
    ├── api.py              Connects route groups
    ├── config.py           Environment and service configuration
    ├── auth/               Checks who the user is
    ├── database/           Creates PostgreSQL sessions
    ├── ircfa_interaction/  Main chat and AI orchestration
    │   ├── router.py       HTTP endpoints
    │   ├── stream2.py      Main AI workflow
    │   ├── tools/          Sensor, PFI, reliability tools
    │   ├── llm/            Remote model adapters
    │   ├── lib/            Context and attachment helpers
    │   ├── system_templates/ Instructions and tool guides
    │   ├── model.py        Database table definitions
    │   ├── schema.py       Request/response shapes
    │   └── service.py      Reusable database/business operations
    ├── rag/                Document retrieval
    ├── root_cause/         Root-cause datasets and routes
    ├── failure_mode/       Failure-mode data and routes
    ├── equipments/         Equipment data and routes
    ├── fmea/               FMEA data and routes
    └── ircfa_alert/        Alert data and routes

An easy analogy:

• run.py is the key that starts the building.
• src/main.py is the building entrance.
• src/api.py is the directory showing where each department is.
• router.py is the receptionist.
• stream2.py is the supervisor.
• tools/ are specialist workers.
• llm/ is the phone connection to Gemma.
• model.py describes database tables.
• service.py contains reusable business actions.

1. The application starts in run.py

The normal local launcher is run.py. It starts Uvicorn and points it to the FastAPI application in src/main.py:

run.py -> src.main:app

The FastAPI application itself is created in src/main.py.

At startup it:

1. creates the FastAPI application;
2. creates database tables if needed;
3. loads model configurations;
4. installs error handling;
5. installs rate limiting;
6. installs database-session middleware;
7. registers all API routes.

The startup function is in main.py.

Your memory-copy backend runs on port 8005.

2. api.py connects the departments

api.py creates the main route collection.

It registers:

/chat
/rag
/equipments
/failure-mode
/fmea
/root-cause
/ircfa-alert

The chat router is connected under /chat and protected by a JWT dependency:

authenticated_api_router.include_router(
    rcfarouter,
    prefix="/chat",
    tags=["chat"],
)

3. Authentication happens before chat logic

Before the chat code runs, the backend checks the login token using auth/service.py.

The token identifies:

• which user is making the request;
• whether the token is valid;
• which private conversations and memories belong to that user.

The basic flow is:

Login token -> JWTBearer -> CurrentUser -> chat request

This is important for Memory Lab because the memory query uses the authenticated user's ID.

4. Database sessions are created for each request

The backend creates database sessions in middleware.

There are two database connections:

Main application database
  - conversations
  - chat history
  - memories
  - model configuration
  - application data

Collector database
  - collector/plant data

These connections are configured in database/core.py.

The request lifecycle is:

Request starts
  ↓
Open main DB session
  ↓
Open collector DB session
  ↓
Run endpoint
  ↓
Commit or rollback
  ↓
Close sessions

The database URLs come from environment variables in config.py:

• SQLALCHEMY_DATABASE_URI is the main application database.
• COLLECTOR_URI is the collector database.
• PFI_API_URL is the PFI HTTP API.
• RP_API_URL is the reliability HTTP API.

5. Database models and API schemas

Database table definitions are in ircfa_interaction/model.py.

Important tables include:

Conversation: rcfa_conversation

Stores the larger chat container:

• user ID;
• conversation ID;
• conversation title;
• RCFA phase;
• RCFA state.

Chat history: rcfa_chat_history

Stores individual messages and answers:

• user message;
• assistant response;
• timestamps;
• token usage;
• model name;
• location tag;
• references and context information.

Memory: rcfa_memory

Stores private long-term memories:

• user ID;
• conversation ID;
• memory summary;
• source type;
• equipment;
• location tag;
• metadata.

Model configuration: rcfa_model_config

Stores model connection details:

• model name;
• remote server URL;
• provider;
• API key;
• streaming setting;
• default model flag;
• generation settings.

API request and response shapes are in schema.py.

The difference is:

model.py  = database shape
schema.py = API message shape

6. router.py receives the chat request

The main streaming endpoint is in router.py:

POST /chat/stream

It:

1. reads the JSON request;
2. finds the last user message;
3. extracts attachments;
4. reads the conversation ID;
5. reads the location tag;
6. reads the selected model;
7. builds a ChatRequest;
8. calls stream_create_v2 from stream2.py.

The important handoff is:

router.py -> stream2.py

router.py mainly receives, validates, and forwards. Most AI orchestration happens elsewhere.

7. stream2.py is the main supervisor

The main orchestration function is stream2.py:

async def stream_create(...)

It coordinates:

• conversation history;
• RCFA state;
• memory retrieval;
• document retrieval;
• model configuration;
• tools;
• streaming;
• database saving.

The flow is:

Receive ChatRequest
  ↓
Find equipment/location
  ↓
Load recent conversation
  ↓
Load relevant memories
  ↓
Load document context
  ↓
Prepare system instructions
  ↓
Create model adapter
  ↓
Create tool list
  ↓
Run AI agent
  ↓
Execute tools when requested
  ↓
Stream answer
  ↓
Save chat and memory

8. Context is prepared before Gemma is called

Before Gemma receives the question, stream2.py builds the complete context. It can include:

System instructions
RCFA workflow instructions
Memory privacy instructions
Conversation history
RCFA state
Relevant memories
Document text
Equipment tag
Current question

Gemma therefore receives more than the user's one sentence. The backend prepares the model's working desk first.

9. Technical tools are controlled backend functions

Tools are located under ircfa_interaction/tools/.

Examples include:

current_sensor_tool.py
kondisi_sensor_historis.py
current_reliability_tool.py
history_reliability_tool.py
per_equipment_pfi_tool.py
next_failure_pfi_tool.py
root_cause_spln_tool.py
failure_mode_tool.py
plant_status_tool.py

Each tool generally:

1. receives arguments from the AI;
2. calls another API or reads data;
3. formats the result;
4. returns the result to the AI.

The tool loop looks like this:

User asks about sensor data
  ↓
Gemma decides it needs the sensor tool
  ↓
i-RCFA backend runs CurrentSensorTool
  ↓
The tool calls the PFI API
  ↓
The tool returns sensor data
  ↓
Gemma explains the result

Gemma does not freely open database connections. It requests a tool, and the backend decides what code runs.

10. PFI integration is usually an API call

The backend configuration contains PFI_API_URL and RP_API_URL in config.py.

For PFI sensor information, the normal relationship is:

Gemma
  ↓
i-RCFA PFI tool
  ↓
PFI HTTP API
  ↓
PFI data source

The i-RCFA backend does not normally query PFI database tables directly.

11. The remote model adapter

The remote model adapter is gemma_llamacpp.py.

It translates the application's internal request into an OpenAI-compatible request:

{
  "model": "...",
  "messages": [],
  "tools": [],
  "tool_choice": "auto",
  "stream": true
}

The remote completion URL is built from the configured model server:

https://gemma.digitaltwin.my.id/v1/chat/completions

The adapter also:

• adds authentication headers;
• converts messages into OpenAI-compatible format;
• includes tools;
• handles streaming;
• reads tool-call fragments;
• handles multimodal image data;
• logs remote errors.

12. The agent loop

Around the middle of stream2.py, the backend creates a qwen_agent assistant with:

• a system message;
• the selected LLM adapter;
• the allowed tool list.

Conceptually:

bot = Assistant(
    system_message=system_prompt,
    llm=llm,
    function_list=tools,
)

Then it runs the assistant with the prepared messages.

One request can contain multiple rounds:

Round 1: Gemma requests a tool
Round 2: backend executes the tool and returns the result
Round 3: Gemma produces the final answer

13. What happens when Gemma calls a tool

Inside the streaming loop, stream2.py handles:

• assistant messages;
• function/tool messages;
• reasoning content;
• final answer content.

When it sees a tool result, it can update RCFA state.

For example:

PFI tool result
  -> save failure prediction state
  -> move RCFA phase forward

Current sensor tool result
  -> save sensor state
  -> move RCFA phase forward

Root-cause tool result
  -> save diagnosis state

This is why the UI can show RCFA progress such as:

1. Failure Prediction
2. Sensor Data Mapping
3. Diagnosis

The backend is not only generating text. It is also maintaining an investigation state.

14. Streaming uses Server-Sent Events

stream2.py yields event strings such as:

data: answer text

event: phase
data: 2. Sensor Data Mapping

event: usage_metadata
data: {...}

event: error
data: {...}

The full path is:

Gemma token
  ↓
Gemma adapter
  ↓
stream2.py
  ↓
StreamingResponse
  ↓
nginx
  ↓
React chat screen

This is why the answer appears gradually.

15. Saving happens after the answer

After the agent finishes, the backend creates a chat-history record.

It saves things such as:

• user message;
• assistant response;
• timestamps;
• model;
• token counts;
• RCFA state;
• references;
• location tag.

Then it calls the memory-save logic if the user explicitly asked to remember something.

The normal save sequence is:

Save chat history
  ↓
Check/save explicit memory
  ↓
Commit transaction

If saving fails, the backend rolls back and logs the error.

16. Memory is a separate backend layer

Memory retrieval happens before Gemma is called:

Database -> memory filter -> relevant memories -> prompt

Memory saving happens after the answer:

Explicit user request
  -> policy check
  -> lifecycle check
  -> database

The memory privacy rules are in memory_policy.py.

The memory lifecycle rules are in memory_lifecycle.py.

The important separation is:

Gemma decides how to answer.
Backend decides what memory is allowed.
Database stores the memory.

17. RAG is another AI-related path

The backend contains a RAG system under rag/.

RAG means:

Retrieve useful documents
  ↓
Give those documents to the AI
  ↓
Ask the AI to answer using them

This is different from text generation:

Generation:
  Gemma writes the answer.

Retrieval:
  Embeddings/vector search find relevant documents.

The text-generation model can work while document retrieval has a separate problem.

18. Legacy model paths

Some older classes in model.py still define Ollama or vLLM-based configurations for older workflows and auxiliary classifiers.

That does not mean the normal i-RCFA chat has switched to local Ollama.

The active chat path is controlled by the model configuration in rcfa_model_config and the selected configuration passed into stream2.py.

Think of it as:

Main chat path:
  router.py -> stream2.py -> configured remote model adapter

Other/legacy paths:
  may contain older Ollama, vLLM, or auxiliary configurations

19. Complete example

Suppose the user asks:

What is the possible root cause of high vibration on 3DP-FDR711F?

The backend flow is:

1. JWTBearer identifies the user.

2. router.py receives /chat/stream.

3. router.py creates ChatRequest.

4. stream2.py loads:
   - recent chat history;
   - private memories;
   - RCFA state;
   - location tag 3DP-FDR711F.

5. stream2.py creates the available tool list.

6. Gemma receives the question and tools.

7. Gemma requests current sensor data.

8. CurrentSensorTool calls the PFI API.

9. The sensor result returns to Gemma.

10. Gemma may request root-cause information.

11. RootCauseSPLNTool searches the root-cause data.

12. Gemma writes the explanation.

13. stream2.py sends answer pieces to the browser.

14. The backend updates RCFA state.

15. The backend saves chat history.

16. If the user explicitly asked to remember something, memory logic handles it.

The most important relationship

router.py
  -> stream2.py
      -> tools/
      -> llm/gemma_llamacpp.py
      -> database models/services
      -> StreamingResponse

That is the backbone of the backend.

Recommended learning order

Read the backend in this order:

1. api.py — where routes are connected.
2. router.py — how HTTP chat requests enter.
3. stream2.py — the main AI workflow.
4. One simple tool, such as current_sensor_tool.py.
5. gemma_llamacpp.py — how the backend talks to Gemma.
6. model.py — how chat, memory, and model settings are stored.
7. database/core.py — how database connections are created.

20. Files not directly involved in the running backend

The normal backend started with python run.py follows this path:

run.py -> src/main.py -> src/api.py -> route modules -> stream2.py -> tools / database / remote Gemma

The items below are not directly used by that normal running path. This does not automatically mean they are safe to delete: some are useful for maintenance, testing, or an older deployment approach.

Item	Current role	Keep or remove?
be-ircfa/src/metrics.py	Old metrics code. The entire file and its import in main.py are commented out.	A clear cleanup candidate after a normal review.
be-ircfa/src/ircfa_interaction/schema.py.orig and router.py.orig	Editor backup copies; Python does not load .orig files.	Review their differences, then they can normally be removed.
be-ircfa/scratch/	Manual experiments, benchmarks, and diagnostic scripts.	Keep for learning/debugging; it is not part of the live backend.
be-ircfa/docs/, .vscode/, and tests/	Documentation, editor settings, and verification files.	Keep; they do not run as part of the backend service.
be-ircfa/docker-compose.yaml, docker-kong.yml, kong.yml, config.yaml, docker_build_push.sh, and Jenkinsfile	Older container, Kong, local-model, build, and CI deployment material.	Not used by the current manual port-8005 setup. Keep until the old deployment is formally retired.
be-ircfa/requirements/requirements-full.txt, requirements-heavy.txt, and requirements-light.txt	Alternative dependency lists with no repository references. Docker uses requirements.txt and requirements/requirements-base.txt instead.	Archive or remove only after confirming no external deployment uses them.
be-ircfa/src/root_cause/sync.py and cleanup_data.py	Manual database loading and cleanup scripts. They are not called when the app starts.	Keep if you may need to rebuild or clean root-cause data.
be-ircfa/src/root_cause/*.json and fmea_data/*.tsv	Source datasets used by manual data-loading work, not read during ordinary chat requests.	Do not remove unless the data is safely stored elsewhere and no future rebuild is needed.
be-ircfa/src/ircfa_interaction/stream.py	Older chat-orchestration implementation. All live chat endpoints call stream2.py. router.py still imports it, however.	Do not remove it alone; first remove the stale import and test the chat flow.
be-ircfa/src/ircfa_interaction/llm/n8n.py	An optional N8N LLM adapter. It is not selected by the current remote Gemma configuration.	Keep as a conditional fallback unless you decide to retire N8N support.

In simple terms: stream2.py is the current engine. The other items above are mostly old tools, spare parts, instruction books, or data boxes. They do not make the normal chat answer a user request today.

Important caution

The legacy Docker and deployment files should be treated as sensitive configuration. Before sharing, committing, or reusing them, move any credentials and connection values into protected environment variables.

21. Visual code maps

These diagrams show the important driver files, rather than every small helper or data class. Solid arrows show the normal running path. Dashed arrows show optional, legacy, or conditional paths.

21.1 Folder map: startup and HTTP routes

flowchart TD
    subgraph ROOT[be-ircfa root]
        RUN[run.py]
        ENV[Environment variables / .env]
    end

    subgraph CORE[src/ core files]
        MAIN[src/main.py]
        API[src/api.py]
        CFG[src/config.py]
        MID[src/middleware.py]
        LOG[src/logging.py]
    end

    subgraph AUTH[src/auth/]
        JWT[service.py: JWTBearer / CurrentUser]
        AUTHMODEL[model.py]
    end

    subgraph INTERACTION[src/ircfa_interaction/]
        CHATROUTER[router.py]
        STREAM2[stream2.py]
    end

    subgraph DOMAINS[Other routed feature folders]
        RAGROUTER[src/rag/router.py]
        ALERTROUTER[src/ircfa_alert/router.py]
        EQUIPROUTER[src/equipments/router.py]
        FMROUTER[src/failure_mode/router.py]
        FMEAROUTER[src/fmea/router.py]
        RCROUTER[src/root_cause/router.py]
    end

    RUN --> MAIN
    ENV --> CFG
    CFG --> MAIN
    MAIN --> LOG
    MAIN --> MID
    MAIN --> API
    API --> JWT
    API --> CHATROUTER
    API --> RAGROUTER
    API --> ALERTROUTER
    API --> EQUIPROUTER
    API --> FMROUTER
    API --> FMEAROUTER
    API --> RCROUTER
    CHATROUTER --> STREAM2

What this means: run.py turns on FastAPI. main.py prepares the application. api.py is the receptionist that sends each URL request to the correct folder. The chat requests go to ircfa_interaction/router.py, then to stream2.py.

21.2 Folder map: the i-RCFA interaction engine

flowchart LR
    subgraph INTERACTION[src/ircfa_interaction/]
        ROUTER[router.py\nchat, conversations, Memory Lab]
        STREAM[stream2.py\nmain orchestration]
        SCHEMA[schema.py\nrequest/response shapes]
        SERVICE[service.py\nchat/model persistence]
        MODEL[model.py\nchat, memory, LLM tables]
        MEMORY[memory_policy.py + memory_lifecycle.py]
        LEGACY[stream.py\nolder orchestration]
    end

    subgraph LLM[src/ircfa_interaction/llm/]
        GEMMA[gemma_llamacpp.py\nOpenAI-compatible Gemma adapter]
        N8N[n8n.py\noptional adapter]
    end

    subgraph TOOLS[src/ircfa_interaction/tools/]
        SENSOR[Sensor / PFI tools]
        RELIABILITY[Reliability tools]
        ROOTCAUSE[RootCauseSPLNTool]
        VECTOR[VectorSearchTool]
        OTHERTOOLS[Other RCFA tools]
    end

    subgraph SUPPORT[Supporting folders]
        RAG[src/rag/\nfiles, vector search, RAG history]
        DB[src/database/\nmain and collector sessions]
        EQUIP[src/equipments/]
        ROOTDATA[src/root_cause/]
    end

    ROUTER --> SCHEMA
    ROUTER --> STREAM
    ROUTER --> SERVICE
    STREAM --> MEMORY
    MEMORY --> MODEL
    STREAM --> MODEL
    SERVICE --> MODEL
    STREAM --> GEMMA
    STREAM -. optional configured path .-> N8N
    STREAM --> SENSOR
    STREAM --> RELIABILITY
    STREAM --> ROOTCAUSE
    STREAM --> VECTOR
    STREAM --> OTHERTOOLS
    STREAM --> RAG
    SERVICE --> DB
    MODEL --> DB
    SENSOR --> EQUIP
    ROOTCAUSE --> ROOTDATA
    VECTOR --> RAG
    ROUTER -. stale import; no active endpoint call .-> LEGACY

What this means: stream2.py is the conductor. It gathers chat context, private memory, RCFA data, attachments, and available tools; it then asks the configured LLM to answer. The LLM can request a tool, and stream2.py returns that tool result to the LLM before streaming the final response back to the browser.

21.3 Cross-folder runtime map: one chat request

sequenceDiagram
    participant FE as Frontend / nginx
    participant MAIN as src/main.py
    participant API as src/api.py
    participant AUTH as src/auth/service.py
    participant ROUTER as ircfa_interaction/router.py
    participant STREAM as ircfa_interaction/stream2.py
    participant DB as database/ + PostgreSQL
    participant TOOLS as tools/ + domain services
    participant EXT as PFI / Reliability APIs
    participant LLM as Remote Gemma

    FE->>MAIN: POST /ircfa/chat/stream
    MAIN->>API: send request to registered router
    API->>AUTH: verify JWT and identify user
    AUTH-->>ROUTER: authenticated user
    ROUTER->>STREAM: ChatRequest + database sessions
    STREAM->>DB: load chat, RCFA state, model config, private memories
    DB-->>STREAM: saved context
    STREAM->>LLM: question + system instructions + available tools
    opt LLM requests a technical tool
        LLM-->>STREAM: tool request
        STREAM->>TOOLS: run selected tool
        TOOLS->>EXT: request sensor, PFI, or reliability data
        EXT-->>TOOLS: technical data
        TOOLS-->>STREAM: tool result
        STREAM->>LLM: tool result
    end
    LLM-->>STREAM: streamed answer pieces
    STREAM->>DB: save chat, RCFA state, and explicit memory
    STREAM-->>FE: streamed answer events

This third diagram is deliberately cross-folder: a real answer passes through authentication, the chat router, orchestration, database storage, tools, external operational APIs, and the remote Gemma service.

21.4 Folder responsibilities at a glance

flowchart TB
    API[src/api.py\nconnects routes] --> AUTH[src/auth/\nwho is this user?]
    API --> CHAT[src/ircfa_interaction/\nchat, memory, LLM orchestration]
    API --> RAG[src/rag/\ndocuments and retrieval]
    API --> EQUIP[src/equipments/\nequipment and maintenance data]
    API --> FM[src/failure_mode/\nfailure-mode data]
    API --> FMEA[src/fmea/\nFMEA records]
    API --> RC[src/root_cause/\nroot-cause matrix data]
    API --> ALERT[src/ircfa_alert/\nalerts]
    CHAT --> DATABASE[src/database/\nPostgreSQL sessions]
    RAG --> DATABASE
    EQUIP --> DATABASE
    FM --> DATABASE
    FMEA --> DATABASE
    RC --> DATABASE

Use the diagrams together:

1. 21.1 answers: “Which folder receives my request?”
2. 21.2 answers: “What happens inside the AI/chat engine?”
3. 21.3 answers: “How do folders and outside services cooperate for one answer?”
4. 21.4 answers: “What is each feature folder responsible for?”