add markdown files

2 months ago · 38ede967b8
parent b9e2378d39
commit 38ede967b8
10 changed files with 2053 additions and 67 deletions
--- a/BE_IRCFA.md
+++ b/BE_IRCFA.md
@ -0,0 +1,897 @@
 # `be-ircfa` Backend Architecture
 This document explains the backend of the memory-enabled i-RCFA application in beginner-friendly terms.
 ## The backend's shape
 ```text
 be-ircfa
 │
 ├── run.py                  Local launcher; starts src.main:app
 ├── pyproject.toml          Python/Poetry project configuration
 ├── requirements/           Dependency groups
 ├── Dockerfile              Backend container definition
 ├── docker-compose.yaml     Local/container orchestration
 ├── docs/                   Design notes and reports
 ├── scratch/                Experiments and diagnostic scripts
 ├── tests/                  Automated and manual tests
 ├── workspace/              Workspace/tool files
 └── src/                    Actual Python application package
    ├── main.py             Creates FastAPI and middleware
    ├── api.py              Connects route groups
    ├── config.py           Environment and service configuration
    ├── auth/               Checks who the user is
    ├── database/           Creates PostgreSQL sessions
    ├── ircfa_interaction/  Main chat and AI orchestration
    │   ├── router.py       HTTP endpoints
    │   ├── stream2.py      Main AI workflow
    │   ├── tools/          Sensor, PFI, reliability tools
    │   ├── llm/            Remote model adapters
    │   ├── lib/            Context and attachment helpers
    │   ├── system_templates/ Instructions and tool guides
    │   ├── model.py        Database table definitions
    │   ├── schema.py       Request/response shapes
    │   └── service.py      Reusable database/business operations
    ├── rag/                Document retrieval
    ├── root_cause/         Root-cause datasets and routes
    ├── failure_mode/       Failure-mode data and routes
    ├── equipments/         Equipment data and routes
    ├── fmea/               FMEA data and routes
    └── ircfa_alert/        Alert data and routes
 ```
 An easy analogy:
 - `run.py` is the key that starts the building.
 - `src/main.py` is the building entrance.
 - `src/api.py` is the directory showing where each department is.
 - `router.py` is the receptionist.
 - `stream2.py` is the supervisor.
 - `tools/` are specialist workers.
 - `llm/` is the phone connection to Gemma.
 - `model.py` describes database tables.
 - `service.py` contains reusable business actions.
 ## 1. The application starts in `run.py`
 The normal local launcher is [`run.py`](be-ircfa/run.py). It starts Uvicorn and points it to the FastAPI application in [`src/main.py`](be-ircfa/src/main.py):
 ```text
 run.py -> src.main:app
 ```
 The FastAPI application itself is created in [`src/main.py`](be-ircfa/src/main.py).
 At startup it:
 1. creates the FastAPI application;
 2. creates database tables if needed;
 3. loads model configurations;
 4. installs error handling;
 5. installs rate limiting;
 6. installs database-session middleware;
 7. registers all API routes.
 The startup function is in [`main.py`](be-ircfa/src/main.py:52).
 Your memory-copy backend runs on port `8005`.
 ## 2. `api.py` connects the departments
 [`api.py`](be-ircfa/src/api.py) creates the main route collection.
 It registers:
 ```text
 /chat
 /rag
 /equipments
 /failure-mode
 /fmea
 /root-cause
 /ircfa-alert
 ```
 The chat router is connected under `/chat` and protected by a JWT dependency:
 ```python
 authenticated_api_router.include_router(
    rcfarouter,
    prefix="/chat",
    tags=["chat"],
 )
 ```
 ## 3. Authentication happens before chat logic
 Before the chat code runs, the backend checks the login token using [`auth/service.py`](be-ircfa/src/auth/service.py).
 The token identifies:
 - which user is making the request;
 - whether the token is valid;
 - which private conversations and memories belong to that user.
 The basic flow is:
 ```text
 Login token -> JWTBearer -> CurrentUser -> chat request
 ```
 This is important for Memory Lab because the memory query uses the authenticated user's ID.
 ## 4. Database sessions are created for each request
 The backend creates database sessions in middleware.
 There are two database connections:
 ```text
 Main application database
  - conversations
  - chat history
  - memories
  - model configuration
  - application data
 Collector database
  - collector/plant data
 ```
 These connections are configured in [`database/core.py`](be-ircfa/src/database/core.py).
 The request lifecycle is:
 ```text
 Request starts
  ↓
 Open main DB session
  ↓
 Open collector DB session
  ↓
 Run endpoint
  ↓
 Commit or rollback
  ↓
 Close sessions
 ```
 The database URLs come from environment variables in [`config.py`](be-ircfa/src/config.py):
 - `SQLALCHEMY_DATABASE_URI` is the main application database.
 - `COLLECTOR_URI` is the collector database.
 - `PFI_API_URL` is the PFI HTTP API.
 - `RP_API_URL` is the reliability HTTP API.
 ## 5. Database models and API schemas
 Database table definitions are in [`ircfa_interaction/model.py`](be-ircfa/src/ircfa_interaction/model.py).
 Important tables include:
 ### Conversation: `rcfa_conversation`
 Stores the larger chat container:
 - user ID;
 - conversation ID;
 - conversation title;
 - RCFA phase;
 - RCFA state.
 ### Chat history: `rcfa_chat_history`
 Stores individual messages and answers:
 - user message;
 - assistant response;
 - timestamps;
 - token usage;
 - model name;
 - location tag;
 - references and context information.
 ### Memory: `rcfa_memory`
 Stores private long-term memories:
 - user ID;
 - conversation ID;
 - memory summary;
 - source type;
 - equipment;
 - location tag;
 - metadata.
 ### Model configuration: `rcfa_model_config`
 Stores model connection details:
 - model name;
 - remote server URL;
 - provider;
 - API key;
 - streaming setting;
 - default model flag;
 - generation settings.
 API request and response shapes are in [`schema.py`](be-ircfa/src/ircfa_interaction/schema.py).
 The difference is:
 ```text
 model.py  = database shape
 schema.py = API message shape
 ```
 ## 6. `router.py` receives the chat request
 The main streaming endpoint is in [`router.py`](be-ircfa/src/ircfa_interaction/router.py):
 ```text
 POST /chat/stream
 ```
 It:
 1. reads the JSON request;
 2. finds the last user message;
 3. extracts attachments;
 4. reads the conversation ID;
 5. reads the location tag;
 6. reads the selected model;
 7. builds a `ChatRequest`;
 8. calls `stream_create_v2` from `stream2.py`.
 The important handoff is:
 ```text
 router.py -> stream2.py
 ```
 `router.py` mainly receives, validates, and forwards. Most AI orchestration happens elsewhere.
 ## 7. `stream2.py` is the main supervisor
 The main orchestration function is [`stream2.py`](be-ircfa/src/ircfa_interaction/stream2.py:603):
 ```python
 async def stream_create(...)
 ```
 It coordinates:
 - conversation history;
 - RCFA state;
 - memory retrieval;
 - document retrieval;
 - model configuration;
 - tools;
 - streaming;
 - database saving.
 The flow is:
 ```text
 Receive ChatRequest
  ↓
 Find equipment/location
  ↓
 Load recent conversation
  ↓
 Load relevant memories
  ↓
 Load document context
  ↓
 Prepare system instructions
  ↓
 Create model adapter
  ↓
 Create tool list
  ↓
 Run AI agent
  ↓
 Execute tools when requested
  ↓
 Stream answer
  ↓
 Save chat and memory
 ```
 ## 8. Context is prepared before Gemma is called
 Before Gemma receives the question, `stream2.py` builds the complete context. It can include:
 ```text
 System instructions
 RCFA workflow instructions
 Memory privacy instructions
 Conversation history
 RCFA state
 Relevant memories
 Document text
 Equipment tag
 Current question
 ```
 Gemma therefore receives more than the user's one sentence. The backend prepares the model's working desk first.
 ## 9. Technical tools are controlled backend functions
 Tools are located under [`ircfa_interaction/tools/`](be-ircfa/src/ircfa_interaction/tools).
 Examples include:
 ```text
 current_sensor_tool.py
 kondisi_sensor_historis.py
 current_reliability_tool.py
 history_reliability_tool.py
 per_equipment_pfi_tool.py
 next_failure_pfi_tool.py
 root_cause_spln_tool.py
 failure_mode_tool.py
 plant_status_tool.py
 ```
 Each tool generally:
 1. receives arguments from the AI;
 2. calls another API or reads data;
 3. formats the result;
 4. returns the result to the AI.
 The tool loop looks like this:
 ```text
 User asks about sensor data
  ↓
 Gemma decides it needs the sensor tool
  ↓
 i-RCFA backend runs CurrentSensorTool
  ↓
 The tool calls the PFI API
  ↓
 The tool returns sensor data
  ↓
 Gemma explains the result
 ```
 Gemma does not freely open database connections. It requests a tool, and the backend decides what code runs.
 ## 10. PFI integration is usually an API call
 The backend configuration contains `PFI_API_URL` and `RP_API_URL` in [`config.py`](be-ircfa/src/config.py:55).
 For PFI sensor information, the normal relationship is:
 ```text
 Gemma
  ↓
 i-RCFA PFI tool
  ↓
 PFI HTTP API
  ↓
 PFI data source
 ```
 The i-RCFA backend does not normally query PFI database tables directly.
 ## 11. The remote model adapter
 The remote model adapter is [`gemma_llamacpp.py`](be-ircfa/src/ircfa_interaction/llm/gemma_llamacpp.py).
 It translates the application's internal request into an OpenAI-compatible request:
 ```json
 {
  "model": "...",
  "messages": [],
  "tools": [],
  "tool_choice": "auto",
  "stream": true
 }
 ```
 The remote completion URL is built from the configured model server:
 ```text
 https://gemma.digitaltwin.my.id/v1/chat/completions
 ```
 The adapter also:
 - adds authentication headers;
 - converts messages into OpenAI-compatible format;
 - includes tools;
 - handles streaming;
 - reads tool-call fragments;
 - handles multimodal image data;
 - logs remote errors.
 ## 12. The agent loop
 Around the middle of `stream2.py`, the backend creates a `qwen_agent` assistant with:
 - a system message;
 - the selected LLM adapter;
 - the allowed tool list.
 Conceptually:
 ```python
 bot = Assistant(
    system_message=system_prompt,
    llm=llm,
    function_list=tools,
 )
 ```
 Then it runs the assistant with the prepared messages.
 One request can contain multiple rounds:
 ```text
 Round 1: Gemma requests a tool
 Round 2: backend executes the tool and returns the result
 Round 3: Gemma produces the final answer
 ```
 ## 13. What happens when Gemma calls a tool
 Inside the streaming loop, `stream2.py` handles:
 - assistant messages;
 - function/tool messages;
 - reasoning content;
 - final answer content.
 When it sees a tool result, it can update RCFA state.
 For example:
 ```text
 PFI tool result
  -> save failure prediction state
  -> move RCFA phase forward
 Current sensor tool result
  -> save sensor state
  -> move RCFA phase forward
 Root-cause tool result
  -> save diagnosis state
 ```
 This is why the UI can show RCFA progress such as:
 ```text
 1. Failure Prediction
 2. Sensor Data Mapping
 3. Diagnosis
 ```
 The backend is not only generating text. It is also maintaining an investigation state.
 ## 14. Streaming uses Server-Sent Events
 `stream2.py` yields event strings such as:
 ```text
 data: answer text
 event: phase
 data: 2. Sensor Data Mapping
 event: usage_metadata
 data: {...}
 event: error
 data: {...}
 ```
 The full path is:
 ```text
 Gemma token
  ↓
 Gemma adapter
  ↓
 stream2.py
  ↓
 StreamingResponse
  ↓
 nginx
  ↓
 React chat screen
 ```
 This is why the answer appears gradually.
 ## 15. Saving happens after the answer
 After the agent finishes, the backend creates a chat-history record.
 It saves things such as:
 - user message;
 - assistant response;
 - timestamps;
 - model;
 - token counts;
 - RCFA state;
 - references;
 - location tag.
 Then it calls the memory-save logic if the user explicitly asked to remember something.
 The normal save sequence is:
 ```text
 Save chat history
  ↓
 Check/save explicit memory
  ↓
 Commit transaction
 ```
 If saving fails, the backend rolls back and logs the error.
 ## 16. Memory is a separate backend layer
 Memory retrieval happens before Gemma is called:
 ```text
 Database -> memory filter -> relevant memories -> prompt
 ```
 Memory saving happens after the answer:
 ```text
 Explicit user request
  -> policy check
  -> lifecycle check
  -> database
 ```
 The memory privacy rules are in [`memory_policy.py`](be-ircfa/src/ircfa_interaction/memory_policy.py).
 The memory lifecycle rules are in [`memory_lifecycle.py`](be-ircfa/src/ircfa_interaction/memory_lifecycle.py).
 The important separation is:
 ```text
 Gemma decides how to answer.
 Backend decides what memory is allowed.
 Database stores the memory.
 ```
 ## 17. RAG is another AI-related path
 The backend contains a RAG system under [`rag/`](be-ircfa/src/rag).
 RAG means:
 ```text
 Retrieve useful documents
  ↓
 Give those documents to the AI
  ↓
 Ask the AI to answer using them
 ```
 This is different from text generation:
 ```text
 Generation:
  Gemma writes the answer.
 Retrieval:
  Embeddings/vector search find relevant documents.
 ```
 The text-generation model can work while document retrieval has a separate problem.
 ## 18. Legacy model paths
 Some older classes in [`model.py`](be-ircfa/src/ircfa_interaction/model.py) still define Ollama or vLLM-based configurations for older workflows and auxiliary classifiers.
 That does not mean the normal i-RCFA chat has switched to local Ollama.
 The active chat path is controlled by the model configuration in `rcfa_model_config` and the selected configuration passed into `stream2.py`.
 Think of it as:
 ```text
 Main chat path:
  router.py -> stream2.py -> configured remote model adapter
 Other/legacy paths:
  may contain older Ollama, vLLM, or auxiliary configurations
 ```
 ## 19. Complete example
 Suppose the user asks:
 ```text
 What is the possible root cause of high vibration on 3DP-FDR711F?
 ```
 The backend flow is:
 ```text
 1. JWTBearer identifies the user.
 2. router.py receives /chat/stream.
 3. router.py creates ChatRequest.
 4. stream2.py loads:
   - recent chat history;
   - private memories;
   - RCFA state;
   - location tag 3DP-FDR711F.
 5. stream2.py creates the available tool list.
 6. Gemma receives the question and tools.
 7. Gemma requests current sensor data.
 8. CurrentSensorTool calls the PFI API.
 9. The sensor result returns to Gemma.
 10. Gemma may request root-cause information.
 11. RootCauseSPLNTool searches the root-cause data.
 12. Gemma writes the explanation.
 13. stream2.py sends answer pieces to the browser.
 14. The backend updates RCFA state.
 15. The backend saves chat history.
 16. If the user explicitly asked to remember something, memory logic handles it.
 ```
 ## The most important relationship
 ```text
 router.py
  -> stream2.py
      -> tools/
      -> llm/gemma_llamacpp.py
      -> database models/services
      -> StreamingResponse
 ```
 That is the backbone of the backend.
 ## Recommended learning order
 Read the backend in this order:
 1. [`api.py`](be-ircfa/src/api.py) — where routes are connected.
 2. [`router.py`](be-ircfa/src/ircfa_interaction/router.py) — how HTTP chat requests enter.
 3. [`stream2.py`](be-ircfa/src/ircfa_interaction/stream2.py) — the main AI workflow.
 4. One simple tool, such as [`current_sensor_tool.py`](be-ircfa/src/ircfa_interaction/tools/current_sensor_tool.py).
 5. [`gemma_llamacpp.py`](be-ircfa/src/ircfa_interaction/llm/gemma_llamacpp.py) — how the backend talks to Gemma.
 6. [`model.py`](be-ircfa/src/ircfa_interaction/model.py) — how chat, memory, and model settings are stored.
 7. [`database/core.py`](be-ircfa/src/database/core.py) — how database connections are created.
 ## 20. Files not directly involved in the running backend
 The normal backend started with `python run.py` follows this path:
 ```text
 run.py -> src/main.py -> src/api.py -> route modules -> stream2.py -> tools / database / remote Gemma
 ```
 The items below are **not directly used by that normal running path**. This does not automatically mean they are safe to delete: some are useful for maintenance, testing, or an older deployment approach.
 | Item | Current role | Keep or remove? |
 |---|---|---|
 | `be-ircfa/src/metrics.py` | Old metrics code. The entire file and its import in `main.py` are commented out. | A clear cleanup candidate after a normal review. |
 | `be-ircfa/src/ircfa_interaction/schema.py.orig` and `router.py.orig` | Editor backup copies; Python does not load `.orig` files. | Review their differences, then they can normally be removed. |
 | `be-ircfa/scratch/` | Manual experiments, benchmarks, and diagnostic scripts. | Keep for learning/debugging; it is not part of the live backend. |
 | `be-ircfa/docs/`, `.vscode/`, and `tests/` | Documentation, editor settings, and verification files. | Keep; they do not run as part of the backend service. |
 | `be-ircfa/docker-compose.yaml`, `docker-kong.yml`, `kong.yml`, `config.yaml`, `docker_build_push.sh`, and `Jenkinsfile` | Older container, Kong, local-model, build, and CI deployment material. | Not used by the current manual port-8005 setup. Keep until the old deployment is formally retired. |
 | `be-ircfa/requirements/requirements-full.txt`, `requirements-heavy.txt`, and `requirements-light.txt` | Alternative dependency lists with no repository references. Docker uses `requirements.txt` and `requirements/requirements-base.txt` instead. | Archive or remove only after confirming no external deployment uses them. |
 | `be-ircfa/src/root_cause/sync.py` and `cleanup_data.py` | Manual database loading and cleanup scripts. They are not called when the app starts. | Keep if you may need to rebuild or clean root-cause data. |
 | `be-ircfa/src/root_cause/*.json` and `fmea_data/*.tsv` | Source datasets used by manual data-loading work, not read during ordinary chat requests. | Do not remove unless the data is safely stored elsewhere and no future rebuild is needed. |
 | `be-ircfa/src/ircfa_interaction/stream.py` | Older chat-orchestration implementation. All live chat endpoints call `stream2.py`. `router.py` still imports it, however. | Do not remove it alone; first remove the stale import and test the chat flow. |
 | `be-ircfa/src/ircfa_interaction/llm/n8n.py` | An optional N8N LLM adapter. It is not selected by the current remote Gemma configuration. | Keep as a conditional fallback unless you decide to retire N8N support. |
 In simple terms: `stream2.py` is the current engine. The other items above are mostly old tools, spare parts, instruction books, or data boxes. They do not make the normal chat answer a user request today.
 ### Important caution
 The legacy Docker and deployment files should be treated as sensitive configuration. Before sharing, committing, or reusing them, move any credentials and connection values into protected environment variables.
 ## 21. Visual code maps
 These diagrams show the **important driver files**, rather than every small helper or data class. Solid arrows show the normal running path. Dashed arrows show optional, legacy, or conditional paths.
 ### 21.1 Folder map: startup and HTTP routes
 ```mermaid
 flowchart TD
    subgraph ROOT[be-ircfa root]
        RUN[run.py]
        ENV[Environment variables / .env]
    end
    subgraph CORE[src/ core files]
        MAIN[src/main.py]
        API[src/api.py]
        CFG[src/config.py]
        MID[src/middleware.py]
        LOG[src/logging.py]
    end
    subgraph AUTH[src/auth/]
        JWT[service.py: JWTBearer / CurrentUser]
        AUTHMODEL[model.py]
    end
    subgraph INTERACTION[src/ircfa_interaction/]
        CHATROUTER[router.py]
        STREAM2[stream2.py]
    end
    subgraph DOMAINS[Other routed feature folders]
        RAGROUTER[src/rag/router.py]
        ALERTROUTER[src/ircfa_alert/router.py]
        EQUIPROUTER[src/equipments/router.py]
        FMROUTER[src/failure_mode/router.py]
        FMEAROUTER[src/fmea/router.py]
        RCROUTER[src/root_cause/router.py]
    end
    RUN --> MAIN
    ENV --> CFG
    CFG --> MAIN
    MAIN --> LOG
    MAIN --> MID
    MAIN --> API
    API --> JWT
    API --> CHATROUTER
    API --> RAGROUTER
    API --> ALERTROUTER
    API --> EQUIPROUTER
    API --> FMROUTER
    API --> FMEAROUTER
    API --> RCROUTER
    CHATROUTER --> STREAM2
 ```
 **What this means:** `run.py` turns on FastAPI. `main.py` prepares the application. `api.py` is the receptionist that sends each URL request to the correct folder. The chat requests go to `ircfa_interaction/router.py`, then to `stream2.py`.
 ### 21.2 Folder map: the i-RCFA interaction engine
 ```mermaid
 flowchart LR
    subgraph INTERACTION[src/ircfa_interaction/]
        ROUTER[router.py\nchat, conversations, Memory Lab]
        STREAM[stream2.py\nmain orchestration]
        SCHEMA[schema.py\nrequest/response shapes]
        SERVICE[service.py\nchat/model persistence]
        MODEL[model.py\nchat, memory, LLM tables]
        MEMORY[memory_policy.py + memory_lifecycle.py]
        LEGACY[stream.py\nolder orchestration]
    end
    subgraph LLM[src/ircfa_interaction/llm/]
        GEMMA[gemma_llamacpp.py\nOpenAI-compatible Gemma adapter]
        N8N[n8n.py\noptional adapter]
    end
    subgraph TOOLS[src/ircfa_interaction/tools/]
        SENSOR[Sensor / PFI tools]
        RELIABILITY[Reliability tools]
        ROOTCAUSE[RootCauseSPLNTool]
        VECTOR[VectorSearchTool]
        OTHERTOOLS[Other RCFA tools]
    end
    subgraph SUPPORT[Supporting folders]
        RAG[src/rag/\nfiles, vector search, RAG history]
        DB[src/database/\nmain and collector sessions]
        EQUIP[src/equipments/]
        ROOTDATA[src/root_cause/]
    end
    ROUTER --> SCHEMA
    ROUTER --> STREAM
    ROUTER --> SERVICE
    STREAM --> MEMORY
    MEMORY --> MODEL
    STREAM --> MODEL
    SERVICE --> MODEL
    STREAM --> GEMMA
    STREAM -. optional configured path .-> N8N
    STREAM --> SENSOR
    STREAM --> RELIABILITY
    STREAM --> ROOTCAUSE
    STREAM --> VECTOR
    STREAM --> OTHERTOOLS
    STREAM --> RAG
    SERVICE --> DB
    MODEL --> DB
    SENSOR --> EQUIP
    ROOTCAUSE --> ROOTDATA
    VECTOR --> RAG
    ROUTER -. stale import; no active endpoint call .-> LEGACY
 ```
 **What this means:** `stream2.py` is the conductor. It gathers chat context, private memory, RCFA data, attachments, and available tools; it then asks the configured LLM to answer. The LLM can request a tool, and `stream2.py` returns that tool result to the LLM before streaming the final response back to the browser.
 ### 21.3 Cross-folder runtime map: one chat request
 ```mermaid
 sequenceDiagram
    participant FE as Frontend / nginx
    participant MAIN as src/main.py
    participant API as src/api.py
    participant AUTH as src/auth/service.py
    participant ROUTER as ircfa_interaction/router.py
    participant STREAM as ircfa_interaction/stream2.py
    participant DB as database/ + PostgreSQL
    participant TOOLS as tools/ + domain services
    participant EXT as PFI / Reliability APIs
    participant LLM as Remote Gemma
    FE->>MAIN: POST /ircfa/chat/stream
    MAIN->>API: send request to registered router
    API->>AUTH: verify JWT and identify user
    AUTH-->>ROUTER: authenticated user
    ROUTER->>STREAM: ChatRequest + database sessions
    STREAM->>DB: load chat, RCFA state, model config, private memories
    DB-->>STREAM: saved context
    STREAM->>LLM: question + system instructions + available tools
    opt LLM requests a technical tool
        LLM-->>STREAM: tool request
        STREAM->>TOOLS: run selected tool
        TOOLS->>EXT: request sensor, PFI, or reliability data
        EXT-->>TOOLS: technical data
        TOOLS-->>STREAM: tool result
        STREAM->>LLM: tool result
    end
    LLM-->>STREAM: streamed answer pieces
    STREAM->>DB: save chat, RCFA state, and explicit memory
    STREAM-->>FE: streamed answer events
 ```
 This third diagram is deliberately cross-folder: a real answer passes through authentication, the chat router, orchestration, database storage, tools, external operational APIs, and the remote Gemma service.
 ### 21.4 Folder responsibilities at a glance
 ```mermaid
 flowchart TB
    API[src/api.py\nconnects routes] --> AUTH[src/auth/\nwho is this user?]
    API --> CHAT[src/ircfa_interaction/\nchat, memory, LLM orchestration]
    API --> RAG[src/rag/\ndocuments and retrieval]
    API --> EQUIP[src/equipments/\nequipment and maintenance data]
    API --> FM[src/failure_mode/\nfailure-mode data]
    API --> FMEA[src/fmea/\nFMEA records]
    API --> RC[src/root_cause/\nroot-cause matrix data]
    API --> ALERT[src/ircfa_alert/\nalerts]
    CHAT --> DATABASE[src/database/\nPostgreSQL sessions]
    RAG --> DATABASE
    EQUIP --> DATABASE
    FM --> DATABASE
    FMEA --> DATABASE
    RC --> DATABASE
 ```
 Use the diagrams together:
 1. **21.1** answers: “Which folder receives my request?”
 2. **21.2** answers: “What happens inside the AI/chat engine?”
 3. **21.3** answers: “How do folders and outside services cooperate for one answer?”
 4. **21.4** answers: “What is each feature folder responsible for?”
--- a/HOW_IT_WORKS.md
+++ b/HOW_IT_WORKS.md
@ -0,0 +1,341 @@
 # How i-RCFA Works
 This document explains how the memory-enabled i-RCFA application connects the chat screen, backend, databases, technical tools, and remote AI model.
 ## The big picture
 The AI is one worker inside the application, not the entire application.
 ```text
 User
  |
  v
 React chat screen
  |
  v
 Frontend nginx container
  |  /ircfa is removed here
  v
 i-RCFA FastAPI backend :8005
  |
  +--> PostgreSQL: chat history, RCFA state, model configuration, memories
  |
  +--> Technical tools: sensor, reliability, PFI, RCFA and plant-status APIs
  |
  +--> Remote Gemma service
  |
  v
 Streaming answer back to the browser
 ```
 The current i-RCFA setup uses the established remote Gemma service. It does not use local Ollama, llama.cpp, or vLLM for the normal i-RCFA chat flow.
 ## 1. The browser
 The React frontend is responsible for:
 - displaying messages;
 - sending the user's question;
 - sending the login token;
 - displaying streamed answer text;
 - displaying tool/status events and suggested questions.
 The main chat request is in [`newChat.tsx`](fe-digitaltwin-v2/src/shared/components/rcfa-app/newChat.tsx). It sends a request to:
 ```text
 POST /ircfa/chat/stream
 ```
 The request contains information such as:
 ```json
 {
  "messages": [
    {
      "role": "user",
      "content": "What is the condition of 3DP-FDR711F?"
    }
  ],
  "conversation_id": "...",
  "location_tag": "3DP-FDR711F"
 }
 ```
 The browser does not call Gemma directly.
 ## 2. Frontend nginx
 The frontend container uses nginx as a proxy. The nginx configuration is in [`nginx.conf`](fe-digitaltwin-v2/nginx.conf).
 When nginx receives:
 ```text
 /ircfa/chat/stream
 ```
 it removes the `/ircfa` prefix and forwards the request to the backend:
 ```text
 http://host.docker.internal:8005/chat/stream
 ```
 Therefore the first part of the real request path is:
 ```text
 Browser -> nginx -> backend port 8005
 ```
 Nginx also keeps the streaming connection open long enough for a remote model response.
 ## 3. FastAPI backend
 The backend route is in [`router.py`](be-ircfa/src/ircfa_interaction/router.py). The `POST /chat/stream` route:
 1. reads the incoming JSON;
 2. extracts the latest user message;
 3. reads the conversation and equipment information;
 4. reads uploaded files, if present;
 5. identifies the authenticated user;
 6. creates a structured `ChatRequest`;
 7. starts the main orchestration function in [`stream2.py`](be-ircfa/src/ircfa_interaction/stream2.py).
 The backend is the control center. It decides what context and tools are available to the AI.
 ## 4. The orchestration layer
 The main function in `stream2.py` is the director of one chat turn. Before calling the model, it gathers:
 - recent conversation history;
 - the current RCFA state;
 - relevant private memories;
 - uploaded document content;
 - equipment and location information;
 - system instructions;
 - available technical tools.
 It then builds the complete model input. Conceptually, the model receives:
 ```text
 System instructions:
  You are an RCFA assistant.
  Follow memory privacy rules.
  Use technical tools when necessary.
 Conversation history:
  Previous questions and answers.
 Relevant memory:
  Active private memories that match the current question.
 Evidence:
  Sensor, reliability, PFI or document data.
 Current question:
  The user's latest message.
 ```
 The model does not receive the entire database. The backend selects and prepares the information first.
 ## 5. The remote Gemma service
 The model adapter is [`gemma_llamacpp.py`](be-ircfa/src/ircfa_interaction/llm/gemma_llamacpp.py).
 It converts the backend request into an OpenAI-compatible request to:
 ```text
 https://gemma.digitaltwin.my.id/v1/chat/completions
 ```
 The request can include:
 ```json
 {
  "model": "...",
  "messages": [],
  "tools": [],
  "tool_choice": "auto",
  "stream": true
 }
 ```
 The active model information comes from the `rcfa_model_config` PostgreSQL table. This includes the model name, server URL, provider, streaming setting, and generation settings.
 The backend therefore communicates with Gemma like this:
 ```text
 i-RCFA backend -> remote Gemma API
 ```
 It does not normally communicate with local Ollama.
 ## 6. How technical tools work
 Gemma cannot magically see the sensor or PFI database. The backend gives it a list of controlled tools.
 Examples include tools for:
 - current sensor data;
 - historical sensor data;
 - current reliability data;
 - PFI predictions;
 - failure modes;
 - root-cause information;
 - plant status.
 The tool loop looks like this:
 ```text
 User asks about sensor data
  |
  v
 Gemma decides it needs sensor information
  |
  v
 Gemma requests a technical tool
  |
  v
 i-RCFA backend executes that tool
  |
  v
 Tool calls the appropriate PFI/reliability API
  |
  v
 Tool result returns to the backend
  |
  v
 Backend gives the result back to Gemma
  |
  v
 Gemma explains the result to the user
 ```
 For PFI sensor information, the usual relationship is:
 ```text
 Gemma
  -> i-RCFA tool
  -> PFI API
  -> PFI database or system
 ```
 The AI normally does not directly query the PFI database.
 ## 7. Where memory fits
 Memory is loaded before the model is called.
 The backend:
 1. searches memories belonging to the signed-in user;
 2. ignores archived memories;
 3. ignores expired memories;
 4. scores memories for relevance;
 5. gives pinned memories a small priority boost;
 6. places relevant memories into the model context.
 Memory retrieval is implemented in [`stream2.py`](be-ircfa/src/ircfa_interaction/stream2.py).
 Memory saving happens after the response is completed. The backend only saves when the user explicitly asks it to remember something.
 The save path:
 ```text
 User explicitly asks to remember something
  |
  v
 Backend checks the request
  |
  +--> blocks obvious passwords, tokens and credentials
  |
  +--> checks for a duplicate
  |
  +--> checks the active-memory limit
  |
  v
 Memory is saved to PostgreSQL
 ```
 The assistant's answer is not automatically turned into a memory. An explicit user request is required.
 ## 8. Streaming the answer
 The backend returns a streaming response instead of waiting for the complete answer.
 During one response it can send:
 - answer text;
 - tool progress;
 - usage information;
 - RCFA state;
 - suggested questions;
 - error events.
 The frontend reads these pieces from the response body and updates the chat screen while the answer is being generated.
 ```text
 Gemma produces a piece
  -> backend receives it
  -> backend sends it through the stream
  -> frontend reads it
  -> chat screen displays it
 ```
 After the response is complete, the backend saves the conversation and any approved memory in the database.
 ## 9. What a 503 means
 A 503 means that a service was temporarily unavailable. In the common i-RCFA case, the failing link is near the remote Gemma service:
 ```text
 Browser -> nginx -> i-RCFA backend -> remote Gemma -> 503
 ```
 Possible causes include:
 - the remote model is overloaded;
 - the remote model is restarting;
 - too many requests arrived at once;
 - the remote proxy timed out;
 - the remote service temporarily rejected the request.
 A Gemma 503 does not automatically mean that the PFI database, sensor timestamp, memory, or frontend is broken.
 The useful debugging question is: which room failed?
 ```text
 Browser room       Did the request leave the browser?
 Nginx room         Did the proxy forward it?
 Backend room       Did FastAPI start orchestration?
 Tool room          Did the PFI/reliability API respond?
 Gemma room         Did the remote AI answer?
 Database room      Did chat/memory saving succeed?
 ```
 ## The complete request loop
 ```text
 1. Prepare context
 2. Ask Gemma
 3. Gemma may request a tool
 4. Backend executes the tool
 5. Backend returns the tool result to Gemma
 6. Gemma writes the explanation
 7. Backend streams the explanation to the browser
 8. Backend saves the conversation
 ```
 The most important architectural idea is that Gemma does not own the application. The backend controls what Gemma can see, which tools it can use, which user's memory it can receive, and how the final answer reaches the browser.
 ## Main files
 | Responsibility | File |
 |---|---|
 | Chat UI and streaming reader | [`newChat.tsx`](fe-digitaltwin-v2/src/shared/components/rcfa-app/newChat.tsx) |
 | Frontend-to-backend proxy | [`nginx.conf`](fe-digitaltwin-v2/nginx.conf) |
 | FastAPI chat routes | [`router.py`](be-ircfa/src/ircfa_interaction/router.py) |
 | Main AI orchestration | [`stream2.py`](be-ircfa/src/ircfa_interaction/stream2.py) |
 | Remote Gemma adapter | [`gemma_llamacpp.py`](be-ircfa/src/ircfa_interaction/llm/gemma_llamacpp.py) |
 | Model configuration | [`config.py`](be-ircfa/src/ircfa_interaction/config.py) and `rcfa_model_config` |
 | Memory privacy rules | [`memory_policy.py`](be-ircfa/src/ircfa_interaction/memory_policy.py) |
 | Memory lifecycle rules | [`memory_lifecycle.py`](be-ircfa/src/ircfa_interaction/memory_lifecycle.py) |
 | Memory Lab UI | [`MemoryLabModal.tsx`](fe-digitaltwin-v2/src/shared/components/rcfa-app/MemoryLabModal.tsx) |
--- a/be-ircfa/README.md
+++ b/be-ircfa/README.md
@ -1,38 +1,306 @@
-# I-RCFA Service
+# i-RCFA Backend (`be-ircfa`)
-## Debug Export
+`be-ircfa` is the Python FastAPI backend for the i-RCFA (Root Cause Failure Analysis) chat application. It receives authenticated chat requests, prepares conversation and RCFA context, retrieves sensor/reliability/PFI information through controlled tools, calls the configured LLM, streams the response, and saves the conversation.
-Debug snapshots are generated to help troubleshoot LLM interactions. By default, debug snapshots are **disabled** in all environments.
+The current memory-copy application is intended to keep using the established remote Gemma service. Do not start local Ollama, llama.cpp, or vLLM services for the normal i-RCFA chat flow.
-### Enabling Debug Export
+## What the backend does
-To enable debug export in **development environment only**, set the `ENABLE_DEBUG_EXPORT` environment variable:
+```text
 Browser
  -> frontend nginx proxy
  -> FastAPI backend
      -> PostgreSQL chat/memory database
      -> collector database session
      -> PFI/reliability HTTP APIs
      -> remote Gemma service
  <- streamed answer
 ```
 The main AI workflow is in [`src/ircfa_interaction/stream2.py`](src/ircfa_interaction/stream2.py). The complete architecture walkthrough is in [`../BE_IRCFA.md`](../BE_IRCFA.md).
 ## Project layout
-**Docker Compose:**
+```text
-```yaml
+be-ircfa/
-environment:
+├── run.py                 Local launcher
-  - ENABLE_DEBUG_EXPORT=true
+├── pyproject.toml         Poetry project/dependencies
 ├── requirements/          Dependency groups
 ├── Dockerfile             Backend image definition
 ├── docker-compose.yaml    Legacy/all-in-one container setup
 ├── docs/                  Design notes and reports
 ├── scratch/               Experiments and diagnostics
 ├── tests/                 Tests
 └── src/
    ├── main.py            FastAPI application and middleware
    ├── api.py             API route registration
    ├── auth/              Authentication
    ├── database/          Async PostgreSQL sessions
    ├── ircfa_interaction/ Chat, AI, tools, memory and model adapters
    ├── rag/               Document retrieval
    ├── root_cause/        Root-cause data and routes
    ├── failure_mode/      Failure-mode data and routes
    ├── equipments/        Equipment routes and services
    ├── fmea/              FMEA routes and models
    └── ircfa_alert/       Alert routes and services
 ```
-**Shell/Terminal:**
+## Prerequisites
 You need:
 - Python `>=3.11,<3.13`;
 - a working virtual environment with the backend dependencies;
 - access to the main PostgreSQL database;
 - access to the collector database, if collector-backed features are used;
 - access to the PFI and reliability APIs;
 - the configured remote Gemma endpoint;
 - a backend `.env` file or equivalent exported environment variables.
 The backend reads environment variables in [`src/config.py`](src/config.py). Do not commit `.env` files or credentials.
 ## Recommended current setup: run the memory-copy backend manually
 This is the setup currently used for the memory-enabled comparison copy.
 ### 1. Start the backend
 Open a terminal and run:
 ```bash
-export ENABLE_DEBUG_EXPORT=true
+cd /home/nvidia/dev-rcfa-memory/be-ircfa
 source /home/nvidia/dev-rcfa/be-ircfa/venv/bin/activate
 set -a
 source /home/nvidia/dev-rcfa/be-ircfa/.env
 set +a
 export HOST=0.0.0.0
 export PORT=8005
 python run.py
 ```
 Why is the `.env` path from `/home/nvidia/dev-rcfa`? The current memory-copy backend setup intentionally reuses the established environment while it is being tested. This also means it may point at the same PostgreSQL database as the baseline application. Isolate the database before serious memory experiments.
 `run.py` starts Uvicorn with:
 ```text
 run.py -> src.main:app
 ```
 The backend should listen on:
 ```text
 http://0.0.0.0:8005
 ```
 ### 2. Check backend health
 In another terminal:
 ```bash
 curl http://127.0.0.1:8005/healthcheck
 ```
 Expected response:
 ```json
 {"status":"ok"}
 ```
 ### 3. Run the matching frontend
 The memory-copy frontend normally runs in Docker and forwards `/ircfa` requests to this backend:
 ```bash
 cd /home/nvidia/dev-rcfa-memory/fe-digitaltwin-v2
 FRONTEND_PORT=8083 \
 BACKEND_URL_IRCFA=http://host.docker.internal:8005 \
 docker compose up --build
 ```
 Open the real chat route:
 ```text
 http://127.0.0.1:8083/i-rcfa-app/chat
 ```
 The request path is:
 ```text
 Browser -> frontend nginx /ircfa proxy -> backend :8005
 ```
 ## Alternative local commands
 ### Run with Make
 From `be-ircfa/`:
 ```bash
 make install
 make run
 ```
 `make install` runs `poetry install`. `make run` runs `python run.py`.
 The Makefile also defines:
 ```bash
 make test
 ```
 which runs the unit tests under `tests/unit/` through Poetry. The environment must have the test dependencies installed.
 ### Run with Poetry directly
 ```bash
 cd be-ircfa
 poetry install
 poetry run python run.py
 ```
 ### Run Uvicorn directly
 ```bash
 cd be-ircfa
 poetry run uvicorn src.main:app --host 0.0.0.0 --port 8005 --reload
 ```
 Do not run multiple backend processes against the same port.
 ## Docker Compose note
 `docker-compose.yaml` contains an older all-in-one deployment definition. It includes:
 - the i-RCFA backend;
 - Kong;
 - vLLM services;
 - llama.cpp services;
 - optional local Gemma services.
 That file is useful as a deployment/reference artifact, but it is not the recommended command for the current memory-copy learning setup. In particular, do not start the local model services just to run i-RCFA. Keep the backend pointed at the established remote Gemma configuration.
 If you intentionally need to inspect the legacy Compose setup, review the file first and verify every environment variable and service before running:
 ```bash
 cd be-ircfa
 docker compose config
 docker compose up --build
 ```
 ## Important environment variables
 The names below are read by [`src/config.py`](src/config.py):
 | Variable | Purpose |
 |---|---|
 | `HOST` | Backend bind address, usually `0.0.0.0` when serving containers |
 | `PORT` | Backend port, `8005` for the current memory-copy setup |
 | `DATABASE_HOSTNAME` | Main PostgreSQL host |
 | `DATABASE_PORT` | Main PostgreSQL port |
 | `DATABASE_CREDENTIAL_USER` | Main PostgreSQL user |
 | `DATABASE_CREDENTIAL_PASSWORD` | Main PostgreSQL password; keep secret |
 | `DATABASE_NAME` | Main application database |
 | `COLLECTOR_HOSTNAME` | Collector database host |
 | `COLLECTOR_PORT` | Collector database port |
 | `COLLECTOR_CREDENTIAL_USER` | Collector database user |
 | `COLLECTOR_CREDENTIAL_PASSWORD` | Collector database password; keep secret |
 | `COLLECTOR_NAME` | Collector database name |
 | `PFI_API_URL` | PFI HTTP API base URL |
 | `RP_API_URL` | Reliability HTTP API base URL |
 | `MODEL_SERVER_URL` | OpenAI-compatible model server base URL |
 | `MODEL_SERVER_API_KEY` | Model-service credential |
 | `PLANT_STATUS_URL` | Plant-status endpoint |
 | `ENABLE_BACKGROUND_ANALYSIS` | Enables background RCFA analysis behavior |
 | `ENABLE_DEBUG_EXPORT` | Enables local debug snapshots; keep off in production |
 The active/default model is stored in the `rcfa_model_config` table. Do not assume a model is selected from Python defaults alone; verify the database configuration used by the running backend.
 ## Main chat request flow
 ```text
 1. Frontend sends POST /chat/stream.
 2. JWT authentication identifies the user.
 3. router.py validates and converts the request into ChatRequest.
 4. stream2.py loads history, RCFA state, memories and documents.
 5. stream2.py creates the available technical tools.
 6. The configured LLM adapter calls remote Gemma.
 7. Gemma may request a tool.
 8. The backend executes the tool and returns its result to Gemma.
 9. The final answer is streamed back as events.
 10. Chat history and approved explicit memories are saved.
 ```
-**`.env` file:**
+The main files are:
 - [`src/ircfa_interaction/router.py`](src/ircfa_interaction/router.py) — HTTP chat routes;
 - [`src/ircfa_interaction/stream2.py`](src/ircfa_interaction/stream2.py) — orchestration and streaming;
 - [`src/ircfa_interaction/llm/gemma_llamacpp.py`](src/ircfa_interaction/llm/gemma_llamacpp.py) — remote model adapter;
 - [`src/ircfa_interaction/tools/`](src/ircfa_interaction/tools) — controlled technical tools;
 - [`src/ircfa_interaction/memory_policy.py`](src/ircfa_interaction/memory_policy.py) — explicit-only memory policy;
 - [`src/ircfa_interaction/memory_lifecycle.py`](src/ircfa_interaction/memory_lifecycle.py) — memory expiry, archive, pinning and limits.
 ## Debugging and logs
 The backend logs incoming requests, completion status, model activity, tool calls and failures. For the running manual backend, keep the terminal open and watch the logs while reproducing a problem.
 To enable detailed debug snapshots in a development environment:
 ```bash
 export ENABLE_DEBUG_EXPORT=true
 ```
-ENABLE_DEBUG_EXPORT=true
+
 Debug snapshots are written under the backend's debug output directory and may include:
 - conversation metadata;
 - LLM configuration;
 - input/output messages;
 - function-call results;
 - retrieved context documents.
 Treat these snapshots as sensitive because they may contain user questions, equipment information, documents, or model context. Keep `ENABLE_DEBUG_EXPORT=false` in production.
 For a `503 Service Unavailable`, inspect the request in this order:
 ```text
 1. Browser: did the request leave the chat screen?
 2. Nginx: did /ircfa forward to the backend?
 3. Backend: did /healthcheck and /chat/stream respond?
 4. Tool/API: did PFI or reliability calls respond?
 5. Remote Gemma: did /v1/chat/completions respond?
 6. Database: did chat/memory saving commit?
 ```
-### Debug Snapshot Location
+A remote Gemma 503 usually means the remote model service was overloaded, restarting, timing out, or temporarily unavailable. It does not automatically mean the PFI database or memory system failed.
 ## Testing and validation
 Useful checks:
 ```bash
 cd be-ircfa
 # Syntax check selected backend files
 poetry run python -m py_compile \
  src/main.py \
  src/api.py \
  src/ircfa_interaction/router.py \
  src/ircfa_interaction/stream2.py
 # Unit tests, when pytest is installed in the Poetry environment
 make test
 ```
-When enabled, debug snapshots are saved to `debug/chat_debug_<timestamp>.md` with detailed information about:
+The repository also contains integration and manual tests under `tests/`, plus exploratory scripts under `scratch/`.
 - Conversation metadata
 - LLM configuration
 - Input/output messages
 - Function call results
 - Retrieved context documents
-### Production Deployment
+## Safety notes for this learning setup
-For production environments, ensure `ENABLE_DEBUG_EXPORT` is **not set** or set to `false` to prevent unnecessary file generation and potential disk space issues.
+- Keep using the established remote Gemma service for i-RCFA.
 - Do not start local Ollama, llama.cpp or vLLM for the normal chat path.
 - Do not modify `/home/nvidia/dev-rcfa`; it is the baseline application.
 - Do not use `/home/nvidia/hermes-webui/pltu_agent_zero` for i-RCFA changes.
 - The memory-copy backend currently may reuse the baseline `.env` and database. Create an isolated database before serious memory testing.
 - The RAG path expects the embedding model `dengcao/Qwen3-Embedding-8B:Q8_0`; normal remote-LLM chat can work even when document retrieval is unavailable.
 - Never commit `.env` files, API keys, passwords, tokens, or generated debug exports.
--- a/be-ircfa/src/ircfa_interaction/memory_lifecycle.py
+++ b/be-ircfa/src/ircfa_interaction/memory_lifecycle.py
@ -0,0 +1,145 @@
 """Small, deterministic lifecycle rules for private i-RCFA memories."""
 from __future__ import annotations
 from datetime import datetime, timedelta, timezone
 from typing import Any
 MAX_ACTIVE_MEMORIES_PER_USER = 100
 MAX_MEMORY_RETRIEVAL_CANDIDATES = 250
 MAX_MEMORY_LAB_ITEMS = 250
 TEST_MEMORY_TTL_DAYS = 7
 MEMORY_ACTIVE = "active"
 MEMORY_ARCHIVED = "archived"
 MEMORY_EXPIRED = "expired"
 def utc_now() -> datetime:
    return datetime.now(timezone.utc)
 def normalise_memory_text(value: str | None) -> str:
    return " ".join((value or "").lower().split())
 def metadata_for(memory: Any) -> dict[str, Any]:
    metadata = getattr(memory, "metadata_json", None)
    return dict(metadata) if isinstance(metadata, dict) else {}
 def is_memory_lab_test(value: str | None) -> bool:
    normalised = normalise_memory_text(value)
    return "memory lab test" in normalised
 def parse_expiry(value: Any) -> datetime | None:
    if not isinstance(value, str) or not value:
        return None
    try:
        parsed = datetime.fromisoformat(value.replace("Z", "+00:00"))
    except ValueError:
        return None
    return parsed if parsed.tzinfo else parsed.replace(tzinfo=timezone.utc)
 def memory_lifecycle(memory: Any, now: datetime | None = None) -> str:
    metadata = metadata_for(memory)
    if metadata.get("lifecycle") == MEMORY_ARCHIVED:
        return MEMORY_ARCHIVED
    expires_at = parse_expiry(metadata.get("expires_at"))
    if expires_at and expires_at <= (now or utc_now()):
        return MEMORY_EXPIRED
    return MEMORY_ACTIVE
 def is_memory_active(memory: Any, now: datetime | None = None) -> bool:
    return memory_lifecycle(memory, now) == MEMORY_ACTIVE
 def is_memory_pinned(memory: Any) -> bool:
    return bool(metadata_for(memory).get("pinned"))
 def memory_expiry(memory: Any) -> str | None:
    return metadata_for(memory).get("expires_at") or None
 def memory_duplicate_count(memory: Any) -> int:
    try:
        return max(1, int(metadata_for(memory).get("remember_count", 1)))
    except (TypeError, ValueError):
        return 1
 def new_memory_metadata(
    *,
    files: list[str] | None,
    saved_images: list[str] | None,
    message_id: str,
    message: str,
    now: datetime | None = None,
 ) -> dict[str, Any]:
    current_time = now or utc_now()
    metadata: dict[str, Any] = {
        "files": files or [],
        "saved_images": saved_images or [],
        "message_id": message_id,
        "lifecycle": MEMORY_ACTIVE,
        "pinned": False,
        "remember_count": 1,
        "last_remembered_at": current_time.isoformat(),
    }
    if is_memory_lab_test(message):
        metadata["expires_at"] = (current_time + timedelta(days=TEST_MEMORY_TTL_DAYS)).isoformat()
        metadata["is_test_memory"] = True
    return metadata
 def refresh_duplicate_metadata(
    memory: Any,
    *,
    message_id: str,
    message: str,
    now: datetime | None = None,
 ) -> dict[str, Any]:
    current_time = now or utc_now()
    metadata = metadata_for(memory)
    metadata["lifecycle"] = MEMORY_ACTIVE
    metadata["message_id"] = message_id
    metadata["remember_count"] = memory_duplicate_count(memory) + 1
    metadata["last_remembered_at"] = current_time.isoformat()
    if is_memory_lab_test(message):
        metadata["expires_at"] = (current_time + timedelta(days=TEST_MEMORY_TTL_DAYS)).isoformat()
        metadata["is_test_memory"] = True
    else:
        metadata.pop("expires_at", None)
        metadata.pop("is_test_memory", None)
    return metadata
 def update_memory_controls(
    memory: Any,
    *,
    lifecycle: str | None = None,
    pinned: bool | None = None,
    expires_in_days: int | None = None,
    clear_expiry: bool = False,
    now: datetime | None = None,
 ) -> dict[str, Any]:
    current_time = now or utc_now()
    metadata = metadata_for(memory)
    if lifecycle is not None:
        metadata["lifecycle"] = lifecycle
        expires_at = parse_expiry(metadata.get("expires_at"))
        if lifecycle == MEMORY_ACTIVE and expires_at and expires_at <= current_time:
            metadata.pop("expires_at", None)
    if pinned is not None:
        metadata["pinned"] = pinned
    if expires_in_days is not None:
        metadata["expires_at"] = (current_time + timedelta(days=expires_in_days)).isoformat()
    if clear_expiry:
        metadata.pop("expires_at", None)
    return metadata
--- a/be-ircfa/src/ircfa_interaction/router.py
+++ b/be-ircfa/src/ircfa_interaction/router.py
@ -3,9 +3,10 @@ import json as json_module
 from typing import Optional, List
 from fastapi import APIRouter, HTTPException, status, Query, File, UploadFile, Request
 from fastapi.responses import StreamingResponse
-from src.ircfa_interaction.schema import ChatResponse, Conversation, ConversationUpdate, ConversationDataPagination, UpdateFeedback, RootCauseRequest, RootCauseResponse, ConversationExportUpdate, ChatRequest, FlowType, ChatHistory, ChatHistoryResponse, ModelRead, AdminModelRead, ModelConfigCreate, ModelConfigUpdate, MemoryLabSnapshot, MemoryRead
+from src.ircfa_interaction.schema import ChatResponse, Conversation, ConversationUpdate, ConversationDataPagination, UpdateFeedback, RootCauseRequest, RootCauseResponse, ConversationExportUpdate, ChatRequest, FlowType, ChatHistory, ChatHistoryResponse, ModelRead, AdminModelRead, ModelConfigCreate, ModelConfigUpdate, MemoryLabSnapshot, MemoryLabUpdate, MemoryRead
 from src.ircfa_interaction.tools import RootCauseSPLNTool
 from src.ircfa_interaction.model import ClassifierConfig, Conversation as ConversationModel, RCFAMemory
 from src.ircfa_interaction.memory_lifecycle import MAX_ACTIVE_MEMORIES_PER_USER, MAX_MEMORY_LAB_ITEMS, MEMORY_ACTIVE, MEMORY_ARCHIVED, is_memory_pinned, memory_duplicate_count, memory_expiry, memory_lifecycle, update_memory_controls, utc_now
 from src.rag.service import get_rag_histories_by_conversation_id, create as rag_create
 from src.ircfa_interaction.service import create, get_conversation, continue_conversation, delete_conversation, update_conversation_title, get_conversation_by_id, get_chat_by_id, update_chat_history_feedback, delete_chat_by_id, update_conversation_export_status, get_all_models_service, toggle_model_service, set_default_model_service, create_model_service, update_model_service, delete_model_service, get_feedback_analytics
 from src.database.service import CommonParameters, search_filter_sort_paginate
@ -18,7 +19,7 @@ from src.models import StandardResponse
 from src.rag.lib.prompt_template_list import searchRAGKeywords
 from uuid import UUID
 from pydantic import Field, BaseModel
-from sqlalchemy import func, select
+from sqlalchemy import func, or_, select
 router = APIRouter()
@ -44,12 +45,16 @@ def _memory_to_read(memory: RCFAMemory) -> MemoryRead:
        id=memory.id,
        conversation_id=memory.conversation_id,
        source_type=memory.source_type,
        lifecycle=memory_lifecycle(memory),
        pinned=is_memory_pinned(memory),
        title=memory.title,
        summary=memory.summary,
        excerpt=memory.excerpt,
        location_tag=memory.location_tag,
        equipment=memory.equipment,
        confidence=memory.confidence,
        expires_at=memory_expiry(memory),
        remember_count=memory_duplicate_count(memory),
        created_at=memory.created_at,
    )
@ -407,18 +412,76 @@ async def get_memory_lab(
        select(RCFAMemory)
        .where(*filters)
        .order_by(RCFAMemory.created_at.desc())
-        .limit(50)
+        .limit(MAX_MEMORY_LAB_ITEMS)
    )
    memories = list(memory_result.scalars())
    total = await db_session.scalar(select(func.count(RCFAMemory.id)).where(*filters)) or 0
    lifecycle_value = RCFAMemory.metadata_json["lifecycle"].as_string()
    expiry_value = RCFAMemory.metadata_json["expires_at"].as_string()
    now = utc_now().isoformat()
    active_filter = (
        or_(lifecycle_value.is_(None), lifecycle_value != MEMORY_ARCHIVED),
        or_(expiry_value.is_(None), expiry_value > now),
    )
    active_count = await db_session.scalar(
        select(func.count(RCFAMemory.id)).where(*filters, *active_filter)
    ) or 0
    archived_count = await db_session.scalar(
        select(func.count(RCFAMemory.id)).where(*filters, lifecycle_value == MEMORY_ARCHIVED)
    ) or 0
    expired_count = await db_session.scalar(
        select(func.count(RCFAMemory.id)).where(
            *filters,
            or_(lifecycle_value.is_(None), lifecycle_value != MEMORY_ARCHIVED),
            expiry_value.is_not(None),
            expiry_value <= now,
        )
    ) or 0
    snapshot = MemoryLabSnapshot(
        items=[_memory_to_read(memory) for memory in memories],
        total=total,
-        retrieval_limit=50,
+        active_count=active_count,
        archived_count=archived_count,
        expired_count=expired_count,
        active_limit=MAX_ACTIVE_MEMORIES_PER_USER,
        retrieval_limit=MAX_MEMORY_LAB_ITEMS,
        capacity_reached=active_count >= MAX_ACTIVE_MEMORIES_PER_USER,
    )
    return StandardResponse(data=snapshot, message="Private memories retrieved successfully")
@router.patch("/memory-lab/{memory_id}", response_model=StandardResponse[MemoryRead], summary="Update one private Memory Lab record")
 async def update_memory_lab_record(
    memory_id: UUID,
    payload: MemoryLabUpdate,
    db_session: DbSession,
    current_user: CurrentUser,
 ):
    memory = await db_session.scalar(
        select(RCFAMemory).where(
            RCFAMemory.id == memory_id,
            RCFAMemory.user_id == current_user.user_id,
            RCFAMemory.scope == "user",
        )
    )
    if memory is None:
        raise HTTPException(
            status_code=status.HTTP_404_NOT_FOUND,
            detail="This private memory does not exist.",
        )
    memory.metadata_json = update_memory_controls(
        memory,
        lifecycle=payload.lifecycle,
        pinned=payload.pinned,
        expires_in_days=payload.expires_in_days,
        clear_expiry=payload.clear_expiry,
    )
    memory.updated_by = str(current_user.name)
    await db_session.commit()
    return StandardResponse(data=_memory_to_read(memory), message="Private memory updated successfully")
@router.delete("/memory-lab/{memory_id}", response_model=StandardResponse[str], summary="Delete one private Memory Lab record")
 async def delete_memory_lab_record(
--- a/be-ircfa/src/ircfa_interaction/schema.py
+++ b/be-ircfa/src/ircfa_interaction/schema.py
@ -281,12 +281,16 @@ class MemoryRead(DefaultBase):
    id: UUID
    conversation_id: Optional[UUID] = None
    source_type: str
    lifecycle: str = "active"
    pinned: bool = False
    title: Optional[str] = None
    summary: str
    excerpt: Optional[str] = None
    location_tag: Optional[str] = None
    equipment: Optional[str] = None
    confidence: Optional[float] = None
    expires_at: Optional[str] = None
    remember_count: int = 1
    created_at: Optional[datetime] = None
@ -295,9 +299,24 @@ class MemoryLabSnapshot(DefaultBase):
    items: List[MemoryRead] = Field(default_factory=list)
    total: int = 0
-    retrieval_limit: int = 50
+    active_count: int = 0
    archived_count: int = 0
    expired_count: int = 0
    active_limit: int = 100
    retrieval_limit: int = 250
    capacity_reached: bool = False
    explicit_only: bool = True
    user_private: bool = True
    secrets_blocked: bool = True
 class MemoryLabUpdate(DefaultBase):
    """User-controlled lifecycle changes for one private memory."""
    lifecycle: Optional[str] = Field(default=None, pattern="^(active|archived)$")
    pinned: Optional[bool] = None
    expires_in_days: Optional[int] = Field(default=None, ge=1, le=365)
    clear_expiry: bool = False
--- a/be-ircfa/src/ircfa_interaction/stream2.py
+++ b/be-ircfa/src/ircfa_interaction/stream2.py
@ -11,9 +11,19 @@ import json
 from datetime import datetime, timedelta
 from pathlib import Path
 from fastapi import HTTPException
-from sqlalchemy import select, delete, update, desc
+from sqlalchemy import select, delete, update, desc, func, or_
 from src.ircfa_interaction.schema import ChatRequest, ChatResponse, ChatMessage, Performance, ChatHistory, Conversation, ClassifierResponse, ChatRequest as ChatRequestRAG, ChatResponse as ChatResponseRAG, ChatHistory as ChatHistorySchema
 from src.ircfa_interaction.model import ChatHistory as ChatHistoryModel, Conversation as ConversationModel, RCFAMemory, LLMConfig, ClassifierConfig, VLLMConfig, RCFAContextExtractorConfig
 from src.ircfa_interaction.memory_lifecycle import (
    MAX_ACTIVE_MEMORIES_PER_USER,
    MEMORY_ARCHIVED,
    utc_now,
    MAX_MEMORY_RETRIEVAL_CANDIDATES,
    is_memory_active,
    is_memory_pinned,
    new_memory_metadata,
    refresh_duplicate_metadata,
 )
 from src.rag.service import create as chat_create
 from src.rag.model import UploadedFile
 from src.equipments.service import get_standard_metrics
@ -274,6 +284,7 @@ async def retrieve_memory_references(
    message: str,
    location_tag: str | None,
 ) -> list[dict[str, Any]]:
    """Find the small set of private memories useful for this chat turn."""
    try:
        query = (
            select(RCFAMemory)
@ -282,7 +293,7 @@ async def retrieve_memory_references(
                RCFAMemory.scope == "user",
            )
            .order_by(desc(RCFAMemory.created_at))
-            .limit(50)
+            .limit(MAX_MEMORY_RETRIEVAL_CANDIDATES)
        )
        result = await db_session.execute(query)
        memories = list(result.scalars())
@ -294,6 +305,10 @@ async def retrieve_memory_references(
    scored: list[tuple[float, RCFAMemory]] = []
    for memory in memories:
        # Archived and expired notes remain visible in Memory Lab, but never go to the LLM.
        if not is_memory_active(memory):
            continue
        searchable = " ".join(
            str(value or "")
            for value in [
@ -312,6 +327,8 @@ async def retrieve_memory_references(
            score += sum(1 for keyword in keywords if keyword in searchable) / max(len(keywords), 1)
        if memory.scope == "global":
            score += 0.05
        if is_memory_pinned(memory):
            score += 0.15
        if score > 0:
            scored.append((score, memory))
@ -333,6 +350,7 @@ async def save_response_memory(
    saved_images: list[str] | None,
    current_rcfa_state: dict[str, Any],
 ) -> None:
    """Save only an explicit, private memory request from the signed-in user."""
    try:
        message = request.message or ""
        if not is_explicit_memory_request(message):
@ -357,8 +375,55 @@ async def save_response_memory(
        if not summary:
            return
        # Repeating the same request refreshes one record; it does not create clutter.
        existing_memory = await db_session.scalar(
            select(RCFAMemory)
            .where(
                RCFAMemory.user_id == current_user.user_id,
                RCFAMemory.scope == scope,
                func.lower(RCFAMemory.summary) == summary.lower(),
            )
            .order_by(desc(RCFAMemory.created_at))
            .limit(1)
        )
        if existing_memory is not None:
            existing_memory.conversation_id = conversation_id
            existing_memory.chat_history_id = chat_history_id
            existing_memory.title = title
            existing_memory.excerpt = excerpt
            existing_memory.filepath = all_files[0] if all_files else None
            existing_memory.location_tag = location_tag
            existing_memory.metadata_json = refresh_duplicate_metadata(
                existing_memory,
                message_id=str(chat_history_id),
                message=message,
            )
            existing_memory.updated_by = str(current_user.name)
            logger.info("Updated duplicate private memory instead of creating another record.")
            return
        # The memory list grows over time, so never silently delete old records. Instead,
        # pause new active saves at the per-user limit until the user archives or deletes one.
        lifecycle_value = RCFAMemory.metadata_json["lifecycle"].as_string()
        expiry_value = RCFAMemory.metadata_json["expires_at"].as_string()
        now = utc_now().isoformat()
        active_memory_count = await db_session.scalar(
            select(func.count(RCFAMemory.id)).where(
                RCFAMemory.user_id == current_user.user_id,
                RCFAMemory.scope == scope,
                or_(lifecycle_value.is_(None), lifecycle_value != MEMORY_ARCHIVED),
                or_(expiry_value.is_(None), expiry_value > now),
            )
        ) or 0
        if active_memory_count >= MAX_ACTIVE_MEMORIES_PER_USER:
            logger.info(
                "Skipping new private memory because the active memory limit (%s) was reached.",
                MAX_ACTIVE_MEMORIES_PER_USER,
            )
            return
        memory = RCFAMemory(
-            user_id=current_user.user_id if scope == "user" else None,
+            user_id=current_user.user_id,
            conversation_id=conversation_id,
            chat_history_id=chat_history_id,
            scope=scope,
@ -369,12 +434,13 @@ async def save_response_memory(
            filepath=all_files[0] if all_files else None,
            location_tag=location_tag,
            equipment=None,
-            confidence=0.75 if scope == "user" else 0.85,
+            confidence=0.75,
-            metadata_json={
+            metadata_json=new_memory_metadata(
-                "files": all_files or [],
+                files=all_files,
-                "saved_images": saved_images or [],
+                saved_images=saved_images,
-                "message_id": str(chat_history_id),
+                message_id=str(chat_history_id),
-            },
+                message=message,
            ),
            created_by=str(current_user.name),
            updated_by=str(current_user.name),
        )
@ -1054,7 +1120,7 @@ async def stream_create(
    # Re-apply any updated summary if compaction occurred
    if compacted:
        system_prompt = (
-            f"{base_system_prompt}\n\n{rcfa_workflow_prompt}\n\n"
+            f"{base_system_prompt}\n\n{MEMORY_POLICY_INSTRUCTIONS}\n\n{rcfa_workflow_prompt}\n\n"
            f"[INFO UI] Progres RCFA saat ini ada di: **{current_rcfa_phase}**\n\n"
            f"Gunakan bahasa Indonesia yang profesional, singkat, dan padat. "
            f"JANGAN menerjemahkan istilah teknis yang sudah baku (Technical Terms) ke dalam bahasa Indonesia.\n"
--- a/be-ircfa/tests/unit/test_memory_lifecycle.py
+++ b/be-ircfa/tests/unit/test_memory_lifecycle.py
@ -0,0 +1,74 @@
 from datetime import timedelta
 from types import SimpleNamespace
 from src.ircfa_interaction.memory_lifecycle import (
    MEMORY_ACTIVE,
    MEMORY_ARCHIVED,
    MEMORY_EXPIRED,
    memory_duplicate_count,
    memory_lifecycle,
    new_memory_metadata,
    refresh_duplicate_metadata,
    update_memory_controls,
    utc_now,
 )
 def memory_with(metadata):
    return SimpleNamespace(metadata_json=metadata)
 def test_memory_lab_test_note_expires_after_seven_days():
    now = utc_now()
    memory = memory_with(
        new_memory_metadata(
            files=None,
            saved_images=None,
            message_id="message-1",
            message="Please remember this harmless Memory Lab test note: bluebird-4829.",
            now=now,
        )
    )
    assert memory_lifecycle(memory, now) == MEMORY_ACTIVE
    assert memory_lifecycle(memory, now + timedelta(days=7, seconds=1)) == MEMORY_EXPIRED
 def test_archived_memory_stays_archived_and_can_be_restored():
    now = utc_now()
    memory = memory_with({"lifecycle": MEMORY_ACTIVE, "pinned": False})
    memory.metadata_json = update_memory_controls(memory, lifecycle=MEMORY_ARCHIVED, now=now)
    assert memory_lifecycle(memory, now) == MEMORY_ARCHIVED
    memory.metadata_json = update_memory_controls(memory, lifecycle=MEMORY_ACTIVE, now=now)
    assert memory_lifecycle(memory, now) == MEMORY_ACTIVE
 def test_restoring_an_expired_memory_removes_its_expiry_date():
    now = utc_now()
    memory = memory_with({
        "lifecycle": MEMORY_ACTIVE,
        "expires_at": (now - timedelta(seconds=1)).isoformat(),
    })
    memory.metadata_json = update_memory_controls(memory, lifecycle=MEMORY_ACTIVE, now=now)
    assert "expires_at" not in memory.metadata_json
    assert memory_lifecycle(memory, now) == MEMORY_ACTIVE
 def test_repeated_memory_request_refreshes_one_record_metadata():
    now = utc_now()
    memory = memory_with({"remember_count": 1, "lifecycle": MEMORY_ACTIVE})
    refreshed = refresh_duplicate_metadata(
        memory,
        message_id="message-2",
        message="Please remember the same technical note.",
        now=now,
    )
    assert refreshed["lifecycle"] == MEMORY_ACTIVE
    assert refreshed["message_id"] == "message-2"
    assert memory_duplicate_count(memory_with(refreshed)) == 2
--- a/be-ircfa/tests/unit/test_memory_policy.py
+++ b/be-ircfa/tests/unit/test_memory_policy.py
@ -1,3 +1,5 @@
 from pathlib import Path
 from src.ircfa_interaction.memory_policy import (
    contains_sensitive_memory_content,
    is_explicit_memory_request,
@ -15,3 +17,11 @@ def test_sensitive_content_is_not_eligible_for_memory():
    assert contains_sensitive_memory_content("Remember this API key.")
    assert contains_sensitive_memory_content("Simpan password saya.")
    assert not contains_sensitive_memory_content("Remember that I prefer concise answers.")
 def test_stream_keeps_memory_policy_after_context_compaction():
    stream_source = Path(__file__).parents[2] / "src" / "ircfa_interaction" / "stream2.py"
    source = stream_source.read_text()
    # One import and two prompt-build paths: normal and after compaction.
    assert source.count("MEMORY_POLICY_INSTRUCTIONS") >= 3
--- a/fe-digitaltwin-v2/src/shared/components/rcfa-app/MemoryLabModal.tsx
+++ b/fe-digitaltwin-v2/src/shared/components/rcfa-app/MemoryLabModal.tsx
@ -10,7 +10,18 @@ import {
  ModalHeader,
  Spinner,
 } from "@heroui/react";
-import { Brain, Copy, RefreshCw, ShieldCheck, Trash2 } from "lucide-react";
+import {
  Archive,
  Brain,
  Clock3,
  Copy,
  Pin,
  PinOff,
  RefreshCw,
  RotateCcw,
  ShieldCheck,
  Trash2,
 } from "lucide-react";
 import { IRCFA_API_URL } from "@/shared/api-url";
 import toast from "@/shared/components/ui/toaster";
@ -18,6 +29,10 @@ type MemoryRecord = {
  id: string;
  conversation_id?: string | null;
  source_type: string;
  lifecycle: "active" | "archived" | "expired";
  pinned: boolean;
  expires_at?: string | null;
  remember_count: number;
  title?: string | null;
  summary: string;
  excerpt?: string | null;
@ -30,12 +45,24 @@ type MemoryRecord = {
 type MemorySnapshot = {
  items: MemoryRecord[];
  total: number;
  active_count: number;
  archived_count: number;
  expired_count: number;
  active_limit: number;
  capacity_reached: boolean;
  retrieval_limit: number;
  explicit_only: boolean;
  user_private: boolean;
  secrets_blocked: boolean;
 };
 type MemoryUpdate = {
  lifecycle?: "active" | "archived";
  pinned?: boolean;
  expires_in_days?: number;
  clear_expiry?: boolean;
 };
 type MemoryLabModalProps = {
  isOpen: boolean;
  onOpenChange: (isOpen: boolean) => void;
@ -57,11 +84,18 @@ const TEST_PROMPTS = [
  },
 ];
 function expiryText(expiresAt?: string | null) {
  if (!expiresAt) return "No expiry date";
  const date = new Date(expiresAt);
  return Number.isNaN(date.getTime()) ? "Expiry date unknown" : `Expires ${date.toLocaleString()}`;
 }
 export function MemoryLabModal({ isOpen, onOpenChange, authToken }: MemoryLabModalProps) {
  const [snapshot, setSnapshot] = useState<MemorySnapshot | null>(null);
  const [isLoading, setIsLoading] = useState(false);
  const [error, setError] = useState<string | null>(null);
  const [pendingDeleteId, setPendingDeleteId] = useState<string | null>(null);
  const [updatingMemoryId, setUpdatingMemoryId] = useState<string | null>(null);
  const loadMemories = useCallback(async () => {
    if (!authToken) {
@ -98,7 +132,35 @@ export function MemoryLabModal({ isOpen, onOpenChange, authToken }: MemoryLabMod
    }
  };
  const updateMemory = async (memoryId: string, update: MemoryUpdate, successMessage: string) => {
    setUpdatingMemoryId(memoryId);
    try {
      const response = await fetch(`${IRCFA_API_URL}/chat/memory-lab/${memoryId}`, {
        method: "PATCH",
        headers: {
          Authorization: `Bearer ${authToken}`,
          "Content-Type": "application/json",
        },
        body: JSON.stringify(update),
      });
      const payload = await response.json();
      if (!response.ok) throw new Error(payload?.detail || "Could not update this memory.");
      setSnapshot((current) => current ? {
        ...current,
        items: current.items.map((memory) => memory.id === memoryId ? payload.data : memory),
      } : current);
      await loadMemories();
      toast.success(successMessage);
    } catch (updateError) {
      toast.error(updateError instanceof Error ? updateError.message : "Could not update this memory.");
    } finally {
      setUpdatingMemoryId(null);
    }
  };
  const deleteMemory = async (memoryId: string) => {
    setUpdatingMemoryId(memoryId);
    try {
      const response = await fetch(`${IRCFA_API_URL}/chat/memory-lab/${memoryId}`, {
        method: "DELETE",
@ -107,15 +169,13 @@ export function MemoryLabModal({ isOpen, onOpenChange, authToken }: MemoryLabMod
      const payload = await response.json();
      if (!response.ok) throw new Error(payload?.detail || "Could not delete this memory.");
      setSnapshot((current) => current ? {
        ...current,
        total: Math.max(0, current.total - 1),
        items: current.items.filter((memory) => memory.id !== memoryId),
      } : current);
      setPendingDeleteId(null);
      await loadMemories();
      toast.success("Private memory deleted");
    } catch (deleteError) {
      toast.error(deleteError instanceof Error ? deleteError.message : "Could not delete this memory.");
    } finally {
      setUpdatingMemoryId(null);
    }
  };
@ -131,14 +191,17 @@ export function MemoryLabModal({ isOpen, onOpenChange, authToken }: MemoryLabMod
                <Chip size="sm" color="primary" variant="flat">Developer</Chip>
              </span>
              <span className="text-xs font-normal text-slate-500 dark:text-slate-400">
-                A safe place to see, test, and remove your own long-term memory entries.
+                A safe place to see, test, organize, and remove your own long-term memory entries.
              </span>
            </ModalHeader>
            <ModalBody>
              <div className="rounded-xl border border-blue-200 bg-blue-50 p-3 text-sm text-blue-900 dark:border-blue-900/60 dark:bg-blue-950/30 dark:text-blue-100">
                <div className="flex gap-2">
                  <ShieldCheck className="mt-0.5 h-4 w-4 flex-none" />
-                  <p>Only an explicit “remember” request can be saved. Your memories are private to this signed-in account, and obvious passwords or tokens are blocked.</p>
+                  <p>
                    Only an explicit “remember” request can be saved. Your memories are private to this signed-in account,
                    and obvious passwords or tokens are blocked. Memory Lab test notes automatically stop being used after 7 days.
                  </p>
                </div>
              </div>
@ -168,41 +231,81 @@ export function MemoryLabModal({ isOpen, onOpenChange, authToken }: MemoryLabMod
                <div>
                  <h3 className="text-sm font-semibold">Your saved memories</h3>
                  <p className="text-xs text-slate-500 dark:text-slate-400">
-                    {snapshot ? `Showing ${snapshot.items.length} of ${snapshot.total} records. Chat retrieval checks the newest ${snapshot.retrieval_limit}.` : "Loading your private records..."}
+                    {snapshot
                      ? `${snapshot.active_count}/${snapshot.active_limit} active · ${snapshot.archived_count} archived · ${snapshot.expired_count} expired`
                      : "Loading your private records..."}
                  </p>
                </div>
                <Chip size="sm" variant="flat">Private</Chip>
              </div>
              {snapshot?.capacity_reached && (
                <p className="rounded-xl border border-warning-200 bg-warning-50 p-3 text-xs text-warning-800 dark:border-warning-900/60 dark:bg-warning-900/20 dark:text-warning-200">
                  Your active-memory box is full. Archive or delete one note before saving a new one. Nothing will be removed automatically.
                </p>
              )}
              {isLoading && !snapshot ? (
                <div className="flex justify-center py-8"><Spinner size="sm" /></div>
              ) : error ? (
                <p className="rounded-xl bg-danger-50 p-3 text-sm text-danger-700 dark:bg-danger-900/20 dark:text-danger-300">{error}</p>
              ) : snapshot?.items.length ? (
                <div className="space-y-2">
-                  {snapshot.items.map((memory) => (
+                  {snapshot.items.map((memory) => {
-                    <div key={memory.id} className="rounded-xl border border-slate-200 p-3 dark:border-slate-700">
+                    const isUpdating = updatingMemoryId === memory.id;
-                      <div className="flex items-start justify-between gap-3">
+                    const isActive = memory.lifecycle === "active";
-                        <div className="min-w-0">
+                    const isExpired = memory.lifecycle === "expired";
-                          <p className="truncate text-sm font-medium">{memory.title || "Untitled memory"}</p>
+                    return (
-                          <p className="mt-1 whitespace-pre-wrap break-words text-xs text-slate-600 dark:text-slate-300">{memory.summary}</p>
+                      <div key={memory.id} className="rounded-xl border border-slate-200 p-3 dark:border-slate-700">
-                          <p className="mt-2 text-[10px] text-slate-400">
+                        <div className="flex items-start justify-between gap-3">
-                            {memory.source_type.replaceAll("_", " ")} · {memory.created_at ? new Date(memory.created_at).toLocaleString() : "Unknown date"}
+                          <div className="min-w-0">
-                          </p>
+                            <div className="flex flex-wrap items-center gap-1.5">
-                        </div>
+                              <p className="truncate text-sm font-medium">{memory.title || "Untitled memory"}</p>
-                        {pendingDeleteId === memory.id ? (
+                              <Chip size="sm" variant="flat">{memory.lifecycle}</Chip>
-                          <div className="flex shrink-0 flex-col gap-1">
+                              {memory.pinned && <Chip size="sm" color="primary" variant="flat">Pinned</Chip>}
-                            <Button size="sm" variant="light" onPress={() => setPendingDeleteId(null)}>Cancel</Button>
+                            </div>
-                            <Button size="sm" color="danger" onPress={() => void deleteMemory(memory.id)}>Delete</Button>
+                            <p className="mt-1 whitespace-pre-wrap break-words text-xs text-slate-600 dark:text-slate-300">{memory.summary}</p>
                            <p className="mt-2 text-[10px] text-slate-400">
                              {memory.source_type.replaceAll("_", " ")} · {memory.created_at ? new Date(memory.created_at).toLocaleString() : "Unknown date"}
                              {memory.remember_count > 1 ? ` · saved ${memory.remember_count} times` : ""}
                            </p>
                            <p className="mt-0.5 flex items-center gap-1 text-[10px] text-slate-400">
                              <Clock3 className="h-3 w-3" /> {expiryText(memory.expires_at)}
                            </p>
                          </div>
-                        ) : (
+                          {pendingDeleteId === memory.id ? (
-                          <Button isIconOnly size="sm" variant="light" color="danger" aria-label="Delete private memory" onPress={() => setPendingDeleteId(memory.id)}>
+                            <div className="flex shrink-0 flex-col gap-1">
-                            <Trash2 className="h-4 w-4" />
+                              <Button size="sm" variant="light" onPress={() => setPendingDeleteId(null)} isDisabled={isUpdating}>Cancel</Button>
-                          </Button>
+                              <Button size="sm" color="danger" onPress={() => void deleteMemory(memory.id)} isLoading={isUpdating}>Delete</Button>
-                        )}
+                            </div>
                          ) : (
                            <Button isIconOnly size="sm" variant="light" color="danger" aria-label="Delete private memory" onPress={() => setPendingDeleteId(memory.id)} isDisabled={isUpdating}>
                              <Trash2 className="h-4 w-4" />
                            </Button>
                          )}
                        </div>
                        <div className="mt-3 flex flex-wrap gap-1.5">
                          {isActive ? (
                            <>
                              <Button size="sm" variant="flat" startContent={memory.pinned ? <PinOff className="h-3.5 w-3.5" /> : <Pin className="h-3.5 w-3.5" />} onPress={() => void updateMemory(memory.id, { pinned: !memory.pinned }, memory.pinned ? "Memory unpinned" : "Memory pinned")} isDisabled={isUpdating}>
                                {memory.pinned ? "Unpin" : "Pin"}
                              </Button>
                              <Button size="sm" variant="flat" startContent={<Clock3 className="h-3.5 w-3.5" />} onPress={() => void updateMemory(memory.id, memory.expires_at ? { clear_expiry: true } : { expires_in_days: 7 }, memory.expires_at ? "Expiry removed" : "Memory will expire in 7 days")} isDisabled={isUpdating}>
                                {memory.expires_at ? "Keep forever" : "Expire in 7 days"}
                              </Button>
                              <Button size="sm" variant="flat" startContent={<Archive className="h-3.5 w-3.5" />} onPress={() => void updateMemory(memory.id, { lifecycle: "archived" }, "Memory archived")} isDisabled={isUpdating}>
                                Archive
                              </Button>
                            </>
                          ) : (
                            <Button size="sm" variant="flat" startContent={<RotateCcw className="h-3.5 w-3.5" />} onPress={() => void updateMemory(memory.id, { lifecycle: "active", clear_expiry: isExpired }, "Memory restored")} isDisabled={isUpdating}>
                              Restore
                            </Button>
                          )}
                        </div>
                      </div>
-                    </div>
+                    );
-                  ))}
+                  })}
                </div>
              ) : (
                <p className="rounded-xl border border-dashed border-slate-300 p-4 text-center text-sm text-slate-500 dark:border-slate-700 dark:text-slate-400">