You cannot select more than 25 topics
Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
41 lines
2.5 KiB
Markdown
41 lines
2.5 KiB
Markdown
# RCFA Memory Variant Comparison
|
|
|
|
## Versions
|
|
|
|
- `/home/nvidia/dev-rcfa` is the restored application as it existed before the private-memory changes.
|
|
- `/home/nvidia/dev-rcfa-memory` is the comparison copy with private, explicit-only memory behavior.
|
|
|
|
The original application already contained an `RCFAMemory` table and retrieval path. The memory variant changes that existing behavior; it does not introduce memory into an application that had none.
|
|
|
|
## Observable differences
|
|
|
|
| Behavior | Restored original | Private-memory variant |
|
|
| --- | --- | --- |
|
|
| Ordinary chat of 25+ characters | May be saved automatically | Is not saved as durable memory |
|
|
| Explicit “remember/ingat” request | Can be stored | Stored for the authenticated user only |
|
|
| Knowledge/global memory | May be retrieved across users | Global records are ignored |
|
|
| Secret-like memory | No dedicated rejection policy | API keys, passwords, tokens, and credentials are rejected |
|
|
| Prompt ordering | Memory is appended after the newest question | Memory is placed before the newest question and marked as untrusted reference data |
|
|
|
|
## Suggested comparison
|
|
|
|
Use isolated test databases for the two backends. Do not run both variants against production data.
|
|
|
|
1. Start a fresh conversation in each version.
|
|
2. Send: `For this investigation, discuss Boiler Feed Pump in concise bullet points.`
|
|
3. Start another conversation and ask: `What do you remember about Boiler Feed Pump?`
|
|
4. The restored original may retrieve the first message because it automatically persisted a sufficiently long chat. The private-memory variant should not persist it.
|
|
5. In a fresh conversation, send: `Remember that I prefer concise bullet-point answers.`
|
|
6. Start another conversation and ask for an RCFA explanation. The private-memory variant should retrieve that explicit preference for the same authenticated user.
|
|
7. With dummy data only, try: `Remember my API key is DEMO-NOT-A-REAL-KEY.` The private-memory variant should refuse to store it.
|
|
|
|
For an ownership test, use two disposable test users. A memory saved by the first user must not be retrieved for the second user in the private-memory variant.
|
|
|
|
## Verification
|
|
|
|
- Restored original backend: 15 unit tests pass.
|
|
- Private-memory backend: 17 unit tests pass, including the two memory-policy tests.
|
|
- Both variants pass Python syntax compilation for the affected backend modules.
|
|
|
|
The comparison copy excludes `.git`, dependency directories, caches, logs, and `.env` files. Install/reuse dependencies and provide a separate local environment configuration before running it.
|