Compare commits
1 Commits
main
...
feature/re
| Author | SHA1 | Date |
|---|---|---|
 Cizz22
|
aa106a5a85 | 9 months ago |
@ -1,208 +0,0 @@
|
||||
# DigitalTwin - Optimum Overhaul App (be-optimumoh) API Documentation
|
||||
|
||||
This document maps all the API endpoints, route handlers, database models, and external service layers inside the `be-optimumoh` microservice. It is a FastAPI-based backend engine managing Overhaul schedules, standard scopes, spare parts readiness, Temporal workflow executions, and optimization/constraint calculations.
|
||||
|
||||
---
|
||||
|
||||
## 📊 Summary of Services
|
||||
|
||||
| Service / Module | Purpose | Endpoints Count | Primary Database Tables |
|
||||
| :--- | :--- | :---: | :--- |
|
||||
| **Overhauls** (`overhauls`) | Core overhaul session status, critical parts, schedules, and systems overview. | 4 | `oh_ms_overhaul` |
|
||||
| **Standard Scope** (`scope-equipments`) | Fleet scopes configuration, master equipment availability, and historical overhaul logs. | 4 | `oh_ms_standard_scope`, `oh_ms_equipment_oh_history` |
|
||||
| **Overhaul Activities** (`overhaul-activity`) | Operations/activities scheduling and batch additions to active overhaul sessions. | 4 | `oh_tr_overhaul_activity` |
|
||||
| **Workscope Groups** (`workscopes`) | Maintenance activity groups, tasks, and overhaul scheduling. | 5 | `oh_ms_workscope_group`, `oh_ms_workscope_task` |
|
||||
| **Spare Parts** (`spareparts`) | Global inventory levels, procurement planning status, and planning remarks. | 2 | `oh_ms_sparepart`, `oh_ms_sparepart_remark` |
|
||||
| **Equipment Workscopes** (`equipment-workscopes`) | Equipment-specific maintenance work scope groups mapping and jobs tracking. | 3 | `oh_tr_equipment_workscope_group` |
|
||||
| **Equipment Spare Parts** (`equipment-spareparts`) | Material part catalog tracking per asset location tag. | 1 | `oh_ms_scope_equipment_part` |
|
||||
| **Monitoring Gantt** (`overhaul-gantt`) | Google Sheets Gantt performance charts extraction and sync triggers. | 3 | `oh_ms_monitoring_spreadsheet` |
|
||||
| **Overhaul Schedules** (`overhaul-schedules`) | Detailed calendar session creation, history tracking, updates, and deletes. | 6 | `oh_ms_overhaul` |
|
||||
| **Time Constraints** (`time-constraint`) | Scheduling simulations under a rigid time window restriction. | 9 | `oh_ms_calculation_param`, `oh_tr_calculation_result` |
|
||||
| **Optimized Time Constraints** | Simulations for ideal schedules under a variable optimal calendar logic. | 9 | `oh_tr_calculation_data`, `oh_tr_calculation_equipment_result` |
|
||||
| **Target Reliability** | Background RBD Monte Carlo simulations executed via Temporal workflows. | 2 | None (Leverages Temporal Client SDK) |
|
||||
| **Budget Constraints** | Financial threshold filtering and consequence calculation matrix. | 1 | `oh_tr_calculation_result` |
|
||||
|
||||
---
|
||||
|
||||
## 🛡️ App Middlewares & Framework Architecture
|
||||
|
||||
### 1. Security Headers Middleware
|
||||
FastAPI adds rigid HTTP security headers to all incoming connections inside `src/main.py`. In production, it enforces `Strict-Transport-Security`, `X-Frame-Options` (DENY), and full CSP blockades. In development mode, it supports relaxed parameters.
|
||||
|
||||
### 2. Request Sanitization & Slowapi Limiting
|
||||
All JSON and path structures pass through `RequestValidationMiddleware` (`slowapi` integration) to enforce rate limiting thresholds globally, returning standard payload error formatting on breaches.
|
||||
|
||||
### 3. Multi-Session Context Scoping
|
||||
Scoped database sessions are generated per request context dynamically utilizing ContextVars in `src/context.py` and SQLAlchemy's async connection pool engine:
|
||||
* `db` session: Optimum overhaul local PostgreSQL dataset (`default`).
|
||||
* `collector_db` session: Asset metadata and equipment configurations.
|
||||
|
||||
---
|
||||
|
||||
## 🛠️ Complete API Endpoints Catalog
|
||||
|
||||
```mermaid
|
||||
graph TD
|
||||
UI[Frontend Client Application] -->|JWTBearer token| G[FastAPI Gateway Router]
|
||||
G -->|Overhaul Master| OH[Overhaul Service]
|
||||
G -->|Standard Scopes| SS[Standard Scope Service]
|
||||
G -->|Spare Parts| SP[Spare Parts Service]
|
||||
G -->|Gantt Tracking| GT[Gantt Chart Service]
|
||||
G -->|Simulations API| TC[Constraint Calculations Service]
|
||||
TC -->|Simulation Worker| TMP[Temporalio Cluster Engine]
|
||||
```
|
||||
|
||||
### 1. 📂 Overhauls Service (`overhauls`)
|
||||
Prefix: `/overhauls` — Exposes master overhaul dashboard summaries, schedules, critical parts status, and system maps.
|
||||
|
||||
| Service | Endpoint | Method | Type | DB Tables Touched | Called By | LOC | Protect? (Y/N) | Dependencies |
|
||||
| :--- | :--- | :---: | :--- | :--- | :--- | :---: | :---: | :--- |
|
||||
| **Overhauls** | `/overhauls` | `GET` | Algorithm | `oh_ms_overhaul` | Frontend | ~28 | **Y** | JWTBearer |
|
||||
| **Overhauls** | `/overhauls/schedules` | `GET` | CRUD | `oh_ms_overhaul` | Frontend | ~8 | **Y** | JWTBearer |
|
||||
| **Overhauls** | `/overhauls/critical-parts` | `GET` | CRUD | `oh_ms_sparepart`, `oh_ms_sparepart_procurement` | Frontend | ~8 | **Y** | JWTBearer |
|
||||
| **Overhauls** | `/overhauls/system-components` | `GET` | CRUD | None (Static mapping) | Frontend | ~10 | **Y** | JWTBearer |
|
||||
|
||||
---
|
||||
|
||||
### 2. 📋 Standard Scope Service (`scope-equipments`)
|
||||
Prefix: `/scope-equipments` — Manages fleet-wide equipment checklists and standard scopes configurations.
|
||||
|
||||
| Service | Endpoint | Method | Type | DB Tables Touched | Called By | LOC | Protect? (Y/N) | Dependencies |
|
||||
| :--- | :--- | :---: | :--- | :--- | :--- | :---: | :---: | :--- |
|
||||
| **Standard Scope** | `/scope-equipments` | `GET` | CRUD | `oh_ms_standard_scope` | Frontend | ~10 | **Y** | JWTBearer |
|
||||
| **Standard Scope** | `/scope-equipments/available/{scope_name}` | `GET` | CRUD | `ms_equipment_master`, `oh_ms_standard_scope` | Frontend | ~9 | **Y** | JWTBearer |
|
||||
| **Standard Scope** | `/scope-equipments` | `POST` | CRUD | `oh_ms_standard_scope` | Frontend | ~8 | **Y** | JWTBearer |
|
||||
| **Standard Scope** | `/scope-equipments/history/{oh_session_id}`| `GET` | CRUD | `oh_ms_equipment_oh_history`, `ms_equipment_master` | Frontend | ~10 | **Y** | JWTBearer |
|
||||
|
||||
---
|
||||
|
||||
### 3. 🏃♂️ Overhaul Activity Service (`overhaul-activity`)
|
||||
Prefix: `/overhaul-activity` — Controls tasks assigned to individual overhaul sessions.
|
||||
|
||||
| Service | Endpoint | Method | Type | DB Tables Touched | Called By | LOC | Protect? (Y/N) | Dependencies |
|
||||
| :--- | :--- | :---: | :--- | :--- | :--- | :---: | :---: | :--- |
|
||||
| **Overhaul Activity** | `/overhaul-activity/{overhaul_session}` | `GET` | CRUD | `oh_tr_overhaul_activity` | Frontend | ~25 | **Y** | JWTBearer |
|
||||
| **Overhaul Activity** | `/overhaul-activity/{overhaul_session_id}`| `POST` | CRUD | `oh_tr_overhaul_activity` | Frontend | ~15 | **Y** | JWTBearer |
|
||||
| **Overhaul Activity** | `/overhaul-activity/{overhaul_session}/{assetnum}`| `GET` | CRUD | `oh_tr_overhaul_activity` | Frontend | ~17 | **Y** | JWTBearer |
|
||||
| **Overhaul Activity** | `/overhaul-activity/delete/{overhaul_session}/{location_tag}`| `POST` | CRUD | `oh_tr_overhaul_activity` | Frontend | ~8 | **Y** | JWTBearer |
|
||||
|
||||
---
|
||||
|
||||
### 4. 🔨 Workscope Group Service (`workscopes`)
|
||||
Prefix: `/workscopes` — Defines tasks and maintenance groupings.
|
||||
|
||||
| Service | Endpoint | Method | Type | DB Tables Touched | Called By | LOC | Protect? (Y/N) | Dependencies |
|
||||
| :--- | :--- | :---: | :--- | :--- | :--- | :---: | :---: | :--- |
|
||||
| **Workscope Group** | `/workscopes` | `GET` | CRUD | `oh_ms_workscope_group` | Frontend | ~12 | **Y** | JWTBearer |
|
||||
| **Workscope Group** | `/workscopes` | `POST` | CRUD | `oh_ms_workscope_group` | Frontend | ~8 | **Y** | JWTBearer |
|
||||
| **Workscope Group** | `/workscopes/{scope_equipment_activity_id}`| `GET` | CRUD | `oh_ms_workscope_task` | Frontend | ~12 | **Y** | JWTBearer |
|
||||
| **Workscope Group** | `/workscopes/update/{scope_equipment_activity_id}`| `POST` | CRUD | `oh_ms_workscope_task` | Frontend | ~17 | **Y** | JWTBearer |
|
||||
| **Workscope Group** | `/workscopes/delete/{scope_equipment_activity_id}`| `POST` | CRUD | `oh_ms_workscope_task` | Frontend | ~14 | **Y** | JWTBearer |
|
||||
|
||||
---
|
||||
|
||||
### 5. 📦 Spare Parts Service (`spareparts`)
|
||||
Prefix: `/spareparts` — Manages part catalogs and remark notations.
|
||||
|
||||
| Service | Endpoint | Method | Type | DB Tables Touched | Called By | LOC | Protect? (Y/N) | Dependencies |
|
||||
| :--- | :--- | :---: | :--- | :--- | :--- | :---: | :---: | :--- |
|
||||
| **Spare Parts** | `/spareparts` | `GET` | CRUD | `oh_ms_sparepart`, `oh_ms_sparepart_procurement` | Frontend | ~12 | **Y** | JWTBearer |
|
||||
| **Spare Parts** | `/spareparts` | `POST` | CRUD | `oh_ms_sparepart_remark` | Frontend | ~8 | **Y** | JWTBearer |
|
||||
|
||||
---
|
||||
|
||||
### 6. 🔌 Equipment Workscopes Service (`equipment-workscopes`)
|
||||
Prefix: `/equipment-workscopes` — Maps equipment directly to defined work scopes.
|
||||
|
||||
| Service | Endpoint | Method | Type | DB Tables Touched | Called By | LOC | Protect? (Y/N) | Dependencies |
|
||||
| :--- | :--- | :---: | :--- | :--- | :--- | :---: | :---: | :--- |
|
||||
| **Equipment Workscopes**| `/equipment-workscopes/{location_tag}`| `GET` | CRUD | `oh_tr_equipment_workscope_group` | Frontend | ~12 | **Y** | JWTBearer |
|
||||
| **Equipment Workscopes**| `/equipment-workscopes/{assetnum}` | `POST` | CRUD | `oh_tr_equipment_workscope_group` | Frontend | ~11 | **Y** | JWTBearer |
|
||||
| **Equipment Workscopes**| `/equipment-workscopes/delete/{scope_job_id}`| `POST` | CRUD | `oh_tr_equipment_workscope_group` | Frontend | ~9 | **Y** | JWTBearer |
|
||||
|
||||
---
|
||||
|
||||
### 7. 🔩 Equipment Spare Parts Service (`equipment-spareparts`)
|
||||
Prefix: `/equipment-spareparts` — Lists active spare parts mapping per equipment tag.
|
||||
|
||||
| Service | Endpoint | Method | Type | DB Tables Touched | Called By | LOC | Protect? (Y/N) | Dependencies |
|
||||
| :--- | :--- | :---: | :--- | :--- | :--- | :---: | :---: | :--- |
|
||||
| **Equipment Spareparts**| `/equipment-spareparts/{location_tag}`| `GET` | CRUD | `oh_ms_scope_equipment_part` | Frontend | ~10 | **Y** | JWTBearer |
|
||||
|
||||
---
|
||||
|
||||
### 8. 📊 Overhaul Gantt Chart Service (`overhaul-gantt`)
|
||||
Prefix: `/overhaul-gantt` — Coordinates Google Sheets sync actions to build overhaul performance monitoring dashboards.
|
||||
|
||||
| Service | Endpoint | Method | Type | DB Tables Touched | Called By | LOC | Protect? (Y/N) | Dependencies |
|
||||
| :--- | :--- | :---: | :--- | :--- | :--- | :---: | :---: | :--- |
|
||||
| **Overhaul Gantt** | `/overhaul-gantt` | `GET` | Integration | `oh_ms_monitoring_spreadsheet` | Frontend | ~19 | **Y** | Google Sheets APIs, cell parsers |
|
||||
| **Overhaul Gantt** | `/overhaul-gantt/spreadsheet`| `GET` | CRUD | `oh_ms_monitoring_spreadsheet` | Frontend | ~22 | **Y** | JWTBearer |
|
||||
| **Overhaul Gantt** | `/overhaul-gantt/spreadsheet`| `POST` | CRUD | `oh_ms_monitoring_spreadsheet` | Frontend | ~36 | **Y** | Google Sheets URL validator |
|
||||
|
||||
---
|
||||
|
||||
### 9. 📅 Overhaul Session Schedules Service (`overhaul-schedules`)
|
||||
Prefix: `/overhaul-schedules` — Controls scheduler calendar timelines.
|
||||
|
||||
| Service | Endpoint | Method | Type | DB Tables Touched | Called By | LOC | Protect? (Y/N) | Dependencies |
|
||||
| :--- | :--- | :---: | :--- | :--- | :--- | :---: | :---: | :--- |
|
||||
| **Schedules** | `/overhaul-schedules` | `GET` | CRUD | `oh_ms_overhaul` | Frontend | ~10 | **Y** | JWTBearer |
|
||||
| **Schedules** | `/overhaul-schedules/history`| `GET` | CRUD | `oh_ms_overhaul` | Frontend | ~4 | **Y** | JWTBearer |
|
||||
| **Schedules** | `/overhaul-schedules/{overhaul_session_id}`| `GET` | CRUD | `oh_ms_overhaul` | Frontend | ~10 | **Y** | JWTBearer |
|
||||
| **Schedules** | `/overhaul-schedules` | `POST` | CRUD | `oh_ms_overhaul` | Frontend | ~5 | **Y** | JWTBearer |
|
||||
| **Schedules** | `/overhaul-schedules/update/{scope_id}`| `POST` | CRUD | `oh_ms_overhaul` | Frontend | ~12 | **Y** | JWTBearer |
|
||||
| **Schedules** | `/overhaul-schedules/delete/{scope_id}`| `POST` | CRUD | `oh_ms_overhaul` | Frontend | ~13 | **Y** | JWTBearer |
|
||||
|
||||
---
|
||||
|
||||
### 10. ⏱️ Time Constraints Calculation Service (`time-constraint`)
|
||||
Prefix: `/calculation/time-constraint` — Simulates failure counts and risks under strict overhaul downtime constraints.
|
||||
|
||||
| Service | Endpoint | Method | Type | DB Tables Touched | Called By | LOC | Protect? (Y/N) | Dependencies |
|
||||
| :--- | :--- | :---: | :--- | :--- | :--- | :---: | :---: | :--- |
|
||||
| **Time Constraints** | `/calculation/time-constraint` | `POST` | Algorithm | `oh_ms_calculation_param` | Frontend | ~36 | **Y** | RBD risk simulation logic |
|
||||
| **Time Constraints** | `/calculation/time-constraint` | `GET` | CRUD | `oh_tr_calculation_data` | Frontend | ~18 | **Y** | JWTBearer |
|
||||
| **Time Constraints** | `/calculation/time-constraint/parameters`| `GET` | CRUD | `oh_ms_calculation_param` | Frontend | ~13 | **Y** | JWTBearer |
|
||||
| **Time Constraints** | `/calculation/time-constraint/{calculation_id}`| `GET` | Algorithm | `oh_tr_calculation_result` | Frontend | ~18 | **N** | Internal Auth Key (Unprotected endpoint) |
|
||||
| **Time Constraints** | `/calculation/time-constraint/{calculation_id}/{assetnum}`| `GET` | CRUD | `oh_tr_calculation_equipment_result` | Frontend | ~9 | **Y** | JWTBearer |
|
||||
| **Time Constraints** | `/calculation/time-constraint/{calculation_id}/simulation`| `POST` | Algorithm | `oh_tr_calculation_result` | Frontend | ~13 | **Y** | Temporal timelines calculator |
|
||||
| **Time Constraints** | `/calculation/time-constraint/update/{calculation_id}`| `POST` | CRUD | `oh_tr_calculation_equipment_result` | Frontend | ~18 | **Y** | Bulk parameters updates |
|
||||
| **Time Constraints** | `/calculation/time-constraint/{calculation_id}/refresh-spareparts`| `POST` | Integration | `oh_ms_sparepart`, `oh_ms_calculation_param` | Frontend | ~15 | **Y** | Inventory sync parameters |
|
||||
| **Time Constraints** | `/calculation/time-constraint/{calculation_id}/recalculate`| `POST` | Algorithm / Int | `oh_tr_calculation_result` | Frontend | ~17 | **Y** | Temporal workflows trigger |
|
||||
|
||||
---
|
||||
|
||||
### 11. ⚙️ Optimized Time Constraints Service (`optimum-time-constraint`)
|
||||
Prefix: `/calculation/optimum-time-constraint` — Solves for optimized downtime intervals.
|
||||
|
||||
| Service | Endpoint | Method | Type | DB Tables Touched | Called By | LOC | Protect? (Y/N) | Dependencies |
|
||||
| :--- | :--- | :---: | :--- | :--- | :--- | :---: | :---: | :--- |
|
||||
| **Optimum Constraints** | `/calculation/optimum-time-constraint` | `POST` | Algorithm | `oh_ms_calculation_param` | Frontend | ~29 | **Y** | Optimum solver iterations |
|
||||
| **Optimum Constraints** | `/calculation/optimum-time-constraint` | `GET` | CRUD | `oh_tr_calculation_data` | Frontend | ~15 | **Y** | JWTBearer |
|
||||
| **Optimum Constraints** | `/calculation/optimum-time-constraint/parameters`| `GET` | CRUD | `oh_ms_calculation_param` | Frontend | ~13 | **Y** | JWTBearer |
|
||||
| **Optimum Constraints** | `/calculation/optimum-time-constraint/{calculation_id}`| `GET` | Algorithm | `oh_tr_calculation_result` | Frontend | ~19 | **N** | Internal Auth Key (Unprotected endpoint) |
|
||||
| **Optimum Constraints** | `/calculation/optimum-time-constraint/{calculation_id}/{assetnum}`| `GET` | CRUD | `oh_tr_calculation_equipment_result` | Frontend | ~8 | **Y** | JWTBearer |
|
||||
| **Optimum Constraints** | `/calculation/optimum-time-constraint/{calculation_id}/simulation`| `POST` | Algorithm | `oh_tr_calculation_result` | Frontend | ~13 | **Y** | Optimization simulation matrix |
|
||||
| **Optimum Constraints** | `/calculation/optimum-time-constraint/update/{calculation_id}`| `POST` | CRUD | `oh_tr_calculation_equipment_result` | Frontend | ~18 | **Y** | JWTBearer |
|
||||
| **Optimum Constraints** | `/calculation/optimum-time-constraint/{calculation_id}/refresh-spareparts`| `POST` | Integration | `oh_ms_sparepart` | Frontend | ~15 | **Y** | Inventory sync parameters |
|
||||
| **Optimum Constraints** | `/calculation/optimum-time-constraint/{calculation_id}/recalculate`| `POST` | Algorithm / Int | `oh_tr_calculation_result` | Frontend | ~17 | **Y** | Temporal workflows trigger |
|
||||
|
||||
---
|
||||
|
||||
### 12. ⚡ Target Reliability Simulation Service (`target-reliability`)
|
||||
Prefix: `/calculation/target-reliability` — Leverages Temporal client triggers to dispatch heavy RBD Monte Carlo workflows.
|
||||
|
||||
| Service | Endpoint | Method | Type | DB Tables Touched | Called By | LOC | Protect? (Y/N) | Dependencies |
|
||||
| :--- | :--- | :---: | :--- | :--- | :--- | :---: | :---: | :--- |
|
||||
| **Target Reliability** | `/calculation/target-reliability/simulate`| `POST` | Integration | None (Triggers Temporal workflow) | Frontend | ~18 | **Y** | Temporal IO client runner |
|
||||
| **Target Reliability** | `/calculation/target-reliability` | `GET` | Algorithm | `oh_ms_overhaul` | Frontend | ~88 | **Y** | Temporal client status tracking, EAF math |
|
||||
|
||||
---
|
||||
|
||||
### 13. 💰 Budget Constraints Service (`budget-constraint`)
|
||||
Prefix: `/calculation/budget-constraint` — Computes maintenance strategies according to cost limit thresholds.
|
||||
|
||||
| Service | Endpoint | Method | Type | DB Tables Touched | Called By | LOC | Protect? (Y/N) | Dependencies |
|
||||
| :--- | :--- | :---: | :--- | :--- | :--- | :---: | :---: | :--- |
|
||||
| **Budget Constraints** | `/calculation/budget-constraint/{session_id}`| `GET` | Algorithm | `oh_ms_overhaul`, `oh_tr_calculation_result` | Frontend | ~25 | **Y** | Risk-cost matrix calculator |
|
||||
@ -1,96 +1,107 @@
|
||||
pipeline {
|
||||
agent any
|
||||
|
||||
|
||||
environment {
|
||||
// Replace with your Docker Hub username/organization
|
||||
DOCKER_HUB_USERNAME = 'aimodocker'
|
||||
// This creates DOCKER_AUTH_USR and DOCKER_AUTH_PSW
|
||||
DOCKER_AUTH = credentials('aimodocker')
|
||||
// Use credentials for Docker Hub
|
||||
DOCKER_CREDENTIALS = credentials('aimodocker')
|
||||
// Replace with your image name
|
||||
IMAGE_NAME = 'oh-service'
|
||||
SERVICE_NAME = 'ahm-app'
|
||||
|
||||
SECURITY_PREFIX = 'security'
|
||||
// Replace with your docker compose service name
|
||||
SERVICE_NAME = 'oh-app'
|
||||
// Variable for Git commit hash
|
||||
GIT_COMMIT_HASH = ''
|
||||
|
||||
// Initialize variables to be updated in script blocks
|
||||
GIT_COMMIT_HASH = ""
|
||||
IMAGE_TAG = ""
|
||||
SECONDARY_TAG = ""
|
||||
// Replace with the SSH credentials for development server
|
||||
// SSH_CREDENTIALS = credentials('backend-server-digitaltwin')
|
||||
// SSH_CREDENTIALS_USR = 'aimo'
|
||||
// SSH_SERVER_IP = '192.168.1.82'
|
||||
}
|
||||
|
||||
|
||||
stages {
|
||||
stage('Checkout & Setup') {
|
||||
stage('Checkout') {
|
||||
steps {
|
||||
script {
|
||||
// Checkout and get git commit hash
|
||||
checkout scm
|
||||
GIT_COMMIT_HASH = sh(script: 'git rev-parse --short HEAD', returnStdout: true).trim()
|
||||
|
||||
// Use env.BRANCH_NAME or logic to handle detached HEAD if necessary
|
||||
def branch = env.BRANCH_NAME ?: 'unknown'
|
||||
echo "Current Branch: ${branch}"
|
||||
|
||||
if (branch == 'main') {
|
||||
IMAGE_TAG = GIT_COMMIT_HASH
|
||||
SECONDARY_TAG = 'latest'
|
||||
} else if (branch == 'oh_security') {
|
||||
IMAGE_TAG = "${SECURITY_PREFIX}-${GIT_COMMIT_HASH}"
|
||||
SECONDARY_TAG = "${SECURITY_PREFIX}-latest"
|
||||
} else {
|
||||
IMAGE_TAG = "temp-${GIT_COMMIT_HASH}"
|
||||
SECONDARY_TAG = "" // Ensure it's empty for other branches
|
||||
}
|
||||
|
||||
echo "Primary Tag: ${IMAGE_TAG}"
|
||||
def commitHash = sh(script: 'git rev-parse --short HEAD', returnStdout: true).trim()
|
||||
GIT_COMMIT_HASH = commitHash
|
||||
echo "Git commit hash: ${GIT_COMMIT_HASH}"
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
stage('Docker Login') {
|
||||
steps {
|
||||
// Fixed variable names based on the 'DOCKER_AUTH' environment key
|
||||
sh "echo ${DOCKER_AUTH_PSW} | docker login -u ${DOCKER_AUTH_USR} --password-stdin"
|
||||
sh '''
|
||||
echo ${DOCKER_CREDENTIALS_PSW} | docker login -u ${DOCKER_CREDENTIALS_USR} --password-stdin
|
||||
'''
|
||||
}
|
||||
}
|
||||
|
||||
stage('Build & Tag') {
|
||||
|
||||
stage('Build Docker Image') {
|
||||
steps {
|
||||
script {
|
||||
def fullImageName = "${DOCKER_HUB_USERNAME}/${IMAGE_NAME}"
|
||||
sh "docker build -t ${fullImageName}:${IMAGE_TAG} ."
|
||||
|
||||
if (SECONDARY_TAG) {
|
||||
sh "docker tag ${fullImageName}:${IMAGE_TAG} ${fullImageName}:${SECONDARY_TAG}"
|
||||
}
|
||||
// Build with commit hash tag
|
||||
sh """
|
||||
docker build -t ${DOCKER_HUB_USERNAME}/${IMAGE_NAME}:latest .
|
||||
docker tag ${DOCKER_HUB_USERNAME}/${IMAGE_NAME}:latest ${DOCKER_HUB_USERNAME}/${IMAGE_NAME}:${GIT_COMMIT_HASH}
|
||||
"""
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
stage('Push to Docker Hub') {
|
||||
steps {
|
||||
script {
|
||||
def fullImageName = "${DOCKER_HUB_USERNAME}/${IMAGE_NAME}"
|
||||
sh "docker push ${fullImageName}:${IMAGE_TAG}"
|
||||
|
||||
if (SECONDARY_TAG) {
|
||||
sh "docker push ${fullImageName}:${SECONDARY_TAG}"
|
||||
}
|
||||
}
|
||||
sh """
|
||||
# Push both tags
|
||||
docker push ${DOCKER_HUB_USERNAME}/${IMAGE_NAME}:${GIT_COMMIT_HASH}
|
||||
docker push ${DOCKER_HUB_USERNAME}/${IMAGE_NAME}:latest
|
||||
"""
|
||||
}
|
||||
}
|
||||
|
||||
// stage('Deploy') {
|
||||
// steps {
|
||||
// script {
|
||||
// sshagent(credentials: ['backend-server-digitaltwin']) {
|
||||
// sh """
|
||||
// ssh -o StrictHostKeyChecking=no -p 12558 aimo@0.tcp.ap.ngrok.io '
|
||||
// cd ~/digital-twin/Docker
|
||||
// sudo docker compose pull ${SERVICE_NAME}
|
||||
// sudo docker compose up -d ${SERVICE_NAME}
|
||||
// '
|
||||
// """
|
||||
// }
|
||||
// }
|
||||
// }
|
||||
// }
|
||||
}
|
||||
|
||||
|
||||
post {
|
||||
always {
|
||||
// Clean up
|
||||
sh 'docker logout'
|
||||
|
||||
// Clean up local images
|
||||
script {
|
||||
sh 'docker logout'
|
||||
def fullImageName = "${DOCKER_HUB_USERNAME}/${IMAGE_NAME}"
|
||||
// Clean up images to save agent disk space
|
||||
sh "docker rmi ${fullImageName}:${IMAGE_TAG} || true"
|
||||
if (SECONDARY_TAG) {
|
||||
sh "docker rmi ${fullImageName}:${SECONDARY_TAG} || true"
|
||||
try {
|
||||
sh """
|
||||
# Push both tags
|
||||
docker rmi ${DOCKER_HUB_USERNAME}/${IMAGE_NAME}:${GIT_COMMIT_HASH}
|
||||
docker rmi ${DOCKER_HUB_USERNAME}/${IMAGE_NAME}:latest
|
||||
"""
|
||||
} catch (err) {
|
||||
echo "Failed to clean up images: ${err}"
|
||||
}
|
||||
}
|
||||
}
|
||||
success {
|
||||
echo "Successfully processed ${env.BRANCH_NAME}."
|
||||
echo "Successfully built, pushed, and deployed Docker image with tags: latest and ${GIT_COMMIT_HASH}"
|
||||
}
|
||||
failure {
|
||||
echo 'Failed to build/push/deploy Docker image!'
|
||||
}
|
||||
}
|
||||
}
|
||||
@ -1,44 +0,0 @@
|
||||
# Unit Testing Guide - be-optimumoh
|
||||
|
||||
This document provides instructions on how to set up and run unit tests for the **be-optimumoh** project.
|
||||
|
||||
## 1. Preparation
|
||||
|
||||
### Install Dependencies
|
||||
Ensure you have all dependencies installed. This project uses `poetry`.
|
||||
|
||||
```bash
|
||||
# Install dependencies
|
||||
poetry install
|
||||
```
|
||||
|
||||
## 2. Configuration
|
||||
|
||||
### Pytest Configuration
|
||||
Ensure the `pytest.ini` file in the root directory points to the `unit` test folder:
|
||||
|
||||
```ini
|
||||
[pytest]
|
||||
testpaths = tests/unit
|
||||
python_files = test_*.py
|
||||
asyncio_mode = auto
|
||||
```
|
||||
|
||||
## 3. Running Tests
|
||||
|
||||
### Run Unit Tests
|
||||
To run all unit tests in the project:
|
||||
|
||||
```bash
|
||||
poetry run pytest tests/unit
|
||||
```
|
||||
|
||||
### Run Specific Unit Test File
|
||||
```bash
|
||||
poetry run pytest tests/unit/test_specific_feature.py
|
||||
```
|
||||
|
||||
## 4. Best Practices
|
||||
|
||||
- **Isolation**: Unit tests should be isolated from external services. Use mocking for APIs and databases.
|
||||
- **Async Testing**: Use `@pytest.mark.asyncio` for asynchronous test functions.
|
||||
@ -1,41 +0,0 @@
|
||||
# Equipment Level Overhaul Optimization
|
||||
|
||||
This document explains the mathematical theory and implementation used to determine the optimal overhaul interval for individual assets.
|
||||
|
||||
## 1. Theoretical Foundation
|
||||
The optimization model is based on the **Expected Total Cost per Unit Time (CPUT)** theory, a standard approach in reliability engineering for age-replacement and block-replacement policies.
|
||||
|
||||
### Objective Function
|
||||
The goal is to find an overhaul interval ($T$) that minimizes the total expected cost amortized over time:
|
||||
|
||||
$$C(T) = \frac{C_{Preventive} + C_{Corrective} \cdot E[N(T)]}{T}$$
|
||||
|
||||
Where:
|
||||
* **$T$**: The overhaul interval (e.g., month 12, 24, etc.).
|
||||
* **$C_{Preventive}$**: The cost of a planned overhaul (materials, labor, and procurement).
|
||||
* **$C_{Corrective}$**: The cost of an unplanned failure (repairs + risk cost of downtime).
|
||||
* **$E[N(T)]$**: The expected number of failures occurring in the interval $(0, T]$. This is derived from the **NHPP (Non-Homogeneous Poisson Process)** reliability model.
|
||||
|
||||
## 2. The Cost Balance (The "U-Curve")
|
||||
Optimization works by balancing two competing costs:
|
||||
1. **Overhaul Cost ($C_{PM}/T$)**: As the interval $T$ increases, the amortized cost of the overhaul decreases (economy of waiting).
|
||||
2. **Failure Risk ($C_{CM} \cdot E[N(T)]/T$)**: As the interval $T$ increases, the probability and expected frequency of failure increase (cost of wear-out).
|
||||
|
||||
The point where these two lines intersect typically represents the **Global Minimum** of the total cost curve, known as the **Optimum Overhaul Month**.
|
||||
|
||||
## 3. Implementation Logic
|
||||
In the `service.py` engine, the search is performed as follows:
|
||||
|
||||
1. **Failure Projection**: The system fetches reliability prediction data (cumulative failures) for the analysis window.
|
||||
2. **Sparepart Simulation**: For every potential month $T$, the system simulates the procurement process to calculate the real $C_{Preventive}$ (including any shortage penalties).
|
||||
3. **Cost Amortization**:
|
||||
```python
|
||||
total_cycle_cost = total_expected_failure_cost + total_preventive_cost
|
||||
cput = total_cycle_cost / month_index
|
||||
```
|
||||
4. **Grid Search**: The system iterates through all months in the analysis window and identifies the index with the lowest `cput` value.
|
||||
|
||||
## 4. Interpretation in UI
|
||||
* **Analysis Window Card**: Shows the CPUT value at your currently selected month.
|
||||
* **Optimum Target Card**: Shows the month $T$ where the curve is at its lowest point.
|
||||
* **Potential Benefit**: Calculated as $CPUT(current) - CPUT(optimum)$. This represents the real monthly cash-flow saving if the maintenance interval is adjusted.
|
||||
@ -1,47 +0,0 @@
|
||||
# Plant Level Overhaul Optimization
|
||||
|
||||
This document explains how individual asset optimizations are aggregated to find the best economic strategy for the entire fleet or plant.
|
||||
|
||||
## 1. Fleet Aggregation Theory
|
||||
At the plant level, the objective is to minimize the **Total Fleet Cost per Unit Time**. This assumes a **Uniform Interval Policy** or a **Synchronized Overhaul Strategy** where the plant looks for a common maintenance rhythm.
|
||||
|
||||
### Plant Objective Function
|
||||
The plant-level cost function is the summation of individual equipment cost functions ($C_i$):
|
||||
|
||||
$$C_{Plant}(T) = \sum_{i=1}^{N} C_i(T) = \frac{\sum_{i=1}^{N} (C_{p,i} + C_{f,i} \cdot E[N_i(T)])}{T}$$
|
||||
|
||||
Where:
|
||||
* **$N$**: Total number of equipments in the scope.
|
||||
* **$C_{p,i}$**: Preventive cost for equipment $i$.
|
||||
* **$C_{f,i}$**: Failure cost for equipment $i$.
|
||||
* **$E[N_i(T)]$**: Expected failures for equipment $i$ until time $T$.
|
||||
|
||||
## 2. Searching for the Fleet Optimum
|
||||
The "actual theory" used by the engine involves a two-phase search:
|
||||
|
||||
### Phase 1: Unconstrained Summation
|
||||
The system calculates the CPUT curve for every piece of equipment independently. It then sums these curves to create a "Fleet U-Curve." The minimum of this sum represents the **Theoretical Fleet Optimum**.
|
||||
|
||||
### Phase 2: Sparepart Interaction & Constraints
|
||||
Unlike a simple sum, the real plant optimum must account for **shared resources** (e.g., a limited budget or limited spare parts).
|
||||
1. **Sparepart Conflicts**: If multiple equipments reach their optimal interval at the same time, the `SparepartManager` checks if there are enough parts in the warehouse.
|
||||
2. **Constraint Penalty**: If parts are missing, a "Procurement Penalty" is added to the $C_{p,i}$ for that specific month, effectively shifting the "U-curve" to the right (delaying) or left (earlier) depending on availability.
|
||||
3. **Final Selection**: The system chooses the Month $T$ that minimizes the *constrained* total fleet cost.
|
||||
|
||||
## 3. Implementation in `service.py`
|
||||
The code uses `numpy` to perform vector addition of the cost curves:
|
||||
|
||||
```python
|
||||
# Aggregate amortized costs for fleet analysis
|
||||
total_corrective_costs += np.array(corrective_costs)
|
||||
total_preventive_costs += np.array(preventive_costs)
|
||||
total_costs += np.array(total_costs_equipment)
|
||||
|
||||
# Find the month T that minimizes the sum
|
||||
fleet_optimal_index = np.argmin(total_costs)
|
||||
```
|
||||
|
||||
## 4. Key Metrics for Decision Makers
|
||||
* **Fleet CPUT**: The average monthly budget required to maintain the plant at the chosen interval.
|
||||
* **Accumulation Control**: By using amortized costs (Rp/Month) instead of total cumulative costs, the plant chart remains stable and allows for direct comparison between intervals regardless of the number of assets.
|
||||
* **Risk vs. Cost**: The plant chart shows the trade-off between the *Fleet Failure Risk* (increasing line) and *Fixed Overhaul Costs* (decreasing line).
|
||||
@ -1,26 +0,0 @@
|
||||
import asyncio
|
||||
from src.database.core import engine
|
||||
from sqlalchemy import text
|
||||
|
||||
async def alter_table():
|
||||
async with engine.begin() as conn:
|
||||
try:
|
||||
await conn.execute(text("ALTER TABLE oh_tr_calculation_data ADD COLUMN plant_results JSONB"))
|
||||
print("Added plant_results")
|
||||
except Exception as e:
|
||||
print(f"plant_results exists: {e}")
|
||||
|
||||
try:
|
||||
await conn.execute(text("ALTER TABLE oh_tr_calculation_data ADD COLUMN fleet_statistics JSONB"))
|
||||
print("Added fleet_statistics")
|
||||
except Exception as e:
|
||||
print(f"fleet_statistics exists: {e}")
|
||||
|
||||
try:
|
||||
await conn.execute(text("ALTER TABLE oh_tr_calculation_data ADD COLUMN analysis_metadata JSONB"))
|
||||
print("Added analysis_metadata")
|
||||
except Exception as e:
|
||||
print(f"analysis_metadata exists: {e}")
|
||||
|
||||
if __name__ == "__main__":
|
||||
asyncio.run(alter_table())
|
||||
@ -1,155 +0,0 @@
|
||||
|
||||
import sys
|
||||
|
||||
with open('/home/atra/Development/be-optimumoh/src/calculation_time_constrains/service.py', 'r') as f:
|
||||
lines = f.readlines()
|
||||
|
||||
# Find the point where it broke
|
||||
# Line 159 is ' \'month_index\': i + 1,\n'
|
||||
# We want to keep up to line 159 (index 158)
|
||||
|
||||
new_lines = lines[:159]
|
||||
new_lines.append(" 'source': source\n")
|
||||
new_lines.append(" }\n")
|
||||
new_lines.append(" \n")
|
||||
new_lines.append(" return monthly_data\n")
|
||||
new_lines.append("\n")
|
||||
new_lines.append(" async def get_simulation_results(self, simulation_id: str = \"default\"):\n")
|
||||
new_lines.append(" \"\"\"Get simulation results for Birnbaum importance calculations\"\"\"\n")
|
||||
new_lines.append(" headers = {\n")
|
||||
new_lines.append(" \"Authorization\": f\"Bearer {self.token}\",\n")
|
||||
new_lines.append(" \"Content-Type\": \"application/json\"\n")
|
||||
new_lines.append(" }\n")
|
||||
new_lines.append("\n")
|
||||
new_lines.append(" calc_result_url = f\"{self.api_base_url}/aeros/simulation/result/calc/{simulation_id}?nodetype=RegularNode\"\n")
|
||||
new_lines.append(" plant_result_url = f\"{self.api_base_url}/aeros/simulation/result/calc/{simulation_id}/plant\"\n")
|
||||
new_lines.append("\n")
|
||||
new_lines.append(" async with httpx.AsyncClient(timeout=300.0) as client:\n")
|
||||
new_lines.append(" calc_task = client.get(calc_result_url, headers=headers)\n")
|
||||
new_lines.append(" plant_task = client.get(plant_result_url, headers=headers)\n")
|
||||
new_lines.append("\n")
|
||||
new_lines.append(" calc_response, plant_response = await asyncio.gather(calc_task, plant_task)\n")
|
||||
new_lines.append("\n")
|
||||
new_lines.append(" calc_response.raise_for_status()\n")
|
||||
new_lines.append(" plant_response.raise_for_status()\n")
|
||||
new_lines.append("\n")
|
||||
new_lines.append(" calc_data = calc_response.json()[\"data\"]\n")
|
||||
new_lines.append(" plant_data = plant_response.json()[\"data\"]\n")
|
||||
new_lines.append("\n")
|
||||
new_lines.append(" return {\n")
|
||||
new_lines.append(" \"calc_result\": calc_data,\n")
|
||||
new_lines.append(" \"plant_result\": plant_data\n")
|
||||
new_lines.append(" }\n")
|
||||
new_lines.append("\n")
|
||||
new_lines.append(" def _calculate_equipment_costs_with_spareparts(self, failures_prediction: Dict, birnbaum_importance: float,\n")
|
||||
new_lines.append(" preventive_cost: float, failure_replacement_cost: float, ecs,\n")
|
||||
new_lines.append(" location_tag: str, planned_overhauls: List = None, loss_production_permonth=0) -> List[Dict]:\n")
|
||||
new_lines.append(" \"\"\"Calculate costs for each month including sparepart costs and availability\"\"\"\n")
|
||||
new_lines.append(" \n")
|
||||
new_lines.append(" if not failures_prediction:\n")
|
||||
new_lines.append(" self.logger.warning(f\"No failure prediction data for {location_tag}\")\n")
|
||||
new_lines.append(" return []\n")
|
||||
new_lines.append("\n")
|
||||
new_lines.append(" results = []\n")
|
||||
new_lines.append(" months = list(failures_prediction.keys())\n")
|
||||
new_lines.append(" num_months = len(months)\n")
|
||||
new_lines.append(" failure_counts = []\n")
|
||||
new_lines.append(" \n")
|
||||
new_lines.append(" monthly_risk_cost_per_failure = 0\n")
|
||||
new_lines.append(" \n")
|
||||
new_lines.append(" if ecs:\n")
|
||||
new_lines.append(" is_trip = 1 if ecs.get(\"Diskripsi Operasional Akibat Equip. Failure\") == \"Trip\" else 0\n")
|
||||
new_lines.append(" is_series = 0 if not birnbaum_importance else math.floor(birnbaum_importance)\n")
|
||||
new_lines.append(" if is_trip:\n")
|
||||
new_lines.append(" downtime = ecs.get(\"Estimasi Waktu Maint. / Downtime / Gangguan (Jam)\")\n")
|
||||
new_lines.append(" monthly_risk_cost_per_failure = 660 * 1000000 * is_trip * downtime * is_series\n")
|
||||
new_lines.append(" \n")
|
||||
new_lines.append(" for month_key in months:\n")
|
||||
new_lines.append(" data = failures_prediction[month_key]\n")
|
||||
new_lines.append(" failure_counts.append(data['cumulative_failures'])\n")
|
||||
new_lines.append(" \n")
|
||||
new_lines.append(" for i in range(num_months):\n")
|
||||
new_lines.append(" month_index = i + 1\n")
|
||||
new_lines.append(" \n")
|
||||
new_lines.append(" # Use only months within the analysis window\n")
|
||||
new_lines.append(" if month_index > self.time_window_months:\n")
|
||||
new_lines.append(" continue\n")
|
||||
new_lines.append("\n")
|
||||
new_lines.append(" # Check sparepart availability for this month\n")
|
||||
new_lines.append(" sparepart_analysis = self._analyze_sparepart_availability(\n")
|
||||
new_lines.append(" location_tag, month_index - 1, planned_overhauls or []\n")
|
||||
new_lines.append(" )\n")
|
||||
new_lines.append(" \n")
|
||||
new_lines.append(" # THEORY: Total Expected Cost per Unit Time (CPUT)\n")
|
||||
new_lines.append(" # Reference: Maintenance Optimization Models (Age/Block Replacement)\n")
|
||||
new_lines.append(" # C(T) = [Total Preventive Cost + Total Expected Corrective Cost(T)] / T\n")
|
||||
new_lines.append(" \n")
|
||||
new_lines.append(" # 1. Total Expected Corrective Cost until month_index (Expected number of failures * cost per failure)\n")
|
||||
new_lines.append(" # In NHPP model, Expected Failures E[N(T)] = Cumulative Failures\n")
|
||||
new_lines.append(" total_expected_failure_cost = failure_counts[i] * (failure_replacement_cost + monthly_risk_cost_per_failure)\n")
|
||||
new_lines.append(" \n")
|
||||
new_lines.append(" # 2. Total Preventive Cost (One-time cost at month_index)\n")
|
||||
new_lines.append(" # Includes labor, materials, and procurement delays\n")
|
||||
new_lines.append(" procurement_cost = sparepart_analysis['total_procurement_cost']\n")
|
||||
new_lines.append(" total_preventive_cost = preventive_cost + procurement_cost\n")
|
||||
new_lines.append(" \n")
|
||||
new_lines.append(" # 3. Expected Total Cycle Cost\n")
|
||||
new_lines.append(" total_cycle_cost = total_expected_failure_cost + total_preventive_cost\n")
|
||||
new_lines.append(" \n")
|
||||
new_lines.append(" # 4. Expected Cost Per Unit Time (Optimization Target)\n")
|
||||
new_lines.append(" # This value forms the U-shaped curve\n")
|
||||
new_lines.append(" cput = total_cycle_cost / month_index\n")
|
||||
new_lines.append(" \n")
|
||||
new_lines.append(" # Store both absolute and amortized components for visualization\n")
|
||||
new_lines.append(" results.append({\n")
|
||||
new_lines.append(" 'month': month_index,\n")
|
||||
new_lines.append(" 'number_of_failures': failure_counts[i],\n")
|
||||
new_lines.append(" 'is_actual': failures_prediction[months[i]].get('source') == 'actual',\n")
|
||||
new_lines.append(" \n")
|
||||
new_lines.append(" # Amortized components (for the \"U\" chart)\n")
|
||||
new_lines.append(" 'failure_cost': total_expected_failure_cost / month_index,\n")
|
||||
new_lines.append(" 'preventive_cost': preventive_cost / month_index,\n")
|
||||
new_lines.append(" 'procurement_cost': procurement_cost / month_index,\n")
|
||||
new_lines.append(" 'total_cost': cput,\n")
|
||||
new_lines.append(" \n")
|
||||
new_lines.append(" # Absolute values (for breakdown analysis)\n")
|
||||
new_lines.append(" 'absolute_failure_cost': total_expected_failure_cost,\n")
|
||||
new_lines.append(" 'absolute_overhaul_cost': preventive_cost,\n")
|
||||
new_lines.append(" 'absolute_procurement_cost': procurement_cost,\n")
|
||||
new_lines.append(" 'total_cycle_cost': total_cycle_cost,\n")
|
||||
new_lines.append("\n")
|
||||
new_lines.append(" 'is_after_planned_oh': month_index > self.planned_oh_months,\n")
|
||||
new_lines.append(" 'delay_months': max(0, month_index - self.planned_oh_months),\n")
|
||||
new_lines.append(" 'procurement_details': sparepart_analysis,\n")
|
||||
new_lines.append(" 'sparepart_available': sparepart_analysis['available'],\n")
|
||||
new_lines.append(" 'sparepart_status': sparepart_analysis['message'],\n")
|
||||
new_lines.append(" 'can_proceed': sparepart_analysis['can_proceed_with_delays']\n")
|
||||
new_lines.append(" })\n")
|
||||
new_lines.append(" \n")
|
||||
new_lines.append(" return results\n")
|
||||
new_lines.append("\n")
|
||||
new_lines.append(" def _analyze_sparepart_availability(self, equipment_tag: str, target_month: int, \n")
|
||||
new_lines.append(" planned_overhauls: List) -> Dict:\n")
|
||||
new_lines.append(" \"\"\"Analyze sparepart availability for equipment at target month\"\"\"\n")
|
||||
new_lines.append(" if not self.sparepart_manager:\n")
|
||||
new_lines.append(" return {\n")
|
||||
new_lines.append(" 'available': True,\n")
|
||||
new_lines.append(" 'message': 'Sparepart manager not initialized',\n")
|
||||
new_lines.append(" 'total_procurement_cost': 0,\n")
|
||||
new_lines.append(" 'procurement_needed': [],\n")
|
||||
new_lines.append(" 'can_proceed_with_delays': True\n")
|
||||
new_lines.append(" }\n")
|
||||
new_lines.append(" \n")
|
||||
new_lines.append(" # Convert planned overhauls to format expected by sparepart manager\n")
|
||||
new_lines.append(" other_overhauls = [(eq_tag, month) for eq_tag, month in planned_overhauls\n")
|
||||
new_lines.append(" if eq_tag != equipment_tag and month <= target_month]\n")
|
||||
new_lines.append("\n")
|
||||
new_lines.append(" return self.sparepart_manager.check_sparepart_availability(\n")
|
||||
new_lines.append(" equipment_tag, target_month, other_overhauls\n")
|
||||
new_lines.append(" )\n")
|
||||
new_lines.append("\n")
|
||||
|
||||
new_lines.extend(lines[159:])
|
||||
|
||||
with open('/home/atra/Development/be-optimumoh/src/calculation_time_constrains/service.py', 'w') as f:
|
||||
f.writelines(new_lines)
|
||||
@ -1,263 +0,0 @@
|
||||
import re
|
||||
|
||||
with open('src/calculation_time_constrains/service.py', 'r') as f:
|
||||
content = f.read()
|
||||
|
||||
# Replacement 1: run_simulation_with_spareparts
|
||||
replacement_1 = """ finally:
|
||||
await optimum_oh_model._close_session()
|
||||
|
||||
# Re-fetch calculation_data with equipment_results to ensure they are loaded
|
||||
from sqlalchemy import select
|
||||
from sqlalchemy.orm import selectinload
|
||||
calculation_query = await db_session.execute(
|
||||
select(CalculationData)
|
||||
.options(selectinload(CalculationData.equipment_results), selectinload(CalculationData.parameter))
|
||||
.where(CalculationData.id == calculation.id)
|
||||
)
|
||||
scope_calculation = calculation_query.scalar_one_or_none()
|
||||
|
||||
data_num = scope_calculation.max_interval
|
||||
all_equipment = scope_calculation.equipment_results
|
||||
included_equipment = [eq for eq in all_equipment if eq.is_included]
|
||||
|
||||
calculation_results = []
|
||||
fleet_statistics = {
|
||||
'total_equipment': len(all_equipment),
|
||||
'included_equipment': len(included_equipment),
|
||||
'excluded_equipment': len(all_equipment) - len(included_equipment),
|
||||
'equipment_with_sparepart_constraints': 0,
|
||||
'total_procurement_items': 0,
|
||||
'critical_procurement_items': 0,
|
||||
'total_spareparts': 745
|
||||
}
|
||||
|
||||
avg_failure_cost = sum((eq.material_cost or 0) + (3 * 111000 * 3) for eq in all_equipment) / len(all_equipment) if all_equipment else 0
|
||||
|
||||
rbd_marginal_fails = [0] * data_num
|
||||
try:
|
||||
if scope_calculation.rbd_simulation_id:
|
||||
from src.config import RBD_SERVICE_API
|
||||
import httpx
|
||||
plant_result_url = f"{RBD_SERVICE_API}/aeros/simulation/result/calc/{scope_calculation.rbd_simulation_id}/plant"
|
||||
async with httpx.AsyncClient(timeout=30.0) as client:
|
||||
response = await client.get(
|
||||
plant_result_url,
|
||||
headers={"Authorization": f"Bearer {token}", "Content-Type": "application/json"}
|
||||
)
|
||||
if response.status_code == 200:
|
||||
plant_data = response.json().get("data", {})
|
||||
timestamp_outs = plant_data.get("timestamp_outs", [])
|
||||
if timestamp_outs:
|
||||
from src.calculation_time_constrains.utils import create_time_series_data, calculate_failures_per_month
|
||||
hourly_data = create_time_series_data(timestamp_outs, max_hours=data_num * 720)
|
||||
cumulative_rbd_fails = calculate_failures_per_month(hourly_data)
|
||||
rbd_fails_map = {m['month']: m['failures'] for m in cumulative_rbd_fails}
|
||||
prev_fail = 0
|
||||
for m in range(1, data_num + 1):
|
||||
curr_fail = rbd_fails_map.get(m, prev_fail)
|
||||
rbd_marginal_fails[m-1] = curr_fail - prev_fail
|
||||
prev_fail = curr_fail
|
||||
except Exception as e:
|
||||
import logging
|
||||
logger = logging.getLogger(__name__)
|
||||
logger.warning(f"Failed to fetch plant simulation: {e}")
|
||||
|
||||
cumulative_plant_failures = 0
|
||||
import numpy as np
|
||||
from .schema import CalculationResultsRead
|
||||
for month_index in range(data_num):
|
||||
historical_marginal_fail = 0
|
||||
for eq in all_equipment:
|
||||
if eq.is_actual and month_index < len(eq.is_actual) and eq.is_actual[month_index]:
|
||||
curr_fail = eq.daily_failures[month_index] if month_index < len(eq.daily_failures) else 0
|
||||
prev_fail = eq.daily_failures[month_index-1] if month_index > 0 and (month_index - 1) < len(eq.daily_failures) else 0
|
||||
historical_marginal_fail += max(0, curr_fail - prev_fail)
|
||||
|
||||
marginal_fail = rbd_marginal_fails[month_index] + historical_marginal_fail
|
||||
cumulative_plant_failures += marginal_fail
|
||||
|
||||
month_result = {
|
||||
"overhaul_cost": 0.0,
|
||||
"corrective_cost": 0.0,
|
||||
"procurement_cost": 0.0,
|
||||
"num_failures": cumulative_plant_failures,
|
||||
"day": month_index + 1,
|
||||
"month": month_index + 1,
|
||||
"procurement_details": {},
|
||||
"sparepart_summary": {
|
||||
"total_procurement_cost": 0.0,
|
||||
"equipment_requiring_procurement": 0,
|
||||
"critical_shortages": 0,
|
||||
"existing_orders_value": 0.0,
|
||||
"new_orders_required": 0,
|
||||
"urgent_procurements": 0
|
||||
}
|
||||
}
|
||||
|
||||
equipment_requiring_procurement = 0
|
||||
total_existing_orders_value = 0.0
|
||||
total_new_orders_value = 0.0
|
||||
critical_shortages = 0
|
||||
urgent_procurements = 0
|
||||
|
||||
for eq in all_equipment:
|
||||
if month_index >= len(eq.procurement_details):
|
||||
continue
|
||||
procurement_detail = eq.procurement_details[month_index]
|
||||
if (procurement_detail and isinstance(procurement_detail, dict) and procurement_detail.get("procurement_needed")):
|
||||
equipment_requiring_procurement += 1
|
||||
pr_po_summary = procurement_detail.get("pr_po_summary", {})
|
||||
existing_orders_value = pr_po_summary.get("total_existing_value", 0)
|
||||
total_existing_orders_value += existing_orders_value
|
||||
new_orders_value = pr_po_summary.get("total_new_orders_value", 0)
|
||||
total_new_orders_value += new_orders_value
|
||||
critical_missing = procurement_detail.get("critical_missing_parts", 0)
|
||||
if critical_missing > 0:
|
||||
critical_shortages += 1
|
||||
recommendations = procurement_detail.get("recommendations", [])
|
||||
urgent_items = [r for r in recommendations if r.get("priority") == "CRITICAL"]
|
||||
if urgent_items:
|
||||
urgent_procurements += 1
|
||||
is_included_eq = False if eq.is_initial else eq.is_included
|
||||
month_result["procurement_details"][eq.location_tag] = {
|
||||
"is_included": is_included_eq,
|
||||
"location_tag": eq.location_tag,
|
||||
"details": procurement_detail.get("procurement_needed", []),
|
||||
"detailed_message": procurement_detail.get("detailed_message", ""),
|
||||
"pr_po_summary": pr_po_summary,
|
||||
"recommendations": recommendations,
|
||||
"sparepart_available": procurement_detail.get("sparepart_available", True),
|
||||
"can_proceed": procurement_detail.get("can_proceed_with_delays", True),
|
||||
"critical_missing_parts": critical_missing,
|
||||
"existing_orders_value": existing_orders_value,
|
||||
"new_orders_value": new_orders_value
|
||||
}
|
||||
if eq.is_included:
|
||||
if (month_index < len(eq.overhaul_costs) and month_index < len(eq.procurement_costs)):
|
||||
month_result["overhaul_cost"] += float(eq.overhaul_costs[month_index])
|
||||
month_result["procurement_cost"] += float(eq.procurement_costs[month_index])
|
||||
|
||||
month_result["corrective_cost"] = (cumulative_plant_failures * avg_failure_cost) / (month_index + 1)
|
||||
month_result["sparepart_summary"].update({
|
||||
"total_procurement_cost": month_result["procurement_cost"],
|
||||
"equipment_requiring_procurement": equipment_requiring_procurement,
|
||||
"critical_shortages": critical_shortages,
|
||||
"existing_orders_value": total_existing_orders_value,
|
||||
"new_orders_required": len([eq for eq in all_equipment if month_index < len(eq.procurement_details) and eq.procurement_details[month_index] and eq.procurement_details[month_index].get("procurement_needed")]),
|
||||
"urgent_procurements": urgent_procurements
|
||||
})
|
||||
month_result["total_cost"] = month_result["corrective_cost"] + month_result["overhaul_cost"] + month_result["procurement_cost"]
|
||||
calculation_results.append(month_result)
|
||||
|
||||
optimum_day = np.argmin([month["total_cost"] for month in calculation_results])
|
||||
scope_calculation.optimum_oh_day = int(optimum_day)
|
||||
|
||||
fleet_statistics['equipment_with_sparepart_constraints'] = len([
|
||||
eq for eq in all_equipment if any(detail and detail.get("procurement_needed") for detail in eq.procurement_details if detail)
|
||||
])
|
||||
fleet_statistics['total_procurement_items'] = sum([
|
||||
len(detail.get("procurement_needed", [])) for eq in all_equipment for detail in eq.procurement_details if detail and isinstance(detail, dict)
|
||||
])
|
||||
|
||||
analysis_metadata = {
|
||||
"planned_month": (scope.start_date.year - prev_oh_scope.end_date.year) * 12 + (scope.start_date.month - prev_oh_scope.end_date.month) if prev_oh_scope and scope else 0,
|
||||
"total_fleet_procurement_cost": sum([eq.procurement_costs[int(scope_calculation.optimum_oh_day)] for eq in all_equipment if eq.procurement_costs]),
|
||||
"last_oh_date": prev_oh_scope.end_date.isoformat() if prev_oh_scope else None,
|
||||
"next_oh_date": scope.start_date.isoformat() if scope else None,
|
||||
"optimal_stat": None
|
||||
}
|
||||
|
||||
calc_results_read = [CalculationResultsRead(**r) for r in calculation_results]
|
||||
optimal_analysis = _analyze_optimal_timing(
|
||||
calc_results_read, scope_calculation.optimum_oh_day, prev_oh_scope, scope
|
||||
)
|
||||
|
||||
scope_calculation.plant_results = calculation_results
|
||||
scope_calculation.fleet_statistics = fleet_statistics
|
||||
scope_calculation.analysis_metadata = analysis_metadata
|
||||
scope_calculation.optimum_analysis = optimal_analysis
|
||||
|
||||
await db_session.commit()
|
||||
|
||||
return {
|
||||
"id": calculation.id,
|
||||
"optimum": optimal_analysis
|
||||
}"""
|
||||
|
||||
pattern_1 = re.compile(r" finally:\n await optimum_oh_model\._close_session\(\).*?return \{\n \"id\": calculation_data\.id,\n \"optimum\": stats\n \}", re.DOTALL)
|
||||
if pattern_1.search(content):
|
||||
content = pattern_1.sub(replacement_1, content)
|
||||
else:
|
||||
print("Could not find Replacement 1 target")
|
||||
|
||||
# Replacement 2: get_calculation_result
|
||||
replacement_2 = """async def get_calculation_result(db_session: DbSession, calculation_id: str, token, include_risk_cost):
|
||||
\"\"\"
|
||||
Get calculation results from DB, returning pre-calculated plant and equipment results.
|
||||
\"\"\"
|
||||
try:
|
||||
# Get calculation data with equipment results
|
||||
calculation_query = await db_session.execute(
|
||||
select(CalculationData)
|
||||
.options(selectinload(CalculationData.equipment_results))
|
||||
.where(CalculationData.id == calculation_id)
|
||||
)
|
||||
scope_calculation = calculation_query.scalar_one_or_none()
|
||||
|
||||
if not scope_calculation:
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_404_NOT_FOUND,
|
||||
detail=f"Calculation with id {calculation_id} does not exist.",
|
||||
)
|
||||
|
||||
scope_overhaul = await get_scope(
|
||||
db_session=db_session,
|
||||
overhaul_session_id=scope_calculation.overhaul_session_id
|
||||
)
|
||||
if not scope_overhaul:
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_404_NOT_FOUND,
|
||||
detail=f"Overhaul scope for session {scope_calculation.overhaul_session_id} does not exist.",
|
||||
)
|
||||
|
||||
# Parse pre-calculated plant results
|
||||
plant_results_raw = scope_calculation.plant_results or []
|
||||
calculation_results = [CalculationResultsRead(**r) for r in plant_results_raw]
|
||||
|
||||
# Return comprehensive result
|
||||
return CalculationTimeConstrainsRead(
|
||||
id=scope_calculation.id,
|
||||
reference=scope_calculation.overhaul_session_id,
|
||||
scope=scope_overhaul.maintenance_type.name,
|
||||
results=calculation_results,
|
||||
optimum_oh=scope_calculation.optimum_oh_day,
|
||||
optimum_oh_month=scope_calculation.optimum_oh_day + 1 if scope_calculation.optimum_oh_day is not None else None,
|
||||
equipment_results=scope_calculation.equipment_results,
|
||||
fleet_statistics=scope_calculation.fleet_statistics or {},
|
||||
optimal_analysis=scope_calculation.optimum_analysis or {},
|
||||
analysis_metadata=scope_calculation.analysis_metadata or {}
|
||||
)
|
||||
|
||||
except HTTPException:
|
||||
raise
|
||||
except Exception as e:
|
||||
import logging
|
||||
logger = logging.getLogger(__name__)
|
||||
logger.error(f"Error in get_calculation_result for calculation_id {calculation_id}: {str(e)}")
|
||||
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_500_INTERNAL_SERVER_ERROR,
|
||||
detail=f"Internal error processing calculation results: {str(e)}",
|
||||
)"""
|
||||
|
||||
pattern_2 = re.compile(r"async def get_calculation_result\(db_session: DbSession, calculation_id: str, token, include_risk_cost\):.*?raise HTTPException\(\n status_code=status\.HTTP_500_INTERNAL_SERVER_ERROR,\n detail=f\"Internal error processing calculation results: \{str\(e\)\}\",\n \)", re.DOTALL)
|
||||
if pattern_2.search(content):
|
||||
content = pattern_2.sub(replacement_2, content)
|
||||
else:
|
||||
print("Could not find Replacement 2 target")
|
||||
|
||||
with open('src/calculation_time_constrains/service.py', 'w') as f:
|
||||
f.write(content)
|
||||
|
||||
print("Patch applied.")
|
||||
Binary file not shown.
|
Before Width: | Height: | Size: 63 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 72 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 63 KiB |
@ -1,6 +0,0 @@
|
||||
[pytest]
|
||||
asyncio_mode = auto
|
||||
testpaths = tests/unit
|
||||
python_files = test_*.py
|
||||
filterwarnings =
|
||||
ignore::pydantic.PydanticDeprecatedSince20
|
||||
@ -1,35 +0,0 @@
|
||||
import asyncio
|
||||
import os
|
||||
from temporalio.client import Client
|
||||
from temporalio.worker import Worker
|
||||
|
||||
from temporal.temporal_workflows import OptimumOHCalculationWorkflow
|
||||
from temporal.temporal_workflows import create_optimum_oh_calculation, request_rbd_simulation, run_optimum_oh_calculation
|
||||
|
||||
TEMPORAL_URL = os.environ.get("TEMPORAL_URL", "http://192.168.1.86:7233")
|
||||
|
||||
async def main():
|
||||
client = await Client.connect(TEMPORAL_URL)
|
||||
|
||||
try:
|
||||
worker = Worker(
|
||||
client,
|
||||
task_queue="oh-sim-queue",
|
||||
workflows=[OptimumOHCalculationWorkflow],
|
||||
activities=[
|
||||
create_optimum_oh_calculation,
|
||||
request_rbd_simulation,
|
||||
run_optimum_oh_calculation
|
||||
],
|
||||
max_concurrent_workflow_tasks=50,
|
||||
max_concurrent_activities=12
|
||||
)
|
||||
await worker.run()
|
||||
except Exception as e:
|
||||
print(f"Worker failed: {e}")
|
||||
import traceback
|
||||
traceback.print_exc()
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
asyncio.run(main())
|
||||
File diff suppressed because it is too large
Load Diff
File diff suppressed because it is too large
Load Diff
@ -1,42 +0,0 @@
|
||||
from contextvars import ContextVar
|
||||
from typing import Optional, Final
|
||||
|
||||
REQUEST_ID_CTX_KEY: Final[str] = "request_id"
|
||||
USER_ID_CTX_KEY: Final[str] = "user_id"
|
||||
USERNAME_CTX_KEY: Final[str] = "username"
|
||||
ROLE_CTX_KEY: Final[str] = "role"
|
||||
|
||||
_request_id_ctx_var: ContextVar[Optional[str]] = ContextVar(REQUEST_ID_CTX_KEY, default=None)
|
||||
_user_id_ctx_var: ContextVar[Optional[str]] = ContextVar(USER_ID_CTX_KEY, default=None)
|
||||
_username_ctx_var: ContextVar[Optional[str]] = ContextVar(USERNAME_CTX_KEY, default=None)
|
||||
_role_ctx_var: ContextVar[Optional[str]] = ContextVar(ROLE_CTX_KEY, default=None)
|
||||
|
||||
|
||||
def get_request_id() -> Optional[str]:
|
||||
return _request_id_ctx_var.get()
|
||||
|
||||
|
||||
def set_request_id(request_id: str):
|
||||
return _request_id_ctx_var.set(request_id)
|
||||
|
||||
|
||||
def reset_request_id(token):
|
||||
_request_id_ctx_var.reset(token)
|
||||
|
||||
def get_user_id() -> Optional[str]:
|
||||
return _user_id_ctx_var.get()
|
||||
|
||||
def set_user_id(user_id: str):
|
||||
return _user_id_ctx_var.set(user_id)
|
||||
|
||||
def get_username() -> Optional[str]:
|
||||
return _username_ctx_var.get()
|
||||
|
||||
def set_username(username: str):
|
||||
return _username_ctx_var.set(username)
|
||||
|
||||
def get_role() -> Optional[str]:
|
||||
return _role_ctx_var.get()
|
||||
|
||||
def set_role(role: str):
|
||||
return _role_ctx_var.set(role)
|
||||
@ -1,22 +0,0 @@
|
||||
from typing import List, Optional
|
||||
|
||||
from pydantic import Field
|
||||
from src.models import DefultBase
|
||||
|
||||
|
||||
class CommonParams(DefultBase):
|
||||
# This ensures no extra query params are allowed
|
||||
current_user: Optional[str] = Field(None, alias="currentUser")
|
||||
page: int = Field(1, gt=0, lt=2147483647)
|
||||
items_per_page: int = Field(5, gt=0, le=50, multiple_of=5, alias="itemsPerPage")
|
||||
query_str: Optional[str] = Field(None, alias="q")
|
||||
filter_spec: Optional[str] = Field(None, alias="filter")
|
||||
sort_by: List[str] = Field(default_factory=list, alias="sortBy[]")
|
||||
descending: List[bool] = Field(default_factory=list, alias="descending[]")
|
||||
exclude: List[str] = Field(default_factory=list, alias="exclude[]")
|
||||
all_params: int = Field(0, alias="all")
|
||||
|
||||
# Property to mirror your original return dict's bool conversion
|
||||
@property
|
||||
def is_all(self) -> bool:
|
||||
return bool(self.all_params)
|
||||
@ -1,332 +1,79 @@
|
||||
from datetime import datetime
|
||||
from typing import Optional, Union
|
||||
from sqlalchemy import select, func, cast, Numeric, text
|
||||
from sqlalchemy import select, func, cast, Numeric
|
||||
from sqlalchemy.orm import Session
|
||||
from sqlalchemy import and_
|
||||
from sqlalchemy.sql import not_
|
||||
from src.maximo.model import WorkOrderData # Assuming this is where your model is
|
||||
from src.database.core import CollectorDbSession, DbSession
|
||||
from src.overhaul_scope.model import OverhaulScope
|
||||
from src.database.core import CollectorDbSession
|
||||
|
||||
|
||||
async def get_cm_cost_summary(collector_db: CollectorDbSession, last_oh_date:datetime, upcoming_oh_date:datetime):
|
||||
query = text("""WITH part_costs AS (
|
||||
SELECT
|
||||
mu.wonum,
|
||||
SUM(mu.itemqty * COALESCE(inv.avgcost, po.unit_cost, 0)) AS parts_total_cost
|
||||
FROM maximo_workorder_materials mu
|
||||
LEFT JOIN maximo_inventory inv
|
||||
ON mu.itemnum = inv.itemnum
|
||||
LEFT JOIN (
|
||||
SELECT item_num, AVG(unit_cost) AS unit_cost
|
||||
FROM maximo_sparepart_pr_po_line
|
||||
GROUP BY item_num
|
||||
) po
|
||||
ON mu.itemnum = po.item_num
|
||||
GROUP BY mu.wonum
|
||||
),
|
||||
wo_costs AS (
|
||||
SELECT
|
||||
w.wonum,
|
||||
w.asset_location,
|
||||
(COALESCE(w.mat_cost_max, 0) + COALESCE(pc.parts_total_cost, 0)) AS total_wo_cost
|
||||
FROM wo_staging_maximo_2 w
|
||||
LEFT JOIN part_costs pc
|
||||
ON w.wonum = pc.wonum
|
||||
WHERE
|
||||
w.worktype IN ('CM', 'EM', 'PROACTIVE')
|
||||
AND w.asset_system IN (
|
||||
'HPB','AH','APC','SCR','CL','DM','CRH','ASH','BAD','DS','WTP',
|
||||
'MT','SUP','DCS','FF','EG','AI','SPS','EVM','SCW','KLH','CH',
|
||||
'TUR','LOT','HRH','ESP','CAE','GMC','BFT','LSH','CHB','BSS',
|
||||
'LOS','LPB','SAC','CP','EHS','RO','GG','MS','CW','SO','ATT',
|
||||
'AFG','EHB','RP','FO','PC','APE','AF','DMW','BRS','GEN','ABS',
|
||||
'CHA','TR','H2','BDW','LOM','ACR','AL','FW','COND','CCCW','IA',
|
||||
'GSS','BOL','SSB','CO','OA','CTH-UPD','AS','DP'
|
||||
query = select(
|
||||
WorkOrderData.location,
|
||||
(func.sum(WorkOrderData.total_cost_max).cast(Numeric) / func.count(WorkOrderData.wonum)).label('avg_cost')
|
||||
).where(
|
||||
and_(
|
||||
# WorkOrderData.wo_start >= last_oh_date,
|
||||
# WorkOrderData.wo_start <= upcoming_oh_date,
|
||||
WorkOrderData.worktype.in_(['CM', 'EM', 'PROACTIVE']),
|
||||
WorkOrderData.system_tag.in_(['HPB', 'AH', 'APC', 'SCR', 'CL', 'DM', 'CRH', 'ASH', 'BAD', 'DS', 'WTP',
|
||||
'MT', 'SUP', 'DCS', 'FF', 'EG', 'AI', 'SPS', 'EVM', 'SCW', 'KLH', 'CH',
|
||||
'TUR', 'LOT', 'HRH', 'ESP', 'CAE', 'GMC', 'BFT', 'LSH', 'CHB', 'BSS',
|
||||
'LOS', 'LPB', 'SAC', 'CP', 'EHS', 'RO', 'GG', 'MS', 'CW', 'SO', 'ATT',
|
||||
'AFG', 'EHB', 'RP', 'FO', 'PC', 'APE', 'AF', 'DMW', 'BRS', 'GEN', 'ABS',
|
||||
'CHA', 'TR', 'H2', 'BDW', 'LOM', 'ACR', 'AL', 'FW', 'COND', 'CCCW', 'IA',
|
||||
'GSS', 'BOL', 'SSB', 'CO', 'OA', 'CTH-UPD', 'AS', 'DP']),
|
||||
WorkOrderData.reportdate.is_not(None),
|
||||
WorkOrderData.actstart.is_not(None),
|
||||
WorkOrderData.actfinish.is_not(None),
|
||||
WorkOrderData.unit.in_([3, 0]),
|
||||
WorkOrderData.reportdate >= datetime.strptime('2015-01-01', '%Y-%m-%d'),
|
||||
not_(WorkOrderData.wonum.like('T%'))
|
||||
)
|
||||
AND w.reportdate IS NOT NULL
|
||||
AND w.actstart IS NOT NULL
|
||||
AND w.actfinish IS NOT NULL
|
||||
AND w.asset_unit IN ('3','00')
|
||||
AND w.reportdate >= '2015-01-01'
|
||||
AND w.wonum NOT LIKE 'T%'
|
||||
),
|
||||
-- find max cost per location
|
||||
location_max AS (
|
||||
SELECT asset_location, MAX(total_wo_cost) AS max_cost
|
||||
FROM wo_costs
|
||||
WHERE total_wo_cost > 0
|
||||
GROUP BY asset_location
|
||||
),
|
||||
-- filter WO costs to only reasonable range (e.g. >0 and >=10% of max)
|
||||
filtered_wo AS (
|
||||
SELECT w.*
|
||||
FROM wo_costs w
|
||||
JOIN location_max lm ON w.asset_location = lm.asset_location
|
||||
WHERE w.total_wo_cost > 0
|
||||
)
|
||||
SELECT
|
||||
asset_location,
|
||||
SUM(total_wo_cost)::numeric / COUNT(wonum) AS avg_cost
|
||||
FROM filtered_wo
|
||||
GROUP BY asset_location
|
||||
ORDER BY avg_cost DESC;
|
||||
""")
|
||||
results = await collector_db.execute(query)
|
||||
data = []
|
||||
|
||||
for row in results:
|
||||
data.append({
|
||||
"location_tag": row.asset_location,
|
||||
"avg_cost": row.avg_cost
|
||||
})
|
||||
|
||||
).group_by(
|
||||
WorkOrderData.location
|
||||
).order_by(
|
||||
func.count(WorkOrderData.wonum).desc()
|
||||
)
|
||||
result = await collector_db.execute(query)
|
||||
data = result.all()
|
||||
|
||||
return {
|
||||
item["location_tag"]: item["avg_cost"] for item in data
|
||||
data.location: data.avg_cost for data in data
|
||||
}
|
||||
|
||||
|
||||
# async def get_oh_cost_summary(collector_db: CollectorDbSession, last_oh_date:datetime, upcoming_oh_date:datetime):
|
||||
# query = text("""
|
||||
# WITH target_wo AS (
|
||||
# -- Get work orders under a specific parent(s)
|
||||
# SELECT
|
||||
# wonum,
|
||||
# xx_parent,
|
||||
# assetnum,
|
||||
# location_tag AS asset_location,
|
||||
# actmatcost,
|
||||
# actservcost,
|
||||
# reportdate
|
||||
# FROM public.wo_maxim
|
||||
# WHERE xx_parent = ANY(:parent_nums)
|
||||
# ),
|
||||
# part_costs AS (
|
||||
# -- Calculate parts cost per WO if actmatcost = 0
|
||||
# SELECT
|
||||
# wm.wonum,
|
||||
# SUM(
|
||||
# wm.itemqty *
|
||||
# COALESCE(wm.inv_avgcost, po.unit_cost, 0)
|
||||
# ) AS parts_total_cost
|
||||
# FROM public.wo_maxim_material wm
|
||||
# LEFT JOIN (
|
||||
# SELECT item_num, AVG(unit_cost) AS unit_cost
|
||||
# FROM public.maximo_sparepart_pr_po_line
|
||||
# GROUP BY item_num
|
||||
# ) po ON wm.itemnum = po.item_num
|
||||
# WHERE wm.itemnum IS NOT NULL
|
||||
# GROUP BY wm.wonum
|
||||
# ),
|
||||
# wo_costs AS (
|
||||
# SELECT
|
||||
# w.wonum,
|
||||
# w.asset_location,
|
||||
# CASE
|
||||
# WHEN COALESCE(w.actmatcost, 0) > 0 THEN COALESCE(w.actmatcost, 0)
|
||||
# ELSE COALESCE(pc.parts_total_cost, 0)
|
||||
# END AS material_cost,
|
||||
# COALESCE(w.actservcost, 0) AS service_cost
|
||||
# FROM target_wo w
|
||||
# LEFT JOIN part_costs pc ON w.wonum = pc.wonum
|
||||
# )
|
||||
# SELECT
|
||||
# asset_location,
|
||||
# ROUND(SUM(material_cost + service_cost)::numeric / COUNT(wonum), 2) AS avg_cost,
|
||||
# COUNT(wonum) AS total_wo_count
|
||||
# FROM wo_costs
|
||||
# GROUP BY asset_location
|
||||
# ORDER BY total_wo_count DESC;
|
||||
# """)
|
||||
|
||||
# parent_nums = []
|
||||
|
||||
# result = await collector_db.execute(query, {"parent_nums": parent_nums})
|
||||
# data = []
|
||||
|
||||
# for row in result:
|
||||
# data.append({
|
||||
# "location_tag": row.asset_location,
|
||||
# "avg_cost": float(row.avg_cost or 0.0),
|
||||
# "total_wo_count": row.total_wo_count,
|
||||
# })
|
||||
|
||||
# return {item["location_tag"]: item["avg_cost"] for item in data}
|
||||
|
||||
|
||||
async def get_oh_cost_summary(collector_db: CollectorDbSession, last_oh_date:datetime, upcoming_oh_date:datetime):
|
||||
# query = text("""
|
||||
# WITH part_costs AS (
|
||||
# SELECT
|
||||
# wm.wonum,
|
||||
# SUM(wm.itemqty * COALESCE(wm.inv_avgcost, po.unit_cost, 0)) AS parts_total_cost
|
||||
# FROM public.wo_maxim_material wm
|
||||
# LEFT JOIN (
|
||||
# SELECT item_num, AVG(unit_cost) AS unit_cost
|
||||
# FROM public.maximo_sparepart_pr_po_line
|
||||
# GROUP BY item_num
|
||||
# ) po ON wm.itemnum = po.item_num
|
||||
# WHERE wm.itemnum IS NOT NULL
|
||||
# GROUP BY wm.wonum
|
||||
# ),
|
||||
# wo_costs AS (
|
||||
# SELECT
|
||||
# w.wonum,
|
||||
# w.asset_location,
|
||||
# -- Use mat_cost_max if parts_total_cost = 0
|
||||
# CASE
|
||||
# WHEN COALESCE(pc.parts_total_cost, 0) = 0 THEN COALESCE(w.mat_cost_max , 0)
|
||||
# ELSE COALESCE(pc.parts_total_cost, 0)
|
||||
# END AS total_wo_cost
|
||||
# FROM wo_staging_maximo_2 w
|
||||
# LEFT JOIN part_costs pc
|
||||
# ON w.wonum = pc.wonum
|
||||
# WHERE
|
||||
# w.worktype = 'OH'
|
||||
# AND w.reportdate IS NOT NULL
|
||||
# AND w.actstart IS NOT NULL
|
||||
# AND w.actfinish IS NOT NULL
|
||||
# AND w.asset_unit IN ('3', '00')
|
||||
# AND w.wonum NOT LIKE 'T%'
|
||||
# )
|
||||
# SELECT
|
||||
# asset_location,
|
||||
# AVG(total_wo_cost) AS avg_cost
|
||||
# FROM wo_costs
|
||||
# GROUP BY asset_location
|
||||
# ORDER BY COUNT(wonum) DESC;
|
||||
# """)
|
||||
|
||||
query = text("""
|
||||
WITH part_costs AS (
|
||||
SELECT
|
||||
wm.wonum,
|
||||
SUM(wm.itemqty * COALESCE(inv.avgcost, po.unit_cost, 0)) AS parts_total_cost
|
||||
FROM public.maximo_workorder_materials wm
|
||||
JOIN public.maximo_inventory AS inv on inv.itemnum = wm.itemnum
|
||||
LEFT JOIN (
|
||||
SELECT item_num, AVG(unit_cost) AS unit_cost
|
||||
FROM public.maximo_sparepart_pr_po_line
|
||||
GROUP BY item_num
|
||||
) po ON wm.itemnum = po.item_num
|
||||
WHERE wm.itemnum IS NOT NULL
|
||||
GROUP BY wm.wonum
|
||||
),
|
||||
wo_costs AS (
|
||||
SELECT
|
||||
w.wonum,
|
||||
w.asset_location,
|
||||
-- Use mat_cost_max if parts_total_cost = 0
|
||||
CASE
|
||||
WHEN COALESCE(pc.parts_total_cost, 0) = 0 THEN COALESCE(w.mat_cost_max , 0)
|
||||
ELSE COALESCE(pc.parts_total_cost, 0)
|
||||
END AS total_wo_cost
|
||||
FROM wo_staging_maximo_2 w
|
||||
LEFT JOIN part_costs pc
|
||||
ON w.wonum = pc.wonum
|
||||
WHERE
|
||||
w.worktype = 'OH'
|
||||
AND w.reportdate IS NOT NULL
|
||||
AND w.actstart IS NOT NULL
|
||||
AND w.actfinish IS NOT NULL
|
||||
AND w.asset_unit IN ('3', '00')
|
||||
AND w.wonum NOT LIKE 'T%'
|
||||
query = select(
|
||||
WorkOrderData.location,
|
||||
(func.sum(WorkOrderData.total_cost_max).cast(Numeric) / func.count(WorkOrderData.wonum)).label('avg_cost')
|
||||
).where(
|
||||
and_(
|
||||
# WorkOrderData.wo_start >= last_oh_date,
|
||||
# WorkOrderData.wo_start <= upcoming_oh_date,
|
||||
WorkOrderData.worktype.in_(['OH']),
|
||||
WorkOrderData.system_tag.in_(['HPB', 'AH', 'APC', 'SCR', 'CL', 'DM', 'CRH', 'ASH', 'BAD', 'DS', 'WTP',
|
||||
'MT', 'SUP', 'DCS', 'FF', 'EG', 'AI', 'SPS', 'EVM', 'SCW', 'KLH', 'CH',
|
||||
'TUR', 'LOT', 'HRH', 'ESP', 'CAE', 'GMC', 'BFT', 'LSH', 'CHB', 'BSS',
|
||||
'LOS', 'LPB', 'SAC', 'CP', 'EHS', 'RO', 'GG', 'MS', 'CW', 'SO', 'ATT',
|
||||
'AFG', 'EHB', 'RP', 'FO', 'PC', 'APE', 'AF', 'DMW', 'BRS', 'GEN', 'ABS',
|
||||
'CHA', 'TR', 'H2', 'BDW', 'LOM', 'ACR', 'AL', 'FW', 'COND', 'CCCW', 'IA',
|
||||
'GSS', 'BOL', 'SSB', 'CO', 'OA', 'CTH-UPD', 'AS', 'DP']),
|
||||
WorkOrderData.reportdate.is_not(None),
|
||||
WorkOrderData.actstart.is_not(None),
|
||||
WorkOrderData.actfinish.is_not(None),
|
||||
WorkOrderData.unit.in_([3, 0]),
|
||||
WorkOrderData.reportdate >= datetime.strptime('2015-01-01', '%Y-%m-%d'),
|
||||
not_(WorkOrderData.wonum.like('T%'))
|
||||
)
|
||||
).group_by(
|
||||
WorkOrderData.location
|
||||
).order_by(
|
||||
func.count(WorkOrderData.wonum).desc()
|
||||
)
|
||||
SELECT
|
||||
asset_location,
|
||||
AVG(total_wo_cost) AS avg_cost
|
||||
FROM wo_costs
|
||||
GROUP BY asset_location
|
||||
ORDER BY COUNT(wonum) DESC;
|
||||
""")
|
||||
|
||||
result = await collector_db.execute(query)
|
||||
data = []
|
||||
|
||||
for row in result:
|
||||
data.append({
|
||||
"location_tag": row.asset_location,
|
||||
"avg_cost": row.avg_cost
|
||||
})
|
||||
data = result.all()
|
||||
|
||||
return {
|
||||
item["location_tag"]: item["avg_cost"] for item in data
|
||||
data.location: data.avg_cost for data in data
|
||||
}
|
||||
|
||||
|
||||
from uuid import UUID
|
||||
|
||||
async def get_history_oh_wo(*, db_session: DbSession, collector_db_session: CollectorDbSession, oh_session_id: UUID, parent_wo_num: Optional[Union[str, list]] = None):
|
||||
## Get Parent wo num from oh session table
|
||||
if not parent_wo_num:
|
||||
query = select(OverhaulScope.wo_parent).where(OverhaulScope.id == oh_session_id)
|
||||
result = await db_session.execute(query)
|
||||
parent_wo_num = result.scalar()
|
||||
|
||||
if not parent_wo_num:
|
||||
return []
|
||||
|
||||
# Ensure parent_wo_num is a list and removed duplicates if any
|
||||
if isinstance(parent_wo_num, str):
|
||||
parent_wo_num = [parent_wo_num]
|
||||
else:
|
||||
parent_wo_num = list(set(parent_wo_num))
|
||||
|
||||
sql_query = text("""
|
||||
WITH target_wos AS (
|
||||
SELECT
|
||||
w.wonum,
|
||||
w.assetnum,
|
||||
COALESCE(w.actmatcost, 0) as actmatcost,
|
||||
COALESCE(w.actservcost, 0) as actservcost
|
||||
FROM public.wo_maximo w
|
||||
WHERE w.xx_parent = ANY(:parent_wo_num)
|
||||
),
|
||||
wo_tasks AS (
|
||||
SELECT
|
||||
t.xx_parent AS parent_wonum,
|
||||
JSON_AGG(t.description) AS task_list
|
||||
FROM public.wo_maximo t
|
||||
JOIN target_wos tw ON t.xx_parent = tw.wonum
|
||||
GROUP BY t.xx_parent
|
||||
)
|
||||
SELECT
|
||||
w.assetnum,
|
||||
e.name AS equipment_name,
|
||||
e.location_tag,
|
||||
JSON_OBJECT_AGG(w.wonum, COALESCE(wt.task_list, '[]'::json)) AS wonum_list,
|
||||
COUNT(w.wonum) AS total_wo_count,
|
||||
COALESCE(SUM(w.actmatcost), 0) AS total_material_cost,
|
||||
COALESCE(SUM(w.actservcost), 0) AS total_service_cost,
|
||||
COALESCE(SUM(w.actmatcost + w.actservcost), 0) AS total_actual_cost
|
||||
FROM target_wos w
|
||||
INNER JOIN public.ms_equipment_master e
|
||||
ON w.assetnum = e.assetnum
|
||||
LEFT JOIN wo_tasks wt
|
||||
ON w.wonum = wt.parent_wonum
|
||||
GROUP BY
|
||||
w.assetnum,
|
||||
e.name,
|
||||
e.location_tag
|
||||
ORDER BY total_actual_cost DESC;
|
||||
""")
|
||||
|
||||
results = await collector_db_session.execute(sql_query, {"parent_wo_num": parent_wo_num})
|
||||
|
||||
return [
|
||||
{
|
||||
"assetnum": row.assetnum,
|
||||
"equipment_name": row.equipment_name,
|
||||
"location_tag": row.location_tag,
|
||||
"wonum_list": row.wonum_list,
|
||||
"total_wo_count": row.total_wo_count,
|
||||
"total_material_cost": float(row.total_material_cost),
|
||||
"total_service_cost": float(row.total_service_cost),
|
||||
"total_actual_cost": float(row.total_actual_cost)
|
||||
}
|
||||
for row in results
|
||||
]
|
||||
|
||||
|
||||
|
||||
|
||||
@ -1,426 +0,0 @@
|
||||
import json
|
||||
import re
|
||||
import logging
|
||||
from collections import Counter
|
||||
from fastapi import Request, HTTPException
|
||||
from starlette.middleware.base import BaseHTTPMiddleware
|
||||
|
||||
# =========================
|
||||
# Configuration
|
||||
# =========================
|
||||
|
||||
ALLOWED_MULTI_PARAMS = {
|
||||
"sortBy[]",
|
||||
"descending[]",
|
||||
"exclude[]",
|
||||
"assetnums",
|
||||
"plant_ids",
|
||||
"job_ids",
|
||||
}
|
||||
|
||||
ALLOWED_DATA_PARAMS = {
|
||||
"actual_shutdown", "all_params", "analysis_metadata", "asset_contributions",
|
||||
"assetnum", "assetnums", "assigned_date", "availability", "availableScopes",
|
||||
"avg_cost", "birbaum", "calculation_type", "capacity_weight", "code",
|
||||
"contribution", "corrective_cost", "corrective_costs", "cost", "costPerFailure",
|
||||
"cost_savings_vs_planned", "cost_threshold", "cost_trend", "created_at",
|
||||
"crew_number", "criticalParts", "critical_procurement_items", "current_eaf",
|
||||
"current_plant_eaf", "current_stock", "current_user", "cut_hours",
|
||||
"daily_failures", "data", "datetime", "day", "days", "descending",
|
||||
"description", "down_time", "duration", "duration_oh", "eaf_gap",
|
||||
"eaf_improvement_text", "eaf_input", "efficiency", "end_date",
|
||||
"equipment_name", "equipment_results", "equipment_with_sparepart_constraints",
|
||||
"exclude", "excluded_equipment", "expected_delivery_date", "filter_spec",
|
||||
"finish", "fleet_statistics", "id", "improvement_impact", "included_equipment",
|
||||
"included_in_optimization", "intervalDays", "is_included", "itemnum",
|
||||
"items", "itemsPerPage", "items_per_page", "job", "job_ids",
|
||||
"last_overhaul_date", "lead_time", "location", "location_tag", "location_tags",
|
||||
"maintenance_type", "master_equipment", "material_cost", "max_interval",
|
||||
"max_interval_months", "message", "month", "months_from_planned", "name",
|
||||
"next_planned_overhaul", "node", "num_failures", "num_of_failures",
|
||||
"ohSessionId", "oh_scope", "oh_session_id", "oh_type", "oh_types",
|
||||
"optimal_analysis", "optimal_breakdown", "optimal_month", "optimal_total_cost",
|
||||
"optimization_success", "optimum_analysis", "optimum_day", "optimum_oh",
|
||||
"optimum_oh_day", "optimum_oh_month", "order_date", "overhaulCost",
|
||||
"overhaul_activity", "overhaul_cost", "overhaul_costs",
|
||||
"overhaul_reference_type", "overhaul_scope", "overhaul_scope_id", "overview",
|
||||
"page", "parent", "parent_id", "plan_duration", "planned_month",
|
||||
"planned_outage", "plant_level_benefit", "po_pr_id", "po_vendor_delivery_date",
|
||||
"possible_plant_eaf", "priority_score", "procurement_cost", "procurement_costs",
|
||||
"procurement_details", "projected_eaf_improvement", "quantity",
|
||||
"quantity_required", "query_str", "recommendedScope", "recommended_reduced_outage",
|
||||
"reference", "reference_id", "remark", "removal_date", "required_improvement",
|
||||
"results", "schedules", "scope", "scope_calculation_id", "scope_equipment_job",
|
||||
"scope_name", "scope_overhaul", "service_cost", "session", "simulation",
|
||||
"simulation_id", "sort_by", "sortBy[]", "descending[]", "exclude[]",
|
||||
"sparepart_id", "sparepart_impact", "sparepart_name", "sparepart_summary",
|
||||
"spreadsheet_link", "start", "start_date", "status", "subsystem", "system",
|
||||
"systemComponents", "target_plant_eaf", "tasks", "timing_recommendation",
|
||||
"total", "totalPages", "total_cost", "total_equipment", "total_equipment_analyzed",
|
||||
"total_procurement_items", "type", "unit_cost", "warning_message", "with_results",
|
||||
"workscope", "workscope_group", "year", "_", "t", "timestamp",
|
||||
"q", "filter", "currentUser", "risk_cost", "all", "with_results",
|
||||
"eaf_threshold", "simulation_id", "scope_calculation_id", "calculation_id", "simulationRBDId"
|
||||
}
|
||||
|
||||
ALLOWED_HEADERS = {
|
||||
"host",
|
||||
"user-agent",
|
||||
"accept",
|
||||
"accept-language",
|
||||
"accept-encoding",
|
||||
"connection",
|
||||
"upgrade-insecure-requests",
|
||||
"if-modified-since",
|
||||
"if-none-match",
|
||||
"cache-control",
|
||||
"authorization",
|
||||
"content-type",
|
||||
"content-length",
|
||||
"origin",
|
||||
"referer",
|
||||
"sec-fetch-dest",
|
||||
"sec-fetch-mode",
|
||||
"sec-fetch-site",
|
||||
"sec-fetch-user",
|
||||
"sec-ch-ua",
|
||||
"sec-ch-ua-mobile",
|
||||
"sec-ch-ua-platform",
|
||||
"pragma",
|
||||
"dnt",
|
||||
"priority",
|
||||
"x-forwarded-for",
|
||||
"x-forwarded-proto",
|
||||
"x-forwarded-host",
|
||||
"x-forwarded-port",
|
||||
"x-real-ip",
|
||||
"x-request-id",
|
||||
"x-correlation-id",
|
||||
"x-requested-with",
|
||||
"x-csrf-token",
|
||||
"x-xsrf-token",
|
||||
"postman-token",
|
||||
"x-forwarded-path",
|
||||
"x-forwarded-prefix",
|
||||
"cookie",
|
||||
"x-kong-request-id"
|
||||
}
|
||||
|
||||
MAX_QUERY_PARAMS = 50
|
||||
MAX_QUERY_LENGTH = 2000
|
||||
MAX_JSON_BODY_SIZE = 1024 * 500 # 500 KB
|
||||
|
||||
XSS_PATTERN = re.compile(
|
||||
r"("
|
||||
r"<(script|iframe|embed|object|svg|img|video|audio|base|link|meta|form|button|details|animate)\b|"
|
||||
r"javascript\s*:|vbscript\s*:|data\s*:[^,]*base64[^,]*|data\s*:text/html|"
|
||||
r"\bon[a-z]+\s*=|" # Catch-all for any 'on' event (onerror, onclick, etc.)
|
||||
r"style\s*=.*expression\s*\(|" # Old IE specific
|
||||
r"\b(eval|setTimeout|setInterval|Function)\s*\("
|
||||
r")",
|
||||
re.IGNORECASE,
|
||||
)
|
||||
|
||||
SQLI_PATTERN = re.compile(
|
||||
r"("
|
||||
# 1. Keywords followed by whitespace and common SQL characters
|
||||
r"\b(UNION|SELECT|INSERT|UPDATE|DELETE|DROP|ALTER|CREATE|TRUNCATE|EXEC(UTE)?|DECLARE)\b\s+[\w\*\(\']|"
|
||||
|
||||
# 2. Time-based attacks (more specific than just 'SLEEP')
|
||||
r"\b(WAITFOR\b\s+DELAY|PG_SLEEP|SLEEP\s*\()|"
|
||||
|
||||
# 3. System tables/functions
|
||||
r"\b(INFORMATION_SCHEMA|SYS\.|SYSOBJECTS|XP_CMDSHELL|LOAD_FILE|INTO\s+OUTFILE)\b|"
|
||||
|
||||
# 4. Logical Tautologies (OR 1=1) - Optimized for boundaries
|
||||
r"\b(OR|AND)\b\s+['\"]?\d+['\"]?\s*=\s*['\"]?\d+|"
|
||||
|
||||
# 5. Comments
|
||||
# Match '--' if at start or preceded by whitespace
|
||||
r"(?<!\S)--|"
|
||||
# Match block comments, ensuring they aren't part of mime patterns like */*
|
||||
r"(?<!\*)/\*|(?<!\*)\*/(?!\*)|"
|
||||
# Match '#' if at start or preceded by whitespace
|
||||
r"(?<!\S)#|"
|
||||
|
||||
# 6. Hex / Stacked Queries
|
||||
r";\s*\b(SELECT|DROP|DELETE|UPDATE|INSERT)\b"
|
||||
r")",
|
||||
re.IGNORECASE
|
||||
)
|
||||
|
||||
RCE_PATTERN = re.compile(
|
||||
r"("
|
||||
r"\$\(.*\)|`.*`|" # Command substitution $(...) or `...`
|
||||
r"[;&|]\s*(cat|ls|id|whoami|pwd|ifconfig|ip|netstat|nc|netcat|nmap|curl|wget|python|php|perl|ruby|bash|sh|cmd|powershell|pwsh|sc\s+|tasklist|taskkill|base64|sudo|crontab|ssh|ftp|tftp)|"
|
||||
# Only flag naked commands if they are clearly standalone or system paths
|
||||
r"(/etc/passwd|/etc/shadow|/etc/group|/etc/issue|/proc/self/|/windows/system32/|C:\\Windows\\)\b"
|
||||
r")",
|
||||
re.IGNORECASE,
|
||||
)
|
||||
|
||||
TRAVERSAL_PATTERN = re.compile(
|
||||
r"(\.\.[/\\]|%2e%2e%2f|%2e%2e/|\.\.%2f|%2e%2e%5c|%252e%252e%252f|\\00)",
|
||||
re.IGNORECASE,
|
||||
)
|
||||
|
||||
FORBIDDEN_JSON_KEYS = {"__proto__", "constructor", "prototype"}
|
||||
|
||||
DYNAMIC_KEYS = {
|
||||
"data",
|
||||
"results",
|
||||
"analysis_metadata",
|
||||
"asset_contributions",
|
||||
"equipment_results",
|
||||
"optimal_analysis",
|
||||
"optimum_analysis",
|
||||
"schedules",
|
||||
"tasks",
|
||||
"all_params",
|
||||
"parameters",
|
||||
"program_data"
|
||||
}
|
||||
|
||||
log = logging.getLogger("security_logger")
|
||||
|
||||
|
||||
def has_control_chars(value: str) -> bool:
|
||||
return any(ord(c) < 32 and c not in ("\n", "\r", "\t") for c in value)
|
||||
|
||||
|
||||
def inspect_value(value: str, source: str):
|
||||
if not isinstance(value, str) or value == "*/*":
|
||||
return
|
||||
|
||||
|
||||
if XSS_PATTERN.search(value):
|
||||
log.warning(f"Security violation: Potential XSS payload detected in {source}, value: {value}")
|
||||
raise HTTPException(
|
||||
status_code=422,
|
||||
detail="Invalid request parameters",
|
||||
)
|
||||
|
||||
if SQLI_PATTERN.search(value):
|
||||
log.warning(f"Security violation: Potential SQL injection payload detected in {source}, value: {value}")
|
||||
raise HTTPException(
|
||||
status_code=422,
|
||||
detail="Invalid request parameters",
|
||||
)
|
||||
|
||||
if RCE_PATTERN.search(value):
|
||||
log.warning(f"Security violation: Potential RCE payload detected in {source}, value: {value}")
|
||||
raise HTTPException(
|
||||
status_code=422,
|
||||
detail="Invalid request parameters",
|
||||
)
|
||||
|
||||
if TRAVERSAL_PATTERN.search(value):
|
||||
log.warning(f"Security violation: Potential Path Traversal payload detected in {source}, value: {value}")
|
||||
raise HTTPException(
|
||||
status_code=422,
|
||||
detail="Invalid request parameters",
|
||||
)
|
||||
|
||||
if has_control_chars(value):
|
||||
log.warning(f"Security violation: Invalid control characters detected in {source}")
|
||||
raise HTTPException(
|
||||
status_code=422,
|
||||
detail="Invalid request parameters",
|
||||
)
|
||||
|
||||
|
||||
def inspect_json(obj, path="body", check_whitelist=True):
|
||||
if isinstance(obj, dict):
|
||||
for key, value in obj.items():
|
||||
if key in FORBIDDEN_JSON_KEYS:
|
||||
log.warning(f"Security violation: Forbidden JSON key detected: {path}.{key}")
|
||||
raise HTTPException(
|
||||
status_code=422,
|
||||
detail="Invalid request parameters",
|
||||
)
|
||||
|
||||
# if check_whitelist and key not in ALLOWED_DATA_PARAMS:
|
||||
# log.warning(f"Security violation: Unknown JSON key detected: {path}.{key}")
|
||||
# raise HTTPException(
|
||||
# status_code=422,
|
||||
# detail="Invalid request parameters",
|
||||
# )
|
||||
|
||||
# Recurse. If the key is a dynamic container, we stop whitelist checking for children.
|
||||
should_check_subkeys = check_whitelist and (key not in DYNAMIC_KEYS)
|
||||
inspect_json(value, f"{path}.{key}", check_whitelist=should_check_subkeys)
|
||||
elif isinstance(obj, list):
|
||||
for i, item in enumerate(obj):
|
||||
inspect_json(item, f"{path}[{i}]", check_whitelist=check_whitelist)
|
||||
elif isinstance(obj, str):
|
||||
inspect_value(obj, path)
|
||||
|
||||
|
||||
# =========================
|
||||
# Middleware
|
||||
# =========================
|
||||
|
||||
class RequestValidationMiddleware(BaseHTTPMiddleware):
|
||||
async def dispatch(self, request: Request, call_next):
|
||||
# -------------------------
|
||||
# 0. Header validation
|
||||
# -------------------------
|
||||
header_keys = [key.lower() for key, _ in request.headers.items()]
|
||||
|
||||
# Check for duplicate headers
|
||||
header_counter = Counter(header_keys)
|
||||
duplicate_headers = [key for key, count in header_counter.items() if count > 1]
|
||||
|
||||
ALLOW_DUPLICATE_HEADERS = {'accept', 'accept-encoding', 'accept-language', 'accept-charset', 'cookie'}
|
||||
real_duplicates = [h for h in duplicate_headers if h not in ALLOW_DUPLICATE_HEADERS]
|
||||
if real_duplicates:
|
||||
log.warning(f"Security violation: Duplicate headers detected: {real_duplicates}")
|
||||
raise HTTPException(
|
||||
status_code=422,
|
||||
detail="Invalid request parameters",
|
||||
)
|
||||
|
||||
# Whitelist headers
|
||||
unknown_headers = [key for key in header_keys if key not in ALLOWED_HEADERS]
|
||||
if unknown_headers:
|
||||
filtered_unknown = [h for h in unknown_headers if not h.startswith('sec-')]
|
||||
if filtered_unknown:
|
||||
log.warning(f"Security violation: Unknown headers detected: {filtered_unknown}")
|
||||
raise HTTPException(
|
||||
status_code=422,
|
||||
detail="Invalid request parameters",
|
||||
)
|
||||
|
||||
# Inspect header values
|
||||
for key, value in request.headers.items():
|
||||
if value:
|
||||
inspect_value(value, f"header '{key}'")
|
||||
|
||||
# -------------------------
|
||||
# 1. Query string limits
|
||||
# -------------------------
|
||||
if len(request.url.query) > MAX_QUERY_LENGTH:
|
||||
log.warning(f"Security violation: Query string too long")
|
||||
raise HTTPException(
|
||||
status_code=422,
|
||||
detail="Invalid request parameters",
|
||||
)
|
||||
|
||||
params = request.query_params.multi_items()
|
||||
|
||||
if len(params) > MAX_QUERY_PARAMS:
|
||||
log.warning(f"Security violation: Too many query parameters")
|
||||
raise HTTPException(
|
||||
status_code=422,
|
||||
detail="Invalid request parameters",
|
||||
)
|
||||
|
||||
# Check for unknown query parameters
|
||||
# unknown_params = [key for key, _ in params if key not in ALLOWED_DATA_PARAMS]
|
||||
# if unknown_params:
|
||||
# log.warning(f"Security violation: Unknown query parameters detected: {unknown_params}")
|
||||
# raise HTTPException(
|
||||
# status_code=422,
|
||||
# detail="Invalid request parameters",
|
||||
# )
|
||||
|
||||
# -------------------------
|
||||
# 2. Duplicate parameters
|
||||
# -------------------------
|
||||
counter = Counter(key for key, _ in params)
|
||||
duplicates = [
|
||||
key for key, count in counter.items()
|
||||
if count > 1 and key not in ALLOWED_MULTI_PARAMS
|
||||
]
|
||||
|
||||
if duplicates:
|
||||
log.warning(f"Security violation: Duplicate query parameters detected: {duplicates}")
|
||||
raise HTTPException(
|
||||
status_code=422,
|
||||
detail="Invalid request parameters",
|
||||
)
|
||||
|
||||
# -------------------------
|
||||
# 3. Query param inspection & Pagination
|
||||
# -------------------------
|
||||
pagination_size_keys = {"size", "itemsPerPage", "per_page", "limit", "items_per_page"}
|
||||
for key, value in params:
|
||||
if value:
|
||||
inspect_value(value, f"query param '{key}'")
|
||||
|
||||
if key in pagination_size_keys and value:
|
||||
try:
|
||||
size_val = int(value)
|
||||
if size_val > 100:
|
||||
log.warning(f"Security violation: Pagination size too large ({size_val})")
|
||||
raise HTTPException(
|
||||
status_code=422,
|
||||
detail="Invalid request parameters",
|
||||
)
|
||||
if size_val % 5 != 0:
|
||||
log.warning(f"Security violation: Pagination size not multiple of 5 ({size_val})")
|
||||
raise HTTPException(
|
||||
status_code=422,
|
||||
detail="Invalid request parameters",
|
||||
)
|
||||
except ValueError:
|
||||
log.warning(f"Security violation: Pagination size invalid value")
|
||||
raise HTTPException(
|
||||
status_code=422,
|
||||
detail="Invalid request parameters",
|
||||
)
|
||||
|
||||
# -------------------------
|
||||
# 4. Content-Type sanity
|
||||
# -------------------------
|
||||
content_type = request.headers.get("content-type", "")
|
||||
if content_type and not any(
|
||||
content_type.startswith(t)
|
||||
for t in ("application/json", "multipart/form-data", "application/x-www-form-urlencoded")
|
||||
):
|
||||
log.warning(f"Security violation: Unsupported Content-Type: {content_type}")
|
||||
raise HTTPException(status_code=422, detail="Invalid request parameters")
|
||||
|
||||
# -------------------------
|
||||
# 5. Single source check (Query vs JSON Body)
|
||||
# -------------------------
|
||||
has_query = len(params) > 0
|
||||
has_body = False
|
||||
|
||||
if content_type.startswith("application/json"):
|
||||
# We can't easily check body existence without consuming it,
|
||||
# so we check if Content-Length > 0
|
||||
content_length = request.headers.get("content-length")
|
||||
if content_length and int(content_length) > 0:
|
||||
has_body = True
|
||||
|
||||
if has_query and has_body:
|
||||
log.warning(f"Security violation: Mixed parameters (query + JSON body)")
|
||||
raise HTTPException(
|
||||
status_code=422,
|
||||
detail="Invalid request parameters",
|
||||
)
|
||||
|
||||
# -------------------------
|
||||
# 6. JSON body inspection
|
||||
# -------------------------
|
||||
if content_type.startswith("application/json"):
|
||||
body = await request.body()
|
||||
# if len(body) > MAX_JSON_BODY_SIZE:
|
||||
# raise HTTPException(status_code=422, detail="JSON body too large")
|
||||
|
||||
if body:
|
||||
try:
|
||||
payload = json.loads(body)
|
||||
except json.JSONDecodeError:
|
||||
log.warning(f"Security violation: Invalid JSON body")
|
||||
raise HTTPException(status_code=422, detail="Invalid request parameters")
|
||||
|
||||
inspect_json(payload)
|
||||
|
||||
# Re-inject body for downstream handlers
|
||||
async def receive():
|
||||
return {"type": "http.request", "body": body}
|
||||
request._receive = receive
|
||||
|
||||
return await call_next(request)
|
||||
@ -1 +0,0 @@
|
||||
# Empty init file
|
||||
@ -1,172 +0,0 @@
|
||||
from typing import Optional
|
||||
from uuid import UUID
|
||||
|
||||
from fastapi import HTTPException, status
|
||||
from sqlalchemy import func, select
|
||||
|
||||
from src.auth.service import Token
|
||||
from src.config import TC_RBD_ID
|
||||
from src.database.core import DbSession, CollectorDbSession
|
||||
from src.overhaul_scope.service import get_all
|
||||
from src.standard_scope.model import StandardScope
|
||||
from src.workorder.model import MasterWorkOrder
|
||||
|
||||
from src.calculation_time_constrains.schema import (
|
||||
CalculationTimeConstrainsParametersCreate,
|
||||
CalculationTimeConstrainsParametersRead,
|
||||
CalculationTimeConstrainsParametersRetrive,
|
||||
)
|
||||
from src.optimum_time_constraint.service import (
|
||||
get_calculation_data_by_id,
|
||||
get_calculation_result,
|
||||
)
|
||||
|
||||
|
||||
async def get_create_calculation_parameters(
|
||||
*, db_session: DbSession, calculation_id: Optional[str] = None
|
||||
):
|
||||
if calculation_id is not None:
|
||||
calculation = await get_calculation_data_by_id(
|
||||
calculation_id=calculation_id, db_session=db_session
|
||||
)
|
||||
|
||||
if not calculation:
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_404_NOT_FOUND,
|
||||
detail="A data with this id does not exist.",
|
||||
)
|
||||
|
||||
return CalculationTimeConstrainsParametersRead(
|
||||
costPerFailure=calculation.parameter.avg_failure_cost,
|
||||
overhaulCost=calculation.parameter.overhaul_cost,
|
||||
reference=calculation,
|
||||
)
|
||||
|
||||
stmt = (
|
||||
select(
|
||||
StandardScope,
|
||||
func.avg(MasterWorkOrder.total_cost_max).label("average_cost"),
|
||||
)
|
||||
.outerjoin(MasterWorkOrder, StandardScope.location_tag == MasterWorkOrder.location_tag)
|
||||
.group_by(StandardScope.id)
|
||||
)
|
||||
|
||||
results = await db_session.execute(stmt)
|
||||
costFailure = results.all()
|
||||
scopes = await get_all(db_session=db_session)
|
||||
avaiableScopes = {scope.id: scope.scope_name for scope in scopes}
|
||||
costFailurePerScope = {
|
||||
avaiableScopes.get(costPerFailure[0]): costPerFailure[1]
|
||||
for costPerFailure in costFailure
|
||||
}
|
||||
|
||||
return CalculationTimeConstrainsParametersRetrive(
|
||||
costPerFailure=costFailurePerScope,
|
||||
availableScopes=avaiableScopes.values(),
|
||||
recommendedScope="A",
|
||||
)
|
||||
|
||||
|
||||
async def create_calculation(
|
||||
*,
|
||||
token: str,
|
||||
db_session: DbSession,
|
||||
collector_db_session: CollectorDbSession,
|
||||
calculation_time_constrains_in: CalculationTimeConstrainsParametersCreate,
|
||||
created_by: str,
|
||||
simulation_id
|
||||
):
|
||||
from temporalio.client import Client
|
||||
from src.config import TEMPORAL_URL
|
||||
from temporal.temporal_workflows import OptimumOHCalculationWorkflow
|
||||
import uuid
|
||||
|
||||
temporal_client = await Client.connect(TEMPORAL_URL)
|
||||
|
||||
workflow_id = f"optimum-oh-calc-{uuid.uuid4()}"
|
||||
|
||||
args = {
|
||||
"token": token.token if hasattr(token, 'token') else str(token),
|
||||
"calculation_in": calculation_time_constrains_in.model_dump(mode="json"),
|
||||
"created_by": created_by
|
||||
}
|
||||
|
||||
handle = await temporal_client.start_workflow(
|
||||
OptimumOHCalculationWorkflow.run,
|
||||
args,
|
||||
id=workflow_id,
|
||||
task_queue="oh-task-queue"
|
||||
)
|
||||
|
||||
return {
|
||||
"data": workflow_id,
|
||||
"status": "success",
|
||||
"message": "Calculation workflow started successfully"
|
||||
}
|
||||
|
||||
|
||||
async def recalculate_calculation(
|
||||
*,
|
||||
token: str,
|
||||
db_session: DbSession,
|
||||
collector_db_session: CollectorDbSession,
|
||||
calculation_id: str,
|
||||
simulation_id: Optional[str] = None
|
||||
):
|
||||
calculation_data = await get_calculation_data_by_id(
|
||||
db_session=db_session, calculation_id=calculation_id
|
||||
)
|
||||
|
||||
if not calculation_data:
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_404_NOT_FOUND,
|
||||
detail="Calculation not found",
|
||||
)
|
||||
|
||||
rbd_simulation_id = simulation_id or calculation_data.rbd_simulation_id or TC_RBD_ID
|
||||
|
||||
from sqlalchemy import delete
|
||||
from src.calculation_time_constrains.model import CalculationEquipmentResult, CalculationResult
|
||||
|
||||
await db_session.execute(
|
||||
delete(CalculationResult).where(CalculationResult.calculation_data_id == calculation_data.id)
|
||||
)
|
||||
await db_session.execute(
|
||||
delete(CalculationEquipmentResult).where(CalculationEquipmentResult.calculation_data_id == calculation_data.id)
|
||||
)
|
||||
await db_session.commit()
|
||||
|
||||
from src.optimum_time_constraint.optimizer import run_simulation_with_spareparts
|
||||
await run_simulation_with_spareparts(
|
||||
db_session=db_session,
|
||||
calculation=calculation_data,
|
||||
token=token,
|
||||
collector_db_session=collector_db_session,
|
||||
simulation_id=rbd_simulation_id
|
||||
)
|
||||
|
||||
return await get_calculation_result(
|
||||
db_session=db_session,
|
||||
calculation_id=calculation_id,
|
||||
token=token,
|
||||
include_risk_cost=1
|
||||
)
|
||||
|
||||
|
||||
async def get_or_create_scope_equipment_calculation(
|
||||
*,
|
||||
db_session: DbSession,
|
||||
scope_calculation_id,
|
||||
calculation_time_constrains_in: Optional[CalculationTimeConstrainsParametersCreate]
|
||||
):
|
||||
scope_calculation = await get_calculation_data_by_id(
|
||||
db_session=db_session, calculation_id=scope_calculation_id
|
||||
)
|
||||
|
||||
if not scope_calculation:
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_404_NOT_FOUND,
|
||||
detail="A data with this id does not exist.",
|
||||
)
|
||||
|
||||
return scope_calculation.id
|
||||
@ -1,917 +0,0 @@
|
||||
import asyncio
|
||||
import calendar
|
||||
import datetime
|
||||
import json
|
||||
import logging
|
||||
import math
|
||||
from datetime import date, timedelta
|
||||
from typing import Dict, List, Optional, Tuple
|
||||
|
||||
import aiohttp
|
||||
import httpx
|
||||
import numpy as np
|
||||
import requests
|
||||
from sqlalchemy import select
|
||||
from sqlalchemy.orm import selectinload
|
||||
|
||||
from src.config import REALIBILITY_SERVICE_API, RBD_SERVICE_API
|
||||
from src.database.core import CollectorDbSession, DbSession
|
||||
from src.calculation_time_constrains.model import CalculationData, CalculationEquipmentResult
|
||||
from src.calculation_time_constrains.schema import CalculationResultsRead
|
||||
from src.calculation_time_constrains.utils import (
|
||||
calculate_failures_per_month,
|
||||
create_time_series_data,
|
||||
get_months_between,
|
||||
)
|
||||
from src.overhaul_scope.service import get as get_scope, get_prev_oh
|
||||
from src.sparepart.service import load_sparepart_data_from_db
|
||||
|
||||
log = logging.getLogger(__name__)
|
||||
|
||||
|
||||
class OptimumCostModelWithSpareparts:
|
||||
def __init__(
|
||||
self,
|
||||
token: str,
|
||||
last_oh_date: date,
|
||||
next_oh_date: date,
|
||||
sparepart_manager,
|
||||
time_window_months: Optional[int] = None,
|
||||
base_url: str = "http://192.168.1.82:8000",
|
||||
):
|
||||
"""
|
||||
Initialize the Optimum Cost Model with sparepart management
|
||||
"""
|
||||
self.api_base_url = base_url
|
||||
self.token = token
|
||||
self.last_oh_date = last_oh_date
|
||||
self.next_oh_date = next_oh_date
|
||||
self.session = None
|
||||
self.sparepart_manager = sparepart_manager
|
||||
|
||||
# Calculate planned overhaul interval in months
|
||||
self.planned_oh_months = get_months_between(last_oh_date, next_oh_date)
|
||||
|
||||
# Set analysis time window: next OH + 6 months
|
||||
self.time_window_months = time_window_months or (self.planned_oh_months + 6)
|
||||
|
||||
# Pre-calculate date range for API calls
|
||||
self.date_range = self._generate_date_range()
|
||||
|
||||
self.logger = log
|
||||
|
||||
def _generate_date_range(self) -> List[datetime.datetime]:
|
||||
"""Generate date range for analysis based on time window"""
|
||||
dates = []
|
||||
current_date = datetime.datetime.combine(self.last_oh_date, datetime.datetime.min.time())
|
||||
end_date = current_date + timedelta(days=self.time_window_months * 30)
|
||||
|
||||
while current_date <= end_date:
|
||||
dates.append(current_date)
|
||||
current_date += timedelta(days=31)
|
||||
|
||||
return dates
|
||||
|
||||
async def _create_session(self):
|
||||
"""Create aiohttp session with connection pooling"""
|
||||
if self.session is None:
|
||||
timeout = aiohttp.ClientTimeout(total=300)
|
||||
connector = aiohttp.TCPConnector(
|
||||
limit=500,
|
||||
limit_per_host=200,
|
||||
ttl_dns_cache=300,
|
||||
use_dns_cache=True,
|
||||
force_close=False,
|
||||
enable_cleanup_closed=True,
|
||||
)
|
||||
self.session = aiohttp.ClientSession(
|
||||
timeout=timeout,
|
||||
connector=connector,
|
||||
headers={"Authorization": f"Bearer {self.token}"},
|
||||
)
|
||||
|
||||
async def _close_session(self):
|
||||
"""Close aiohttp session"""
|
||||
if self.session:
|
||||
await self.session.close()
|
||||
self.session = None
|
||||
|
||||
async def get_failures_prediction(self, location_tag: str):
|
||||
"""Get failure predictions for equipment from Reliability Predict service"""
|
||||
|
||||
start_date = self.last_oh_date.strftime("%Y-%m-%d")
|
||||
end_date_val = self.next_oh_date + timedelta(days=6 * 30)
|
||||
end_date = end_date_val.strftime("%Y-%m-%d")
|
||||
|
||||
predict_url = f"{REALIBILITY_SERVICE_API}/main/predict/{location_tag}/{start_date}/{end_date}"
|
||||
|
||||
try:
|
||||
response = requests.get(
|
||||
predict_url,
|
||||
headers={
|
||||
"Content-Type": "application/json",
|
||||
"Authorization": f"Bearer {self.token}",
|
||||
},
|
||||
timeout=30,
|
||||
)
|
||||
response.raise_for_status()
|
||||
prediction_data = response.json()
|
||||
except (requests.RequestException, ValueError) as e:
|
||||
self.logger.error(f"Failed to fetch prediction data for {location_tag}: {e}")
|
||||
return None
|
||||
|
||||
predictions = (
|
||||
prediction_data.get("data", {}).get("predictions")
|
||||
if prediction_data.get("data")
|
||||
else None
|
||||
)
|
||||
|
||||
if not predictions:
|
||||
self.logger.warning(f"No prediction data available for {location_tag}")
|
||||
return None
|
||||
|
||||
monthly_data = {}
|
||||
cumulative = 0.0
|
||||
for i, pred in enumerate(predictions):
|
||||
month_key = pred["month"]
|
||||
monthly_fail = pred["predicted_failures"]
|
||||
source = pred.get("source", "predicted")
|
||||
cumulative += monthly_fail
|
||||
monthly_data[month_key] = {
|
||||
"cumulative_failures": cumulative,
|
||||
"monthly_failures": monthly_fail,
|
||||
"month_index": i + 1,
|
||||
"source": source,
|
||||
}
|
||||
|
||||
return monthly_data
|
||||
|
||||
async def get_simulation_results(self, simulation_id: str = "default"):
|
||||
"""Get simulation results for Birnbaum importance calculations"""
|
||||
headers = {
|
||||
"Authorization": f"Bearer {self.token}",
|
||||
"Content-Type": "application/json",
|
||||
}
|
||||
|
||||
calc_result_url = f"{self.api_base_url}/aeros/simulation/result/calc/{simulation_id}?nodetype=RegularNode"
|
||||
plant_result_url = f"{self.api_base_url}/aeros/simulation/result/calc/{simulation_id}/plant"
|
||||
|
||||
async with httpx.AsyncClient(timeout=300.0) as client:
|
||||
calc_task = client.get(calc_result_url, headers=headers)
|
||||
plant_task = client.get(plant_result_url, headers=headers)
|
||||
|
||||
calc_response, plant_response = await asyncio.gather(calc_task, plant_task)
|
||||
|
||||
calc_response.raise_for_status()
|
||||
plant_response.raise_for_status()
|
||||
|
||||
calc_data = calc_response.json()["data"]
|
||||
plant_data = plant_response.json()["data"]
|
||||
|
||||
return {"calc_result": calc_data, "plant_result": plant_data}
|
||||
|
||||
def _calculate_equipment_costs_with_spareparts(
|
||||
self,
|
||||
failures_prediction: Dict,
|
||||
birnbaum_importance: float,
|
||||
preventive_cost: float,
|
||||
failure_replacement_cost: float,
|
||||
ecs,
|
||||
location_tag: str,
|
||||
planned_overhauls: List = None,
|
||||
loss_production_permonth=0,
|
||||
) -> List[Dict]:
|
||||
"""Calculate costs for each month including sparepart costs and availability"""
|
||||
|
||||
if not failures_prediction:
|
||||
self.logger.warning(f"No failure prediction data for {location_tag}")
|
||||
return []
|
||||
|
||||
results = []
|
||||
months = list(failures_prediction.keys())
|
||||
num_months = len(months)
|
||||
failure_counts = []
|
||||
|
||||
monthly_risk_cost_per_failure = 0
|
||||
|
||||
if ecs:
|
||||
is_trip = 1 if ecs.get("Diskripsi Operasional Akibat Equip. Failure") == "Trip" else 0
|
||||
is_series = 0 if not birnbaum_importance else math.floor(birnbaum_importance)
|
||||
if is_trip:
|
||||
downtime = ecs.get("Estimasi Waktu Maint. / Downtime / Gangguan (Jam)")
|
||||
monthly_risk_cost_per_failure = 660 * 1000000 * is_trip * downtime * is_series
|
||||
|
||||
for month_key in months:
|
||||
data = failures_prediction[month_key]
|
||||
failure_counts.append(data["cumulative_failures"])
|
||||
|
||||
for i in range(num_months):
|
||||
month_index = i + 1
|
||||
|
||||
if month_index > self.time_window_months:
|
||||
continue
|
||||
|
||||
sparepart_analysis = self._analyze_sparepart_availability(
|
||||
location_tag, month_index - 1, planned_overhauls or []
|
||||
)
|
||||
|
||||
total_expected_failure_cost = failure_counts[i] * (
|
||||
failure_replacement_cost + monthly_risk_cost_per_failure
|
||||
)
|
||||
|
||||
procurement_cost = sparepart_analysis["total_procurement_cost"]
|
||||
total_preventive_cost = preventive_cost + procurement_cost
|
||||
|
||||
total_cycle_cost = total_expected_failure_cost + total_preventive_cost
|
||||
|
||||
cput = total_cycle_cost / month_index
|
||||
|
||||
results.append(
|
||||
{
|
||||
"month": month_index,
|
||||
"number_of_failures": failure_counts[i],
|
||||
"is_actual": failures_prediction[months[i]].get("source") == "actual",
|
||||
"failure_cost": total_expected_failure_cost / month_index,
|
||||
"preventive_cost": preventive_cost / month_index,
|
||||
"procurement_cost": procurement_cost / month_index,
|
||||
"total_cost": cput,
|
||||
"absolute_failure_cost": total_expected_failure_cost,
|
||||
"absolute_overhaul_cost": preventive_cost,
|
||||
"absolute_procurement_cost": procurement_cost,
|
||||
"total_cycle_cost": total_cycle_cost,
|
||||
"is_after_planned_oh": month_index > self.planned_oh_months,
|
||||
"delay_months": max(0, month_index - self.planned_oh_months),
|
||||
"procurement_details": sparepart_analysis,
|
||||
"sparepart_available": sparepart_analysis["available"],
|
||||
"sparepart_status": sparepart_analysis["message"],
|
||||
"can_proceed": sparepart_analysis["can_proceed_with_delays"],
|
||||
}
|
||||
)
|
||||
|
||||
return results
|
||||
|
||||
def _analyze_sparepart_availability(
|
||||
self, equipment_tag: str, target_month: int, planned_overhauls: List
|
||||
) -> Dict:
|
||||
"""Analyze sparepart availability for equipment at target month"""
|
||||
if not self.sparepart_manager:
|
||||
return {
|
||||
"available": True,
|
||||
"message": "Sparepart manager not initialized",
|
||||
"total_procurement_cost": 0,
|
||||
"procurement_needed": [],
|
||||
"can_proceed_with_delays": True,
|
||||
}
|
||||
|
||||
other_overhauls = [
|
||||
(eq_tag, month)
|
||||
for eq_tag, month in planned_overhauls
|
||||
if eq_tag != equipment_tag and month <= target_month
|
||||
]
|
||||
|
||||
return self.sparepart_manager.check_sparepart_availability(
|
||||
equipment_tag, target_month, other_overhauls
|
||||
)
|
||||
|
||||
def _find_optimal_timing_with_spareparts(
|
||||
self, cost_results: List[Dict], location_tag: str
|
||||
) -> Optional[Dict]:
|
||||
"""Find optimal timing considering sparepart constraints"""
|
||||
if not cost_results:
|
||||
return None
|
||||
|
||||
feasible_results = [r for r in cost_results if r["can_proceed"]]
|
||||
|
||||
min_cost = float("inf")
|
||||
optimal_result = None
|
||||
|
||||
for i, result in enumerate(cost_results):
|
||||
if result in feasible_results and result["total_cost"] < min_cost:
|
||||
min_cost = result["total_cost"]
|
||||
optimal_result = result
|
||||
|
||||
if optimal_result is None:
|
||||
return None
|
||||
|
||||
return self._create_optimal_result(optimal_result, location_tag, "OPTIMAL")
|
||||
|
||||
def _create_optimal_result(
|
||||
self, optimal_result: Dict, location_tag: str, status: str
|
||||
) -> Dict:
|
||||
"""Create standardized optimal result dictionary"""
|
||||
return {
|
||||
"location_tag": location_tag,
|
||||
"optimal_month": optimal_result["month"],
|
||||
"optimal_index": optimal_result["month"] - 1,
|
||||
"optimal_cost": optimal_result["total_cost"],
|
||||
"failure_cost": optimal_result["failure_cost"],
|
||||
"preventive_cost": optimal_result["preventive_cost"],
|
||||
"procurement_cost": optimal_result["procurement_cost"],
|
||||
"number_of_failures": optimal_result["number_of_failures"],
|
||||
"is_delayed": optimal_result["is_after_planned_oh"],
|
||||
"delay_months": optimal_result["delay_months"],
|
||||
"planned_oh_month": self.planned_oh_months,
|
||||
"planned_cost": None,
|
||||
"cost_vs_planned": None,
|
||||
"savings_from_delay": 0,
|
||||
"cost_of_delay": 0,
|
||||
"sparepart_available": optimal_result["sparepart_available"],
|
||||
"sparepart_status": optimal_result["sparepart_status"],
|
||||
"procurement_details": optimal_result["procurement_details"],
|
||||
"optimization_status": status,
|
||||
"all_monthly_costs": [],
|
||||
}
|
||||
|
||||
async def calculate_cost_all_equipment_with_spareparts(
|
||||
self,
|
||||
db_session,
|
||||
collector_db_session,
|
||||
equipments: List,
|
||||
calculation,
|
||||
preventive_cost: float,
|
||||
simulation_id: str = "default",
|
||||
):
|
||||
"""
|
||||
Calculate optimal overhaul timing for entire fleet considering sparepart constraints
|
||||
"""
|
||||
|
||||
self.logger.info(
|
||||
f"Starting fleet optimization with reliability prediction for {len(equipments)} equipment"
|
||||
)
|
||||
max_interval = self.time_window_months
|
||||
|
||||
# Phase 1: Calculate individual optimal timings without considering interactions
|
||||
individual_results = {}
|
||||
|
||||
try:
|
||||
with open("src/calculation_time_constrains/full_equipment_with_downtime_opdesc.json", "r") as f:
|
||||
data = json.load(f)
|
||||
ecs_tags = {eq["Location"]: eq for eq in data}
|
||||
except FileNotFoundError:
|
||||
ecs_tags = {}
|
||||
|
||||
for equipment in equipments:
|
||||
location_tag = equipment.location_tag
|
||||
contribution_factor = 1.0
|
||||
ecs = ecs_tags.get(location_tag, None)
|
||||
|
||||
monthly_data = await self.get_failures_prediction(location_tag)
|
||||
|
||||
if not monthly_data:
|
||||
continue
|
||||
|
||||
equipment_preventive_cost = equipment.overhaul_cost + equipment.service_cost
|
||||
failure_replacement_cost = equipment.material_cost + (3 * 111000 * 3)
|
||||
|
||||
cost_results = self._calculate_equipment_costs_with_spareparts(
|
||||
failures_prediction=monthly_data,
|
||||
birnbaum_importance=contribution_factor,
|
||||
preventive_cost=equipment_preventive_cost,
|
||||
failure_replacement_cost=failure_replacement_cost,
|
||||
location_tag=location_tag,
|
||||
planned_overhauls=[],
|
||||
ecs=ecs,
|
||||
loss_production_permonth=0,
|
||||
)
|
||||
|
||||
if not cost_results:
|
||||
continue
|
||||
|
||||
optimal_timing = self._find_optimal_timing_with_spareparts(cost_results, location_tag)
|
||||
|
||||
if optimal_timing:
|
||||
optimal_timing["all_monthly_costs"] = cost_results
|
||||
individual_results[location_tag] = optimal_timing
|
||||
self.logger.info(
|
||||
f"Individual optimal for {location_tag}: Month {optimal_timing['optimal_month']}"
|
||||
)
|
||||
|
||||
# Phase 2: Optimize considering sparepart interactions
|
||||
self.logger.info("Phase 2: Optimizing with sparepart interactions...")
|
||||
|
||||
improved_plan = self._optimize_fleet_with_sparepart_constraints(
|
||||
individual_results, equipments, simulation_id
|
||||
)
|
||||
|
||||
# Phase 3: Generate final results and database objects
|
||||
fleet_results = []
|
||||
total_corrective_costs = np.zeros(max_interval)
|
||||
total_preventive_costs = np.zeros(max_interval)
|
||||
total_procurement_costs = np.zeros(max_interval)
|
||||
total_costs = np.zeros(max_interval)
|
||||
|
||||
total_fleet_procurement_cost = 0
|
||||
|
||||
for equipment in equipments:
|
||||
location_tag = equipment.location_tag
|
||||
|
||||
if location_tag not in individual_results:
|
||||
continue
|
||||
|
||||
equipment_timing = next(
|
||||
(month for tag, month in improved_plan if tag == location_tag),
|
||||
individual_results[location_tag]["optimal_month"],
|
||||
)
|
||||
|
||||
cost_data = individual_results[location_tag]["all_monthly_costs"][equipment_timing - 1]
|
||||
|
||||
all_costs = individual_results[location_tag]["all_monthly_costs"]
|
||||
|
||||
corrective_costs = [r["failure_cost"] for r in all_costs]
|
||||
preventive_costs = [r["preventive_cost"] for r in all_costs]
|
||||
procurement_costs = [r["procurement_cost"] for r in all_costs]
|
||||
failures = [r["number_of_failures"] for r in all_costs]
|
||||
total_costs_equipment = [r["total_cost"] for r in all_costs]
|
||||
procurement_details = [r["procurement_details"] for r in all_costs]
|
||||
|
||||
def pad_array(arr, target_length):
|
||||
if len(arr) < target_length:
|
||||
return arr + [arr[-1]] * (target_length - len(arr))
|
||||
return arr[:target_length]
|
||||
|
||||
corrective_costs = pad_array(corrective_costs, max_interval)
|
||||
preventive_costs = pad_array(preventive_costs, max_interval)
|
||||
procurement_costs = pad_array(procurement_costs, max_interval)
|
||||
failures = pad_array(failures, max_interval)
|
||||
total_costs_equipment = pad_array(total_costs_equipment, max_interval)
|
||||
procurement_details = pad_array(procurement_details, max_interval)
|
||||
|
||||
is_actual_list = [r.get("is_actual", False) for r in all_costs]
|
||||
is_actual_list = pad_array(is_actual_list, max_interval)
|
||||
|
||||
equipment_result = CalculationEquipmentResult(
|
||||
corrective_costs=corrective_costs,
|
||||
overhaul_costs=preventive_costs,
|
||||
procurement_costs=procurement_costs,
|
||||
daily_failures=failures,
|
||||
is_actual=is_actual_list,
|
||||
location_tag=equipment.location_tag,
|
||||
material_cost=equipment.material_cost,
|
||||
service_cost=equipment.service_cost,
|
||||
optimum_day=equipment_timing - 1,
|
||||
calculation_data_id=calculation.id,
|
||||
procurement_details=procurement_details,
|
||||
)
|
||||
|
||||
fleet_results.append(equipment_result)
|
||||
|
||||
total_corrective_costs += np.array(corrective_costs)
|
||||
total_preventive_costs += np.array(preventive_costs)
|
||||
total_procurement_costs += np.array(procurement_costs)
|
||||
total_costs += np.array(total_costs_equipment)
|
||||
|
||||
total_fleet_procurement_cost += cost_data["procurement_cost"]
|
||||
|
||||
fleet_optimal_index = np.argmin(total_costs)
|
||||
|
||||
calculation.optimum_oh_day = int(fleet_optimal_index)
|
||||
calculation.max_interval = max_interval
|
||||
calculation.rbd_simulation_id = simulation_id
|
||||
|
||||
db_session.add_all(fleet_results)
|
||||
await db_session.commit()
|
||||
|
||||
return int(fleet_optimal_index)
|
||||
|
||||
def _optimize_fleet_with_sparepart_constraints(
|
||||
self, individual_results: Dict, equipments: List, simulation_id: str
|
||||
) -> List[Tuple[str, int]]:
|
||||
"""
|
||||
Optimize fleet overhaul timing considering sparepart sharing constraints
|
||||
"""
|
||||
current_plan = [(tag, result["optimal_month"]) for tag, result in individual_results.items()]
|
||||
current_plan.sort(key=lambda x: x[1])
|
||||
|
||||
improved_plan = []
|
||||
processed_equipment = []
|
||||
|
||||
for equipment_tag, optimal_month in current_plan:
|
||||
sparepart_analysis = self.sparepart_manager.check_sparepart_availability(
|
||||
equipment_tag, optimal_month - 1, processed_equipment
|
||||
)
|
||||
|
||||
if sparepart_analysis["available"] or sparepart_analysis["can_proceed_with_delays"]:
|
||||
improved_plan.append((equipment_tag, optimal_month))
|
||||
processed_equipment.append((equipment_tag, optimal_month))
|
||||
else:
|
||||
alternative_month = self._find_alternative_timing(
|
||||
equipment_tag,
|
||||
optimal_month,
|
||||
individual_results[equipment_tag]["all_monthly_costs"],
|
||||
processed_equipment,
|
||||
)
|
||||
|
||||
if alternative_month:
|
||||
improved_plan.append((equipment_tag, alternative_month))
|
||||
processed_equipment.append((equipment_tag, alternative_month))
|
||||
else:
|
||||
improved_plan.append((equipment_tag, optimal_month))
|
||||
processed_equipment.append((equipment_tag, optimal_month))
|
||||
|
||||
return improved_plan
|
||||
|
||||
def _find_alternative_timing(
|
||||
self,
|
||||
equipment_tag: str,
|
||||
preferred_month: int,
|
||||
cost_results: List[Dict],
|
||||
processed_equipment: List[Tuple[str, int]],
|
||||
) -> Optional[int]:
|
||||
"""
|
||||
Find alternative timing when preferred month has sparepart constraints
|
||||
"""
|
||||
search_range = 6
|
||||
|
||||
candidates = []
|
||||
|
||||
for offset in range(-search_range // 2, search_range // 2 + 1):
|
||||
candidate_month = preferred_month + offset
|
||||
|
||||
if candidate_month <= 0 or candidate_month > len(cost_results):
|
||||
continue
|
||||
|
||||
if candidate_month == preferred_month:
|
||||
continue
|
||||
|
||||
sparepart_analysis = self.sparepart_manager.check_sparepart_availability(
|
||||
equipment_tag, candidate_month - 1, processed_equipment
|
||||
)
|
||||
|
||||
if sparepart_analysis["available"] or sparepart_analysis["can_proceed_with_delays"]:
|
||||
cost_data = cost_results[candidate_month - 1]
|
||||
candidates.append((candidate_month, cost_data["total_cost"]))
|
||||
|
||||
if not candidates:
|
||||
return None
|
||||
|
||||
candidates.sort(key=lambda x: x[1])
|
||||
return candidates[0][0]
|
||||
|
||||
async def __aenter__(self):
|
||||
await self._create_session()
|
||||
return self
|
||||
|
||||
async def __aexit__(self, exc_type, exc_val, exc_tb):
|
||||
await self._close_session()
|
||||
|
||||
|
||||
def _analyze_optimal_timing(
|
||||
calculation_results: List, optimum_oh_day: int, prev_oh_scope, scope_overhaul
|
||||
) -> Dict:
|
||||
"""Analyze optimal timing and provide recommendations"""
|
||||
|
||||
if not calculation_results:
|
||||
return {}
|
||||
|
||||
optimal_result = None
|
||||
if 0 <= optimum_oh_day < len(calculation_results):
|
||||
optimal_result = calculation_results[optimum_oh_day]
|
||||
|
||||
planned_oh_months = None
|
||||
if prev_oh_scope and scope_overhaul:
|
||||
planned_oh_months = get_months_between(prev_oh_scope.end_date, scope_overhaul.start_date)
|
||||
|
||||
timing_recommendation = "OPTIMAL"
|
||||
if planned_oh_months:
|
||||
if optimum_oh_day + 1 < planned_oh_months:
|
||||
timing_recommendation = "EARLY"
|
||||
elif optimum_oh_day + 1 > planned_oh_months:
|
||||
timing_recommendation = "DELAYED"
|
||||
else:
|
||||
timing_recommendation = "ON_SCHEDULE"
|
||||
|
||||
cost_trend = "STABLE"
|
||||
if len(calculation_results) > 1:
|
||||
early_costs = [r.total_cost for r in calculation_results[: len(calculation_results) // 3]]
|
||||
late_costs = [r.total_cost for r in calculation_results[-len(calculation_results) // 3 :]]
|
||||
|
||||
avg_early = sum(early_costs) / len(early_costs) if early_costs else 0
|
||||
avg_late = sum(late_costs) / len(late_costs) if late_costs else 0
|
||||
|
||||
if avg_late > avg_early * 1.2:
|
||||
cost_trend = "INCREASING"
|
||||
elif avg_late < avg_early * 0.8:
|
||||
cost_trend = "DECREASING"
|
||||
|
||||
return {
|
||||
"optimal_month": optimum_oh_day + 1,
|
||||
"planned_month": planned_oh_months,
|
||||
"timing_recommendation": timing_recommendation,
|
||||
"optimal_total_cost": optimal_result.total_cost if optimal_result else 0,
|
||||
"optimal_breakdown": {
|
||||
"corrective_cost": optimal_result.corrective_cost if optimal_result else 0,
|
||||
"overhaul_cost": optimal_result.overhaul_cost if optimal_result else 0,
|
||||
"procurement_cost": optimal_result.procurement_cost if optimal_result else 0,
|
||||
"num_failures": optimal_result.num_failures if optimal_result else 0,
|
||||
},
|
||||
"cost_trend": cost_trend,
|
||||
"months_from_planned": (optimum_oh_day + 1 - planned_oh_months)
|
||||
if planned_oh_months
|
||||
else None,
|
||||
"cost_savings_vs_planned": None,
|
||||
"sparepart_impact": {
|
||||
"equipment_with_constraints": optimal_result.sparepart_summary["equipment_requiring_procurement"]
|
||||
if optimal_result
|
||||
else 0,
|
||||
"critical_shortages": optimal_result.sparepart_summary["critical_shortages"]
|
||||
if optimal_result
|
||||
else 0,
|
||||
"procurement_investment": optimal_result.sparepart_summary["total_procurement_cost"]
|
||||
if optimal_result
|
||||
else 0,
|
||||
},
|
||||
}
|
||||
|
||||
|
||||
async def run_simulation_with_spareparts(
|
||||
*,
|
||||
db_session: DbSession,
|
||||
calculation,
|
||||
token: str,
|
||||
collector_db_session: CollectorDbSession,
|
||||
time_window_months: Optional[int] = None,
|
||||
simulation_id: str = "default",
|
||||
) -> Dict:
|
||||
"""
|
||||
Run complete overhaul optimization simulation with sparepart management
|
||||
"""
|
||||
from src.optimum_time_constraint.service import get_calculation_data_by_id
|
||||
from src.overhaul_activity.service import get_standard_scope_by_session_id
|
||||
|
||||
equipments = await get_standard_scope_by_session_id(
|
||||
db_session=db_session,
|
||||
overhaul_session_id=calculation.overhaul_session_id,
|
||||
collector_db=collector_db_session,
|
||||
)
|
||||
|
||||
scope = await get_scope(
|
||||
db_session=db_session, overhaul_session_id=calculation.overhaul_session_id
|
||||
)
|
||||
prev_oh_scope = await get_prev_oh(db_session=db_session, overhaul_session=scope)
|
||||
|
||||
calculation_data = await get_calculation_data_by_id(
|
||||
db_session=db_session, calculation_id=calculation.id
|
||||
)
|
||||
|
||||
time_window_months = get_months_between(prev_oh_scope.end_date, scope.start_date) + 6
|
||||
|
||||
sparepart_manager = await load_sparepart_data_from_db(
|
||||
scope=scope,
|
||||
prev_oh_scope=prev_oh_scope,
|
||||
db_session=collector_db_session,
|
||||
app_db_session=db_session,
|
||||
analysis_window_months=time_window_months,
|
||||
)
|
||||
|
||||
optimum_oh_model = OptimumCostModelWithSpareparts(
|
||||
token=token,
|
||||
last_oh_date=prev_oh_scope.end_date,
|
||||
next_oh_date=scope.start_date,
|
||||
base_url=RBD_SERVICE_API,
|
||||
sparepart_manager=sparepart_manager,
|
||||
)
|
||||
|
||||
try:
|
||||
fleet_optimal_index = await optimum_oh_model.calculate_cost_all_equipment_with_spareparts(
|
||||
db_session=db_session,
|
||||
collector_db_session=collector_db_session,
|
||||
equipments=equipments,
|
||||
calculation=calculation_data,
|
||||
preventive_cost=calculation_data.parameter.overhaul_cost,
|
||||
simulation_id=simulation_id,
|
||||
)
|
||||
finally:
|
||||
await optimum_oh_model._close_session()
|
||||
|
||||
calculation_query = await db_session.execute(
|
||||
select(CalculationData)
|
||||
.options(
|
||||
selectinload(CalculationData.equipment_results),
|
||||
selectinload(CalculationData.parameter),
|
||||
)
|
||||
.where(CalculationData.id == calculation.id)
|
||||
)
|
||||
scope_calculation = calculation_query.scalar_one_or_none()
|
||||
|
||||
data_num = scope_calculation.max_interval
|
||||
all_equipment = scope_calculation.equipment_results
|
||||
included_equipment = [eq for eq in all_equipment if eq.is_included]
|
||||
|
||||
calculation_results = []
|
||||
fleet_statistics = {
|
||||
"total_equipment": len(all_equipment),
|
||||
"included_equipment": len(included_equipment),
|
||||
"excluded_equipment": len(all_equipment) - len(included_equipment),
|
||||
"equipment_with_sparepart_constraints": 0,
|
||||
"total_procurement_items": 0,
|
||||
"critical_procurement_items": 0,
|
||||
"total_spareparts": 745,
|
||||
}
|
||||
|
||||
avg_failure_cost = (
|
||||
sum((eq.material_cost or 0) + (3 * 111000 * 3) for eq in all_equipment) / len(all_equipment)
|
||||
if all_equipment
|
||||
else 0
|
||||
)
|
||||
|
||||
rbd_marginal_fails = [0] * data_num
|
||||
try:
|
||||
if scope_calculation.rbd_simulation_id:
|
||||
plant_result_url = f"{RBD_SERVICE_API}/aeros/simulation/result/calc/{scope_calculation.rbd_simulation_id}/plant"
|
||||
async with httpx.AsyncClient(timeout=30.0) as client:
|
||||
response = await client.get(
|
||||
plant_result_url,
|
||||
headers={
|
||||
"Authorization": f"Bearer {token}",
|
||||
"Content-Type": "application/json",
|
||||
},
|
||||
)
|
||||
if response.status_code == 200:
|
||||
plant_data = response.json().get("data", {})
|
||||
timestamp_outs = plant_data.get("timestamp_outs", [])
|
||||
if timestamp_outs:
|
||||
hourly_data = create_time_series_data(
|
||||
timestamp_outs, max_hours=data_num * 720
|
||||
)
|
||||
cumulative_rbd_fails = calculate_failures_per_month(hourly_data)
|
||||
rbd_fails_map = {m["month"]: m["failures"] for m in cumulative_rbd_fails}
|
||||
prev_fail = 0
|
||||
for m in range(1, data_num + 1):
|
||||
curr_fail = rbd_fails_map.get(m, prev_fail)
|
||||
rbd_marginal_fails[m - 1] = curr_fail - prev_fail
|
||||
prev_fail = curr_fail
|
||||
except Exception as e:
|
||||
logger = logging.getLogger(__name__)
|
||||
logger.warning(f"Failed to fetch plant simulation: {e}")
|
||||
|
||||
cumulative_plant_failures = 0
|
||||
|
||||
for month_index in range(data_num):
|
||||
historical_marginal_fail = 0
|
||||
for eq in all_equipment:
|
||||
if eq.is_actual and month_index < len(eq.is_actual) and eq.is_actual[month_index]:
|
||||
curr_fail = (
|
||||
eq.daily_failures[month_index]
|
||||
if month_index < len(eq.daily_failures)
|
||||
else 0
|
||||
)
|
||||
prev_fail = (
|
||||
eq.daily_failures[month_index - 1]
|
||||
if month_index > 0 and (month_index - 1) < len(eq.daily_failures)
|
||||
else 0
|
||||
)
|
||||
historical_marginal_fail += max(0, curr_fail - prev_fail)
|
||||
|
||||
marginal_fail = rbd_marginal_fails[month_index] + historical_marginal_fail
|
||||
cumulative_plant_failures += marginal_fail
|
||||
|
||||
month_result = {
|
||||
"overhaul_cost": 0.0,
|
||||
"corrective_cost": 0.0,
|
||||
"procurement_cost": 0.0,
|
||||
"num_failures": cumulative_plant_failures,
|
||||
"day": month_index + 1,
|
||||
"month": month_index + 1,
|
||||
"procurement_details": {},
|
||||
"sparepart_summary": {
|
||||
"total_procurement_cost": 0.0,
|
||||
"equipment_requiring_procurement": 0,
|
||||
"critical_shortages": 0,
|
||||
"existing_orders_value": 0.0,
|
||||
"new_orders_required": 0,
|
||||
"urgent_procurements": 0,
|
||||
},
|
||||
}
|
||||
|
||||
equipment_requiring_procurement = 0
|
||||
total_existing_orders_value = 0.0
|
||||
total_new_orders_value = 0.0
|
||||
critical_shortages = 0
|
||||
urgent_procurements = 0
|
||||
|
||||
for eq in all_equipment:
|
||||
if month_index >= len(eq.procurement_details):
|
||||
continue
|
||||
procurement_detail = eq.procurement_details[month_index]
|
||||
if (
|
||||
procurement_detail
|
||||
and isinstance(procurement_detail, dict)
|
||||
and procurement_detail.get("procurement_needed")
|
||||
):
|
||||
equipment_requiring_procurement += 1
|
||||
pr_po_summary = procurement_detail.get("pr_po_summary", {})
|
||||
existing_orders_value = pr_po_summary.get("total_existing_value", 0)
|
||||
total_existing_orders_value += existing_orders_value
|
||||
new_orders_value = pr_po_summary.get("total_new_orders_value", 0)
|
||||
total_new_orders_value += new_orders_value
|
||||
critical_missing = procurement_detail.get("critical_missing_parts", 0)
|
||||
if critical_missing > 0:
|
||||
critical_shortages += 1
|
||||
recommendations = procurement_detail.get("recommendations", [])
|
||||
urgent_items = [
|
||||
r for r in recommendations if r.get("priority") == "CRITICAL"
|
||||
]
|
||||
if urgent_items:
|
||||
urgent_procurements += 1
|
||||
is_included_eq = False if eq.is_initial else eq.is_included
|
||||
month_result["procurement_details"][eq.location_tag] = {
|
||||
"is_included": is_included_eq,
|
||||
"location_tag": eq.location_tag,
|
||||
"details": procurement_detail.get("procurement_needed", []),
|
||||
"detailed_message": procurement_detail.get("detailed_message", ""),
|
||||
"pr_po_summary": pr_po_summary,
|
||||
"recommendations": recommendations,
|
||||
"sparepart_available": procurement_detail.get("sparepart_available", True),
|
||||
"can_proceed": procurement_detail.get("can_proceed_with_delays", True),
|
||||
"critical_missing_parts": critical_missing,
|
||||
"existing_orders_value": existing_orders_value,
|
||||
"new_orders_value": new_orders_value,
|
||||
}
|
||||
if eq.is_included:
|
||||
if month_index < len(eq.overhaul_costs) and month_index < len(
|
||||
eq.procurement_costs
|
||||
):
|
||||
month_result["overhaul_cost"] += float(eq.overhaul_costs[month_index])
|
||||
month_result["procurement_cost"] += float(eq.procurement_costs[month_index])
|
||||
|
||||
month_result["corrective_cost"] = (
|
||||
cumulative_plant_failures * avg_failure_cost
|
||||
) / (month_index + 1)
|
||||
month_result["sparepart_summary"].update(
|
||||
{
|
||||
"total_procurement_cost": month_result["procurement_cost"],
|
||||
"equipment_requiring_procurement": equipment_requiring_procurement,
|
||||
"critical_shortages": critical_shortages,
|
||||
"existing_orders_value": total_existing_orders_value,
|
||||
"new_orders_required": len(
|
||||
[
|
||||
eq
|
||||
for eq in all_equipment
|
||||
if month_index < len(eq.procurement_details)
|
||||
and eq.procurement_details[month_index]
|
||||
and eq.procurement_details[month_index].get("procurement_needed")
|
||||
]
|
||||
),
|
||||
"urgent_procurements": urgent_procurements,
|
||||
}
|
||||
)
|
||||
month_result["total_cost"] = (
|
||||
month_result["corrective_cost"]
|
||||
+ month_result["overhaul_cost"]
|
||||
+ month_result["procurement_cost"]
|
||||
)
|
||||
calculation_results.append(month_result)
|
||||
|
||||
optimum_day = np.argmin([month["total_cost"] for month in calculation_results])
|
||||
scope_calculation.optimum_oh_day = int(optimum_day)
|
||||
|
||||
fleet_statistics["equipment_with_sparepart_constraints"] = len(
|
||||
[
|
||||
eq
|
||||
for eq in all_equipment
|
||||
if any(
|
||||
detail and detail.get("procurement_needed")
|
||||
for detail in eq.procurement_details
|
||||
if detail
|
||||
)
|
||||
]
|
||||
)
|
||||
fleet_statistics["total_procurement_items"] = sum(
|
||||
[
|
||||
len(detail.get("procurement_needed", []))
|
||||
for eq in all_equipment
|
||||
for detail in eq.procurement_details
|
||||
if detail and isinstance(detail, dict)
|
||||
]
|
||||
)
|
||||
|
||||
analysis_metadata = {
|
||||
"planned_month": (scope.start_date.year - prev_oh_scope.end_date.year) * 12
|
||||
+ (scope.start_date.month - prev_oh_scope.end_date.month)
|
||||
if prev_oh_scope and scope
|
||||
else 0,
|
||||
"total_fleet_procurement_cost": sum(
|
||||
[
|
||||
eq.procurement_costs[int(scope_calculation.optimum_oh_day)]
|
||||
for eq in all_equipment
|
||||
if eq.procurement_costs
|
||||
]
|
||||
),
|
||||
"last_oh_date": prev_oh_scope.end_date.isoformat() if prev_oh_scope else None,
|
||||
"next_oh_date": scope.start_date.isoformat() if scope else None,
|
||||
"optimal_stat": None,
|
||||
}
|
||||
|
||||
calc_results_read = [CalculationResultsRead(**r) for r in calculation_results]
|
||||
optimal_analysis = _analyze_optimal_timing(
|
||||
calc_results_read, scope_calculation.optimum_oh_day, prev_oh_scope, scope
|
||||
)
|
||||
|
||||
scope_calculation.plant_results = calculation_results
|
||||
scope_calculation.fleet_statistics = fleet_statistics
|
||||
scope_calculation.analysis_metadata = analysis_metadata
|
||||
scope_calculation.optimum_analysis = optimal_analysis
|
||||
|
||||
await db_session.commit()
|
||||
|
||||
return {"id": calculation.id, "optimum": optimal_analysis}
|
||||
@ -1,233 +0,0 @@
|
||||
from typing import Annotated, List, Optional, Union
|
||||
|
||||
from fastapi import APIRouter
|
||||
from fastapi.params import Query
|
||||
|
||||
from src.auth.service import CurrentUser, InternalKey, Token
|
||||
from src.config import DEFAULT_TC_ID
|
||||
from src.database.core import DbSession, CollectorDbSession
|
||||
from src.models import StandardResponse
|
||||
|
||||
from src.optimum_time_constraint.flows import (
|
||||
create_calculation,
|
||||
get_create_calculation_parameters,
|
||||
get_or_create_scope_equipment_calculation,
|
||||
recalculate_calculation,
|
||||
)
|
||||
from src.calculation_time_constrains.schema import (
|
||||
CalculationResultsRead,
|
||||
CalculationSelectedEquipmentUpdate,
|
||||
CalculationTimeConstrainsCreate,
|
||||
CalculationTimeConstrainsParametersCreate,
|
||||
CalculationTimeConstrainsParametersRead,
|
||||
CalculationTimeConstrainsParametersRetrive,
|
||||
CalculationTimeConstrainsRead,
|
||||
CreateCalculationQuery,
|
||||
EquipmentResult,
|
||||
CalculationTimeConstrainsReadNoResult,
|
||||
)
|
||||
from src.optimum_time_constraint.service import (
|
||||
bulk_update_equipment,
|
||||
get_calculation_result,
|
||||
get_calculation_result_by_day,
|
||||
get_calculation_by_assetnum,
|
||||
get_all_calculations,
|
||||
refresh_spareparts_service,
|
||||
)
|
||||
|
||||
router = APIRouter()
|
||||
get_calculation = APIRouter()
|
||||
|
||||
|
||||
@router.post(
|
||||
"", response_model=StandardResponse[Union[dict, CalculationTimeConstrainsRead]]
|
||||
)
|
||||
async def create_calculation_time_constrains(
|
||||
token: Token,
|
||||
db_session: DbSession,
|
||||
collector_db_session: CollectorDbSession,
|
||||
current_user: CurrentUser,
|
||||
calculation_time_constrains_in: CalculationTimeConstrainsParametersCreate,
|
||||
params: Annotated[CreateCalculationQuery, Query()],
|
||||
):
|
||||
"""Save calculation time constrains Here"""
|
||||
scope_calculation_id = params.scope_calculation_id
|
||||
simulation_id = params.simulation_id
|
||||
|
||||
if scope_calculation_id:
|
||||
results = await get_or_create_scope_equipment_calculation(
|
||||
db_session=db_session,
|
||||
scope_calculation_id=scope_calculation_id,
|
||||
calculation_time_constrains_in=calculation_time_constrains_in,
|
||||
)
|
||||
else:
|
||||
results = await create_calculation(
|
||||
token=token,
|
||||
db_session=db_session,
|
||||
collector_db_session=collector_db_session,
|
||||
calculation_time_constrains_in=calculation_time_constrains_in,
|
||||
created_by=current_user.name,
|
||||
simulation_id=simulation_id,
|
||||
)
|
||||
|
||||
return StandardResponse(data=results, message="Data created successfully")
|
||||
|
||||
|
||||
@router.get(
|
||||
"", response_model=StandardResponse[List[CalculationTimeConstrainsReadNoResult]]
|
||||
)
|
||||
async def get_all_simulation_calculations(
|
||||
db_session: DbSession,
|
||||
token: Token,
|
||||
current_user: CurrentUser,
|
||||
):
|
||||
"""Get all calculation time constrains Here"""
|
||||
|
||||
calculations = await get_all_calculations(db_session=db_session)
|
||||
|
||||
return StandardResponse(
|
||||
data=calculations,
|
||||
message="Data retrieved successfully",
|
||||
)
|
||||
|
||||
|
||||
@router.get(
|
||||
"/parameters",
|
||||
response_model=StandardResponse[
|
||||
Union[
|
||||
CalculationTimeConstrainsParametersRetrive,
|
||||
CalculationTimeConstrainsParametersRead,
|
||||
]
|
||||
],
|
||||
)
|
||||
async def get_calculation_parameters(
|
||||
db_session: DbSession, calculation_id: Optional[str] = Query(default=None)
|
||||
):
|
||||
"""Get all calculation parameter."""
|
||||
|
||||
parameters = await get_create_calculation_parameters(
|
||||
db_session=db_session, calculation_id=calculation_id
|
||||
)
|
||||
|
||||
return StandardResponse(
|
||||
data=parameters,
|
||||
message="Data retrieved successfully",
|
||||
)
|
||||
|
||||
|
||||
@get_calculation.get(
|
||||
"/{calculation_id}", response_model=StandardResponse[CalculationTimeConstrainsRead]
|
||||
)
|
||||
async def get_calculation_results(
|
||||
db_session: DbSession,
|
||||
calculation_id,
|
||||
token: InternalKey,
|
||||
include_risk_cost: int = Query(1, alias="risk_cost"),
|
||||
):
|
||||
if calculation_id == "default":
|
||||
calculation_id = DEFAULT_TC_ID
|
||||
|
||||
results = await get_calculation_result(
|
||||
db_session=db_session,
|
||||
calculation_id=calculation_id,
|
||||
token=token,
|
||||
include_risk_cost=include_risk_cost,
|
||||
)
|
||||
|
||||
return StandardResponse(
|
||||
data=results,
|
||||
message="Data retrieved successfully",
|
||||
)
|
||||
|
||||
|
||||
@router.get(
|
||||
"/{calculation_id}/{assetnum}", response_model=StandardResponse[EquipmentResult]
|
||||
)
|
||||
async def get_calculation_per_equipment(db_session: DbSession, calculation_id, assetnum):
|
||||
results = await get_calculation_by_assetnum(
|
||||
db_session=db_session, assetnum=assetnum, calculation_id=calculation_id
|
||||
)
|
||||
|
||||
return StandardResponse(
|
||||
data=results,
|
||||
message="Data retrieved successfully",
|
||||
)
|
||||
|
||||
|
||||
@router.post(
|
||||
"/{calculation_id}/simulation",
|
||||
response_model=StandardResponse[CalculationResultsRead],
|
||||
)
|
||||
async def get_simulation_result(
|
||||
db_session: DbSession,
|
||||
calculation_id,
|
||||
calculation_simuation_in: CalculationTimeConstrainsCreate,
|
||||
):
|
||||
simulation_result = await get_calculation_result_by_day(
|
||||
db_session=db_session,
|
||||
calculation_id=calculation_id,
|
||||
simulation_day=calculation_simuation_in.intervalDays,
|
||||
)
|
||||
|
||||
return StandardResponse(
|
||||
data=simulation_result, message="Data retrieved successfully"
|
||||
)
|
||||
|
||||
|
||||
@router.post("/update/{calculation_id}", response_model=StandardResponse[List[str]])
|
||||
async def update_selected_equipment(
|
||||
db_session: DbSession,
|
||||
calculation_id,
|
||||
calculation_time_constrains_in: List[CalculationSelectedEquipmentUpdate],
|
||||
):
|
||||
if calculation_id == "default":
|
||||
calculation_id = "3b9a73a2-bde6-418c-9e2f-19046f501a05"
|
||||
|
||||
results = await bulk_update_equipment(
|
||||
db=db_session,
|
||||
selected_equipments=calculation_time_constrains_in,
|
||||
calculation_data_id=calculation_id,
|
||||
)
|
||||
|
||||
return StandardResponse(
|
||||
data=results,
|
||||
message="Data retrieved successfully",
|
||||
)
|
||||
|
||||
|
||||
@router.post("/{calculation_id}/refresh-spareparts", response_model=StandardResponse[dict])
|
||||
async def refresh_spareparts(
|
||||
db_session: DbSession,
|
||||
collector_db_session: CollectorDbSession,
|
||||
calculation_id: str,
|
||||
current_user: CurrentUser,
|
||||
):
|
||||
"""Refresh sparepart availability for an existing calculation"""
|
||||
|
||||
await refresh_spareparts_service(
|
||||
db_session=db_session,
|
||||
collector_db_session=collector_db_session,
|
||||
calculation_id=calculation_id,
|
||||
)
|
||||
|
||||
return StandardResponse(data={}, message="Spareparts refreshed successfully")
|
||||
|
||||
|
||||
@router.post("/{calculation_id}/recalculate", response_model=StandardResponse[CalculationTimeConstrainsRead])
|
||||
async def recalculate_calculation_api(
|
||||
db_session: DbSession,
|
||||
collector_db_session: CollectorDbSession,
|
||||
calculation_id: str,
|
||||
token: Token,
|
||||
current_user: CurrentUser,
|
||||
):
|
||||
"""Recalculate an existing simulation with fresh data"""
|
||||
|
||||
results = await recalculate_calculation(
|
||||
token=token,
|
||||
db_session=db_session,
|
||||
collector_db_session=collector_db_session,
|
||||
calculation_id=calculation_id,
|
||||
)
|
||||
|
||||
return StandardResponse(data=results, message="Calculation updated with fresh data")
|
||||
@ -1,252 +0,0 @@
|
||||
import datetime
|
||||
from typing import Coroutine, List, Optional, Tuple, Dict
|
||||
from uuid import UUID
|
||||
|
||||
from fastapi import HTTPException, status
|
||||
from sqlalchemy import and_, case, func, select, update
|
||||
from sqlalchemy.orm import joinedload, selectinload
|
||||
|
||||
from src.database.core import DbSession, CollectorDbSession
|
||||
from src.workorder.model import MasterWorkOrder
|
||||
from src.calculation_time_constrains.model import (CalculationData, CalculationEquipmentResult, CalculationResult)
|
||||
from src.calculation_time_constrains.schema import (
|
||||
CalculationResultsRead,
|
||||
CalculationSelectedEquipmentUpdate,
|
||||
CalculationTimeConstrainsParametersCreate,
|
||||
CalculationTimeConstrainsRead
|
||||
)
|
||||
from src.overhaul_scope.service import get as get_scope, get_prev_oh
|
||||
|
||||
|
||||
async def create_param_and_data(
|
||||
*,
|
||||
db_session: DbSession,
|
||||
calculation_param_in: CalculationTimeConstrainsParametersCreate,
|
||||
created_by: str,
|
||||
parameter_id: Optional[UUID] = None,
|
||||
):
|
||||
"""Creates a new document."""
|
||||
if calculation_param_in.ohSessionId is None:
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_400_BAD_REQUEST,
|
||||
detail="overhaul_session_id is required",
|
||||
)
|
||||
|
||||
calculationData = await CalculationData.create_with_param(
|
||||
db=db_session,
|
||||
overhaul_session_id=calculation_param_in.ohSessionId,
|
||||
avg_failure_cost=calculation_param_in.costPerFailure,
|
||||
overhaul_cost=calculation_param_in.overhaulCost,
|
||||
created_by=created_by,
|
||||
params_id=parameter_id,
|
||||
)
|
||||
return calculationData
|
||||
|
||||
|
||||
async def get_calculation_result(db_session: DbSession, calculation_id: str, token, include_risk_cost):
|
||||
"""
|
||||
Get calculation results from DB, returning pre-calculated plant and equipment results.
|
||||
"""
|
||||
try:
|
||||
# Get calculation data with equipment results
|
||||
calculation_query = await db_session.execute(
|
||||
select(CalculationData)
|
||||
.options(selectinload(CalculationData.equipment_results))
|
||||
.where(CalculationData.id == calculation_id)
|
||||
)
|
||||
scope_calculation = calculation_query.scalar_one_or_none()
|
||||
|
||||
if not scope_calculation:
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_404_NOT_FOUND,
|
||||
detail=f"Calculation with id {calculation_id} does not exist.",
|
||||
)
|
||||
|
||||
scope_overhaul = await get_scope(
|
||||
db_session=db_session,
|
||||
overhaul_session_id=scope_calculation.overhaul_session_id
|
||||
)
|
||||
if not scope_overhaul:
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_404_NOT_FOUND,
|
||||
detail=f"Overhaul scope for session {scope_calculation.overhaul_session_id} does not exist.",
|
||||
)
|
||||
|
||||
# Parse pre-calculated plant results
|
||||
plant_results_raw = scope_calculation.plant_results or []
|
||||
calculation_results = [CalculationResultsRead(**r) for r in plant_results_raw]
|
||||
|
||||
# Return comprehensive result
|
||||
return CalculationTimeConstrainsRead(
|
||||
id=scope_calculation.id,
|
||||
reference=scope_calculation.overhaul_session_id,
|
||||
scope=scope_overhaul.maintenance_type.name,
|
||||
results=calculation_results,
|
||||
optimum_oh=scope_calculation.optimum_oh_day,
|
||||
optimum_oh_month=scope_calculation.optimum_oh_day + 1 if scope_calculation.optimum_oh_day is not None else None,
|
||||
equipment_results=scope_calculation.equipment_results,
|
||||
fleet_statistics=scope_calculation.fleet_statistics or {},
|
||||
optimal_analysis=scope_calculation.optimum_analysis or {},
|
||||
analysis_metadata=scope_calculation.analysis_metadata or {}
|
||||
)
|
||||
|
||||
except HTTPException:
|
||||
raise
|
||||
except Exception as e:
|
||||
import logging
|
||||
logger = logging.getLogger(__name__)
|
||||
logger.error(f"Error in get_calculation_result for calculation_id {calculation_id}: {str(e)}")
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_500_INTERNAL_SERVER_ERROR,
|
||||
detail=f"Internal error processing calculation results: {str(e)}",
|
||||
)
|
||||
|
||||
|
||||
async def get_calculation_data_by_id(db_session: DbSession, calculation_id) -> CalculationData:
|
||||
stmt = (
|
||||
select(CalculationData)
|
||||
.filter(CalculationData.id == calculation_id)
|
||||
.options(
|
||||
joinedload(CalculationData.equipment_results),
|
||||
joinedload(CalculationData.parameter),
|
||||
)
|
||||
)
|
||||
result = await db_session.execute(stmt)
|
||||
return result.unique().scalar()
|
||||
|
||||
|
||||
async def get_all_calculations(db_session: DbSession) -> List[CalculationData]:
|
||||
stmt = (
|
||||
select(CalculationData)
|
||||
.options(selectinload(CalculationData.session))
|
||||
.where(
|
||||
CalculationData.optimum_oh_day.isnot(None),
|
||||
CalculationData.max_interval.isnot(None),
|
||||
CalculationData.optimum_analysis.isnot(None),
|
||||
)
|
||||
.order_by(CalculationData.created_at.desc())
|
||||
)
|
||||
result = await db_session.execute(stmt)
|
||||
return result.scalars().all()
|
||||
|
||||
|
||||
async def get_calculation_by_assetnum(*, db_session: DbSession, assetnum: str, calculation_id: str):
|
||||
stmt = (
|
||||
select(CalculationEquipmentResult)
|
||||
.where(CalculationEquipmentResult.assetnum == assetnum)
|
||||
.where(CalculationEquipmentResult.calculation_data_id == calculation_id)
|
||||
)
|
||||
result = await db_session.execute(stmt)
|
||||
return result.scalar()
|
||||
|
||||
|
||||
async def get_calculation_by_reference_and_parameter(
|
||||
*, db_session: DbSession, calculation_reference_id, parameter_id
|
||||
):
|
||||
stmt = select(CalculationData).filter(
|
||||
and_(
|
||||
CalculationData.reference_id == calculation_reference_id,
|
||||
CalculationData.parameter_id == parameter_id,
|
||||
)
|
||||
)
|
||||
result = await db_session.execute(stmt)
|
||||
return result.scalar()
|
||||
|
||||
|
||||
async def get_calculation_result_by_day(*, db_session: DbSession, calculation_id, simulation_day):
|
||||
stmt = select(CalculationData).filter(CalculationData.id == calculation_id)
|
||||
result = await db_session.execute(stmt)
|
||||
calculation_data = result.scalar_one_or_none()
|
||||
|
||||
if not calculation_data or not calculation_data.plant_results:
|
||||
return None
|
||||
|
||||
for res in calculation_data.plant_results:
|
||||
if res.get("day") == simulation_day:
|
||||
return res
|
||||
|
||||
return None
|
||||
|
||||
|
||||
async def get_avg_cost_by_asset(*, db_session: DbSession, assetnum: str):
|
||||
stmt = select(func.avg(MasterWorkOrder.total_cost_max).label("average_cost")).where(
|
||||
MasterWorkOrder.assetnum == assetnum
|
||||
)
|
||||
result = await db_session.execute(stmt)
|
||||
return result.scalar_one_or_none()
|
||||
|
||||
|
||||
async def bulk_update_equipment(
|
||||
*,
|
||||
db: DbSession,
|
||||
selected_equipments: List[CalculationSelectedEquipmentUpdate],
|
||||
calculation_data_id: UUID,
|
||||
):
|
||||
case_mappings = {asset.location_tag: asset.is_included for asset in selected_equipments}
|
||||
location_tags = list(case_mappings.keys())
|
||||
|
||||
when_clauses = [
|
||||
(CalculationEquipmentResult.location_tag == location_tag, is_included)
|
||||
for location_tag, is_included in case_mappings.items()
|
||||
]
|
||||
|
||||
stmt = (
|
||||
update(CalculationEquipmentResult)
|
||||
.where(CalculationEquipmentResult.calculation_data_id == calculation_data_id)
|
||||
.where(CalculationEquipmentResult.location_tag.in_(location_tags))
|
||||
.values(
|
||||
{
|
||||
"is_included": case(*when_clauses),
|
||||
"is_initial": False
|
||||
}
|
||||
)
|
||||
)
|
||||
|
||||
await db.execute(stmt)
|
||||
await db.commit()
|
||||
return location_tags
|
||||
|
||||
|
||||
async def refresh_spareparts_service(db_session: DbSession, collector_db_session: CollectorDbSession, calculation_id: str):
|
||||
stmt = select(CalculationData).where(CalculationData.id == calculation_id).options(
|
||||
joinedload(CalculationData.equipment_results)
|
||||
)
|
||||
result = await db_session.execute(stmt)
|
||||
calculation = result.scalar_one_or_none()
|
||||
|
||||
if not calculation:
|
||||
raise HTTPException(status_code=404, detail="Calculation not found")
|
||||
|
||||
scope = await get_scope(db_session=db_session, overhaul_session_id=calculation.overhaul_session_id)
|
||||
prev_oh_scope = await get_prev_oh(db_session=db_session, overhaul_session=scope)
|
||||
|
||||
last_oh_date = prev_oh_scope.end_date
|
||||
next_oh_date = scope.start_date
|
||||
time_window_months = ((next_oh_date.year - last_oh_date.year) * 12 +
|
||||
(next_oh_date.month - last_oh_date.month) + 6)
|
||||
|
||||
from src.sparepart.service import load_sparepart_data_from_db
|
||||
sparepart_manager = await load_sparepart_data_from_db(
|
||||
scope=scope,
|
||||
prev_oh_scope=prev_oh_scope,
|
||||
db_session=collector_db_session,
|
||||
app_db_session=db_session,
|
||||
analysis_window_months=time_window_months
|
||||
)
|
||||
|
||||
for eq in calculation.equipment_results:
|
||||
procurement_details = []
|
||||
procurement_costs = []
|
||||
for month_index in range(time_window_months):
|
||||
status = sparepart_manager.check_sparepart_availability(eq.location_tag, month_index)
|
||||
procurement_details.append(status)
|
||||
|
||||
if not status['available']:
|
||||
procurement_costs.append(status['total_procurement_cost'])
|
||||
else:
|
||||
procurement_costs.append(0.0)
|
||||
|
||||
eq.procurement_details = procurement_details
|
||||
eq.procurement_costs = procurement_costs
|
||||
|
||||
await db_session.commit()
|
||||
@ -1,140 +0,0 @@
|
||||
import asyncio
|
||||
from datetime import timedelta
|
||||
import httpx
|
||||
from temporalio import activity, workflow
|
||||
|
||||
from src.config import RBD_SERVICE_API
|
||||
from src.database.core import get_main_session, get_collector_session
|
||||
from src.optimum_time_constraint.service import (
|
||||
create_param_and_data,
|
||||
get_calculation_data_by_id
|
||||
)
|
||||
from src.optimum_time_constraint.optimizer import run_simulation_with_spareparts
|
||||
from src.calculation_time_constrains.schema import CalculationTimeConstrainsParametersCreate
|
||||
from src.overhaul_scope.service import get as get_scope, get_prev_oh
|
||||
from src.calculation_time_constrains.utils import get_months_between
|
||||
from src.calculation_target_reliability.utils import wait_for_workflow
|
||||
|
||||
@activity.defn
|
||||
async def create_optimum_oh_calculation(args: dict) -> str:
|
||||
token = args["token"]
|
||||
calc_in_dict = args["calculation_in"]
|
||||
created_by = args["created_by"]
|
||||
|
||||
calc_in = CalculationTimeConstrainsParametersCreate(**calc_in_dict)
|
||||
|
||||
async with get_main_session() as db_session:
|
||||
calc_data = await create_param_and_data(
|
||||
db_session=db_session,
|
||||
calculation_param_in=calc_in,
|
||||
created_by=created_by
|
||||
)
|
||||
return str(calc_data.id)
|
||||
|
||||
@activity.defn
|
||||
async def request_rbd_simulation(args: dict) -> str:
|
||||
calc_id = args["calc_id"]
|
||||
token = args["token"]
|
||||
|
||||
async with get_main_session() as db_session:
|
||||
calc_data = await get_calculation_data_by_id(db_session=db_session, calculation_id=calc_id)
|
||||
|
||||
scope = await get_scope(db_session=db_session, overhaul_session_id=calc_data.overhaul_session_id)
|
||||
prev_oh_scope = await get_prev_oh(db_session=db_session, overhaul_session=scope)
|
||||
|
||||
time_window_months = get_months_between(prev_oh_scope.end_date, scope.start_date) + 6
|
||||
sim_duration_hours = time_window_months * 720
|
||||
|
||||
sim_input = {
|
||||
"SchematicName": "- TJB - Unit 3 -",
|
||||
"SimulationName": f"OptimumOH_Calc_{calc_id}",
|
||||
"SimDuration": sim_duration_hours,
|
||||
"DurationUnit": "UHour",
|
||||
"OffSet": 0,
|
||||
"SimSeed": 99,
|
||||
"SimNumRun": 1,
|
||||
"OverhaulInterval": 0,
|
||||
"MaintenanceOutages": 0,
|
||||
"IsDefault": False,
|
||||
"OverhaulDuration": 0,
|
||||
"AhmJobId": None
|
||||
}
|
||||
|
||||
async with httpx.AsyncClient(timeout=30.0) as client:
|
||||
resp = await client.post(
|
||||
f"{RBD_SERVICE_API}/aeros/simulation/run",
|
||||
json=sim_input,
|
||||
headers={"Authorization": f"Bearer {token}"}
|
||||
)
|
||||
resp.raise_for_status()
|
||||
sim_id = resp.json()["data"]
|
||||
|
||||
calc_data.rbd_simulation_id = sim_id
|
||||
return str(sim_id)
|
||||
|
||||
@activity.defn
|
||||
async def wait_for_rbd_simulation(sim_id: str) -> str:
|
||||
await wait_for_workflow(sim_id)
|
||||
return sim_id
|
||||
|
||||
@activity.defn
|
||||
async def run_optimum_oh_calculation(args: dict) -> dict:
|
||||
calc_id = args["calc_id"]
|
||||
token = args["token"]
|
||||
sim_id = args["sim_id"]
|
||||
|
||||
async with get_main_session() as db_session:
|
||||
async with get_collector_session() as collector_db:
|
||||
calc_data = await get_calculation_data_by_id(db_session=db_session, calculation_id=calc_id)
|
||||
results = await run_simulation_with_spareparts(
|
||||
db_session=db_session,
|
||||
calculation=calc_data,
|
||||
token=token,
|
||||
collector_db_session=collector_db,
|
||||
simulation_id=sim_id
|
||||
)
|
||||
return {"id": str(results["id"]), "optimum": results["optimum"]}
|
||||
|
||||
@workflow.defn
|
||||
class OptimumOHCalculationWorkflow:
|
||||
def __init__(self):
|
||||
self.done = False
|
||||
self.result = None
|
||||
|
||||
@workflow.signal
|
||||
def notify_done(self, result: dict):
|
||||
self.done = True
|
||||
self.result = result
|
||||
|
||||
@workflow.run
|
||||
async def run(self, args: dict) -> dict:
|
||||
calc_id = await workflow.execute_activity(
|
||||
create_optimum_oh_calculation,
|
||||
args,
|
||||
start_to_close_timeout=timedelta(minutes=1)
|
||||
)
|
||||
|
||||
args["calc_id"] = calc_id
|
||||
|
||||
sim_id = await workflow.execute_activity(
|
||||
request_rbd_simulation,
|
||||
args,
|
||||
start_to_close_timeout=timedelta(minutes=2)
|
||||
)
|
||||
|
||||
args["sim_id"] = sim_id
|
||||
|
||||
# await workflow.execute_activity(
|
||||
# wait_for_rbd_simulation,
|
||||
# sim_id,
|
||||
# start_to_close_timeout=timedelta(hours=2)
|
||||
# )
|
||||
await workflow.wait_condition(lambda: self.done)
|
||||
|
||||
result = await workflow.execute_activity(
|
||||
run_optimum_oh_calculation,
|
||||
args,
|
||||
start_to_close_timeout=timedelta(minutes=30)
|
||||
)
|
||||
|
||||
return result
|
||||
@ -1,17 +0,0 @@
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
from sqlalchemy import Column, String
|
||||
from src.database.core import Base
|
||||
from src.models import DefaultMixin
|
||||
|
||||
|
||||
class OverhaulGantt(Base, DefaultMixin):
|
||||
__tablename__ = "oh_ms_monitoring_spreadsheet"
|
||||
|
||||
spreadsheet_id = Column(String, nullable=True)
|
||||
spreadsheet_link = Column(String, nullable=True)
|
||||
|
||||
@ -1,7 +1,4 @@
|
||||
from slowapi import Limiter
|
||||
from slowapi.util import get_remote_address
|
||||
|
||||
limiter = Limiter(
|
||||
key_func=get_remote_address,
|
||||
default_limits=["20000 per hour", "100000 per day"]
|
||||
)
|
||||
limiter = Limiter(key_func=get_remote_address)
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
@ -1,159 +0,0 @@
|
||||
import asyncio
|
||||
from datetime import timedelta
|
||||
from temporalio import activity, workflow
|
||||
|
||||
# with workflow.unsafe.imports_passed_through():
|
||||
# import httpx
|
||||
# from src.config import RBD_SERVICE_API
|
||||
# from src.database.core import get_main_session, get_collector_session
|
||||
# from src.optimum_time_constraint.service import (
|
||||
# create_param_and_data,
|
||||
# get_calculation_data_by_id
|
||||
# )
|
||||
# from src.optimum_time_constraint.optimizer import run_simulation_with_spareparts
|
||||
# from src.overhaul_scope.service import get as get_scope, get_prev_oh
|
||||
# from src.calculation_time_constrains.utils import get_months_between
|
||||
# from src.calculation_target_reliability.utils import wait_for_workflow
|
||||
|
||||
@activity.defn
|
||||
async def create_optimum_oh_calculation(args: dict) -> str:
|
||||
from src.calculation_time_constrains.schema import CalculationTimeConstrainsParametersCreate
|
||||
from src.calculation_time_constrains.service import create_param_and_data
|
||||
from src.database.core import get_main_session
|
||||
|
||||
token = args["token"]
|
||||
calc_in_dict = args["calculation_in"]
|
||||
created_by = args["created_by"]
|
||||
|
||||
calc_in = CalculationTimeConstrainsParametersCreate(**calc_in_dict)
|
||||
|
||||
async with get_main_session() as db_session:
|
||||
calc_data = await create_param_and_data(
|
||||
db_session=db_session,
|
||||
calculation_param_in=calc_in,
|
||||
created_by=created_by
|
||||
)
|
||||
return str(calc_data.id)
|
||||
|
||||
@activity.defn
|
||||
async def request_rbd_simulation(args: dict) -> dict:
|
||||
from src.calculation_time_constrains.service import get_calculation_data_by_id
|
||||
from src.database.core import get_main_session
|
||||
from src.overhaul_scope.service import get as get_scope, get_prev_oh
|
||||
from src.calculation_time_constrains.utils import get_months_between
|
||||
from src.calculation_target_reliability.utils import wait_for_workflow
|
||||
import httpx
|
||||
from src.config import RBD_SERVICE_API
|
||||
|
||||
calc_id = args["calc_id"]
|
||||
token = args["token"]
|
||||
callback_workflow_id = args["callback_workflow_id"]
|
||||
|
||||
async with get_main_session() as db_session:
|
||||
calc_data = await get_calculation_data_by_id(db_session=db_session, calculation_id=calc_id)
|
||||
|
||||
scope = await get_scope(db_session=db_session, overhaul_session_id=calc_data.overhaul_session_id)
|
||||
prev_oh_scope = await get_prev_oh(db_session=db_session, overhaul_session=scope)
|
||||
|
||||
# Calculate time window
|
||||
time_window_months = get_months_between(prev_oh_scope.end_date, scope.start_date) + 6
|
||||
sim_duration_hours = time_window_months * 720
|
||||
|
||||
sim_input = {
|
||||
"SchematicName": "- TJB - Unit 3 -",
|
||||
"SimulationName": f"OptimumOH_Calc_{calc_id}",
|
||||
"SimDuration": sim_duration_hours,
|
||||
"DurationUnit": "UHour",
|
||||
"OffSet": 0,
|
||||
"SimSeed": 99,
|
||||
"SimNumRun": 1,
|
||||
"OverhaulInterval": sim_duration_hours + 1,
|
||||
"MaintenanceOutages": 0,
|
||||
"IsDefault": False,
|
||||
"OverhaulDuration": 1200,
|
||||
"AhmJobId": None,
|
||||
"CallbackWorkflowId": callback_workflow_id,
|
||||
}
|
||||
|
||||
async with httpx.AsyncClient(timeout=30.0) as client:
|
||||
resp = await client.post(
|
||||
f"{RBD_SERVICE_API}/aeros/simulation/run",
|
||||
json=sim_input,
|
||||
headers={"Authorization": f"Bearer {token}"}
|
||||
)
|
||||
resp.raise_for_status()
|
||||
sim_id = resp.json()["data"]
|
||||
|
||||
calc_data.rbd_simulation_id = sim_id
|
||||
return sim_id
|
||||
|
||||
# @activity.defn
|
||||
# async def wait_for_rbd_simulation(sim_id: str) -> str:
|
||||
|
||||
# await wait_for_workflow(sim_id)
|
||||
# return sim_id
|
||||
|
||||
@activity.defn
|
||||
async def run_optimum_oh_calculation(args: dict) -> dict:
|
||||
from src.database.core import get_main_session, get_collector_session
|
||||
from src.calculation_time_constrains.service import get_calculation_data_by_id, run_simulation_with_spareparts
|
||||
from src.config import RBD_SERVICE_API
|
||||
|
||||
calc_id = args["calc_id"]
|
||||
token = args["token"]
|
||||
sim_id = args["sim_id"]
|
||||
|
||||
async with get_main_session() as db_session:
|
||||
async with get_collector_session() as collector_db:
|
||||
calc_data = await get_calculation_data_by_id(db_session=db_session, calculation_id=calc_id)
|
||||
results = await run_simulation_with_spareparts(
|
||||
db_session=db_session,
|
||||
calculation=calc_data,
|
||||
token=token,
|
||||
collector_db_session=collector_db,
|
||||
simulation_id=sim_id,
|
||||
rbd_service_api=RBD_SERVICE_API
|
||||
)
|
||||
# return simplified result since temporal handles JSON
|
||||
return {"id": str(results["id"]), "optimum": results["optimum"]}
|
||||
|
||||
@workflow.defn
|
||||
class OptimumOHCalculationWorkflow:
|
||||
def __init__(self):
|
||||
self.done = False
|
||||
|
||||
@workflow.signal
|
||||
def notify_done(self):
|
||||
self.done = True
|
||||
|
||||
@workflow.run
|
||||
async def run(self, args: dict) -> dict:
|
||||
# 1. Create calculation
|
||||
calc_id = await workflow.execute_activity(
|
||||
create_optimum_oh_calculation,
|
||||
args,
|
||||
start_to_close_timeout=timedelta(minutes=1)
|
||||
)
|
||||
|
||||
args["calc_id"] = calc_id
|
||||
|
||||
# 2. Request RBD simulation
|
||||
sim_id = await workflow.execute_activity(
|
||||
request_rbd_simulation,
|
||||
args,
|
||||
start_to_close_timeout=timedelta(minutes=2)
|
||||
)
|
||||
|
||||
args["sim_id"] = sim_id
|
||||
|
||||
# Wait for RBD simulation to finish using signal
|
||||
await workflow.wait_condition(lambda: self.done)
|
||||
|
||||
# 4. Run Optimum OH calculation
|
||||
result = await workflow.execute_activity(
|
||||
run_optimum_oh_calculation,
|
||||
args,
|
||||
start_to_close_timeout=timedelta(minutes=30)
|
||||
)
|
||||
|
||||
return result
|
||||
@ -1,21 +0,0 @@
|
||||
from sqlalchemy import Column, JSON, String, select
|
||||
from sqlalchemy.orm import declarative_base
|
||||
|
||||
Base = declarative_base()
|
||||
|
||||
class Test(Base):
|
||||
__tablename__ = 'test'
|
||||
id = Column(String, primary_key=True)
|
||||
meta = Column(JSON)
|
||||
|
||||
try:
|
||||
s = select(Test).where(Test.meta["type"].as_string() == "target_reliability")
|
||||
print("as_string works")
|
||||
except Exception as e:
|
||||
print("as_string error:", e)
|
||||
|
||||
try:
|
||||
s = select(Test).where(Test.meta["type"].astext == "target_reliability")
|
||||
print("astext works")
|
||||
except Exception as e:
|
||||
print("astext error:", e)
|
||||
@ -1,6 +0,0 @@
|
||||
import asyncio
|
||||
import httpx
|
||||
from src.config import RBD_SERVICE_API
|
||||
|
||||
async def main():
|
||||
token = "your_token_here" # I don't have a token. I'll need a token.
|
||||
@ -1,68 +1,68 @@
|
||||
# import asyncio
|
||||
# from typing import AsyncGenerator, Generator
|
||||
import asyncio
|
||||
from typing import AsyncGenerator, Generator
|
||||
|
||||
# import pytest
|
||||
# from httpx import AsyncClient
|
||||
# from sqlalchemy.ext.asyncio import AsyncSession, create_async_engine
|
||||
# from sqlalchemy.orm import sessionmaker
|
||||
# from sqlalchemy.pool import StaticPool
|
||||
# from sqlalchemy_utils import database_exists, drop_database
|
||||
# from starlette.config import environ
|
||||
# from starlette.testclient import TestClient
|
||||
import pytest
|
||||
from httpx import AsyncClient
|
||||
from sqlalchemy.ext.asyncio import AsyncSession, create_async_engine
|
||||
from sqlalchemy.orm import sessionmaker
|
||||
from sqlalchemy.pool import StaticPool
|
||||
from sqlalchemy_utils import database_exists, drop_database
|
||||
from starlette.config import environ
|
||||
from starlette.testclient import TestClient
|
||||
|
||||
# # from src.database import Base, get_db
|
||||
# # from src.main import app
|
||||
# from src.database import Base, get_db
|
||||
# from src.main import app
|
||||
|
||||
# # Test database URL
|
||||
# TEST_DATABASE_URL = "sqlite+aiosqlite:///:memory:"
|
||||
# Test database URL
|
||||
TEST_DATABASE_URL = "sqlite+aiosqlite:///:memory:"
|
||||
|
||||
# engine = create_async_engine(
|
||||
# TEST_DATABASE_URL,
|
||||
# connect_args={"check_same_thread": False},
|
||||
# poolclass=StaticPool,
|
||||
# )
|
||||
engine = create_async_engine(
|
||||
TEST_DATABASE_URL,
|
||||
connect_args={"check_same_thread": False},
|
||||
poolclass=StaticPool,
|
||||
)
|
||||
|
||||
# async_session = sessionmaker(
|
||||
# engine,
|
||||
# class_=AsyncSession,
|
||||
# expire_on_commit=False,
|
||||
# autocommit=False,
|
||||
# autoflush=False,
|
||||
# )
|
||||
async_session = sessionmaker(
|
||||
engine,
|
||||
class_=AsyncSession,
|
||||
expire_on_commit=False,
|
||||
autocommit=False,
|
||||
autoflush=False,
|
||||
)
|
||||
|
||||
|
||||
# async def override_get_db() -> AsyncGenerator[AsyncSession, None]:
|
||||
# async with async_session() as session:
|
||||
# try:
|
||||
# yield session
|
||||
# await session.commit()
|
||||
# except Exception:
|
||||
# await session.rollback()
|
||||
# raise
|
||||
# finally:
|
||||
# await session.close()
|
||||
async def override_get_db() -> AsyncGenerator[AsyncSession, None]:
|
||||
async with async_session() as session:
|
||||
try:
|
||||
yield session
|
||||
await session.commit()
|
||||
except Exception:
|
||||
await session.rollback()
|
||||
raise
|
||||
finally:
|
||||
await session.close()
|
||||
|
||||
|
||||
# app.dependency_overrides[get_db] = override_get_db
|
||||
app.dependency_overrides[get_db] = override_get_db
|
||||
|
||||
|
||||
# @pytest.fixture(scope="session")
|
||||
# def event_loop() -> Generator:
|
||||
# loop = asyncio.get_event_loop_policy().new_event_loop()
|
||||
# yield loop
|
||||
# loop.close()
|
||||
@pytest.fixture(scope="session")
|
||||
def event_loop() -> Generator:
|
||||
loop = asyncio.get_event_loop_policy().new_event_loop()
|
||||
yield loop
|
||||
loop.close()
|
||||
|
||||
|
||||
# @pytest.fixture(autouse=True)
|
||||
# async def setup_db() -> AsyncGenerator[None, None]:
|
||||
# async with engine.begin() as conn:
|
||||
# await conn.run_sync(Base.metadata.create_all)
|
||||
# yield
|
||||
# async with engine.begin() as conn:
|
||||
# await conn.run_sync(Base.metadata.drop_all)
|
||||
@pytest.fixture(autouse=True)
|
||||
async def setup_db() -> AsyncGenerator[None, None]:
|
||||
async with engine.begin() as conn:
|
||||
await conn.run_sync(Base.metadata.create_all)
|
||||
yield
|
||||
async with engine.begin() as conn:
|
||||
await conn.run_sync(Base.metadata.drop_all)
|
||||
|
||||
|
||||
# @pytest.fixture
|
||||
# async def client() -> AsyncGenerator[AsyncClient, None]:
|
||||
# async with AsyncClient(app=app, base_url="http://test") as client:
|
||||
# yield client
|
||||
@pytest.fixture
|
||||
async def client() -> AsyncGenerator[AsyncClient, None]:
|
||||
async with AsyncClient(app=app, base_url="http://test") as client:
|
||||
yield client
|
||||
|
||||
@ -0,0 +1,3 @@
|
||||
from sqlalchemy.orm import scoped_session, sessionmaker
|
||||
|
||||
Session = scoped_session(sessionmaker())
|
||||
@ -0,0 +1,28 @@
|
||||
import uuid
|
||||
from datetime import datetime
|
||||
|
||||
from factory import (LazyAttribute, LazyFunction, SelfAttribute, Sequence,
|
||||
SubFactory, post_generation)
|
||||
from factory.alchemy import SQLAlchemyModelFactory
|
||||
from factory.fuzzy import FuzzyChoice, FuzzyDateTime, FuzzyInteger, FuzzyText
|
||||
from faker import Faker
|
||||
from faker.providers import misc
|
||||
|
||||
from .database import Session
|
||||
|
||||
# from pytz import UTC
|
||||
|
||||
|
||||
fake = Faker()
|
||||
fake.add_provider(misc)
|
||||
|
||||
|
||||
class BaseFactory(SQLAlchemyModelFactory):
|
||||
"""Base Factory."""
|
||||
|
||||
class Meta:
|
||||
"""Factory configuration."""
|
||||
|
||||
abstract = True
|
||||
sqlalchemy_session = Session
|
||||
sqlalchemy_session_persistence = "commit"
|
||||
@ -1,44 +0,0 @@
|
||||
import pytest
|
||||
from src.calculation_budget_constrains.service import greedy_selection, knapsack_selection
|
||||
|
||||
def test_greedy_selection():
|
||||
equipments = [
|
||||
{"id": 1, "total_cost": 100, "priority_score": 10, "cost": 100},
|
||||
{"id": 2, "total_cost": 50, "priority_score": 20, "cost": 50},
|
||||
{"id": 3, "total_cost": 60, "priority_score": 15, "cost": 60},
|
||||
]
|
||||
budget = 120
|
||||
# Items sorted by priority_score: id 2 (20), id 3 (15), id 1 (10)
|
||||
# 2 (50) + 3 (60) = 110. Item 1 (100) won't fit.
|
||||
selected, excluded = greedy_selection(equipments, budget)
|
||||
|
||||
selected_ids = [e["id"] for e in selected]
|
||||
assert 2 in selected_ids
|
||||
assert 3 in selected_ids
|
||||
assert len(selected) == 2
|
||||
assert excluded[0]["id"] == 1
|
||||
|
||||
def test_knapsack_selection_basic():
|
||||
# Similar items but where greedy might fail if cost/value ratio is tricky
|
||||
# item 1: value 10, cost 60
|
||||
# item 2: value 7, cost 35
|
||||
# item 3: value 7, cost 35
|
||||
# budget: 70
|
||||
# Greedy would take item 1 (value 10, remaining budget 10, can't take more)
|
||||
# Optimal would take item 2 and 3 (value 14, remaining budget 0)
|
||||
|
||||
scale = 1 # No scaling for simplicity in this test
|
||||
equipments = [
|
||||
{"id": 1, "total_cost": 60, "priority_score": 10},
|
||||
{"id": 2, "total_cost": 35, "priority_score": 7},
|
||||
{"id": 3, "total_cost": 35, "priority_score": 7},
|
||||
]
|
||||
budget = 70
|
||||
|
||||
selected, excluded = knapsack_selection(equipments, budget, scale=1)
|
||||
|
||||
selected_ids = [e["id"] for e in selected]
|
||||
assert 2 in selected_ids
|
||||
assert 3 in selected_ids
|
||||
assert len(selected) == 2
|
||||
assert 1 not in selected_ids
|
||||
@ -1,14 +0,0 @@
|
||||
from src.context import set_request_id, get_request_id, set_user_id, get_user_id
|
||||
|
||||
def test_request_id_context():
|
||||
test_id = "test-request-id-123"
|
||||
set_request_id(test_id)
|
||||
assert get_request_id() == test_id
|
||||
|
||||
def test_user_id_context():
|
||||
test_uid = "user-456"
|
||||
set_user_id(test_uid)
|
||||
assert get_user_id() == test_uid
|
||||
|
||||
def test_context_default_none():
|
||||
assert get_request_id() is None or get_request_id() != ""
|
||||
@ -1,53 +0,0 @@
|
||||
import pytest
|
||||
from decimal import Decimal
|
||||
from src.contribution_util import prod, system_availability, get_all_components, birnbaum_importance
|
||||
|
||||
def test_prod():
|
||||
assert prod([1, 2, 3]) == 6.0
|
||||
assert prod([0.5, 0.5]) == 0.25
|
||||
assert prod([]) == 1.0
|
||||
|
||||
def test_system_availability_series():
|
||||
structure = {"series": ["A", "B"]}
|
||||
availabilities = {"A": 0.9, "B": 0.8}
|
||||
# 0.9 * 0.8 = 0.72
|
||||
assert system_availability(structure, availabilities) == pytest.approx(0.72)
|
||||
|
||||
def test_system_availability_parallel():
|
||||
structure = {"parallel": ["A", "B"]}
|
||||
availabilities = {"A": 0.9, "B": 0.8}
|
||||
# 1 - (1-0.9)*(1-0.8) = 1 - 0.1*0.2 = 1 - 0.02 = 0.98
|
||||
assert system_availability(structure, availabilities) == pytest.approx(0.98)
|
||||
|
||||
def test_system_availability_nested():
|
||||
# (A in series with (B in parallel with C))
|
||||
structure = {
|
||||
"series": [
|
||||
"A",
|
||||
{"parallel": ["B", "C"]}
|
||||
]
|
||||
}
|
||||
availabilities = {"A": 0.9, "B": 0.8, "C": 0.7}
|
||||
# B||C = 1 - (1-0.8)*(1-0.7) = 1 - 0.2*0.3 = 0.94
|
||||
# A && (B||C) = 0.9 * 0.94 = 0.846
|
||||
assert system_availability(structure, availabilities) == pytest.approx(0.846)
|
||||
|
||||
def test_get_all_components():
|
||||
structure = {
|
||||
"series": [
|
||||
"A",
|
||||
{"parallel": ["B", "C"]}
|
||||
]
|
||||
}
|
||||
assert get_all_components(structure) == {"A", "B", "C"}
|
||||
|
||||
def test_birnbaum_importance():
|
||||
structure = {"series": ["A", "B"]}
|
||||
availabilities = {"A": 0.9, "B": 0.8}
|
||||
# I_B(A) = A_sys(A=1) - A_sys(A=0)
|
||||
# A_sys(A=1, B=0.8) = 1 * 0.8 = 0.8
|
||||
# A_sys(A=0, B=0.8) = 0 * 0.8 = 0
|
||||
# I_B(A) = 0.8
|
||||
assert birnbaum_importance(structure, availabilities, "A") == pytest.approx(0.8)
|
||||
# I_B(B) = A_sys(B=1, A=0.9) - A_sys(B=0, A=0.9) = 0.9 - 0 = 0.9
|
||||
assert birnbaum_importance(structure, availabilities, "B") == pytest.approx(0.9)
|
||||
@ -1,31 +0,0 @@
|
||||
import pytest
|
||||
from sqlalchemy.exc import IntegrityError, DataError, DBAPIError
|
||||
from src.exceptions import handle_sqlalchemy_error
|
||||
|
||||
def test_handle_sqlalchemy_error_unique_constraint():
|
||||
err = IntegrityError("Unique constraint", params=None, orig=Exception("unique constraint violation"))
|
||||
msg, status = handle_sqlalchemy_error(err)
|
||||
assert status == 409
|
||||
assert "already exists" in msg
|
||||
|
||||
def test_handle_sqlalchemy_error_foreign_key():
|
||||
err = IntegrityError("Foreign key constraint", params=None, orig=Exception("foreign key constraint violation"))
|
||||
msg, status = handle_sqlalchemy_error(err)
|
||||
assert status == 400
|
||||
assert "Related record not found" in msg
|
||||
|
||||
def test_handle_sqlalchemy_error_data_error():
|
||||
err = DataError("Invalid data", params=None, orig=None)
|
||||
msg, status = handle_sqlalchemy_error(err)
|
||||
assert status == 400
|
||||
assert "Invalid data" in msg
|
||||
|
||||
def test_handle_sqlalchemy_error_generic_dbapi():
|
||||
class MockError:
|
||||
def __str__(self):
|
||||
return "Some generic database error"
|
||||
|
||||
err = DBAPIError("Generic error", params=None, orig=MockError())
|
||||
msg, status = handle_sqlalchemy_error(err)
|
||||
assert status == 500
|
||||
assert "Database error" in msg
|
||||
@ -1,58 +0,0 @@
|
||||
import pytest
|
||||
from unittest.mock import AsyncMock, MagicMock
|
||||
from fastapi import HTTPException
|
||||
from src.middleware import RequestValidationMiddleware
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_request_validation_middleware_query_length():
|
||||
middleware = RequestValidationMiddleware(app=MagicMock())
|
||||
request = MagicMock()
|
||||
request.url.query = "a" * 2001
|
||||
|
||||
with pytest.raises(HTTPException) as excinfo:
|
||||
await middleware.dispatch(request, AsyncMock())
|
||||
assert excinfo.value.status_code == 422
|
||||
assert "Invalid request parameters" in excinfo.value.detail
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_request_validation_middleware_too_many_params():
|
||||
middleware = RequestValidationMiddleware(app=MagicMock())
|
||||
request = MagicMock()
|
||||
request.url.query = "a=1"
|
||||
request.query_params.multi_items.return_value = [("param", "val")] * 51
|
||||
|
||||
with pytest.raises(HTTPException) as excinfo:
|
||||
await middleware.dispatch(request, AsyncMock())
|
||||
assert excinfo.value.status_code == 422
|
||||
assert "Invalid request parameters" in excinfo.value.detail
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_request_validation_middleware_xss_detection():
|
||||
middleware = RequestValidationMiddleware(app=MagicMock())
|
||||
request = MagicMock()
|
||||
request.url.query = "q=<script>"
|
||||
request.query_params.multi_items.return_value = [("q", "<script>")]
|
||||
|
||||
with pytest.raises(HTTPException) as excinfo:
|
||||
await middleware.dispatch(request, AsyncMock())
|
||||
assert excinfo.value.status_code == 422
|
||||
assert "Invalid request parameters" in excinfo.value.detail
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_request_validation_middleware_pagination_logic():
|
||||
middleware = RequestValidationMiddleware(app=MagicMock())
|
||||
request = MagicMock()
|
||||
request.url.query = "size=105"
|
||||
request.query_params.multi_items.return_value = [("size", "105")]
|
||||
request.headers = {}
|
||||
|
||||
with pytest.raises(HTTPException) as excinfo:
|
||||
await middleware.dispatch(request, AsyncMock())
|
||||
assert excinfo.value.status_code == 422
|
||||
assert "Invalid request parameters" in excinfo.value.detail
|
||||
|
||||
request.query_params.multi_items.return_value = [("size", "7")]
|
||||
with pytest.raises(HTTPException) as excinfo:
|
||||
await middleware.dispatch(request, AsyncMock())
|
||||
assert excinfo.value.status_code == 422
|
||||
assert "Invalid request parameters" in excinfo.value.detail
|
||||
@ -1,64 +0,0 @@
|
||||
import pytest
|
||||
import math
|
||||
from src.calculation_target_reliability.service import calculate_asset_eaf_contributions
|
||||
|
||||
def test_calculate_asset_eaf_contributions_basic():
|
||||
# Mock plant result
|
||||
plant_result = {
|
||||
"total_uptime": 7000,
|
||||
"total_downtime": 1000,
|
||||
"eaf": 85.0
|
||||
}
|
||||
# total_hours = 8000
|
||||
|
||||
# Mock equipment results
|
||||
eq_results = [
|
||||
{
|
||||
"aeros_node": {"node_name": "Asset1"},
|
||||
"num_events": 5,
|
||||
"contribution_factor": 0.5,
|
||||
"contribution": 0.1, # Birnbaum
|
||||
"availability": 0.9,
|
||||
"total_downtime": 100
|
||||
},
|
||||
{
|
||||
"aeros_node": {"node_name": "Asset2"},
|
||||
"num_events": 2,
|
||||
"contribution_factor": 0.3,
|
||||
"contribution": 0.05,
|
||||
"availability": 0.95,
|
||||
"total_downtime": 50
|
||||
}
|
||||
]
|
||||
|
||||
standard_scope = ["Asset1", "Asset2"]
|
||||
eaf_gap = 2.0 # 2% gap
|
||||
scheduled_outage = 500
|
||||
|
||||
results = calculate_asset_eaf_contributions(
|
||||
plant_result, eq_results, standard_scope, eaf_gap, scheduled_outage
|
||||
)
|
||||
|
||||
assert len(results) == 2
|
||||
# Check sorting (highest birnbaum first)
|
||||
assert results[0].node["node_name"] == "Asset1"
|
||||
assert results[0].birbaum > results[1].birbaum
|
||||
|
||||
# Check that required_improvement is positive
|
||||
assert results[0].required_improvement > 0
|
||||
assert results[0].improvement_impact > 0
|
||||
|
||||
def test_calculate_asset_eaf_contributions_skipping():
|
||||
plant_result = {"total_uptime": 1000, "total_downtime": 0, "eaf": 100}
|
||||
eq_results = [{
|
||||
"aeros_node": {"node_name": "Asset1"},
|
||||
"num_events": 0,
|
||||
"contribution_factor": 0.5,
|
||||
"contribution": 0.1,
|
||||
"availability": 1.0,
|
||||
"total_downtime": 0
|
||||
}]
|
||||
results = calculate_asset_eaf_contributions(
|
||||
plant_result, eq_results, ["Asset1"], 1.0, 0
|
||||
)
|
||||
assert len(results) == 0
|
||||
@ -1,49 +0,0 @@
|
||||
import pytest
|
||||
from pydantic import ValidationError
|
||||
from src.database.schema import CommonParams
|
||||
from src.overhaul.schema import OverhaulCriticalParts
|
||||
|
||||
def test_common_params_valid():
|
||||
params = CommonParams(
|
||||
page=1,
|
||||
itemsPerPage=10,
|
||||
q="search test",
|
||||
all=1
|
||||
)
|
||||
assert params.page == 1
|
||||
assert params.items_per_page == 10
|
||||
assert params.query_str == "search test"
|
||||
assert params.is_all is True
|
||||
|
||||
def test_common_params_page_constraints():
|
||||
# Test page must be > 0
|
||||
with pytest.raises(ValidationError):
|
||||
CommonParams(page=0)
|
||||
|
||||
with pytest.raises(ValidationError):
|
||||
CommonParams(page=-1)
|
||||
|
||||
def test_common_params_items_per_page_constraints():
|
||||
# Test items_per_page must be multiple of 5
|
||||
with pytest.raises(ValidationError):
|
||||
CommonParams(itemsPerPage=7)
|
||||
|
||||
# Test items_per_page maximum
|
||||
with pytest.raises(ValidationError):
|
||||
CommonParams(itemsPerPage=55)
|
||||
|
||||
# Valid multiples of 5
|
||||
assert CommonParams(itemsPerPage=50).items_per_page == 50
|
||||
assert CommonParams(itemsPerPage=5).items_per_page == 5
|
||||
|
||||
def test_overhaul_critical_parts_valid():
|
||||
parts = OverhaulCriticalParts(criticalParts=["Part A", "Part B"])
|
||||
assert parts.criticalParts == ["Part A", "Part B"]
|
||||
|
||||
def test_overhaul_critical_parts_invalid():
|
||||
# criticalParts is required and must be a list
|
||||
with pytest.raises(ValidationError):
|
||||
OverhaulCriticalParts()
|
||||
|
||||
with pytest.raises(ValidationError):
|
||||
OverhaulCriticalParts(criticalParts="Not a list")
|
||||
@ -1,55 +0,0 @@
|
||||
import pytest
|
||||
from fastapi import HTTPException
|
||||
from src.middleware import (
|
||||
inspect_value,
|
||||
inspect_json,
|
||||
has_control_chars,
|
||||
XSS_PATTERN,
|
||||
SQLI_PATTERN
|
||||
)
|
||||
|
||||
def test_xss_patterns():
|
||||
# Test common XSS payloads in be-optimumoh
|
||||
payloads = [
|
||||
"<script>alert(1)</script>",
|
||||
"javascript:alert(1)",
|
||||
"onerror=alert(1)",
|
||||
"onload=alert(1)",
|
||||
"<svg",
|
||||
"<img"
|
||||
]
|
||||
for payload in payloads:
|
||||
assert XSS_PATTERN.search(payload) is not None
|
||||
|
||||
def test_sqli_patterns():
|
||||
# Test common SQLi payloads in be-optimumoh
|
||||
payloads = [
|
||||
"UNION SELECT * FROM users",
|
||||
"OR 1=1",
|
||||
"WAITFOR DELAY '0:0:5'",
|
||||
"INFORMATION_SCHEMA.TABLES",
|
||||
]
|
||||
for payload in payloads:
|
||||
assert SQLI_PATTERN.search(payload) is not None
|
||||
|
||||
def test_inspect_value_raises():
|
||||
with pytest.raises(HTTPException) as excinfo:
|
||||
inspect_value("<script>", "source")
|
||||
assert excinfo.value.status_code == 422
|
||||
assert "Invalid request parameters" in excinfo.value.detail
|
||||
|
||||
with pytest.raises(HTTPException) as excinfo:
|
||||
inspect_value("UNION SELECT * FROM users", "source")
|
||||
assert excinfo.value.status_code == 422
|
||||
assert "Invalid request parameters" in excinfo.value.detail
|
||||
|
||||
def test_inspect_json_raises():
|
||||
with pytest.raises(HTTPException) as excinfo:
|
||||
inspect_json({"__proto__": "polluted"})
|
||||
assert excinfo.value.status_code == 422
|
||||
assert "Invalid request parameters" in excinfo.value.detail
|
||||
|
||||
def test_has_control_chars():
|
||||
assert has_control_chars("normal string") is False
|
||||
assert has_control_chars("string with \x00 null") is True
|
||||
assert has_control_chars("string with \n newline") is False
|
||||
@ -1,33 +0,0 @@
|
||||
import pytest
|
||||
from datetime import datetime, timedelta
|
||||
from src.calculation_target_reliability.utils import generate_down_periods
|
||||
|
||||
def test_generate_down_periods_count():
|
||||
start = datetime(2025, 1, 1)
|
||||
end = datetime(2025, 1, 31)
|
||||
# Test fixed number of periods
|
||||
periods = generate_down_periods(start, end, num_periods=5)
|
||||
# It attempts to generate 5, but might be fewer due to overlaps
|
||||
assert len(periods) <= 5
|
||||
|
||||
# Check they are within range
|
||||
for p_start, p_end in periods:
|
||||
assert p_start >= start
|
||||
assert p_end <= end
|
||||
assert p_start < p_end
|
||||
|
||||
def test_generate_down_periods_no_overlap():
|
||||
start = datetime(2025, 1, 1)
|
||||
end = datetime(2025, 1, 31)
|
||||
periods = generate_down_periods(start, end, num_periods=10)
|
||||
|
||||
# Sort and check gaps
|
||||
for i in range(len(periods) - 1):
|
||||
assert periods[i][1] <= periods[i+1][0]
|
||||
|
||||
def test_generate_down_periods_too_small_range():
|
||||
start = datetime(2025, 1, 1)
|
||||
end = datetime(2025, 1, 2)
|
||||
# Requesting 5 days duration in 1 day range
|
||||
periods = generate_down_periods(start, end, num_periods=1, min_duration=5)
|
||||
assert len(periods) == 0
|
||||
@ -1,36 +0,0 @@
|
||||
import pytest
|
||||
from datetime import datetime, timedelta
|
||||
from src.utils import parse_relative_expression, parse_date_string
|
||||
|
||||
def test_parse_relative_expression_days():
|
||||
# Test T, T+n, T-n
|
||||
result = parse_relative_expression("T")
|
||||
assert result is not None
|
||||
assert isinstance(result, datetime)
|
||||
|
||||
result_plus = parse_relative_expression("T+5")
|
||||
assert result_plus is not None
|
||||
|
||||
result_minus = parse_relative_expression("T-3")
|
||||
assert result_minus is not None
|
||||
|
||||
def test_parse_relative_expression_invalid():
|
||||
assert parse_relative_expression("abc") is None
|
||||
assert parse_relative_expression("123") is None
|
||||
assert parse_relative_expression("T++1") is None
|
||||
|
||||
def test_parse_date_string_formats():
|
||||
# Test various ISO and common formats
|
||||
dt = parse_date_string("2024-11-08")
|
||||
assert dt.year == 2024
|
||||
assert dt.month == 11
|
||||
assert dt.day == 8
|
||||
|
||||
dt = parse_date_string("08-11-2024")
|
||||
assert dt.year == 2024
|
||||
assert dt.month == 11
|
||||
assert dt.day == 8
|
||||
|
||||
def test_parse_date_string_invalid():
|
||||
with pytest.raises(ValueError):
|
||||
parse_date_string("invalid-date")
|
||||
Loading…
Reference in New Issue