Spaces:
Sleeping
Sleeping
Upload 26 files
Browse files- COC_END_TO_END_WORKFLOW.md +43 -0
- DETERMINISTIC_EXECUTION_GATE.md +12 -9
- REV28_CHANGELOG.md +31 -0
- agent.py +1 -0
- report_engine.py +292 -5
- requirements.txt +1 -0
- screening_logic.py +35 -12
- server.py +1 -1
- test_regression.py +20 -2
- tools.py +167 -12
COC_END_TO_END_WORKFLOW.md
ADDED
|
@@ -0,0 +1,43 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# Generic City of Calgary SWMR workflow
|
| 2 |
+
|
| 3 |
+
Kincora Phase 2 is a regression fixture only. Production functions do not
|
| 4 |
+
contain Kincora object IDs, values, geometry, or report conclusions.
|
| 5 |
+
|
| 6 |
+
## MCP sequence
|
| 7 |
+
|
| 8 |
+
1. `upload_model` — immutable EPA SWMM INP upload and inventory.
|
| 9 |
+
2. `set_report_details` — project, consultant, professional, site, objective,
|
| 10 |
+
methodology and administrative information.
|
| 11 |
+
3. `set_report_configuration` — project-specific criteria, major routes,
|
| 12 |
+
special flow limits, facility classifications, drawing inventory,
|
| 13 |
+
applicable reports and checklist overrides.
|
| 14 |
+
4. `run_simulation` — legacy-default normalization where required, isolated
|
| 15 |
+
execution, execution-integrity gate, database creation and RPT reconciliation.
|
| 16 |
+
5. `get_table_catalog` / `query_results` — complete tokenized input and result
|
| 17 |
+
review rather than top-N-only evidence.
|
| 18 |
+
6. `preliminary_design_review`, `get_reconciliation`, `calgary_screening` —
|
| 19 |
+
deterministic QA/QC and preliminary City screening.
|
| 20 |
+
7. `get_timeseries` and controlled `run_scenario` calls where authorized.
|
| 21 |
+
8. `generate_report` — editable SWMR DOCX plus immutable audit ZIP, model
|
| 22 |
+
listings, model schematic, depth-velocity figure and readiness registers.
|
| 23 |
+
|
| 24 |
+
## Execution and readiness gates
|
| 25 |
+
|
| 26 |
+
- An invalid hydraulic run produces an Input/Configuration Review and no
|
| 27 |
+
hydraulic screening conclusions.
|
| 28 |
+
- Missing drawings, approved criteria, boundary conditions, current-source
|
| 29 |
+
verification, or professional fields remain explicit readiness gaps.
|
| 30 |
+
- Generated figures do not replace drawing-to-model reconciliation.
|
| 31 |
+
- The output is never represented as approved, authenticated, or submission
|
| 32 |
+
ready solely because the automated workflow completed.
|
| 33 |
+
|
| 34 |
+
## Project-neutral design
|
| 35 |
+
|
| 36 |
+
- Object selection is derived from each uploaded INP or explicit project
|
| 37 |
+
configuration.
|
| 38 |
+
- Unit labels and effective solver defaults follow the model unit system.
|
| 39 |
+
- Coordinates and vertices drive schematic geometry; incomplete coordinate
|
| 40 |
+
sets receive a deterministic topology fallback.
|
| 41 |
+
- Criteria and facility classifications are session-scoped and archived with
|
| 42 |
+
the report package.
|
| 43 |
+
- Kincora-specific numerical pins remain only in regression tests.
|
DETERMINISTIC_EXECUTION_GATE.md
CHANGED
|
@@ -5,9 +5,11 @@ successful SWMR results.
|
|
| 5 |
|
| 6 |
## Changes
|
| 7 |
|
| 8 |
-
1. Dynamic-wave preflight
|
| 9 |
-
`HEAD_TOLERANCE
|
| 10 |
-
|
|
|
|
|
|
|
| 11 |
2. Post-run integrity classification records `valid`, `limited`, or `invalid`
|
| 12 |
using routing convergence and flow-routing continuity evidence.
|
| 13 |
3. A run is invalid when every routing step fails, at least 5% of routing
|
|
@@ -22,21 +24,22 @@ successful SWMR results.
|
|
| 22 |
|
| 23 |
## Kincora failure reproduced
|
| 24 |
|
| 25 |
-
The
|
| 26 |
|
| 27 |
```ini
|
| 28 |
MAX_TRIALS 0
|
| 29 |
HEAD_TOLERANCE 0
|
| 30 |
```
|
| 31 |
|
| 32 |
-
|
| 33 |
-
|
| 34 |
-
|
| 35 |
-
|
|
|
|
| 36 |
|
| 37 |
## Validation
|
| 38 |
|
| 39 |
- Python compilation passed for all packaged modules.
|
| 40 |
-
-
|
| 41 |
- Preflight integration requires the Space dependencies and isolated
|
| 42 |
OpenSWMM worker environment described in the existing README.
|
|
|
|
| 5 |
|
| 6 |
## Changes
|
| 7 |
|
| 8 |
+
1. Dynamic-wave preflight recognizes explicit zero values for `MAX_TRIALS`,
|
| 9 |
+
`HEAD_TOLERANCE`, and `MIN_SURFAREA` as legacy/default sentinels. It creates
|
| 10 |
+
an immutable derivative execution copy using unit-aware effective defaults,
|
| 11 |
+
while preserving and hashing the original upload. Negative and non-numeric
|
| 12 |
+
values remain blocking errors. Omitted values remain untouched.
|
| 13 |
2. Post-run integrity classification records `valid`, `limited`, or `invalid`
|
| 14 |
using routing convergence and flow-routing continuity evidence.
|
| 15 |
3. A run is invalid when every routing step fails, at least 5% of routing
|
|
|
|
| 24 |
|
| 25 |
## Kincora failure reproduced
|
| 26 |
|
| 27 |
+
The legacy input explicitly contained:
|
| 28 |
|
| 29 |
```ini
|
| 30 |
MAX_TRIALS 0
|
| 31 |
HEAD_TOLERANCE 0
|
| 32 |
```
|
| 33 |
|
| 34 |
+
The EPA SWMM 5.2 desktop interface resolves these values to effective defaults.
|
| 35 |
+
For SI models the derivative uses `MAX_TRIALS=8`, `HEAD_TOLERANCE=0.0015 m`,
|
| 36 |
+
and `MIN_SURFAREA=1.167 m2`. Every substitution, both hashes, and both INP files
|
| 37 |
+
are retained in the audit package. This is a disclosed compatibility action,
|
| 38 |
+
not a mutation of the uploaded engineering model.
|
| 39 |
|
| 40 |
## Validation
|
| 41 |
|
| 42 |
- Python compilation passed for all packaged modules.
|
| 43 |
+
- 35 deterministic unit checks passed.
|
| 44 |
- Preflight integration requires the Space dependencies and isolated
|
| 45 |
OpenSWMM worker environment described in the existing README.
|
REV28_CHANGELOG.md
ADDED
|
@@ -0,0 +1,31 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# Rev 28 change log
|
| 2 |
+
|
| 3 |
+
## Fixed
|
| 4 |
+
|
| 5 |
+
- Corrected the deterministic preflight gate for legacy EPA SWMM dynamic-wave
|
| 6 |
+
options explicitly stored as zero.
|
| 7 |
+
- `MAX_TRIALS=0`, `HEAD_TOLERANCE=0`, and `MIN_SURFAREA=0` now resolve on an
|
| 8 |
+
immutable derivative execution copy using unit-aware defaults.
|
| 9 |
+
- Preserved the original uploaded model and added original/execution filenames,
|
| 10 |
+
SHA-256 hashes, and substitution records to run metadata and report identity.
|
| 11 |
+
- Included both original and derivative INP files in the SWMR audit ZIP.
|
| 12 |
+
- Retained blocking behavior for negative and non-numeric solver values.
|
| 13 |
+
- Added a deterministic Alberta/Calgary depth-velocity criteria figure directly
|
| 14 |
+
below SWMR Table 9, including model depth/velocity points, peak-flow colour,
|
| 15 |
+
the tabulated criterion, and audited PNG/CSV source artifacts.
|
| 16 |
+
- Added deterministic SWMM model-schematic automation using INP coordinates,
|
| 17 |
+
vertices, subcatchment routing and hydraulic topology, with a stable fallback
|
| 18 |
+
layout plus archived PNG and generation manifest.
|
| 19 |
+
- Exposed the report engine's project-specific City of Calgary configuration
|
| 20 |
+
through `set_report_configuration`, removing the workflow gap that prevented
|
| 21 |
+
non-sample projects from supplying major routes, criteria, classifications,
|
| 22 |
+
drawing inventories, applicable reports and checklist overrides.
|
| 23 |
+
|
| 24 |
+
## Validation
|
| 25 |
+
|
| 26 |
+
- Python compilation passed.
|
| 27 |
+
- All 35 deterministic unit checks passed.
|
| 28 |
+
- Exact Kincora legacy-zero normalization was verified without modifying the
|
| 29 |
+
uploaded INP.
|
| 30 |
+
- Full worker integration must be rerun in the Hugging Face Space image, which
|
| 31 |
+
supplies the isolated OpenSWMM worker dependencies.
|
agent.py
CHANGED
|
@@ -66,6 +66,7 @@ AGENT_TOOLS_DEFAULT = [
|
|
| 66 |
"get_link_results", "get_subcatchment_results", "get_timeseries",
|
| 67 |
"query_results", "get_table_catalog", "calgary_screening",
|
| 68 |
"preliminary_design_review", "get_reconciliation", "run_scenario",
|
|
|
|
| 69 |
]
|
| 70 |
|
| 71 |
_JSON_TYPES = {str: "string", int: "integer", float: "number", bool: "boolean",
|
|
|
|
| 66 |
"get_link_results", "get_subcatchment_results", "get_timeseries",
|
| 67 |
"query_results", "get_table_catalog", "calgary_screening",
|
| 68 |
"preliminary_design_review", "get_reconciliation", "run_scenario",
|
| 69 |
+
"set_report_details", "set_report_configuration",
|
| 70 |
]
|
| 71 |
|
| 72 |
_JSON_TYPES = {str: "string", int: "integer", float: "number", bool: "boolean",
|
report_engine.py
CHANGED
|
@@ -32,6 +32,246 @@ from calgary_rules import (
|
|
| 32 |
)
|
| 33 |
|
| 34 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 35 |
@dataclass
|
| 36 |
class ReportCriteria:
|
| 37 |
node_depth_ratio: float = 0.80
|
|
@@ -779,15 +1019,17 @@ def _build_swmr_checklist(metadata: ReportMetadata, criteria: ReportCriteria, *,
|
|
| 779 |
add("SWMR-08","Minor-system routed flows, cumulative design flows, pipe capacities and spare capacity", "Not applicable", "No conduits identified in uploaded model", "Confirm whether a separate minor-system model is within the report scope", "Minor System")
|
| 780 |
add("SWMR-09","HGL, surcharge, rim clearance, downstream HWL and backwater assessment", "Requires professional confirmation", "Storage/outfall HGL table; no conduit network identified", "Confirm whether a separate minor-system model and downstream HWL assessment are required", "Minor System")
|
| 781 |
add("SWMR-10","Catchbasin, inlet, ICD and outlet rating information", "Complete" if has_controls else "Not applicable / missing", "Hydraulic controls table" if has_controls else "—", "Confirm inlet types, rating curves and drawing locations", "Minor System")
|
| 782 |
-
add("SWMR-11","Critical overland flows, depths, velocities, spill routes and escape routes", "Partially complete" if has_overland else "Requires drawing review", "
|
| 783 |
add("SWMR-12","Trap-low and storage volume, depth, spill, entrance grade and restrictive covenant information", "Partially complete" if has_storage else "Not applicable", "Storage assessment" if has_storage else "—", "Add spill elevations, building grades and covenant requirements", "Storage")
|
| 784 |
add("SWMR-13","Minor and major boundary inflows/outflows and supporting source", "Partially complete" if has_outfalls else "Missing", "Boundary outflow tables" if has_outfalls else "—", "Confirm downstream capacity and external inflows", "Boundary Conditions")
|
| 785 |
add("SWMR-14","Private-site permissible discharge and on-site storage requirements", "Missing", "—", "Provide applicable release rates and private-site storage criteria", "Private Sites")
|
| 786 |
add("SWMR-15","Water-quality treatment, BMPs, downstream treatment and source controls", "Missing", "—", "Document water-quality strategy and downstream treatment", "Water Quality")
|
| 787 |
required_drawings={"site location","study area","catchment plan","model schematic","overland drainage","storm drainage"}
|
| 788 |
found=len(required_drawings & drawings)
|
| 789 |
-
|
| 790 |
-
|
|
|
|
|
|
|
| 791 |
return pd.DataFrame(rows)
|
| 792 |
|
| 793 |
def _readiness_scores(checklist: pd.DataFrame) -> pd.DataFrame:
|
|
@@ -1202,6 +1444,7 @@ def generate_report_package(
|
|
| 1202 |
checklist_table.loc[_m17, "Status"] = "Included (Appendix D listings + digital package)"
|
| 1203 |
checklist_table.loc[_m17, "Outstanding Action"] = "Verify appendix listings match the issued digital model package"
|
| 1204 |
readiness_table = _readiness_scores(checklist_table)
|
|
|
|
| 1205 |
|
| 1206 |
doc = Document()
|
| 1207 |
sec = doc.sections[0]
|
|
@@ -1258,14 +1501,21 @@ def generate_report_package(
|
|
| 1258 |
doc.add_paragraph("Draft status: suitable for consultant/client discussion and structured engineering review. Human-in-the-loop completion, technical verification, drawing coordination, and professional authentication are required before municipal submission.")
|
| 1259 |
if model_identity:
|
| 1260 |
doc.add_heading("1.0a Model Identity and Execution Provenance", level=2)
|
| 1261 |
-
|
| 1262 |
{"Item": "Model file", "Value": str(model_identity.get("inp_name", "—"))},
|
| 1263 |
{"Item": "Model SHA-256", "Value": str(model_identity.get("sha256", "—"))},
|
| 1264 |
{"Item": "Session / run ID", "Value": f"{model_identity.get('session_id', '—')} / {model_identity.get('run_id', '—')}"},
|
| 1265 |
{"Item": "Engine", "Value": str(model_identity.get("engine", "EPA SWMM / OpenSWMM (crash-isolated worker)"))},
|
| 1266 |
{"Item": "Execution status", "Value": str(model_identity.get("status", "Completed"))},
|
| 1267 |
{"Item": "Generated", "Value": str(model_identity.get("generated", metadata.report_date))},
|
| 1268 |
-
]
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1269 |
_add_df_table(doc, "Table 0A - Model Identity", _id_tbl, font_size=8.0)
|
| 1270 |
_rev_tbl = pd.DataFrame([{"Rev": "P0", "Date": metadata.report_date,
|
| 1271 |
"Description": "Preliminary server-generated draft for engineering review",
|
|
@@ -1326,6 +1576,17 @@ def generate_report_package(
|
|
| 1326 |
)
|
| 1327 |
if options.get("FLOW_ROUTING", "").upper() == "KINWAVE":
|
| 1328 |
doc.add_paragraph("Modelling limitation: kinematic-wave routing does not fully represent backwater, pressurization, reverse flow, or complex surcharge interactions. Dynamic-wave routing should be considered where these effects are material.")
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1329 |
doc.add_heading("3.3 Major-Minor System", level=2)
|
| 1330 |
doc.add_paragraph("Open channels, swales, gutters, culverts, and closed conduits represented in the model were reviewed. Final major/minor classification must be confirmed against the approved drainage concept and drawings.")
|
| 1331 |
if scenario_comparison is not None and not scenario_comparison.empty:
|
|
@@ -1393,6 +1654,25 @@ def generate_report_package(
|
|
| 1393 |
overland_compliance[col] = "Not assessed - hydraulic routing solution invalid"
|
| 1394 |
if "Spill Active" in overland_compliance:
|
| 1395 |
overland_compliance["Spill Active"] = "Not assessed"
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1396 |
if not overland_compliance.empty:
|
| 1397 |
_add_df_table(doc, "Table 9A - Calgary Major-System Depth-Velocity Screening", overland_compliance, landscape=True, font_size=7.2)
|
| 1398 |
|
|
@@ -1613,6 +1893,10 @@ def generate_report_package(
|
|
| 1613 |
zf.writestr("tables/calgary_criteria_register.csv", criteria_table.to_csv(index=False))
|
| 1614 |
zf.writestr("tables/calgary_minor_system_capacity.csv", minor_capacity.to_csv(index=False))
|
| 1615 |
zf.writestr("tables/calgary_major_system_compliance.csv", overland_compliance.to_csv(index=False))
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1616 |
zf.writestr("tables/calgary_storage_assessment.csv", storage_calgary.to_csv(index=False))
|
| 1617 |
zf.writestr("tables/calgary_swmr_completeness_register.csv", checklist_table.to_csv(index=False))
|
| 1618 |
zf.writestr("tables/swmr_draft_readiness.csv", readiness_table.to_csv(index=False))
|
|
@@ -1637,6 +1921,9 @@ def generate_report_package(
|
|
| 1637 |
if model_listings.get("inp_text"):
|
| 1638 |
zf.writestr(f"model/{_safe_name(model_listings.get('inp_name', 'model.inp'))}",
|
| 1639 |
model_listings["inp_text"])
|
|
|
|
|
|
|
|
|
|
| 1640 |
if model_listings.get("rpt_text"):
|
| 1641 |
zf.writestr(f"model/{_safe_name(model_listings.get('rpt_name', 'model.rpt'))}",
|
| 1642 |
model_listings["rpt_text"])
|
|
|
|
| 32 |
)
|
| 33 |
|
| 34 |
|
| 35 |
+
def _depth_velocity_figure(
|
| 36 |
+
compliance: pd.DataFrame,
|
| 37 |
+
curve: tuple[tuple[float, float], ...],
|
| 38 |
+
*,
|
| 39 |
+
results_usable: bool,
|
| 40 |
+
flow_unit: str,
|
| 41 |
+
) -> bytes:
|
| 42 |
+
"""Render the auditable Alberta/Calgary depth-velocity screening figure."""
|
| 43 |
+
from matplotlib.backends.backend_agg import FigureCanvasAgg
|
| 44 |
+
from matplotlib.figure import Figure
|
| 45 |
+
|
| 46 |
+
points = sorted((float(v), float(d)) for v, d in curve)
|
| 47 |
+
fig = Figure(figsize=(7.2, 5.3), dpi=180, facecolor="white")
|
| 48 |
+
FigureCanvasAgg(fig)
|
| 49 |
+
ax = fig.add_subplot(111)
|
| 50 |
+
|
| 51 |
+
# Extend the first permissible depth horizontally to zero velocity. Do
|
| 52 |
+
# not extrapolate beyond the final criterion point (3.0 m/s).
|
| 53 |
+
boundary_x = [0.0] + [p[0] for p in points]
|
| 54 |
+
boundary_y = [points[0][1]] + [p[1] for p in points]
|
| 55 |
+
ax.axhspan(0.0, max(0.9, max(d for _, d in points) * 1.08),
|
| 56 |
+
color="#f4cccc", alpha=0.42)
|
| 57 |
+
ax.fill_between(boundary_x, 0.0, boundary_y, color="#d9ead3", alpha=0.72,
|
| 58 |
+
label="Acceptable screening region")
|
| 59 |
+
ax.plot(boundary_x, boundary_y, color="#0057b8", linewidth=2.0,
|
| 60 |
+
marker="s", markersize=4.5, markerfacecolor="white",
|
| 61 |
+
label="Alberta/Calgary criterion")
|
| 62 |
+
|
| 63 |
+
plotted = 0
|
| 64 |
+
if results_usable and compliance is not None and not compliance.empty:
|
| 65 |
+
dcol = next((c for c in compliance if c.startswith("Peak Depth (")), None)
|
| 66 |
+
vcol = next((c for c in compliance if c.startswith("Peak Velocity (")), None)
|
| 67 |
+
qcol = next((c for c in compliance if c.startswith("Peak Flow (")), None)
|
| 68 |
+
if dcol and vcol:
|
| 69 |
+
data = compliance.copy()
|
| 70 |
+
data["_d"] = pd.to_numeric(data[dcol], errors="coerce")
|
| 71 |
+
data["_v"] = pd.to_numeric(data[vcol], errors="coerce")
|
| 72 |
+
data["_q"] = pd.to_numeric(data[qcol], errors="coerce") if qcol else float("nan")
|
| 73 |
+
data = data.dropna(subset=["_d", "_v"])
|
| 74 |
+
plotted = len(data)
|
| 75 |
+
if plotted:
|
| 76 |
+
if qcol and data["_q"].notna().any():
|
| 77 |
+
scatter = ax.scatter(data["_v"], data["_d"], c=data["_q"],
|
| 78 |
+
cmap="viridis", marker="x", s=48,
|
| 79 |
+
linewidths=1.4, zorder=4,
|
| 80 |
+
label="Model results")
|
| 81 |
+
cbar = fig.colorbar(scatter, ax=ax, pad=0.02)
|
| 82 |
+
cbar.set_label(f"Peak flow ({flow_unit})", fontsize=8)
|
| 83 |
+
else:
|
| 84 |
+
ax.scatter(data["_v"], data["_d"], color="black", marker="x",
|
| 85 |
+
s=48, linewidths=1.4, zorder=4, label="Model results")
|
| 86 |
+
for _, row in data.iterrows():
|
| 87 |
+
label = str(row.get("Segment", "")).strip()
|
| 88 |
+
if label:
|
| 89 |
+
ax.annotate(label, (row["_v"], row["_d"]), xytext=(3, 3),
|
| 90 |
+
textcoords="offset points", fontsize=6.5)
|
| 91 |
+
|
| 92 |
+
ax.text(0.18, 0.10, "ACCEPTABLE", transform=ax.transAxes, color="#548235",
|
| 93 |
+
fontsize=11, weight="bold", alpha=0.9)
|
| 94 |
+
ax.text(0.68, 0.52, "NOT ACCEPTABLE", transform=ax.transAxes, color="#a61c00",
|
| 95 |
+
fontsize=10, weight="bold", alpha=0.82)
|
| 96 |
+
if not results_usable:
|
| 97 |
+
ax.text(0.5, 0.02, "Model points withheld: execution-integrity gate failed",
|
| 98 |
+
transform=ax.transAxes, ha="center", va="bottom", fontsize=8,
|
| 99 |
+
color="#a61c00", bbox={"facecolor": "white", "edgecolor": "#a61c00", "pad": 3})
|
| 100 |
+
elif plotted == 0:
|
| 101 |
+
ax.text(0.5, 0.02, "No modeled overland-route depth/velocity pairs identified",
|
| 102 |
+
transform=ax.transAxes, ha="center", va="bottom", fontsize=8,
|
| 103 |
+
bbox={"facecolor": "white", "edgecolor": "#666666", "pad": 3})
|
| 104 |
+
|
| 105 |
+
ax.set_xlim(0.0, 4.0)
|
| 106 |
+
ax.set_ylim(0.0, max(0.9, max(d for _, d in points) * 1.08))
|
| 107 |
+
ax.set_xlabel("Velocity (m/s)")
|
| 108 |
+
ax.set_ylabel("Depth (m)")
|
| 109 |
+
ax.set_title("Depth–Velocity Criteria for Overland Flow")
|
| 110 |
+
ax.grid(True, color="#777777", linewidth=0.45, alpha=0.65)
|
| 111 |
+
ax.legend(loc="upper right", fontsize=7.5, frameon=True)
|
| 112 |
+
fig.tight_layout()
|
| 113 |
+
out = io.BytesIO()
|
| 114 |
+
fig.savefig(out, format="png", dpi=180, metadata={"Software": "SWMM Analysis MCP Server"})
|
| 115 |
+
return out.getvalue()
|
| 116 |
+
|
| 117 |
+
|
| 118 |
+
def _model_schematic_figure(inp_sections: Mapping[str, list[list[str]]]) -> tuple[bytes, dict[str, Any]]:
|
| 119 |
+
"""Render a deterministic SWMM topology schematic from tokenized INP data."""
|
| 120 |
+
from collections import defaultdict
|
| 121 |
+
from matplotlib.backends.backend_agg import FigureCanvasAgg
|
| 122 |
+
from matplotlib.figure import Figure
|
| 123 |
+
from matplotlib.lines import Line2D
|
| 124 |
+
|
| 125 |
+
node_sections = {"JUNCTIONS": "Junction", "DIVIDERS": "Junction",
|
| 126 |
+
"STORAGE": "Storage", "OUTFALLS": "Outfall"}
|
| 127 |
+
nodes: dict[str, str] = {}
|
| 128 |
+
for section, kind in node_sections.items():
|
| 129 |
+
for row in inp_sections.get(section, []) or []:
|
| 130 |
+
if row:
|
| 131 |
+
nodes[str(row[0])] = kind
|
| 132 |
+
link_sections = ("CONDUITS", "PUMPS", "ORIFICES", "WEIRS", "OUTLETS")
|
| 133 |
+
links: list[tuple[str, str, str, str]] = []
|
| 134 |
+
for section in link_sections:
|
| 135 |
+
for row in inp_sections.get(section, []) or []:
|
| 136 |
+
if len(row) >= 3:
|
| 137 |
+
links.append((str(row[0]), str(row[1]), str(row[2]), section[:-1].title()))
|
| 138 |
+
|
| 139 |
+
coords: dict[str, tuple[float, float]] = {}
|
| 140 |
+
for row in inp_sections.get("COORDINATES", []) or []:
|
| 141 |
+
if len(row) >= 3:
|
| 142 |
+
try:
|
| 143 |
+
coords[str(row[0])] = (float(row[1]), float(row[2]))
|
| 144 |
+
except (TypeError, ValueError):
|
| 145 |
+
pass
|
| 146 |
+
|
| 147 |
+
# Coordinate-free models receive a stable downstream-rank layout.
|
| 148 |
+
if not all(n in coords for n in nodes):
|
| 149 |
+
outgoing: dict[str, list[str]] = defaultdict(list)
|
| 150 |
+
incoming: dict[str, int] = defaultdict(int)
|
| 151 |
+
for _, source, target, _ in links:
|
| 152 |
+
outgoing[source].append(target)
|
| 153 |
+
incoming[target] += 1
|
| 154 |
+
rank = {n: 0 for n in nodes}
|
| 155 |
+
queue = sorted(n for n in nodes if incoming[n] == 0)
|
| 156 |
+
for source in queue:
|
| 157 |
+
for target in sorted(outgoing[source]):
|
| 158 |
+
rank[target] = max(rank.get(target, 0), rank[source] + 1)
|
| 159 |
+
incoming[target] -= 1
|
| 160 |
+
if incoming[target] == 0:
|
| 161 |
+
queue.append(target)
|
| 162 |
+
groups: dict[int, list[str]] = defaultdict(list)
|
| 163 |
+
for n in sorted(nodes):
|
| 164 |
+
groups[rank.get(n, 0)].append(n)
|
| 165 |
+
for r, names in groups.items():
|
| 166 |
+
for i, name in enumerate(names):
|
| 167 |
+
coords.setdefault(name, (float(r), float(-i)))
|
| 168 |
+
coordinate_basis = "INP coordinates with deterministic topology fallback"
|
| 169 |
+
else:
|
| 170 |
+
coordinate_basis = "INP [COORDINATES]"
|
| 171 |
+
|
| 172 |
+
vertices: dict[str, list[tuple[float, float]]] = defaultdict(list)
|
| 173 |
+
for row in inp_sections.get("VERTICES", []) or []:
|
| 174 |
+
if len(row) >= 3:
|
| 175 |
+
try:
|
| 176 |
+
vertices[str(row[0])].append((float(row[1]), float(row[2])))
|
| 177 |
+
except (TypeError, ValueError):
|
| 178 |
+
pass
|
| 179 |
+
|
| 180 |
+
polygon_points: dict[str, list[tuple[float, float]]] = defaultdict(list)
|
| 181 |
+
for row in inp_sections.get("POLYGONS", inp_sections.get("Polygons", [])) or []:
|
| 182 |
+
if len(row) >= 3:
|
| 183 |
+
try:
|
| 184 |
+
polygon_points[str(row[0])].append((float(row[1]), float(row[2])))
|
| 185 |
+
except (TypeError, ValueError):
|
| 186 |
+
pass
|
| 187 |
+
subcatchments = []
|
| 188 |
+
for row in inp_sections.get("SUBCATCHMENTS", []) or []:
|
| 189 |
+
if len(row) >= 3:
|
| 190 |
+
sid, outlet = str(row[0]), str(row[2])
|
| 191 |
+
pts = polygon_points.get(sid, [])
|
| 192 |
+
if pts:
|
| 193 |
+
pos = (sum(x for x, _ in pts) / len(pts), sum(y for _, y in pts) / len(pts))
|
| 194 |
+
elif sid in coords:
|
| 195 |
+
pos = coords[sid]
|
| 196 |
+
else:
|
| 197 |
+
ox, oy = coords.get(outlet, (0.0, 0.0))
|
| 198 |
+
pos = (ox, oy + 1.0)
|
| 199 |
+
subcatchments.append((sid, outlet, pos))
|
| 200 |
+
|
| 201 |
+
fig = Figure(figsize=(10.5, 7.0), dpi=180, facecolor="white")
|
| 202 |
+
FigureCanvasAgg(fig)
|
| 203 |
+
ax = fig.add_subplot(111)
|
| 204 |
+
link_colors = {"Conduit": "#4472c4", "Outlet": "#7030a0", "Pump": "#ed7d31",
|
| 205 |
+
"Orifice": "#a5a5a5", "Weir": "#ffc000"}
|
| 206 |
+
for lid, source, target, kind in links:
|
| 207 |
+
if source not in coords or target not in coords:
|
| 208 |
+
continue
|
| 209 |
+
path = [coords[source], *vertices.get(lid, []), coords[target]]
|
| 210 |
+
color = link_colors.get(kind, "#4472c4")
|
| 211 |
+
for a, b in zip(path[:-1], path[1:]):
|
| 212 |
+
ax.plot([a[0], b[0]], [a[1], b[1]], color=color, linewidth=1.0, zorder=1)
|
| 213 |
+
a, b = path[-2], path[-1]
|
| 214 |
+
ax.annotate("", xy=b, xytext=a,
|
| 215 |
+
arrowprops={"arrowstyle": "-|>", "color": color, "lw": 1.0,
|
| 216 |
+
"shrinkA": 4, "shrinkB": 6}, zorder=2)
|
| 217 |
+
mid = path[len(path) // 2]
|
| 218 |
+
ax.annotate(lid, mid, xytext=(2, 2), textcoords="offset points",
|
| 219 |
+
fontsize=5.5, color="#333333")
|
| 220 |
+
|
| 221 |
+
node_style = {
|
| 222 |
+
"Junction": ("o", "#ffffff", "#1f4e79"),
|
| 223 |
+
"Storage": ("s", "#fff2cc", "#bf9000"),
|
| 224 |
+
"Outfall": ("v", "#f4cccc", "#990000"),
|
| 225 |
+
}
|
| 226 |
+
for name, kind in nodes.items():
|
| 227 |
+
if name not in coords:
|
| 228 |
+
continue
|
| 229 |
+
x, y = coords[name]
|
| 230 |
+
marker, face, edge = node_style[kind]
|
| 231 |
+
ax.scatter([x], [y], marker=marker, s=48, facecolor=face, edgecolor=edge,
|
| 232 |
+
linewidth=1.1, zorder=4)
|
| 233 |
+
ax.annotate(name, (x, y), xytext=(4, 4), textcoords="offset points",
|
| 234 |
+
fontsize=6.5, zorder=5)
|
| 235 |
+
|
| 236 |
+
for sid, outlet, (x, y) in subcatchments:
|
| 237 |
+
ax.scatter([x], [y], marker="D", s=42, facecolor="#e2f0d9",
|
| 238 |
+
edgecolor="#548235", linewidth=1.0, zorder=3)
|
| 239 |
+
ax.annotate(f"SC {sid}", (x, y), xytext=(4, -8), textcoords="offset points",
|
| 240 |
+
fontsize=6.2, zorder=5)
|
| 241 |
+
if outlet in coords:
|
| 242 |
+
ax.annotate("", xy=coords[outlet], xytext=(x, y),
|
| 243 |
+
arrowprops={"arrowstyle": "->", "color": "#70ad47", "lw": 0.75,
|
| 244 |
+
"linestyle": "--", "shrinkA": 4, "shrinkB": 6}, zorder=1)
|
| 245 |
+
|
| 246 |
+
handles = [
|
| 247 |
+
Line2D([0], [0], marker="D", color="none", markerfacecolor="#e2f0d9",
|
| 248 |
+
markeredgecolor="#548235", label="Subcatchment"),
|
| 249 |
+
*[Line2D([0], [0], marker=node_style[k][0], color="none",
|
| 250 |
+
markerfacecolor=node_style[k][1], markeredgecolor=node_style[k][2], label=k)
|
| 251 |
+
for k in ("Junction", "Storage", "Outfall")],
|
| 252 |
+
Line2D([0], [0], color="#4472c4", label="Hydraulic link / flow direction"),
|
| 253 |
+
Line2D([0], [0], color="#70ad47", linestyle="--", label="Runoff routing"),
|
| 254 |
+
]
|
| 255 |
+
ax.legend(handles=handles, loc="best", fontsize=7, frameon=True)
|
| 256 |
+
ax.set_title("Automated SWMM Model Schematic", fontsize=13, weight="bold")
|
| 257 |
+
ax.set_aspect("equal", adjustable="datalim")
|
| 258 |
+
ax.axis("off")
|
| 259 |
+
ax.margins(0.10)
|
| 260 |
+
fig.tight_layout()
|
| 261 |
+
out = io.BytesIO()
|
| 262 |
+
fig.savefig(out, format="png", dpi=180, bbox_inches="tight",
|
| 263 |
+
metadata={"Software": "SWMM Analysis MCP Server"})
|
| 264 |
+
manifest = {
|
| 265 |
+
"coordinate_basis": coordinate_basis,
|
| 266 |
+
"node_count": len(nodes), "link_count": len(links),
|
| 267 |
+
"subcatchment_count": len(subcatchments),
|
| 268 |
+
"node_types": {kind: sum(1 for v in nodes.values() if v == kind)
|
| 269 |
+
for kind in sorted(set(nodes.values()))},
|
| 270 |
+
"limitations": "Topology schematic only; confirm geometry, crossings, scale, and drawing reconciliation before issue.",
|
| 271 |
+
}
|
| 272 |
+
return out.getvalue(), manifest
|
| 273 |
+
|
| 274 |
+
|
| 275 |
@dataclass
|
| 276 |
class ReportCriteria:
|
| 277 |
node_depth_ratio: float = 0.80
|
|
|
|
| 1019 |
add("SWMR-08","Minor-system routed flows, cumulative design flows, pipe capacities and spare capacity", "Not applicable", "No conduits identified in uploaded model", "Confirm whether a separate minor-system model is within the report scope", "Minor System")
|
| 1020 |
add("SWMR-09","HGL, surcharge, rim clearance, downstream HWL and backwater assessment", "Requires professional confirmation", "Storage/outfall HGL table; no conduit network identified", "Confirm whether a separate minor-system model and downstream HWL assessment are required", "Minor System")
|
| 1021 |
add("SWMR-10","Catchbasin, inlet, ICD and outlet rating information", "Complete" if has_controls else "Not applicable / missing", "Hydraulic controls table" if has_controls else "—", "Confirm inlet types, rating curves and drawing locations", "Minor System")
|
| 1022 |
+
add("SWMR-11","Critical overland flows, depths, velocities, spill routes and escape routes", "Partially complete" if has_overland else "Requires drawing review", "Table 9 and deterministic Figure 9-1 depth-velocity screen" if has_overland else "Figure 9-1 criterion included; no overland route identified in model", "Confirm overland route, grading, containment, escape routes and safety on drawings", "Major System")
|
| 1023 |
add("SWMR-12","Trap-low and storage volume, depth, spill, entrance grade and restrictive covenant information", "Partially complete" if has_storage else "Not applicable", "Storage assessment" if has_storage else "—", "Add spill elevations, building grades and covenant requirements", "Storage")
|
| 1024 |
add("SWMR-13","Minor and major boundary inflows/outflows and supporting source", "Partially complete" if has_outfalls else "Missing", "Boundary outflow tables" if has_outfalls else "—", "Confirm downstream capacity and external inflows", "Boundary Conditions")
|
| 1025 |
add("SWMR-14","Private-site permissible discharge and on-site storage requirements", "Missing", "—", "Provide applicable release rates and private-site storage criteria", "Private Sites")
|
| 1026 |
add("SWMR-15","Water-quality treatment, BMPs, downstream treatment and source controls", "Missing", "—", "Document water-quality strategy and downstream treatment", "Water Quality")
|
| 1027 |
required_drawings={"site location","study area","catchment plan","model schematic","overland drainage","storm drainage"}
|
| 1028 |
found=len(required_drawings & drawings)
|
| 1029 |
+
if has_model and "model schematic" not in drawings:
|
| 1030 |
+
found += 1 # deterministic Figure 3-1 generated from the uploaded INP
|
| 1031 |
+
add("SWMR-16","Required figures and drawings", "Complete" if found==len(required_drawings) else ("Partially complete" if found else "Missing"), f"{found}/{len(required_drawings)} core figures/drawings recorded or generated", "Add missing site, catchment, overland and storm-drainage drawings; reconcile Figure 3-1 to issued drawings", "Drawings")
|
| 1032 |
+
add("SWMR-17","Model input/output files, formatted listings, model schematic, drawing reconciliation and auditable digital package", "Partially complete" if has_model and has_results else "Missing", "Digital model package, Figure 3-1 and structured appendices" if has_model else "—", "Complete drawing-to-model cross-reference, revision metadata and final authenticated files", "Appendices")
|
| 1033 |
return pd.DataFrame(rows)
|
| 1034 |
|
| 1035 |
def _readiness_scores(checklist: pd.DataFrame) -> pd.DataFrame:
|
|
|
|
| 1444 |
checklist_table.loc[_m17, "Status"] = "Included (Appendix D listings + digital package)"
|
| 1445 |
checklist_table.loc[_m17, "Outstanding Action"] = "Verify appendix listings match the issued digital model package"
|
| 1446 |
readiness_table = _readiness_scores(checklist_table)
|
| 1447 |
+
schematic_png, schematic_manifest = _model_schematic_figure(inp_sections)
|
| 1448 |
|
| 1449 |
doc = Document()
|
| 1450 |
sec = doc.sections[0]
|
|
|
|
| 1501 |
doc.add_paragraph("Draft status: suitable for consultant/client discussion and structured engineering review. Human-in-the-loop completion, technical verification, drawing coordination, and professional authentication are required before municipal submission.")
|
| 1502 |
if model_identity:
|
| 1503 |
doc.add_heading("1.0a Model Identity and Execution Provenance", level=2)
|
| 1504 |
+
identity_rows = [
|
| 1505 |
{"Item": "Model file", "Value": str(model_identity.get("inp_name", "—"))},
|
| 1506 |
{"Item": "Model SHA-256", "Value": str(model_identity.get("sha256", "—"))},
|
| 1507 |
{"Item": "Session / run ID", "Value": f"{model_identity.get('session_id', '—')} / {model_identity.get('run_id', '—')}"},
|
| 1508 |
{"Item": "Engine", "Value": str(model_identity.get("engine", "EPA SWMM / OpenSWMM (crash-isolated worker)"))},
|
| 1509 |
{"Item": "Execution status", "Value": str(model_identity.get("status", "Completed"))},
|
| 1510 |
{"Item": "Generated", "Value": str(model_identity.get("generated", metadata.report_date))},
|
| 1511 |
+
]
|
| 1512 |
+
if model_identity.get("legacy_defaults_normalized"):
|
| 1513 |
+
identity_rows.extend([
|
| 1514 |
+
{"Item": "Execution derivative", "Value": str(model_identity.get("execution_inp_name", "—"))},
|
| 1515 |
+
{"Item": "Execution SHA-256", "Value": str(model_identity.get("execution_sha256", "—"))},
|
| 1516 |
+
{"Item": "Legacy option handling", "Value": "Zero/default sentinels normalized in an immutable derivative; original upload preserved"},
|
| 1517 |
+
])
|
| 1518 |
+
_id_tbl = pd.DataFrame(identity_rows)
|
| 1519 |
_add_df_table(doc, "Table 0A - Model Identity", _id_tbl, font_size=8.0)
|
| 1520 |
_rev_tbl = pd.DataFrame([{"Rev": "P0", "Date": metadata.report_date,
|
| 1521 |
"Description": "Preliminary server-generated draft for engineering review",
|
|
|
|
| 1576 |
)
|
| 1577 |
if options.get("FLOW_ROUTING", "").upper() == "KINWAVE":
|
| 1578 |
doc.add_paragraph("Modelling limitation: kinematic-wave routing does not fully represent backwater, pressurization, reverse flow, or complex surcharge interactions. Dynamic-wave routing should be considered where these effects are material.")
|
| 1579 |
+
doc.add_picture(io.BytesIO(schematic_png), width=Inches(7.15))
|
| 1580 |
+
doc.paragraphs[-1].alignment = WD_ALIGN_PARAGRAPH.CENTER
|
| 1581 |
+
schematic_caption = doc.add_paragraph(
|
| 1582 |
+
"Figure 3-1 - Automated SWMM Model Schematic. Solid arrows show hydraulic-link "
|
| 1583 |
+
"direction and dashed arrows show subcatchment runoff routing. This topology figure "
|
| 1584 |
+
"is generated from the uploaded INP and must be reconciled to the issued drainage drawings."
|
| 1585 |
+
)
|
| 1586 |
+
schematic_caption.alignment = WD_ALIGN_PARAGRAPH.CENTER
|
| 1587 |
+
for run in schematic_caption.runs:
|
| 1588 |
+
run.italic = True
|
| 1589 |
+
run.font.size = Pt(8)
|
| 1590 |
doc.add_heading("3.3 Major-Minor System", level=2)
|
| 1591 |
doc.add_paragraph("Open channels, swales, gutters, culverts, and closed conduits represented in the model were reviewed. Final major/minor classification must be confirmed against the approved drainage concept and drawings.")
|
| 1592 |
if scenario_comparison is not None and not scenario_comparison.empty:
|
|
|
|
| 1654 |
overland_compliance[col] = "Not assessed - hydraulic routing solution invalid"
|
| 1655 |
if "Spill Active" in overland_compliance:
|
| 1656 |
overland_compliance["Spill Active"] = "Not assessed"
|
| 1657 |
+
depth_velocity_png = _depth_velocity_figure(
|
| 1658 |
+
overland_compliance, calgary.depth_velocity_curve,
|
| 1659 |
+
results_usable=integrity["results_usable"], flow_unit=units.flow)
|
| 1660 |
+
doc.add_picture(io.BytesIO(depth_velocity_png), width=Inches(6.65))
|
| 1661 |
+
doc.paragraphs[-1].alignment = WD_ALIGN_PARAGRAPH.CENTER
|
| 1662 |
+
caption = doc.add_paragraph(
|
| 1663 |
+
"Figure 9-1 - Alberta/Calgary Depth–Velocity Criteria for Overland Flow. "
|
| 1664 |
+
"Model points are derived from Table 9; marker colour represents peak flow. "
|
| 1665 |
+
"Straight-line interpolation is used between the tabulated points. The 2011 baseline "
|
| 1666 |
+
"envelope must be checked against current and project-specific requirements."
|
| 1667 |
+
)
|
| 1668 |
+
caption.alignment = WD_ALIGN_PARAGRAPH.CENTER
|
| 1669 |
+
for run in caption.runs:
|
| 1670 |
+
run.italic = True
|
| 1671 |
+
run.font.size = Pt(8)
|
| 1672 |
+
curve_table = pd.DataFrame(calgary.depth_velocity_curve,
|
| 1673 |
+
columns=["Water Velocity (m/s)", "Permissible Depth (m)"])
|
| 1674 |
+
_add_df_table(doc, "Figure 9-1A - Permissible Depth and Velocity of Overland Flow",
|
| 1675 |
+
curve_table, landscape=False, font_size=8.0)
|
| 1676 |
if not overland_compliance.empty:
|
| 1677 |
_add_df_table(doc, "Table 9A - Calgary Major-System Depth-Velocity Screening", overland_compliance, landscape=True, font_size=7.2)
|
| 1678 |
|
|
|
|
| 1893 |
zf.writestr("tables/calgary_criteria_register.csv", criteria_table.to_csv(index=False))
|
| 1894 |
zf.writestr("tables/calgary_minor_system_capacity.csv", minor_capacity.to_csv(index=False))
|
| 1895 |
zf.writestr("tables/calgary_major_system_compliance.csv", overland_compliance.to_csv(index=False))
|
| 1896 |
+
zf.writestr("tables/depth_velocity_criterion.csv", curve_table.to_csv(index=False))
|
| 1897 |
+
zf.writestr("figures/figure_9_1_depth_velocity_criteria.png", depth_velocity_png)
|
| 1898 |
+
zf.writestr("figures/figure_3_1_model_schematic.png", schematic_png)
|
| 1899 |
+
zf.writestr("metadata/model_schematic_manifest.json", json.dumps(schematic_manifest, indent=2))
|
| 1900 |
zf.writestr("tables/calgary_storage_assessment.csv", storage_calgary.to_csv(index=False))
|
| 1901 |
zf.writestr("tables/calgary_swmr_completeness_register.csv", checklist_table.to_csv(index=False))
|
| 1902 |
zf.writestr("tables/swmr_draft_readiness.csv", readiness_table.to_csv(index=False))
|
|
|
|
| 1921 |
if model_listings.get("inp_text"):
|
| 1922 |
zf.writestr(f"model/{_safe_name(model_listings.get('inp_name', 'model.inp'))}",
|
| 1923 |
model_listings["inp_text"])
|
| 1924 |
+
if model_listings.get("execution_inp_text"):
|
| 1925 |
+
zf.writestr(f"model/{_safe_name(model_listings.get('execution_inp_name', 'execution_model.inp'))}",
|
| 1926 |
+
model_listings["execution_inp_text"])
|
| 1927 |
if model_listings.get("rpt_text"):
|
| 1928 |
zf.writestr(f"model/{_safe_name(model_listings.get('rpt_name', 'model.rpt'))}",
|
| 1929 |
model_listings["rpt_text"])
|
requirements.txt
CHANGED
|
@@ -6,5 +6,6 @@ httpx>=0.27
|
|
| 6 |
requests>=2.31
|
| 7 |
pandas>=2.0
|
| 8 |
numpy>=1.26
|
|
|
|
| 9 |
python-docx>=1.1
|
| 10 |
PyYAML>=6.0
|
|
|
|
| 6 |
requests>=2.31
|
| 7 |
pandas>=2.0
|
| 8 |
numpy>=1.26
|
| 9 |
+
matplotlib>=3.8
|
| 10 |
python-docx>=1.1
|
| 11 |
PyYAML>=6.0
|
screening_logic.py
CHANGED
|
@@ -26,19 +26,29 @@ import pandas as pd
|
|
| 26 |
# Solver-option and execution-integrity gates
|
| 27 |
# ---------------------------------------------------------------------------
|
| 28 |
|
| 29 |
-
def
|
| 30 |
-
"""
|
| 31 |
|
| 32 |
-
|
| 33 |
-
|
| 34 |
-
|
|
|
|
|
|
|
| 35 |
"""
|
| 36 |
opts = {str(k).upper(): str(v).strip() for k, v in options.items()}
|
| 37 |
if opts.get("FLOW_ROUTING", "").upper() != "DYNWAVE":
|
| 38 |
-
return []
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 39 |
errors: list[str] = []
|
| 40 |
-
for name,
|
| 41 |
-
("HEAD_TOLERANCE", "head tolerance")):
|
| 42 |
if name not in opts:
|
| 43 |
continue # omitted means use the engine default
|
| 44 |
try:
|
|
@@ -46,12 +56,25 @@ def validate_solver_options(options: Mapping[str, Any]) -> list[str]:
|
|
| 46 |
except (TypeError, ValueError):
|
| 47 |
errors.append(f"{name} must be numeric for dynamic-wave routing.")
|
| 48 |
continue
|
| 49 |
-
if value <
|
| 50 |
errors.append(
|
| 51 |
-
f"{name}
|
| 52 |
-
f"the uploaded value {opts[name]!r}
|
| 53 |
)
|
| 54 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 55 |
|
| 56 |
|
| 57 |
def execution_integrity_assessment(metadata: Mapping[str, Any]) -> dict[str, Any]:
|
|
|
|
| 26 |
# Solver-option and execution-integrity gates
|
| 27 |
# ---------------------------------------------------------------------------
|
| 28 |
|
| 29 |
+
def resolve_legacy_solver_options(options: Mapping[str, Any]) -> dict[str, Any]:
|
| 30 |
+
"""Resolve auditable SWMM legacy-zero sentinels for an execution copy.
|
| 31 |
|
| 32 |
+
Older/converted INP files can explicitly serialize zero for dynamic-wave
|
| 33 |
+
options that EPA SWMM displays and executes using unit-aware defaults.
|
| 34 |
+
This function returns substitutions for an immutable derivative; it never
|
| 35 |
+
edits the uploaded source model. Negative and non-numeric values remain
|
| 36 |
+
blocking errors. Omitted values remain omitted for the engine to default.
|
| 37 |
"""
|
| 38 |
opts = {str(k).upper(): str(v).strip() for k, v in options.items()}
|
| 39 |
if opts.get("FLOW_ROUTING", "").upper() != "DYNWAVE":
|
| 40 |
+
return {"effective_options": dict(opts), "substitutions": [], "errors": []}
|
| 41 |
+
flow_units = opts.get("FLOW_UNITS", "").upper()
|
| 42 |
+
si_units = flow_units in {"CMS", "LPS", "MLD"}
|
| 43 |
+
defaults = {
|
| 44 |
+
"MAX_TRIALS": (8.0, "count"),
|
| 45 |
+
"HEAD_TOLERANCE": (0.0015 if si_units else 0.005, "m" if si_units else "ft"),
|
| 46 |
+
"MIN_SURFAREA": (1.167 if si_units else 12.566, "m2" if si_units else "ft2"),
|
| 47 |
+
}
|
| 48 |
+
effective = dict(opts)
|
| 49 |
+
substitutions: list[dict[str, Any]] = []
|
| 50 |
errors: list[str] = []
|
| 51 |
+
for name, (default, units) in defaults.items():
|
|
|
|
| 52 |
if name not in opts:
|
| 53 |
continue # omitted means use the engine default
|
| 54 |
try:
|
|
|
|
| 56 |
except (TypeError, ValueError):
|
| 57 |
errors.append(f"{name} must be numeric for dynamic-wave routing.")
|
| 58 |
continue
|
| 59 |
+
if value < 0:
|
| 60 |
errors.append(
|
| 61 |
+
f"{name} cannot be negative for dynamic-wave routing; "
|
| 62 |
+
f"the uploaded value is {opts[name]!r}."
|
| 63 |
)
|
| 64 |
+
elif value == 0:
|
| 65 |
+
effective[name] = format(default, "g")
|
| 66 |
+
substitutions.append({
|
| 67 |
+
"option": name, "original_value": opts[name],
|
| 68 |
+
"effective_value": default, "units": units,
|
| 69 |
+
"reason": "Recognized legacy zero/default sentinel",
|
| 70 |
+
})
|
| 71 |
+
return {"effective_options": effective,
|
| 72 |
+
"substitutions": substitutions, "errors": errors}
|
| 73 |
+
|
| 74 |
+
|
| 75 |
+
def validate_solver_options(options: Mapping[str, Any]) -> list[str]:
|
| 76 |
+
"""Return only blocking errors after legacy-default resolution."""
|
| 77 |
+
return list(resolve_legacy_solver_options(options)["errors"])
|
| 78 |
|
| 79 |
|
| 80 |
def execution_integrity_assessment(metadata: Mapping[str, Any]) -> dict[str, Any]:
|
server.py
CHANGED
|
@@ -43,7 +43,7 @@ from sessions import SESSION_ROOT, STORE
|
|
| 43 |
from tools import TOOL_REGISTRY
|
| 44 |
|
| 45 |
SERVER_NAME = "swmm-analysis"
|
| 46 |
-
SERVER_VERSION = "1.0.
|
| 47 |
|
| 48 |
# ---------------------------------------------------------------------------
|
| 49 |
# MCP surface
|
|
|
|
| 43 |
from tools import TOOL_REGISTRY
|
| 44 |
|
| 45 |
SERVER_NAME = "swmm-analysis"
|
| 46 |
+
SERVER_VERSION = "1.0.4 (engine Rev 23.2; generic CoC workflow Rev 28)"
|
| 47 |
|
| 48 |
# ---------------------------------------------------------------------------
|
| 49 |
# MCP surface
|
test_regression.py
CHANGED
|
@@ -44,12 +44,24 @@ def unit_tests() -> None:
|
|
| 44 |
execution_integrity_assessment,
|
| 45 |
effective_velocity_table,
|
| 46 |
missing_information_register,
|
|
|
|
| 47 |
validate_solver_options)
|
| 48 |
|
| 49 |
solver_errors = validate_solver_options({
|
| 50 |
"FLOW_ROUTING": "DYNWAVE", "MAX_TRIALS": "0", "HEAD_TOLERANCE": "0"})
|
| 51 |
-
check("zero
|
| 52 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 53 |
check("omitted dynamic-wave overrides use engine defaults",
|
| 54 |
validate_solver_options({"FLOW_ROUTING": "DYNWAVE"}) == [])
|
| 55 |
|
|
@@ -220,6 +232,12 @@ def docx_tests(docx_path: str) -> None:
|
|
| 220 |
check("preliminary disclaimer present", "professional authentication are required" in joined)
|
| 221 |
check("no calibration/validation claims",
|
| 222 |
"calibrated" not in joined.lower() and "validated model" not in joined.lower())
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 223 |
|
| 224 |
# No unsupported storage Pass: scan tables for a bare "Pass" in Calgary Status
|
| 225 |
bare_pass = False
|
|
|
|
| 44 |
execution_integrity_assessment,
|
| 45 |
effective_velocity_table,
|
| 46 |
missing_information_register,
|
| 47 |
+
resolve_legacy_solver_options,
|
| 48 |
validate_solver_options)
|
| 49 |
|
| 50 |
solver_errors = validate_solver_options({
|
| 51 |
"FLOW_ROUTING": "DYNWAVE", "MAX_TRIALS": "0", "HEAD_TOLERANCE": "0"})
|
| 52 |
+
check("legacy zero sentinels are not blocking", solver_errors == [])
|
| 53 |
+
resolved = resolve_legacy_solver_options({
|
| 54 |
+
"FLOW_ROUTING": "DYNWAVE", "FLOW_UNITS": "CMS",
|
| 55 |
+
"MIN_SURFAREA": "0", "MAX_TRIALS": "0", "HEAD_TOLERANCE": "0"})
|
| 56 |
+
effective = resolved["effective_options"]
|
| 57 |
+
check("SI legacy MAX_TRIALS resolves to 8", effective["MAX_TRIALS"] == "8")
|
| 58 |
+
check("SI legacy HEAD_TOLERANCE resolves to 0.0015", effective["HEAD_TOLERANCE"] == "0.0015")
|
| 59 |
+
check("SI legacy MIN_SURFAREA resolves to 1.167", effective["MIN_SURFAREA"] == "1.167")
|
| 60 |
+
check("three substitutions are audited", len(resolved["substitutions"]) == 3)
|
| 61 |
+
check("negative values remain blocked", any("cannot be negative" in x for x in
|
| 62 |
+
validate_solver_options({"FLOW_ROUTING": "DYNWAVE", "MAX_TRIALS": "-1"})))
|
| 63 |
+
check("non-numeric values remain blocked", any("must be numeric" in x for x in
|
| 64 |
+
validate_solver_options({"FLOW_ROUTING": "DYNWAVE", "HEAD_TOLERANCE": "bad"})))
|
| 65 |
check("omitted dynamic-wave overrides use engine defaults",
|
| 66 |
validate_solver_options({"FLOW_ROUTING": "DYNWAVE"}) == [])
|
| 67 |
|
|
|
|
| 232 |
check("preliminary disclaimer present", "professional authentication are required" in joined)
|
| 233 |
check("no calibration/validation claims",
|
| 234 |
"calibrated" not in joined.lower() and "validated model" not in joined.lower())
|
| 235 |
+
check("mandatory depth-velocity figure caption present",
|
| 236 |
+
"Figure 9-1 - Alberta/Calgary Depth–Velocity Criteria" in joined)
|
| 237 |
+
check("depth-velocity criterion table present",
|
| 238 |
+
"Figure 9-1A - Permissible Depth and Velocity" in joined)
|
| 239 |
+
check("automated model schematic caption present",
|
| 240 |
+
"Figure 3-1 - Automated SWMM Model Schematic" in joined)
|
| 241 |
|
| 242 |
# No unsupported storage Pass: scan tables for a bare "Pass" in Calgary Status
|
| 243 |
bare_pass = False
|
tools.py
CHANGED
|
@@ -33,7 +33,7 @@ from results_db import ResultDatabase
|
|
| 33 |
from sessions import STORE
|
| 34 |
from sql_agent import SafeSQLAgent
|
| 35 |
from swmm_core import run_swmm
|
| 36 |
-
from screening_logic import execution_integrity_assessment,
|
| 37 |
|
| 38 |
MAX_ROWS = 60
|
| 39 |
MAX_TS_POINTS = 200
|
|
@@ -59,6 +59,34 @@ def _options_map(sections: dict[str, list[list[str]]]) -> dict[str, str]:
|
|
| 59 |
return {str(r[0]).upper(): str(r[1]) for r in sections.get("OPTIONS", []) if len(r) >= 2}
|
| 60 |
|
| 61 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 62 |
# ---------------------------------------------------------------------------
|
| 63 |
# Model lifecycle
|
| 64 |
# ---------------------------------------------------------------------------
|
|
@@ -108,18 +136,30 @@ def run_simulation(session_id: str) -> dict:
|
|
| 108 |
if not inp_path:
|
| 109 |
raise ValueError("Session has no uploaded model.")
|
| 110 |
sections = session.data["sections"]
|
| 111 |
-
|
|
|
|
| 112 |
if option_errors:
|
| 113 |
session.data["input_validation_errors"] = option_errors
|
| 114 |
raise ValueError(
|
| 115 |
"Input validation failed; simulation was not run. " + " ".join(option_errors)
|
| 116 |
)
|
| 117 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
| 118 |
md = results["metadata"]
|
| 119 |
import hashlib, datetime
|
| 120 |
-
|
|
|
|
| 121 |
run_id = datetime.datetime.utcnow().strftime("%Y%m%dT%H%M%SZ")
|
| 122 |
-
md["model_sha256"] =
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 123 |
md["run_id"] = run_id
|
| 124 |
integrity = execution_integrity_assessment(md)
|
| 125 |
md.update({
|
|
@@ -140,7 +180,7 @@ def run_simulation(session_id: str) -> dict:
|
|
| 140 |
if "Status" in link_df:
|
| 141 |
link_df["Status"] = invalid_label
|
| 142 |
db = ResultDatabase(str(session.workdir / "results.sqlite"))
|
| 143 |
-
db.load(node_df, link_df, sub_df, inp_path=
|
| 144 |
|
| 145 |
recon = {"verdict": "Not performed"}
|
| 146 |
recon_links = recon_nodes = recon_cont = None
|
|
@@ -153,6 +193,7 @@ def run_simulation(session_id: str) -> dict:
|
|
| 153 |
|
| 154 |
session.data.update({
|
| 155 |
"results": results, "node_df": node_df, "link_df": link_df, "sub_df": sub_df,
|
|
|
|
| 156 |
"db": db, "recon_links": recon_links, "recon_nodes": recon_nodes,
|
| 157 |
"recon_continuity": recon_cont, "recon_summary": recon,
|
| 158 |
})
|
|
@@ -163,7 +204,10 @@ def run_simulation(session_id: str) -> dict:
|
|
| 163 |
"simulation": "completed" if integrity["results_usable"] else "completed_invalid",
|
| 164 |
"execution_integrity": integrity,
|
| 165 |
"results_usable": integrity["results_usable"],
|
| 166 |
-
"model_sha256":
|
|
|
|
|
|
|
|
|
|
| 167 |
"run_id": run_id,
|
| 168 |
"flow_units": md.get("flow_units"),
|
| 169 |
"runoff_continuity_error_pct": round(float(md.get("runoff_error", 0.0)), 3),
|
|
@@ -302,22 +346,39 @@ def calgary_screening(session_id: str) -> dict:
|
|
| 302 |
"""
|
| 303 |
session = STORE.get(session_id)
|
| 304 |
_require_results(session)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 305 |
integrity = execution_integrity_assessment(session.data["results"].get("metadata", {}))
|
| 306 |
if not integrity["results_usable"]:
|
| 307 |
return {
|
| 308 |
"velocity_screen_flagged": _df_records(pd.DataFrame(), 30),
|
| 309 |
"storage_classification": _df_records(pd.DataFrame(), 30),
|
| 310 |
-
"criteria_register": _df_records(criteria_register(
|
| 311 |
"status": "Not assessed - hydraulic routing solution invalid",
|
| 312 |
"execution_integrity": integrity,
|
| 313 |
}
|
| 314 |
-
crit = CalgaryCriteria()
|
| 315 |
link_df = session.data["link_df"]
|
| 316 |
node_df = session.data["node_df"]
|
| 317 |
vel_col = next((c for c in link_df.columns if c.startswith("Peak Velocity")), None)
|
| 318 |
lv = link_df[["Link ID", vel_col, "Depth Ratio"]].copy()
|
| 319 |
lv["Screen"] = lv[vel_col].apply(
|
| 320 |
-
lambda v: "CRITICAL >
|
|
|
|
|
|
|
|
|
|
| 321 |
flagged = lv[lv["Screen"] != "OK"].sort_values(vel_col, ascending=False)
|
| 322 |
storage = node_df[node_df["Type"].astype(str).str.lower() == "storage"].copy()
|
| 323 |
storage_class = apply_storage_classification(storage, crit, "m") if not storage.empty else pd.DataFrame()
|
|
@@ -489,6 +550,87 @@ def set_report_details(session_id: str, details: dict | str) -> dict:
|
|
| 489 |
"next_step": "generate_report will include these details; readiness scoring reflects them."}
|
| 490 |
|
| 491 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 492 |
def generate_report(session_id: str, project_name: str, client: str = "",
|
| 493 |
consultant: str = "", prepared_by: str = "",
|
| 494 |
outline_plan_no: str = "",
|
|
@@ -502,7 +644,7 @@ def generate_report(session_id: str, project_name: str, client: str = "",
|
|
| 502 |
Returns download paths served by this Space at /files/{session_id}/{name}.
|
| 503 |
The draft-readiness score honestly reflects missing project information.
|
| 504 |
"""
|
| 505 |
-
from report_engine import ReportMetadata, generate_report_package
|
| 506 |
session = STORE.get(session_id)
|
| 507 |
_require_results(session)
|
| 508 |
stored = dict(session.data.get("report_details", {}))
|
|
@@ -514,11 +656,18 @@ def generate_report(session_id: str, project_name: str, client: str = "",
|
|
| 514 |
meta_kwargs[key] = value
|
| 515 |
narrative = {k: v for k, v in stored.items() if k in _NARRATIVE_FIELDS}
|
| 516 |
meta = ReportMetadata(project_name=project_name, **meta_kwargs)
|
|
|
|
| 517 |
model_listings = None
|
| 518 |
if include_model_appendix:
|
| 519 |
inp_p = Path(session.data["inp_path"])
|
| 520 |
model_listings = {"inp_name": inp_p.name,
|
| 521 |
"inp_text": inp_p.read_text(encoding="utf-8", errors="replace")}
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 522 |
rpt_p = (session.data["results"].get("metadata", {}) or {}).get("report_path")
|
| 523 |
if rpt_p and Path(str(rpt_p)).exists():
|
| 524 |
model_listings["rpt_name"] = Path(str(rpt_p)).name
|
|
@@ -529,6 +678,7 @@ def generate_report(session_id: str, project_name: str, client: str = "",
|
|
| 529 |
node_summary=session.data["node_df"], link_summary=session.data["link_df"],
|
| 530 |
sub_summary=session.data["sub_df"],
|
| 531 |
simulation_metadata=session.data["results"]["metadata"],
|
|
|
|
| 532 |
result_db_bytes=session.data["db"].export_bytes(),
|
| 533 |
preliminary_review_artifacts={
|
| 534 |
"findings": findings, "status": "Preliminary",
|
|
@@ -541,6 +691,10 @@ def generate_report(session_id: str, project_name: str, client: str = "",
|
|
| 541 |
"summary": session.data.get("recon_summary", {})},
|
| 542 |
model_identity={"inp_name": Path(session.data["inp_path"]).name,
|
| 543 |
"sha256": session.data["results"]["metadata"].get("model_sha256", "—"),
|
|
|
|
|
|
|
|
|
|
|
|
|
| 544 |
"session_id": session.id,
|
| 545 |
"run_id": session.data["results"]["metadata"].get("run_id", "—"),
|
| 546 |
"engine": "EPA SWMM / OpenSWMM 6 (crash-isolated worker, engine Rev 23.2)",
|
|
@@ -573,6 +727,7 @@ TOOL_REGISTRY: dict[str, Callable[..., dict]] = {
|
|
| 573 |
get_node_results, get_link_results, get_subcatchment_results,
|
| 574 |
get_timeseries, query_results, get_table_catalog,
|
| 575 |
calgary_screening, preliminary_design_review, get_reconciliation,
|
| 576 |
-
run_scenario, attach_figure, set_report_details,
|
|
|
|
| 577 |
]
|
| 578 |
}
|
|
|
|
| 33 |
from sessions import STORE
|
| 34 |
from sql_agent import SafeSQLAgent
|
| 35 |
from swmm_core import run_swmm
|
| 36 |
+
from screening_logic import execution_integrity_assessment, resolve_legacy_solver_options
|
| 37 |
|
| 38 |
MAX_ROWS = 60
|
| 39 |
MAX_TS_POINTS = 200
|
|
|
|
| 59 |
return {str(r[0]).upper(): str(r[1]) for r in sections.get("OPTIONS", []) if len(r) >= 2}
|
| 60 |
|
| 61 |
|
| 62 |
+
def _write_normalized_execution_copy(source: Path, substitutions: list[dict[str, Any]]) -> Path:
|
| 63 |
+
"""Write a derivative INP containing only audited legacy substitutions."""
|
| 64 |
+
replacements = {str(s["option"]).upper(): format(float(s["effective_value"]), "g")
|
| 65 |
+
for s in substitutions}
|
| 66 |
+
lines = source.read_text(encoding="utf-8", errors="replace").splitlines(keepends=True)
|
| 67 |
+
in_options = False
|
| 68 |
+
changed: set[str] = set()
|
| 69 |
+
for i, line in enumerate(lines):
|
| 70 |
+
stripped = line.strip()
|
| 71 |
+
if stripped.startswith("[") and stripped.endswith("]"):
|
| 72 |
+
in_options = stripped.upper() == "[OPTIONS]"
|
| 73 |
+
continue
|
| 74 |
+
if not in_options or not stripped or stripped.startswith(";"):
|
| 75 |
+
continue
|
| 76 |
+
parts = stripped.split()
|
| 77 |
+
key = parts[0].upper() if parts else ""
|
| 78 |
+
if key in replacements:
|
| 79 |
+
newline = "\r\n" if line.endswith("\r\n") else "\n"
|
| 80 |
+
lines[i] = f"{parts[0]:<20} {replacements[key]}{newline}"
|
| 81 |
+
changed.add(key)
|
| 82 |
+
missing = sorted(set(replacements) - changed)
|
| 83 |
+
if missing:
|
| 84 |
+
raise ValueError("Could not normalize legacy [OPTIONS] values: " + ", ".join(missing))
|
| 85 |
+
target = source.with_name(f"{source.stem}_legacy_defaults_normalized{source.suffix}")
|
| 86 |
+
target.write_text("".join(lines), encoding="utf-8", newline="")
|
| 87 |
+
return target
|
| 88 |
+
|
| 89 |
+
|
| 90 |
# ---------------------------------------------------------------------------
|
| 91 |
# Model lifecycle
|
| 92 |
# ---------------------------------------------------------------------------
|
|
|
|
| 136 |
if not inp_path:
|
| 137 |
raise ValueError("Session has no uploaded model.")
|
| 138 |
sections = session.data["sections"]
|
| 139 |
+
resolution = resolve_legacy_solver_options(_options_map(sections))
|
| 140 |
+
option_errors = resolution["errors"]
|
| 141 |
if option_errors:
|
| 142 |
session.data["input_validation_errors"] = option_errors
|
| 143 |
raise ValueError(
|
| 144 |
"Input validation failed; simulation was not run. " + " ".join(option_errors)
|
| 145 |
)
|
| 146 |
+
original_path = Path(inp_path)
|
| 147 |
+
substitutions = list(resolution["substitutions"])
|
| 148 |
+
execution_path = (_write_normalized_execution_copy(original_path, substitutions)
|
| 149 |
+
if substitutions else original_path)
|
| 150 |
+
results = run_swmm(str(execution_path))
|
| 151 |
md = results["metadata"]
|
| 152 |
import hashlib, datetime
|
| 153 |
+
original_sha256 = hashlib.sha256(original_path.read_bytes()).hexdigest()
|
| 154 |
+
execution_sha256 = hashlib.sha256(execution_path.read_bytes()).hexdigest()
|
| 155 |
run_id = datetime.datetime.utcnow().strftime("%Y%m%dT%H%M%SZ")
|
| 156 |
+
md["model_sha256"] = original_sha256
|
| 157 |
+
md["original_model_sha256"] = original_sha256
|
| 158 |
+
md["execution_model_sha256"] = execution_sha256
|
| 159 |
+
md["original_model_filename"] = original_path.name
|
| 160 |
+
md["execution_model_filename"] = execution_path.name
|
| 161 |
+
md["legacy_defaults_normalized"] = bool(substitutions)
|
| 162 |
+
md["solver_option_substitutions"] = substitutions
|
| 163 |
md["run_id"] = run_id
|
| 164 |
integrity = execution_integrity_assessment(md)
|
| 165 |
md.update({
|
|
|
|
| 180 |
if "Status" in link_df:
|
| 181 |
link_df["Status"] = invalid_label
|
| 182 |
db = ResultDatabase(str(session.workdir / "results.sqlite"))
|
| 183 |
+
db.load(node_df, link_df, sub_df, inp_path=original_path, results=results)
|
| 184 |
|
| 185 |
recon = {"verdict": "Not performed"}
|
| 186 |
recon_links = recon_nodes = recon_cont = None
|
|
|
|
| 193 |
|
| 194 |
session.data.update({
|
| 195 |
"results": results, "node_df": node_df, "link_df": link_df, "sub_df": sub_df,
|
| 196 |
+
"execution_inp_path": str(execution_path),
|
| 197 |
"db": db, "recon_links": recon_links, "recon_nodes": recon_nodes,
|
| 198 |
"recon_continuity": recon_cont, "recon_summary": recon,
|
| 199 |
})
|
|
|
|
| 204 |
"simulation": "completed" if integrity["results_usable"] else "completed_invalid",
|
| 205 |
"execution_integrity": integrity,
|
| 206 |
"results_usable": integrity["results_usable"],
|
| 207 |
+
"model_sha256": original_sha256,
|
| 208 |
+
"execution_model_sha256": execution_sha256,
|
| 209 |
+
"legacy_defaults_normalized": bool(substitutions),
|
| 210 |
+
"solver_option_substitutions": substitutions,
|
| 211 |
"run_id": run_id,
|
| 212 |
"flow_units": md.get("flow_units"),
|
| 213 |
"runoff_continuity_error_pct": round(float(md.get("runoff_error", 0.0)), 3),
|
|
|
|
| 346 |
"""
|
| 347 |
session = STORE.get(session_id)
|
| 348 |
_require_results(session)
|
| 349 |
+
cfg = dict(session.data.get("report_configuration", {}))
|
| 350 |
+
crit = CalgaryCriteria(
|
| 351 |
+
minor_release_rate_lps_ha=cfg.get("minor_release_rate_lps_ha"),
|
| 352 |
+
trap_low_max_depth_m=cfg.get("trap_low_max_depth_m", 0.50),
|
| 353 |
+
entrance_grade_margin_m=cfg.get("entrance_grade_margin_m", 0.30),
|
| 354 |
+
pipe_advisory_velocity_mps=cfg.get("velocity_advisory", 3.0),
|
| 355 |
+
pipe_critical_velocity_mps=cfg.get("velocity_threshold", 4.0),
|
| 356 |
+
conduit_capacity_review_ratio=cfg.get("conduit_depth_ratio", 0.80),
|
| 357 |
+
conduit_capacity_warning_ratio=cfg.get("conduit_capacity_warning_ratio", 0.95),
|
| 358 |
+
continuity_review_pct=cfg.get("continuity_review", 0.50),
|
| 359 |
+
continuity_warning_pct=cfg.get("continuity_warning", 1.00),
|
| 360 |
+
special_link_limits=cfg.get("special_link_limits", {}),
|
| 361 |
+
storage_classification=cfg.get("storage_classification", {}),
|
| 362 |
+
outfall_classification=cfg.get("outfall_classification", {}),
|
| 363 |
+
)
|
| 364 |
integrity = execution_integrity_assessment(session.data["results"].get("metadata", {}))
|
| 365 |
if not integrity["results_usable"]:
|
| 366 |
return {
|
| 367 |
"velocity_screen_flagged": _df_records(pd.DataFrame(), 30),
|
| 368 |
"storage_classification": _df_records(pd.DataFrame(), 30),
|
| 369 |
+
"criteria_register": _df_records(criteria_register(crit), 40),
|
| 370 |
"status": "Not assessed - hydraulic routing solution invalid",
|
| 371 |
"execution_integrity": integrity,
|
| 372 |
}
|
|
|
|
| 373 |
link_df = session.data["link_df"]
|
| 374 |
node_df = session.data["node_df"]
|
| 375 |
vel_col = next((c for c in link_df.columns if c.startswith("Peak Velocity")), None)
|
| 376 |
lv = link_df[["Link ID", vel_col, "Depth Ratio"]].copy()
|
| 377 |
lv["Screen"] = lv[vel_col].apply(
|
| 378 |
+
lambda v: (f"CRITICAL > {crit.pipe_critical_velocity_mps:g}"
|
| 379 |
+
if v > crit.pipe_critical_velocity_mps else
|
| 380 |
+
(f"Advisory > {crit.pipe_advisory_velocity_mps:g}"
|
| 381 |
+
if v > crit.pipe_advisory_velocity_mps else "OK")))
|
| 382 |
flagged = lv[lv["Screen"] != "OK"].sort_values(vel_col, ascending=False)
|
| 383 |
storage = node_df[node_df["Type"].astype(str).str.lower() == "storage"].copy()
|
| 384 |
storage_class = apply_storage_classification(storage, crit, "m") if not storage.empty else pd.DataFrame()
|
|
|
|
| 550 |
"next_step": "generate_report will include these details; readiness scoring reflects them."}
|
| 551 |
|
| 552 |
|
| 553 |
+
_REPORT_CONFIG_FIELDS = {
|
| 554 |
+
"node_depth_ratio", "minimum_freeboard", "conduit_depth_ratio",
|
| 555 |
+
"velocity_threshold", "velocity_advisory", "continuity_review",
|
| 556 |
+
"continuity_warning", "suppress_empty_sections", "major_link_ids",
|
| 557 |
+
"area_classification", "calgary_enabled", "minor_release_rate_lps_ha",
|
| 558 |
+
"trap_low_max_depth_m", "entrance_grade_margin_m",
|
| 559 |
+
"conduit_capacity_warning_ratio", "special_link_limits",
|
| 560 |
+
"storage_classification", "outfall_classification", "checklist_overrides",
|
| 561 |
+
"drawing_inventory", "applicable_reports",
|
| 562 |
+
}
|
| 563 |
+
_REPORT_CONFIG_LISTS = {"major_link_ids", "drawing_inventory", "applicable_reports"}
|
| 564 |
+
_REPORT_CONFIG_MAPS = {
|
| 565 |
+
"area_classification", "special_link_limits", "storage_classification",
|
| 566 |
+
"outfall_classification", "checklist_overrides",
|
| 567 |
+
}
|
| 568 |
+
_REPORT_CONFIG_BOOLS = {"suppress_empty_sections", "calgary_enabled"}
|
| 569 |
+
|
| 570 |
+
|
| 571 |
+
def set_report_configuration(session_id: str, configuration: dict | str) -> dict:
|
| 572 |
+
"""Set project-specific City of Calgary SWMR criteria and evidence inputs.
|
| 573 |
+
|
| 574 |
+
Use this after upload and before generate_report. Supported configuration
|
| 575 |
+
includes major_link_ids, project-specific flow limits, storage/outfall/area
|
| 576 |
+
classifications, numerical screening thresholds, drawing_inventory,
|
| 577 |
+
applicable_reports, and checklist_overrides. Repeated calls merge; JSON
|
| 578 |
+
null removes a field. This configures deterministic screening and report
|
| 579 |
+
completeness—it does not establish municipal acceptance or professional
|
| 580 |
+
authentication.
|
| 581 |
+
"""
|
| 582 |
+
session = STORE.get(session_id)
|
| 583 |
+
if isinstance(configuration, str):
|
| 584 |
+
configuration = json.loads(configuration)
|
| 585 |
+
if not isinstance(configuration, dict):
|
| 586 |
+
raise ValueError("configuration must be a JSON object.")
|
| 587 |
+
unknown = sorted(set(configuration) - _REPORT_CONFIG_FIELDS)
|
| 588 |
+
if unknown:
|
| 589 |
+
raise ValueError(f"Unknown report configuration field(s): {unknown}. "
|
| 590 |
+
f"Supported: {sorted(_REPORT_CONFIG_FIELDS)}")
|
| 591 |
+
normalized: dict[str, Any] = {}
|
| 592 |
+
for key, value in configuration.items():
|
| 593 |
+
if value is None:
|
| 594 |
+
normalized[key] = None
|
| 595 |
+
elif key in _REPORT_CONFIG_LISTS:
|
| 596 |
+
if not isinstance(value, (list, tuple)):
|
| 597 |
+
raise ValueError(f"{key} must be a JSON array.")
|
| 598 |
+
normalized[key] = tuple(str(x).strip() for x in value if str(x).strip())
|
| 599 |
+
elif key in _REPORT_CONFIG_MAPS:
|
| 600 |
+
if not isinstance(value, dict):
|
| 601 |
+
raise ValueError(f"{key} must be a JSON object.")
|
| 602 |
+
if key == "special_link_limits":
|
| 603 |
+
try:
|
| 604 |
+
normalized[key] = {str(k): float(v) for k, v in value.items()}
|
| 605 |
+
except (TypeError, ValueError):
|
| 606 |
+
raise ValueError("special_link_limits values must be numeric.")
|
| 607 |
+
else:
|
| 608 |
+
normalized[key] = {str(k): str(v) for k, v in value.items()}
|
| 609 |
+
elif key in _REPORT_CONFIG_BOOLS:
|
| 610 |
+
if not isinstance(value, bool):
|
| 611 |
+
raise ValueError(f"{key} must be true or false.")
|
| 612 |
+
normalized[key] = value
|
| 613 |
+
else:
|
| 614 |
+
if isinstance(value, bool):
|
| 615 |
+
raise ValueError(f"{key} must be numeric or null.")
|
| 616 |
+
try:
|
| 617 |
+
normalized[key] = float(value)
|
| 618 |
+
except (TypeError, ValueError):
|
| 619 |
+
raise ValueError(f"{key} must be numeric or null.")
|
| 620 |
+
stored = session.data.setdefault("report_configuration", {})
|
| 621 |
+
for key, value in normalized.items():
|
| 622 |
+
if value is None:
|
| 623 |
+
stored.pop(key, None)
|
| 624 |
+
else:
|
| 625 |
+
stored[key] = value
|
| 626 |
+
return {
|
| 627 |
+
"session_id": session.id,
|
| 628 |
+
"report_configuration": stored,
|
| 629 |
+
"next_step": "Run Calgary screening/review as needed, then call generate_report.",
|
| 630 |
+
"disclaimer": "Project criteria and classifications require responsible-engineer verification.",
|
| 631 |
+
}
|
| 632 |
+
|
| 633 |
+
|
| 634 |
def generate_report(session_id: str, project_name: str, client: str = "",
|
| 635 |
consultant: str = "", prepared_by: str = "",
|
| 636 |
outline_plan_no: str = "",
|
|
|
|
| 644 |
Returns download paths served by this Space at /files/{session_id}/{name}.
|
| 645 |
The draft-readiness score honestly reflects missing project information.
|
| 646 |
"""
|
| 647 |
+
from report_engine import ReportCriteria, ReportMetadata, generate_report_package
|
| 648 |
session = STORE.get(session_id)
|
| 649 |
_require_results(session)
|
| 650 |
stored = dict(session.data.get("report_details", {}))
|
|
|
|
| 656 |
meta_kwargs[key] = value
|
| 657 |
narrative = {k: v for k, v in stored.items() if k in _NARRATIVE_FIELDS}
|
| 658 |
meta = ReportMetadata(project_name=project_name, **meta_kwargs)
|
| 659 |
+
report_criteria = ReportCriteria(**dict(session.data.get("report_configuration", {})))
|
| 660 |
model_listings = None
|
| 661 |
if include_model_appendix:
|
| 662 |
inp_p = Path(session.data["inp_path"])
|
| 663 |
model_listings = {"inp_name": inp_p.name,
|
| 664 |
"inp_text": inp_p.read_text(encoding="utf-8", errors="replace")}
|
| 665 |
+
execution_p = Path(session.data.get("execution_inp_path", inp_p))
|
| 666 |
+
if execution_p != inp_p:
|
| 667 |
+
model_listings.update({
|
| 668 |
+
"execution_inp_name": execution_p.name,
|
| 669 |
+
"execution_inp_text": execution_p.read_text(encoding="utf-8", errors="replace"),
|
| 670 |
+
})
|
| 671 |
rpt_p = (session.data["results"].get("metadata", {}) or {}).get("report_path")
|
| 672 |
if rpt_p and Path(str(rpt_p)).exists():
|
| 673 |
model_listings["rpt_name"] = Path(str(rpt_p)).name
|
|
|
|
| 678 |
node_summary=session.data["node_df"], link_summary=session.data["link_df"],
|
| 679 |
sub_summary=session.data["sub_df"],
|
| 680 |
simulation_metadata=session.data["results"]["metadata"],
|
| 681 |
+
criteria=report_criteria,
|
| 682 |
result_db_bytes=session.data["db"].export_bytes(),
|
| 683 |
preliminary_review_artifacts={
|
| 684 |
"findings": findings, "status": "Preliminary",
|
|
|
|
| 691 |
"summary": session.data.get("recon_summary", {})},
|
| 692 |
model_identity={"inp_name": Path(session.data["inp_path"]).name,
|
| 693 |
"sha256": session.data["results"]["metadata"].get("model_sha256", "—"),
|
| 694 |
+
"execution_inp_name": session.data["results"]["metadata"].get("execution_model_filename"),
|
| 695 |
+
"execution_sha256": session.data["results"]["metadata"].get("execution_model_sha256"),
|
| 696 |
+
"legacy_defaults_normalized": session.data["results"]["metadata"].get("legacy_defaults_normalized", False),
|
| 697 |
+
"solver_option_substitutions": session.data["results"]["metadata"].get("solver_option_substitutions", []),
|
| 698 |
"session_id": session.id,
|
| 699 |
"run_id": session.data["results"]["metadata"].get("run_id", "—"),
|
| 700 |
"engine": "EPA SWMM / OpenSWMM 6 (crash-isolated worker, engine Rev 23.2)",
|
|
|
|
| 727 |
get_node_results, get_link_results, get_subcatchment_results,
|
| 728 |
get_timeseries, query_results, get_table_catalog,
|
| 729 |
calgary_screening, preliminary_design_review, get_reconciliation,
|
| 730 |
+
run_scenario, attach_figure, set_report_details, set_report_configuration,
|
| 731 |
+
generate_report,
|
| 732 |
]
|
| 733 |
}
|