feat: add host inventory diagnostics and admin history

dev
admin 2 months ago
parent fad3b4e4fd
commit 12e20d150f

@ -2,6 +2,8 @@
管理员入口:`/admin/operations`。沿用 Venix 管理主题,任务列表由业务数据库直接渲染;运行检查随后异步加载,不等待引擎才显示任务。
宿主机容量/IP 盘点的逐次成功、失败、超时与阶段线索使用独立 [盘点诊断页](HOST_INVENTORY_DIAGNOSTICS.md)(`/admin/host-inventory`)。系统运行页提供跳转;它与下面的后台进程心跳不是同一种健康信号。盘点诊断新增历史表,须按其文档运行 `cloudInit`,不适用本页心跳部分的“无需数据库迁移”。
## 查看范围
- 覆盖创建、电源操作、重装、救援、删除及到期、流量策略任务。

@ -29,12 +29,14 @@
## 失败语义
逐次尝试、分层失败和超时后的引擎线索见 [宿主机盘点诊断](HOST_INVENTORY_DIAGNOSTICS.md)。下表是原有健康/调度状态,不是底层错误的精确分类;多数引擎 HTTP 错误经旧客户端规则折叠后仍会显示 `unavailable`。
任何请求、响应校验或事务失败都保留最后一次容量数值与全部 IP 行,不写零值、不根据缺失响应推断资源已经释放:
| 场景 | host health | 固定诊断码 |
| --- | --- | --- |
| 连接、超时、限流或临时 5xx | `unavailable` | `host_inventory_unavailable` |
| 引擎拒绝读取 | `degraded` | `host_inventory_engine_rejected` |
| 保留为 4xx 的引擎拒绝(例如有效 JSON 404) | `degraded` | `host_inventory_engine_rejected` |
| 容量合同无效 | `degraded` | `host_inventory_capacity_invalid` |
| 网络/IP 合同无效或计数不一致 | `degraded` | `host_inventory_network_invalid` |
| 地址已属于另一宿主机 | `degraded` | `host_inventory_ip_conflict` |

@ -0,0 +1,69 @@
# 宿主机盘点诊断
管理员入口:`/admin/host-inventory`。计算节点列表、资源调度抽屉、系统运行页和管理员导航均提供入口。页面只读取记录,不会因打开、筛选、翻页或刷新而执行 SSH/libvirt 盘点。手动同步仍使用原有的宿主机“手动同步”操作。
## 记录与结果
每次真正开始的自动或手动同步生成一个独立的 32 位 `attempt_id`。业务库 `cloud_host_inventory_attempts` 保存开始/结束时间、来源、代际、实际请求预算、PHP 阶段耗时、固定错误码、上游 HTTP 状态和容量是否已提交;不复制容量响应或 IP 清单。摘要按宿主机统计,结果筛选仅影响列表。
- `succeeded`:两个响应通过校验且容量/IP 事务已经提交。
- `failed` / `timeout`:面板本次尝试未成功;不等于物理宿主机离线。
- `superseded`:新的同步代际已取代该结果,不能覆盖较新的容量或健康状态。
- `skipped`:尝试开始后发现宿主机不再允许同步,未发布容量。
- 超过 5 分钟未写终态的记录显示“中断或结果未知”,不伪造结束时间,也不推断容量一定未提交。
面板成功、失败时间由面板记录;耗时使用本进程单调时钟。引擎时间单独展示,不用跨机器时间差推断网络耗时。面板诊断版本是已加载代码的协议标识,不是磁盘上的 Git HEAD。
历史写入故障不会更改原有盘点结果;缺表或数据库不可写时,用固定请求编号/阶段/错误码的 PHP 服务日志补充诊断。面板完全不可用时仍需通过服务日志排查。
## 超时后的引擎线索
原有 `host/capacity`、`host/network-options` 请求附带 `inventory_attempt_id` 查询参数,整个查询参数受原有 HMAC 保护。一个面板尝试对应容量、网络两份引擎摘要。
引擎在进入可能阻塞的阶段前原子保存私有 JSON 摘要,覆盖:宿主机配置、全局锁等待、在线更新检查、ISO/镜像清理、libvirt 连接、宿主资源、VM 枚举/扫描、存储池、公网池目录及占用。嵌套阶段耗时包含内部阶段,不能直接相加。
详情页“读取引擎线索”仅调用独立的轻量查询:
```text
GET /api/v1/admin/host-inventory/{attempt_id}/engine
GET /internal/v1/hosts/{host_id}/host/inventory-diagnostics/{attempt_id}
```
查询有独立 1 秒引擎请求预算,绕过宿主应用加载、全局更新锁与远程清理;不重试完整盘点。PHP 仅接受经过白名单校验的固定字段。普通用户不可访问,响应禁止缓存。
PHP 超时和引擎最终结果分开展示。HTTP 等待结束不意味着底层工作停止;工作线程晚到的成功只补充引擎证据,不修改面板的超时记录或容量。同步盘点 worker 尚未启动就取消的请求标记为中断,结束时间保持未知;当时仍在执行的清理线程保留最后确认阶段,不声称线程已停止。引擎重启后的未终结摘要同样标记为中断/未知。
只有开始记录、没有结束记录时,页面显示的是“最后确认阶段”,不是已证明的根因。没有对应摘要可能是引擎未收到请求、版本未启用诊断、诊断存储不可用或超过保留期限,不应猜测为宿主故障。
## 错误判读
| 证据 | 能确认的范围 |
| --- | --- |
| `engine_http_timeout` | cURL 明确报告超时,或上游返回 408/504;内部原因需结合引擎摘要 |
| `engine_connection_or_timeout` | 客户端没有可可靠区分连接失败与超时的信息,例如部分非 cURL handler |
| `engine_auth_rejected` | 上游 401/403;检查内部身份/HMAC,不应直接认定宿主离线 |
| `engine_configuration_locked` / `engine_request_conflict` | 上游 423/409;结合阶段排查配置或状态冲突 |
| `storage_scan` + 404 | 失败涉及存储读取;不能把业务 404 当作接口版本不支持并重扫 |
| `capacity_validate` / `network_validate` | 面板未接受引擎返回的数据合同或容量/IP 一致性 |
| `database_apply` | 容量/IP 校验后的本地提交阶段失败或结果已被更新代际取代 |
历史健康映射保持不变,所以宿主上的通用 `unavailable` 与诊断页的细分类别可能不同。诊断不会放宽调度、覆盖预留、延长快照有效期,或重试创建/删除操作。
## 保留与开销
- 面板历史约保留 7 天,每宿主约 25000 条;在后续尝试中每次有界分批清理,不通过浏览页面清理。停止盘点的环境不会主动触发这一清理。
- 引擎全局最多 4000 份请求摘要、最长 7 天、每份最多 32 KiB;数量和时间先到者为准,因此旧面板记录可能已无引擎细节。目录为 `data_dir/inventory-diagnostics`,目录 `0700`、文件 `0600`,不增加第三个生产数据库。
- 不保存凭据、HMAC、libvirt URI、原始异常、完整响应、VM XML 或地址清单。
- 阶段持久化带来少量本地 I/O;应使用可靠的本地数据目录并实机测量开销。记录写入异常采用固定码降级,不吞掉原业务异常。
## 部署与验收
1. 使用现有安全更新流程部署引擎和 PHP;手工升级时在项目 `web/` 下运行 `php xcat cloudInit` 创建新历史表。不要覆盖已有业务数据库。
2. 按既有维护流程重启引擎、lifecycle,并确保 PHP-FPM/OPcache 加载新代码。不要为升级诊断强行打断在途破坏性操作。只执行 `git pull` 不能保证常驻进程已更新。
3. 打开诊断页。历史从启用后开始记录,不追溯补造过去失败。缺表时页面明确提示数据库升级,盘点本身仍沿用旧行为。
4. 对选定测试宿主手动同步,检查成功记录、两份引擎摘要、共同请求编号及容量已更新标志。
5. 在受控测试中验证慢请求/锁等待、认证失败、存储池错误、响应格式错误、数据库失败,以及 HTTP 超时后引擎晚到成功。失败时不能清空已有容量或释放已占用 IP。
本阶段保留原来两次 HTTP 请求、每请求 8 秒预算和全部扫描/调度行为;单连接轻量盘点属于后续依据诊断证据决定的优化。
测试资产:`CloudHostInventoryDiagnosticsTest.php`、`CloudInventoryEngineClientTest.php`、`CloudHostInventorySyncServiceTest.php`、`AdminHostInventoryViewTest.php`、`AdminHostInventoryBehaviorTest.mjs` 和 `engine/tests/test_inventory_diagnostics.py`。模拟和 SQLite/WASM 测试不替代真实 MariaDB、PHP-FPM/systemd 与 SSH/libvirt 联调。

@ -1,5 +1,18 @@
# 本版验证记录
## 2026-10-04:宿主机盘点诊断
本轮只增加诊断历史、管理员页面和跨 PHP/引擎阶段证据;每请求 8 秒预算、容量/IP 提交代际和调度判定不变。部署步骤见 [宿主机盘点诊断](HOST_INVENTORY_DIAGNOSTICS.md)。
- Python 3.12 全量隔离回归:**747 passed、10 skipped、68 subtests passed、1 failed**。新增诊断用例覆盖签名/权限、锁等待、存储失败降级、嵌套阶段、取消请求与 worker 晚完成、重启和保留上限。
- 唯一 Python 失败为 `test_app.py::test_native_public_pool_blocks_silent_default_network_change`:测试创建公网池返回 409“公网地址池目录存储不可用”,预期 200。在修改前的 `fad3b4e` 独立源码快照中复现同一失败;未在本轮扩大范围修改。
- 隔离 PHP 8.5.10/WASM、精确 `composer.lock` 依赖与 SQLite 测试通过:`CloudInventoryEngineClientTest`、`CloudHostInventoryDiagnosticsTest`、`CloudHostInventorySyncServiceTest`、`CloudHostInventorySyncContractTest`、`CloudInventoryRelationshipContractTest`、`CloudNetworkIpLedgerContractTest`、`AdminHostInventoryViewTest`、`AdminHostSchedulingContractTest`、`AdminOperationsViewTest`、`CloudRuntimeHeartbeatTest`。覆盖固定错误分类、HMAC 查询参数、历史缺表降级、超时留痕、过时代际、只读页面和字段脱敏。缺表用例的固定诊断降级日志为预期输出。
- 扩展 PHP 检查另发现两项既有失败,均在 `fad3b4e` 快照复现:`CloudProtocolTest.php:85` 的源码断言插值 `$actor`;`CloudAdminOperationsServiceTest.php:245` 的查询字段断言与原服务选择 `payload_cipher` 不一致。本轮未修改这两个测试或其既有服务。
- 新页面行为测试 `AdminHostInventoryBehaviorTest.mjs`、相关 PHP 语法、两份 JS 语法及 `git diff --check` 通过。Chrome 合成数据检查 1440/390 px 页面和详情抽屉:无页面错误或横向溢出;首次展示不调用 API,不扫描宿主机,只有显式点击才读取引擎线索。摘要、列表、详情统一使用面板时区。
- 未连接真实 MariaDB 或远程宿主机,没有验证生产 PHP-FPM/systemd、cURL 传输、SSH/libvirt 延迟或诊断持久化的实际 I/O 开销;不能据此声称用户当前 `unavailable` 的根因已确定。真实环境启用后,应依据新增记录验证失败层和最后确认阶段。
## 2026-09-27:集成基线
2026-09-27,集成代码的隔离验证,不代表生产验收。
- 当前 macOS 工作区 Python 3.12 回归:515 passed、10 skipped、55 subtests passed。覆盖原虚拟化逻辑及私有 API、签名/重放、多宿主机归属、持久幂等、一次性 VNC、退出后断权、带宽 XML、救援上传摘要,以及到期断网/恢复、同实例变更串行、多网卡 live+config 断网/恢复、退出救援保持计费断网、流量累计检查点和破坏性操作结果核对。生产仍按部署文档使用 Python 3.12 或更新的受支持版本复测。

@ -0,0 +1,406 @@
"""Durable inventory evidence must not mutate or rescan the observed host."""
from __future__ import annotations
import asyncio
import base64
import json
import os
import stat
import threading
from types import SimpleNamespace
from unittest.mock import AsyncMock, patch
import pytest
from test_engine_boundary import (
ADMIN, SECRET, USER, FakeUpdater, MemoryStore, boundary, call,
cloud_engine_request,
)
from virsh_panel.internal import EngineApplication
from virsh_panel.inventory_diagnostics import (
CURRENT, MAX_BYTES, MAX_STAGES, InventoryDiagnostics, inventory_result, inventory_stage,
)
from virsh_panel.libvirt_service import VMError, VMService
ATTEMPT = "a" * 32
OTHER = "b" * 32
CAPACITY = "/internal/v1/hosts/hk/host/capacity"
DIAGNOSTICS = f"/internal/v1/hosts/hk/host/inventory-diagnostics/{ATTEMPT}"
def test_endpoint_authentication_and_host_isolation(boundary):
client, engine, *_ = boundary
assert client.get(DIAGNOSTICS).status_code == 401
assert call(client, "GET", DIAGNOSTICS, USER).status_code == 403
assert call(client, "GET", CAPACITY + "?inventory_attempt_id=" + ATTEMPT, USER).status_code == 403
assert engine.inventory_diagnostics.read("hk", ATTEMPT)["traces"] == []
assert call(client, "GET", CAPACITY + "?inventory_attempt_id=" + ATTEMPT, ADMIN).status_code == 200
assert call(client, "GET", "/internal/v1/hosts/hk/host/network-options?inventory_attempt_id=" + ATTEMPT, ADMIN).status_code == 200
result = call(client, "GET", DIAGNOSTICS, ADMIN).json()
assert result["diagnostics_available"] is True
assert [item["resource"] for item in result["traces"]] == ["capacity", "network"]
assert all(item["state"] == "succeeded" for item in result["traces"])
assert all(item["http_status"] == 200 for item in result["traces"])
assert "public_ip_claims" in [item["stage"] for item in result["traces"][0]["stages"]]
for secret in ("host.example.test", "SHA256:example", "/fake/config", "_process", "_started"):
assert secret not in json.dumps(result)
assert call(client, "GET", DIAGNOSTICS.replace("/hk/", "/other/"), ADMIN).json()["traces"] == []
@pytest.mark.parametrize("query", ["", "A" * 32, "../bad", "a" * 31, ATTEMPT + "&inventory_attempt_id=" + OTHER])
def test_invalid_correlation_is_rejected_after_authentication(boundary, query):
client, engine, *_ = boundary
assert call(client, "GET", CAPACITY + "?inventory_attempt_id=" + query, ADMIN).status_code == 400
assert engine.inventory_diagnostics.read("hk", ATTEMPT)["traces"] == []
def test_query_cannot_be_added_to_unsigned_target(boundary):
client, engine, *_ = boundary
from virsh_panel.engine_security import signed_headers
headers = signed_headers(SECRET, "GET", CAPACITY, b"", ADMIN)
result = client.get(CAPACITY + "?inventory_attempt_id=" + ATTEMPT, headers=headers)
assert result.status_code == 401
assert engine.inventory_diagnostics.read("hk", ATTEMPT)["traces"] == []
def test_read_bypasses_registry_lock_and_gc(boundary):
client, engine, *_ = boundary
with patch.object(engine.registry, "get", side_effect=AssertionError("must not load host")), \
patch.object(engine, "_update_lock", side_effect=AssertionError("must not wait")), \
patch.object(engine, "_cleanup_host_user_iso_tombstones", side_effect=AssertionError("must not gc")), \
patch.object(engine, "_cleanup_host_cloud_template_tombstones", side_effect=AssertionError("must not gc")):
assert call(client, "GET", DIAGNOSTICS, ADMIN).status_code == 200
def test_failed_profile_records_fixed_stage_without_secret(boundary):
client, engine, *_ = boundary
assert call(client, "GET", CAPACITY, ADMIN).status_code == 200
engine.registry.components["hk"].manager.profile["key_locked"] = True
assert call(client, "GET", CAPACITY + "?inventory_attempt_id=" + ATTEMPT, ADMIN).status_code == 423
trace = call(client, "GET", DIAGNOSTICS, ADMIN).json()["traces"][0]
assert (trace["state"], trace["stage"], trace["error_code"], trace["http_status"]) == (
"failed", "profile", "profile_failed", 423)
assert trace["stages"][-1]["state"] == "failed"
def make_engine(tmp_path, app):
registry = SimpleNamespace(get=AsyncMock(return_value=app))
engine = EngineApplication(
{"service_secret": base64.b64encode(SECRET).decode(), "data_dir": str(tmp_path)},
MemoryStore(), MemoryStore(), registry=registry, updater=FakeUpdater(),
)
engine._cleanup_host_user_iso_tombstones = AsyncMock()
engine._cleanup_host_cloud_template_tombstones = AsyncMock()
return engine
def test_lock_wait_is_persisted_before_capacity_endpoint(tmp_path):
async def scenario():
called = False
async def endpoint(scope, receive, send):
nonlocal called
called = True
await send({"type": "http.response.start", "status": 200})
await send({"type": "http.response.body", "body": b"{}"})
engine = make_engine(tmp_path, endpoint)
lock = engine._update_lock()
await lock.acquire()
request = asyncio.create_task(cloud_engine_request(
engine, "GET", CAPACITY + "?inventory_attempt_id=" + ATTEMPT, b"", ADMIN))
try:
for _ in range(200):
traces = engine.inventory_diagnostics.read("hk", ATTEMPT)["traces"]
if traces and traces[0]["stage"] == "update_lock_wait":
break
await asyncio.sleep(0.001)
else:
pytest.fail("lock wait not persisted")
status, result = await asyncio.wait_for(cloud_engine_request(
engine, "GET", DIAGNOSTICS, b"", ADMIN), timeout=1)
assert status == 200
assert result["traces"][0]["state"] == "running"
assert result["traces"][0]["stages"][-1]["state"] == "running"
assert called is False
finally:
lock.release()
await request
assert engine.inventory_diagnostics.read("hk", ATTEMPT)["traces"][0]["state"] == "succeeded"
asyncio.run(scenario())
def test_cancellation_during_lock_wait_is_interrupted_not_forever_running(tmp_path):
async def scenario():
endpoint = AsyncMock()
engine = make_engine(tmp_path, endpoint)
lock = engine._update_lock()
await lock.acquire()
request = asyncio.create_task(cloud_engine_request(
engine, "GET", CAPACITY + "?inventory_attempt_id=" + ATTEMPT, b"", ADMIN))
try:
for _ in range(200):
traces = engine.inventory_diagnostics.read("hk", ATTEMPT)["traces"]
if traces and traces[0]["stage"] == "update_lock_wait":
break
await asyncio.sleep(0.001)
else:
pytest.fail("lock wait not persisted")
request.cancel()
with pytest.raises(asyncio.CancelledError):
await request
result = engine.inventory_diagnostics.read("hk", ATTEMPT)["traces"][0]
assert (result["state"], result["error_code"]) == ("interrupted", "request_interrupted")
assert result["stage"] == "update_lock_wait"
assert result["stages"][-1]["state"] == "interrupted"
assert result["finished_at"] is None and result["http_status"] is None
endpoint.assert_not_called()
finally:
lock.release()
asyncio.run(scenario())
def test_cancellation_during_gc_does_not_claim_worker_stopped(tmp_path):
async def scenario():
started, release, completed = threading.Event(), threading.Event(), threading.Event()
def gc_worker():
try:
with inventory_stage("libvirt_connect"):
started.set()
assert release.wait(2)
finally:
completed.set()
engine = make_engine(tmp_path, AsyncMock())
async def cleanup(_host):
await asyncio.to_thread(gc_worker)
engine._cleanup_host_user_iso_tombstones = cleanup
request = asyncio.create_task(cloud_engine_request(
engine, "GET", CAPACITY + "?inventory_attempt_id=" + ATTEMPT, b"", ADMIN))
try:
assert await asyncio.to_thread(started.wait, 1)
request.cancel()
with pytest.raises(asyncio.CancelledError):
await request
result = engine.inventory_diagnostics.read("hk", ATTEMPT)["traces"][0]
assert result["state"] == "interrupted"
assert result["finished_at"] is None and result["http_status"] is None
assert result["stage"] == "libvirt_connect"
assert result["stages"][-1]["state"] == "running"
assert result["stages"][-2]["state"] == "interrupted"
assert not completed.is_set()
finally:
release.set()
assert await asyncio.to_thread(completed.wait, 1)
result = engine.inventory_diagnostics.read("hk", ATTEMPT)["traces"][0]
assert result["state"] == "interrupted" # GC completion is not inventory success
assert result["finished_at"] is None
assert result["stages"][-1]["state"] == "succeeded"
asyncio.run(scenario())
@pytest.mark.parametrize("name", ["registry_profile", "iso_gc", "template_gc"])
def test_pre_dispatch_failures_record_only_fixed_codes(tmp_path, name):
async def scenario():
engine = make_engine(tmp_path, AsyncMock())
target = engine.registry if name == "registry_profile" else engine
attribute = {"registry_profile": "get", "iso_gc": "_cleanup_host_user_iso_tombstones",
"template_gc": "_cleanup_host_cloud_template_tombstones"}[name]
setattr(target, attribute, AsyncMock(side_effect=RuntimeError("SECRET-password-/private/key")))
status, _ = await cloud_engine_request(
engine, "GET", CAPACITY + "?inventory_attempt_id=" + ATTEMPT, b"", ADMIN)
assert status == 503
trace = engine.inventory_diagnostics.read("hk", ATTEMPT)["traces"][0]
assert trace["stage"] == name
assert trace["error_code"] == name + "_failed"
assert "SECRET" not in json.dumps(trace)
asyncio.run(scenario())
def test_worker_finishes_after_http_waiter_cancelled(tmp_path):
async def scenario():
started, release = threading.Event(), threading.Event()
@inventory_result
def worker():
with inventory_stage("domain_scan"):
started.set()
assert release.wait(2)
return {}
async def endpoint(scope, receive, send):
await asyncio.to_thread(worker)
await send({"type": "http.response.start", "status": 200})
engine = make_engine(tmp_path, endpoint)
request = asyncio.create_task(cloud_engine_request(
engine, "GET", CAPACITY + "?inventory_attempt_id=" + ATTEMPT, b"", ADMIN))
try:
assert await asyncio.to_thread(started.wait, 1)
request.cancel()
with pytest.raises(asyncio.CancelledError):
await request
trace = engine.inventory_diagnostics.read("hk", ATTEMPT)["traces"][0]
assert (trace["state"], trace["stage"]) == ("running", "domain_scan")
finally:
release.set()
for _ in range(200):
trace = engine.inventory_diagnostics.read("hk", ATTEMPT)["traces"][0]
if trace["state"] == "succeeded":
break
await asyncio.sleep(0.001)
assert trace["state"] == "succeeded"
asyncio.run(scenario())
def test_queued_worker_starting_after_waiter_cancelled_records_real_result(tmp_path):
store = InventoryDiagnostics(tmp_path, "unknown")
trace = store.begin("hk", ATTEMPT, "capacity")
trace.waiter_cancelled()
assert trace.value["state"] == "interrupted"
token = CURRENT.set(trace)
try:
@inventory_result
def worker():
with inventory_stage("host_info"):
assert trace.value["state"] == "running"
return {}
worker()
finally:
CURRENT.reset(token)
result = store.read("hk", ATTEMPT)["traces"][0]
assert result["state"] == "succeeded"
assert result["error_code"] == ""
assert result["finished_at"] is not None
def test_atomic_summary_permissions_retention_and_restart(tmp_path):
store = InventoryDiagnostics(tmp_path, "a" * 40, max_traces=2)
first = store.begin("hk", ATTEMPT, "capacity")
first.start("domain_scan")
path = store._path("hk", ATTEMPT, "capacity")
assert stat.S_IMODE(store.directory.stat().st_mode) == 0o700
assert stat.S_IMODE(path.stat().st_mode) == 0o600
assert path.stat().st_size < MAX_BYTES
assert store.begin("hk", ATTEMPT, "capacity") is None
restarted = InventoryDiagnostics(tmp_path, "b" * 40, process_id="another-process")
trace = restarted.read("hk", ATTEMPT)["traces"][0]
assert trace["state"] == "interrupted"
assert trace["error_code"] == "request_interrupted"
assert trace["finished_at"] is None
assert trace["engine_version"] == "a" * 40
store.begin("hk", OTHER, "capacity").finish()
store.begin("other", OTHER, "capacity").finish()
assert len(list(store.directory.glob("*.json"))) == 2
assert store.read("hk", ATTEMPT)["traces"] == []
first.finish()
assert not path.exists() # evicted late work cannot exceed retention bound
for current in store.directory.glob("*.json"):
os.utime(current, (1, 1))
store.last_pruned = 0
store.begin("hk", "c" * 32, "capacity").finish()
assert len(list(store.directory.glob("*.json"))) == 1
def test_storage_failure_does_not_fail_inventory(boundary):
client, engine, *_ = boundary
with patch.object(engine.inventory_diagnostics, "_write", side_effect=OSError("SECRET path")):
result = call(client, "GET", CAPACITY + "?inventory_attempt_id=" + ATTEMPT, ADMIN)
assert result.status_code == 200
assert call(client, "GET", DIAGNOSTICS, ADMIN).json()["traces"] == []
def test_real_service_stages_and_translated_http_status(tmp_path):
from virsh_panel import libvirt_service
connection = SimpleNamespace(
getInfo=lambda: ("x86_64", 8192, 4), listAllDomains=lambda _: [],
storagePoolLookupByName=lambda _: SimpleNamespace(isActive=lambda: True, info=lambda: (0, 100, 0, 100)),
close=lambda: None,
)
store = InventoryDiagnostics(tmp_path, "unknown")
trace = store.begin("hk", ATTEMPT, "capacity")
token = CURRENT.set(trace)
try:
with patch.object(libvirt_service.libvirt, "open", return_value=connection, create=True):
VMService("SECRET-uri", "images", "iso", "default").host_capacity()
trace.finish()
finally:
CURRENT.reset(token)
stages = [item["stage"] for item in store.read("hk", ATTEMPT)["traces"][0]["stages"]]
assert stages == ["libvirt_connect", "host_info", "domain_list", "domain_scan", "storage_scan"]
trace = store.begin("hk", OTHER, "capacity")
token = CURRENT.set(trace)
try:
@inventory_result
def fail():
VMService("SECRET-uri", "images", "iso", "default").host_capacity()
with patch.object(libvirt_service.libvirt, "open", side_effect=libvirt_service.libvirt.libvirtError("SECRET"), create=True):
with pytest.raises(VMError):
fail()
finally:
CURRENT.reset(token)
result = store.read("hk", OTHER)["traces"][0]
assert (result["stage"], result["http_status"]) == ("libvirt_connect", 502)
assert "SECRET" not in json.dumps(result)
def test_nested_failed_span_does_not_fail_caught_gc_request(tmp_path):
store = InventoryDiagnostics(tmp_path, "unknown")
trace = store.begin("hk", ATTEMPT, "capacity")
token = CURRENT.set(trace)
try:
with inventory_stage("iso_gc"):
try:
with inventory_stage("libvirt_connect"):
running = store.read("hk", ATTEMPT)["traces"][0]
assert [item["state"] for item in running["stages"]] == ["running", "running"]
raise VMError("secret connection error", 502)
except VMError:
pass
assert trace.value["state"] == "running"
with inventory_stage("host_info"):
pass
trace.finish(200)
finally:
CURRENT.reset(token)
result = store.read("hk", ATTEMPT)["traces"][0]
assert result["state"] == "succeeded"
assert result["error_code"] == ""
assert [item["state"] for item in result["stages"]] == ["succeeded", "failed", "succeeded"]
assert result["stages"][0]["duration_ms"] >= result["stages"][1]["duration_ms"]
def test_escaping_nested_failure_preserves_deepest_stage(tmp_path):
store = InventoryDiagnostics(tmp_path, "unknown")
trace = store.begin("hk", ATTEMPT, "capacity")
token = CURRENT.set(trace)
try:
with pytest.raises(VMError):
with inventory_stage("iso_gc"):
try:
with inventory_stage("libvirt_connect"):
raise RuntimeError("secret")
except RuntimeError as exc:
raise VMError("redacted", 502) from exc
trace.finish(502)
finally:
CURRENT.reset(token)
result = store.read("hk", ATTEMPT)["traces"][0]
assert result["state"] == "failed"
assert result["stage"] == "libvirt_connect"
assert [item["state"] for item in result["stages"]] == ["failed", "failed"]
def test_stage_limit_keeps_latest_blocking_evidence(tmp_path):
store = InventoryDiagnostics(tmp_path, "unknown")
trace = store.begin("hk", ATTEMPT, "capacity")
token = CURRENT.set(trace)
try:
with inventory_stage("iso_gc"):
for _ in range(MAX_STAGES + 5):
with inventory_stage("libvirt_connect"):
pass
item = trace.start("storage_scan")
result = store.read("hk", ATTEMPT)["traces"][0]
assert len(result["stages"]) == MAX_STAGES
assert result["stages"][-1]["stage"] == "storage_scan"
assert result["stages"][-1]["state"] == "running"
trace.end(item)
trace.finish(200)
finally:
CURRENT.reset(token)

@ -42,6 +42,7 @@ from .console import ConsoleRelayError, relay_console
from .database import DatabaseError, make_components
from .database_bootstrap import DatabaseBootstrap, DatabaseSetupError
from .libvirt_service import VMError, VMService
from .inventory_diagnostics import inventory_result, inventory_stage
from .metrics import (MAX_QUERY_POINTS, RETENTION_SECONDS, RANGES, MetricsCollector,
host_scope, query_interval_seconds)
from .panel_state import PanelState, SetupError
@ -754,14 +755,17 @@ def create_app(
raise VMError("公网地址池中没有可分配的 IP,请检查已用和保留地址", 409)
def public_network_options(profile: dict, service: VMService) -> dict:
pools, compatibility_mode = public_network_pools(profile)
with inventory_stage("public_pool_catalog"):
pools, compatibility_mode = public_network_pools(profile)
existing_vms = service.list_vms() if pools else []
claimed_addresses = public_ip_claim_addresses(profile)
inventories = [public_network_inventory(
profile, pool, service,
existing_vms=existing_vms,
claimed_addresses=claimed_addresses,
) for pool in pools]
with inventory_stage("public_ip_claims"):
claimed_addresses = public_ip_claim_addresses(profile)
with inventory_stage("public_ip_scan"):
inventories = [public_network_inventory(
profile, pool, service,
existing_vms=existing_vms,
claimed_addresses=claimed_addresses,
) for pool in pools]
public_items = []
for inventory in inventories:
available_ips = [
@ -809,14 +813,17 @@ def create_app(
}
def public_ip_capacity(profile: dict, service: VMService) -> dict:
pools, compatibility_mode = public_network_pools(profile)
with inventory_stage("public_pool_catalog"):
pools, compatibility_mode = public_network_pools(profile)
existing_vms = service.list_vms() if pools else []
claimed_addresses = public_ip_claim_addresses(profile)
inventories = [public_network_inventory(
profile, pool, service,
existing_vms=existing_vms,
claimed_addresses=claimed_addresses,
) for pool in pools]
with inventory_stage("public_ip_claims"):
claimed_addresses = public_ip_claim_addresses(profile)
with inventory_stage("public_ip_scan"):
inventories = [public_network_inventory(
profile, pool, service,
existing_vms=existing_vms,
claimed_addresses=claimed_addresses,
) for pool in pools]
public_items = [{
"pool_key": item["pool_key"],
"revision": item["revision"],
@ -1885,9 +1892,12 @@ def create_app(
return _public_host(manager.current())
@app.get("/api/v1/host/network-options")
@inventory_result
def host_network_options(_session: Session = Depends(require_admin)):
profile = current_profile()
return public_network_options(profile, current_service())
with inventory_stage("profile"):
profile = current_profile()
service = current_service()
return public_network_options(profile, service)
@app.get("/api/v1/host/network-pools")
def host_network_pools(_session: Session = Depends(require_admin)):
@ -1925,10 +1935,12 @@ def create_app(
}
@app.get("/api/v1/host/capacity")
@inventory_result
def host_capacity(_session: Session = Depends(require_admin)):
"""Return the scheduler's non-secret observation for this host."""
profile = current_profile()
service = new_service(profile)
with inventory_stage("profile"):
profile = current_profile()
service = new_service(profile)
capacity = service.host_capacity()
return {
"health": "healthy",

@ -47,6 +47,10 @@ from .engine_context import (EngineComponents, HostStore, LogStore, MainStore,
TRUSTED_ACTOR, actor_session)
from .engine_security import (EngineAuthError, authenticate, decode_secret,
signed_headers)
from .inventory_diagnostics import (
ATTEMPT_ID, CURRENT as INVENTORY_TRACE, InventoryDiagnostics,
engine_version, inventory_stage,
)
from .updater import UpdateError, Updater
from .user_isos import UserISORuntime
@ -59,6 +63,7 @@ JSON_LIMIT = 1024 * 1024
CLOUD_UPLOAD_IDLE_SECONDS = 60
CLOUD_UPLOAD_WRITE_BYTES = 1024 * 1024
READ_PATHS = (
r"host/inventory-diagnostics/[0-9a-f]{32}",
r"host", r"host/network-options", r"host/network-pools", r"host/capacity", r"resources", r"vm-templates", r"rescue-templates", r"user-isos", r"vms",
r"vm-states", r"vm-metadata",
r"user-isos/[0-9a-f]{32}",
@ -277,6 +282,8 @@ class EngineApplication:
self.registry = registry or HostRegistry(config, store, log_store)
self.introspection = introspection
project_dir = Path(config.get("project_dir", Path(__file__).resolve().parents[2]))
self.inventory_diagnostics = InventoryDiagnostics(
Path(config["data_dir"]), engine_version(project_dir))
self.updater = updater or Updater(
project_dir, store, worker_active=self._update_service_active)
self.update_trigger = update_trigger or self._start_update_service
@ -322,11 +329,15 @@ class EngineApplication:
if scope["type"] != "http":
return
token = None
inventory_token = None
inventory_trace = None
response_started = False
async def tracked_send(message):
nonlocal response_started
if message["type"] == "http.response.start":
response_started = True
if inventory_trace:
inventory_trace.finish(message["status"])
await send(message)
try:
# Public reverse proxies must never forward these routes. Reject
@ -395,18 +406,47 @@ class EngineApplication:
return await receive()
child_receive = replay_receive
token = TRUSTED_ACTOR.set(delegated)
inventory_match = re.fullmatch(
r"/internal/v1/hosts/([A-Za-z0-9][A-Za-z0-9_-]{0,63})/host/(capacity|network-options)",
scope["path"],
)
if scope["method"] == "GET" and inventory_match:
# The query is part of the authenticated raw target. Never
# let unauthenticated or non-admin callers inject evidence.
self._authorize_route("GET", "host/" + inventory_match[2], delegated)
query = parse_qsl(scope.get("query_string", b"").decode("ascii"), keep_blank_values=True)
identifiers = [value for key, value in query if key == "inventory_attempt_id"]
if identifiers:
if len(identifiers) != 1 or not ATTEMPT_ID.fullmatch(identifiers[0]):
raise EngineAuthError("盘点请求编号无效", 400)
inventory_trace = self.inventory_diagnostics.begin(
inventory_match[1], identifiers[0],
"capacity" if inventory_match[2] == "capacity" else "network")
inventory_token = INVENTORY_TRACE.set(inventory_trace)
await self._dispatch(scope, child_receive, tracked_send, body, delegated)
except asyncio.CancelledError:
if inventory_trace:
inventory_trace.waiter_cancelled()
raise
except EngineAuthError as exc:
if inventory_trace:
inventory_trace.finish(exc.status)
if not response_started:
await json_response(send, exc.status, {"error": str(exc)})
except UpdateError as exc:
if inventory_trace:
inventory_trace.finish(exc.status)
if not response_started:
await json_response(send, exc.status, {"error": str(exc)})
except Exception:
if inventory_trace:
inventory_trace.finish(503)
LOG.exception("Private engine request failed")
if not response_started:
await json_response(send, 503, {"error": "引擎服务暂时不可用,请检查引擎日志"})
finally:
if inventory_token is not None:
INVENTORY_TRACE.reset(inventory_token)
if token is not None:
TRUSTED_ACTOR.reset(token)
@ -520,15 +560,31 @@ class EngineApplication:
# catalog appear to have host-local semantics.
raise EngineAuthError("内部资源不存在", 404)
self._authorize_route(scope["method"], resource, delegated)
app = await self.registry.get(host_id)
if scope["method"] == "GET" and resource.startswith("host/inventory-diagnostics/"):
# Independent evidence retrieval must work while registry restore,
# maintenance locks or libvirt are blocked. It never scans a host.
attempt_id = resource.rsplit("/", 1)[1]
result = await asyncio.to_thread(self.inventory_diagnostics.read, host_id, attempt_id)
await json_response(send, 200, result)
return
with inventory_stage("registry_profile"):
app = await self.registry.get(host_id)
if scope["method"] == "GET":
# Reads normally bypass the update coordination lock, but lazy
# cache GC mutates libvirt. Coordinate that maintenance work with
# online update and skip it while an update is already active.
async with self._update_lock():
if not await asyncio.to_thread(self.updater.active_job):
await self._cleanup_host_user_iso_tombstones(host_id)
await self._cleanup_host_cloud_template_tombstones(host_id)
with inventory_stage("update_lock_wait"):
await self._update_lock().acquire()
try:
with inventory_stage("update_check"):
update_active = await asyncio.to_thread(self.updater.active_job)
if not update_active:
with inventory_stage("iso_gc"):
await self._cleanup_host_user_iso_tombstones(host_id)
with inventory_stage("template_gc"):
await self._cleanup_host_cloud_template_tombstones(host_id)
finally:
self._update_lock().release()
else:
# Every host write reached this method while holding _update_lock
# and after the active-update gate in _dispatch().

@ -0,0 +1,370 @@
"""Bounded, non-secret inventory evidence, independent of HTTP caller lifetime.
Only a signed administrator request can install CURRENT. No resource names,
addresses, exception text or response bodies are accepted by this recorder.
Atomic private summaries publish complete stage transitions before blocking work.
"""
from __future__ import annotations
import asyncio
import contextvars
import functools
import hashlib
import inspect
import json
import logging
import os
import re
import stat
import subprocess
import tempfile
import threading
import time
import uuid
from collections import deque
from contextlib import contextmanager
from pathlib import Path
LOG = logging.getLogger(__name__)
ATTEMPT_ID = re.compile(r"[0-9a-f]{32}\Z")
STAGES = frozenset({
"request", "registry_profile", "update_lock_wait", "update_check",
"iso_gc", "template_gc", "profile", "libvirt_connect", "host_info",
"domain_list", "domain_scan", "storage_scan", "public_pool_catalog",
"public_ip_claims", "public_ip_scan", "response",
})
PROCESS_ID = uuid.uuid4().hex
CURRENT: contextvars.ContextVar = contextvars.ContextVar("inventory_trace", default=None)
MAX_TRACES = 4000
RETENTION_SECONDS = 7 * 86400
MAX_STAGES = 48
MAX_BYTES = 32768
def engine_version(project_dir: Path) -> str:
"""Capture the loaded process revision once, never reread on each request."""
try:
result = subprocess.run(
["git", "rev-parse", "HEAD"], cwd=project_dir, capture_output=True,
text=True, timeout=1, check=True,
).stdout.strip()
return result if re.fullmatch(r"[0-9a-f]{40,64}", result) else "unknown"
except (OSError, ValueError, subprocess.SubprocessError):
return "unknown"
class InventoryDiagnostics:
def __init__(self, data_dir: Path, version: str, *, process_id=PROCESS_ID,
max_traces=MAX_TRACES, retention_seconds=RETENTION_SECONDS):
self.directory = data_dir / "inventory-diagnostics"
self.version = version
self.process_id = process_id
self.max_traces = max_traces
self.retention_seconds = retention_seconds
self.lock = threading.RLock()
self.records = deque()
self.last_pruned = 0.0
self.ready = False
try:
self.directory.mkdir(mode=0o700, parents=True, exist_ok=True)
info = self.directory.lstat()
if not stat.S_ISDIR(info.st_mode) or info.st_uid != os.geteuid():
raise OSError("unsafe directory")
self.directory.chmod(0o700)
self._prune(self.max_traces)
self.ready = True
except Exception:
LOG.warning("inventory_diagnostics_storage_unavailable")
def _path(self, host, attempt, resource):
key = hashlib.sha256((host + "\0" + attempt + "\0" + resource).encode()).hexdigest()
return self.directory / (key + ".json")
def _prune(self, limit):
records = []
now = time.time()
for path in self.directory.iterdir():
if not re.fullmatch(r"[0-9a-f]{64}\.json", path.name):
# Clean only our interrupted atomic-write temporary files.
if path.name.startswith(".inventory-") and path.lstat().st_mtime < now - 3600:
path.unlink(missing_ok=True)
continue
info = path.lstat()
if stat.S_ISREG(info.st_mode):
if info.st_mtime < now - self.retention_seconds:
path.unlink(missing_ok=True)
else:
records.append((info.st_mtime, path))
for _, path in sorted(records, reverse=True)[limit:]:
path.unlink(missing_ok=True)
self.records = deque(path for _, path in sorted(records, reverse=True)[:limit][::-1])
self.last_pruned = time.monotonic()
def _write(self, path, value):
payload = json.dumps(value, separators=(",", ":")).encode()
if len(payload) > MAX_BYTES:
raise ValueError("summary too large")
descriptor, temporary = tempfile.mkstemp(prefix=".inventory-", dir=self.directory)
try:
with os.fdopen(descriptor, "wb") as output:
os.fchmod(output.fileno(), 0o600)
output.write(payload)
output.flush()
os.fsync(output.fileno())
os.replace(temporary, path)
finally:
Path(temporary).unlink(missing_ok=True)
def begin(self, host: str, attempt: str, resource: str):
if not self.ready:
return None
trace = InventoryTrace(self, host, attempt, resource)
try:
with self.lock:
path = self._path(host, attempt, resource)
# Never overwrite evidence from a replayed attempt/resource.
if path.exists() or path.is_symlink():
return None
# The directory is engine-private and has one supported
# writer process. Keep a bounded queue rather than scanning
# thousands of files before every inventory request.
if time.monotonic() - self.last_pruned >= 60:
self._prune(self.max_traces)
while len(self.records) >= self.max_traces:
self.records.popleft().unlink(missing_ok=True)
self._write(path, trace.value)
self.records.append(path)
return trace
except Exception:
LOG.warning("inventory_diagnostics_write_failed attempt=%s resource=%s", attempt, resource)
return None
def save(self, trace):
try:
with self.lock:
if trace.value["state"] == "running":
trace.value["duration_ms"] = max(0, int((time.monotonic() - trace.value["_started"]) * 1000))
path = self._path(trace.host, trace.attempt, trace.value["resource"])
# A retention-evicted request cannot resurrect its old record.
if path.exists() and not path.is_symlink():
self._write(path, trace.value)
except Exception:
LOG.warning("inventory_diagnostics_write_failed attempt=%s resource=%s stage=%s",
trace.attempt, trace.value["resource"], trace.value["stage"])
def read(self, host: str, attempt: str) -> dict:
if not self.ready:
return {"attempt_id": attempt, "traces": [], "diagnostics_available": False}
try:
values = []
with self.lock:
for resource in ("capacity", "network"):
path = self._path(host, attempt, resource)
try:
descriptor = os.open(path, os.O_RDONLY | getattr(os, "O_NOFOLLOW", 0))
except FileNotFoundError:
continue
with os.fdopen(descriptor, "rb") as source:
info = os.fstat(source.fileno())
if not stat.S_ISREG(info.st_mode) or info.st_size > MAX_BYTES:
raise ValueError("invalid summary")
if info.st_mtime < time.time() - self.retention_seconds:
continue
values.append(self._public(json.loads(source.read(MAX_BYTES + 1))))
return {"attempt_id": attempt, "traces": values, "diagnostics_available": True}
except Exception:
LOG.warning("inventory_diagnostics_read_failed attempt=%s", attempt)
return {"attempt_id": attempt, "traces": [], "diagnostics_available": False}
def _public(self, value):
if value["state"] in {"running", "interrupted"} and value.get("_process") != self.process_id:
# A restart is evidence of interruption, not evidence of the old
# thread's completion time. Do not invent a finish timestamp.
value.update(state="interrupted", error_code="request_interrupted")
for item in value["stages"]:
if item["state"] == "running":
item["state"] = "interrupted"
elif value["state"] == "running":
now = time.monotonic()
value["duration_ms"] = max(0, int((now - value["_started"]) * 1000))
for item in value["stages"]:
if item["state"] == "running":
item["duration_ms"] = max(0, int((now - item["_started"]) * 1000))
return {key: ([{k: v for k, v in item.items() if not k.startswith("_")} for item in item_value]
if key == "stages" else item_value)
for key, item_value in value.items() if not key.startswith("_")}
class InventoryTrace:
def __init__(self, store, host, attempt, resource):
self.store, self.host, self.attempt = store, host, attempt
self.lock = threading.RLock()
self.active_stages = []
self.failure_stage = None
self.failure_exception = None
self.endpoint_started = False
self.value = {
"resource": resource, "state": "running", "stage": "request",
"error_code": "", "http_status": None, "started_at": int(time.time()),
"finished_at": None, "duration_ms": 0, "stages": [],
"engine_version": store.version, "_started": time.monotonic(),
"_process": store.process_id,
}
def start(self, name):
with self.lock:
if name not in STAGES or self.value["state"] != "running":
return None
self.failure_stage = None
self.failure_exception = None
self.value["stage"] = name
if len(self.value["stages"]) >= MAX_STAGES:
# Repeated tombstone cleanup can contain many connections.
# Keep the newest evidence bounded without losing the stage
# that is about to block, or removing an active parent span.
completed = next((item for item in self.value["stages"]
if item["state"] != "running"), None)
if completed is None:
return None
self.value["stages"].remove(completed)
item = {"stage": name, "state": "running", "duration_ms": 0, "_started": time.monotonic()}
try:
asyncio.get_running_loop()
item["_async"] = True
except RuntimeError:
item["_async"] = False
self.value["stages"].append(item)
self.active_stages.append(item)
self.store.save(self)
return item
def _close_stage(self, state):
for item in self.active_stages:
if item["state"] == "running":
item.update(state=state, duration_ms=max(0, int((time.monotonic() - item["_started"]) * 1000)))
def end(self, item):
with self.lock:
if item is not None and item["state"] == "running":
item.update(state="succeeded", duration_ms=max(0, int((time.monotonic() - item["_started"]) * 1000)))
self.active_stages.remove(item)
if self.active_stages and self.value["state"] == "running":
self.value["stage"] = self.active_stages[-1]["stage"]
elif self.value["state"] == "running":
self.value["stage"] = item["stage"]
self.store.save(self)
def fail_stage(self, item, exception):
with self.lock:
# A GC helper may catch a failed remote call and still allow the
# inventory to proceed. A failed span is NOT a failed request.
if item is not None and item["state"] == "running":
item.update(state="failed", duration_ms=max(0, int((time.monotonic() - item["_started"]) * 1000)))
self.active_stages.remove(item)
chained = exception
matches = False
seen = set()
while chained is not None and id(chained) not in seen:
if chained is self.failure_exception:
matches = True
break
seen.add(id(chained))
chained = chained.__cause__ or chained.__context__
if not matches:
self.failure_stage = item["stage"] if item else self.value["stage"]
self.failure_exception = exception
self.value["stage"] = self.failure_stage
self.store.save(self)
def worker_started(self):
with self.lock:
self.endpoint_started = True
# A queued worker can start just after its ASGI waiter was
# cancelled. Its actual completion remains authoritative engine
# evidence; the cancelled PHP attempt is still separate.
if self.value["state"] == "interrupted" and self.value.get("_waiter_cancelled"):
self.value.update(state="running", error_code="")
self.store.save(self)
def waiter_cancelled(self):
with self.lock:
if self.value["state"] != "running":
return
self.value["_waiter_cancelled"] = True
if self.endpoint_started:
# Cancelling an await cannot stop the sync endpoint worker.
# Its decorator, not this waiter, will record its result.
self.store.save(self)
return
now = time.monotonic()
self.value.update(
state="interrupted", error_code="request_interrupted",
http_status=None, finished_at=None,
duration_ms=max(0, int((now - self.value["_started"]) * 1000)),
)
for item in tuple(self.active_stages):
if item["state"] == "running" and item.get("_async"):
item.update(state="interrupted", duration_ms=max(0, int((now - item["_started"]) * 1000)))
self.active_stages.remove(item)
# A pre-endpoint GC thread may still run. Preserve its last
# confirmed span as running, with no invented finish timestamp.
self.store.save(self)
def finish(self, status=200, *, interrupted=False):
with self.lock:
if self.value["state"] != "running":
# A lower-level libvirt exception is translated into VMError
# by the service. Preserve its original stage, but use the
# final endpoint status when that translation is known.
if self.value["state"] == "failed" and status >= 400:
self.value["http_status"] = status
self.store.save(self)
return
state = "interrupted" if interrupted else "succeeded" if 200 <= status < 300 else "failed"
if state == "failed" and self.failure_stage:
self.value["stage"] = self.failure_stage
self._close_stage(state)
self.value.update(
state=state, http_status=status, finished_at=int(time.time()),
duration_ms=max(0, int((time.monotonic() - self.value["_started"]) * 1000)),
error_code=("request_interrupted" if interrupted else "" if state == "succeeded" else self.value["stage"] + "_failed"),
)
self.store.save(self)
@contextmanager
def inventory_stage(name):
trace = CURRENT.get()
item = trace.start(name) if trace else None
try:
yield
except Exception as exc:
if trace:
trace.fail_stage(item, exc)
raise
else:
if trace:
trace.end(item)
def inventory_result(function):
"""Finalize in the actual worker, even if its HTTP waiter was cancelled."""
@functools.wraps(function)
def wrapper(*args, **kwargs):
trace = CURRENT.get()
if trace:
trace.worker_started()
try:
result = function(*args, **kwargs)
except Exception as exc:
if trace:
status = getattr(exc, "status", getattr(exc, "status_code", 503))
trace.finish(status if isinstance(status, int) and 400 <= status <= 599 else 503)
raise
if trace:
trace.finish(200)
return result
# FastAPI resolves forward annotations using the callable's globals;
# this wrapper lives in another module, so resolve them at decoration.
wrapper.__signature__ = inspect.signature(function, eval_str=True)
return wrapper

@ -30,6 +30,7 @@ from virsh_panel.cloud_init import (
validate_guest_hostname,
)
from virsh_panel.metrics import RawVMStats
from virsh_panel.inventory_diagnostics import inventory_stage
from virsh_panel.user_isos import (
MAX_USER_ISO_BYTES, MAX_USER_ISOS, MAX_USER_ISO_TOTAL_BYTES,
USER_ISO_ID_RE, USER_ISO_VOLUME_RE, verify_user_iso,
@ -109,9 +110,10 @@ class VMService:
@contextmanager
def _connect(self) -> Iterator[Any]:
try:
conn = libvirt.open(self.uri)
if conn is None:
raise VMError("无法连接虚拟化宿主机", 502)
with inventory_stage("libvirt_connect"):
conn = libvirt.open(self.uri)
if conn is None:
raise VMError("无法连接虚拟化宿主机", 502)
except libvirt.libvirtError as exc:
LOG.warning("libvirt connection failed: %s", exc)
raise VMError("无法连接虚拟化宿主机", 502) from exc
@ -601,10 +603,13 @@ class VMService:
def list_vms(self) -> list[dict[str, Any]]:
with self._connect() as conn:
return sorted(
(self._vm_dict(domain) for domain in conn.listAllDomains(0)),
key=lambda row: row["name"].casefold(),
)
with inventory_stage("domain_list"):
domains = conn.listAllDomains(0)
with inventory_stage("domain_scan"):
return sorted(
(self._vm_dict(domain) for domain in domains),
key=lambda row: row["name"].casefold(),
)
@staticmethod
def _vm_state_dict(domain: Any) -> dict[str, Any]:
@ -875,12 +880,13 @@ class VMService:
counters so callers do not lose precision by rounding to GiB.
"""
with self._connect() as conn:
node = conn.getInfo()
if (not isinstance(node, (tuple, list)) or len(node) < 3
or isinstance(node[1], bool) or not isinstance(node[1], int)
or isinstance(node[2], bool) or not isinstance(node[2], int)
or node[1] < 0 or node[2] < 0):
raise VMError("宿主机容量信息无效", 502)
with inventory_stage("host_info"):
node = conn.getInfo()
if (not isinstance(node, (tuple, list)) or len(node) < 3
or isinstance(node[1], bool) or not isinstance(node[1], int)
or isinstance(node[2], bool) or not isinstance(node[2], int)
or node[1] < 0 or node[2] < 0):
raise VMError("宿主机容量信息无效", 502)
cpu_total = node[2]
# libvirt-python converts virNodeInfo.memory from KiB to MiB
@ -891,26 +897,29 @@ class VMService:
cpu_allocated = 0
memory_allocated_mib = 0
running = 0
domains = conn.listAllDomains(0)
for domain in domains:
info = domain.info()
if (not isinstance(info, (tuple, list)) or len(info) < 4
or isinstance(info[1], bool) or not isinstance(info[1], int)
or isinstance(info[3], bool) or not isinstance(info[3], int)
or info[1] < 0 or info[3] < 0):
raise VMError("宿主机虚拟机容量信息无效", 502)
memory_allocated_mib += info[1] // 1024
cpu_allocated += info[3]
if info[0] in {1, 2, 3, 7}:
running += 1
disk_pool = self._active_pool(conn, self.storage_pool)
pool_info = disk_pool.info()
if (not isinstance(pool_info, (tuple, list)) or len(pool_info) < 4
or any(isinstance(value, bool) or not isinstance(value, int)
or value < 0 for value in pool_info[1:4])):
raise VMError("宿主机存储容量信息无效", 502)
storage_total, storage_allocated, storage_available = pool_info[1:4]
with inventory_stage("domain_list"):
domains = conn.listAllDomains(0)
with inventory_stage("domain_scan"):
for domain in domains:
info = domain.info()
if (not isinstance(info, (tuple, list)) or len(info) < 4
or isinstance(info[1], bool) or not isinstance(info[1], int)
or isinstance(info[3], bool) or not isinstance(info[3], int)
or info[1] < 0 or info[3] < 0):
raise VMError("宿主机虚拟机容量信息无效", 502)
memory_allocated_mib += info[1] // 1024
cpu_allocated += info[3]
if info[0] in {1, 2, 3, 7}:
running += 1
with inventory_stage("storage_scan"):
disk_pool = self._active_pool(conn, self.storage_pool)
pool_info = disk_pool.info()
if (not isinstance(pool_info, (tuple, list)) or len(pool_info) < 4
or any(isinstance(value, bool) or not isinstance(value, int)
or value < 0 for value in pool_info[1:4])):
raise VMError("宿主机存储容量信息无效", 502)
storage_total, storage_allocated, storage_available = pool_info[1:4]
return {
"instances": {"total": len(domains), "running": running},

@ -10,6 +10,7 @@ use App\Services\Cloud\CloudAdminOperationsService;
use App\Services\Cloud\CloudAdminResourceService;
use App\Services\Cloud\CloudCountryCatalog;
use App\Services\Cloud\CloudError;
use App\Services\Cloud\CloudHostInventoryDiagnostics;
/** Admin pages use database-only first paints; JSON APIs repeat authorisation. */
final class CloudPageController extends AdminController
@ -177,4 +178,39 @@ final class CloudPageController extends AdminController
{
return $this->view()->display('admin/update.tpl');
}
public function hostInventory($request, $response, $args)
{
try {
$actor = CloudAccess::actor(false);
} catch (CloudError $exception) {
return $response->withStatus($exception->httpStatus)
->withHeader('Cache-Control', 'private, no-store');
}
if (!($actor['is_admin'] ?? false)) {
return $response->withStatus(403)->withHeader('Cache-Control', 'private, no-store');
}
$payload = null;
$error = '';
$status = 200;
try {
$payload = CloudHostInventoryDiagnostics::page($actor, $request->getQueryParams());
} catch (CloudError $exception) {
$error = $exception->getMessage();
$status = $exception->httpStatus;
} catch (\Throwable $exception) {
error_log('Admin inventory history failed: ' . get_class($exception));
$error = '盘点记录暂时无法读取,请确认数据库已升级后重试。';
$status = 503;
}
$smarty = $this->view()->assign([
'inventory_page' => $payload, 'inventory_error' => $error,
'inventory_timezone' => date_default_timezone_get(),
'inventory_bootstrap_json' => json_encode($payload,
JSON_HEX_TAG | JSON_HEX_AMP | JSON_HEX_APOS | JSON_HEX_QUOT
| JSON_UNESCAPED_UNICODE | JSON_INVALID_UTF8_SUBSTITUTE | JSON_THROW_ON_ERROR),
]);
$response->getBody()->write($smarty->fetch('admin/cloud/host_inventory.tpl'));
return $response->withStatus($status)->withHeader('Cache-Control', 'private, no-store');
}
}

@ -0,0 +1,58 @@
<?php
namespace App\Controllers;
use App\Services\Cloud\CloudAccess;
use App\Services\Cloud\CloudEngineClient;
use App\Services\Cloud\CloudError;
use App\Services\Cloud\CloudHostInventoryDiagnostics;
use App\Services\Cloud\CloudInventoryRequestError;
/** Read-only inventory history. Reading a record never starts a host scan. */
final class CloudAdminHostInventoryController
{
public function index($request, $response)
{
return $response->withJson(CloudHostInventoryDiagnostics::page(
$request->getAttribute('cloud_actor') ?? CloudAccess::actor(false),
$request->getQueryParams()
))->withHeader('Cache-Control', 'private, no-store');
}
public function detail($request, $response, $args)
{
return $response->withJson(CloudHostInventoryDiagnostics::detail(
$request->getAttribute('cloud_actor') ?? CloudAccess::actor(false),
(string) ($args['id'] ?? '')
))->withHeader('Cache-Control', 'private, no-store');
}
public function engine($request, $response, $args)
{
$actor = $request->getAttribute('cloud_actor') ?? CloudAccess::actor(false);
$attemptId = (string) ($args['id'] ?? '');
// This authorizes the read and binds its host to a persisted attempt.
// Never accept an arbitrary host or call the capacity/network endpoints.
$detail = CloudHostInventoryDiagnostics::detail($actor, $attemptId);
try {
$result = (new CloudEngineClient(null, 1.0))->inventoryDiagnostics(
(int) $detail['attempt']['host_id'], $actor, $attemptId
);
$available = ($result['diagnostics_available'] ?? true) === true;
$result = ['available' => $available, 'attempt_id' => $attemptId,
'traces' => $result['traces'],
'message' => $available
? '仅查询引擎已记录的线索;不会发起盘点,也不会改变面板同步结果。'
: '引擎诊断存储暂不可用;无法取得内部阶段线索,不代表宿主机离线。'];
} catch (CloudInventoryRequestError $exception) {
$result = ['available' => false, 'attempt_id' => $attemptId,
'traces' => [], 'error_code' => $exception->diagnosticCode,
'message' => '引擎线索暂不可读取或尚未记录;无法据此判断宿主机是否正常。'];
} catch (CloudError $exception) {
$result = ['available' => false, 'attempt_id' => $attemptId,
'traces' => [], 'error_code' => 'diagnostics_unavailable',
'message' => '引擎线索暂不可读取;请检查引擎版本、认证及连接配置。'];
}
return $response->withJson($result)->withHeader('Cache-Control', 'private, no-store');
}
}

@ -19,6 +19,7 @@ final class CloudApi
|| preg_match('#^/api/v1/admin/billing(?:/|$)#D', $path) === 1
|| preg_match('#^/api/v1/admin/instances(?:/|$)#D', $path) === 1
|| preg_match('#^/api/v1/admin/operations(?:/|$)#D', $path) === 1
|| preg_match('#^/api/v1/admin/host-inventory(?:/|$)#D', $path) === 1
|| preg_match('#^/api/v1/admin/(?:resources|regions|network-lines|resource-pools|resource-reservations)(?:/|$)#D', $path) === 1
|| preg_match('#^/api/v1/admin/hosts/[0-9]+/(?:scheduling|capacity-sync)$#D', $path) === 1
|| preg_match('#^/api/v1/ssh-keys(?:/|$)#D', $path) === 1

@ -0,0 +1,18 @@
<?php
namespace App\Models;
final class CloudHostInventoryAttempt extends Model
{
protected $connection = 'default';
protected $table = 'cloud_host_inventory_attempts';
protected $primaryKey = 'attempt_id';
protected $keyType = 'string';
public $incrementing = false;
protected $casts = [
'sequence_id' => 'int', 'host_id' => 'int', 'generation' => 'int', 'started_at' => 'int',
'finished_at' => 'int', 'updated_at' => 'int', 'duration_ms' => 'int',
'http_status' => 'int', 'applied' => 'boolean', 'timeout_ms' => 'int',
];
}

@ -6,11 +6,14 @@ use GuzzleHttp\Client;
final class CloudEngineClient
{
private float $timeoutSeconds;
public function __construct(private ?Client $client = null, float $timeoutSeconds = 20.0)
{
if ($timeoutSeconds < 1 || $timeoutSeconds > 900) {
throw new \InvalidArgumentException('Engine timeout is outside the supported range');
}
$this->timeoutSeconds = $timeoutSeconds;
$this->client ??= new Client(['connect_timeout' => 3, 'timeout' => $timeoutSeconds, 'http_errors' => false,
'allow_redirects' => false]);
}
@ -343,4 +346,165 @@ final class CloudEngineClient
return $this->request($method, '/internal/v1/hosts/' . $hostId . '/' . $resource,
$actor, $payload, $expectedOperationId);
}
/** Read-only inventory keeps diagnostic categories without changing mutation errors. */
public function inventoryTimeoutSeconds(): float
{
return $this->timeoutSeconds;
}
public function hostInventory(int $hostId, string $resource, array $actor,
string $attemptId): array
{
if (!in_array($resource, ['host/capacity', 'host/network-options'], true)) {
throw new \InvalidArgumentException('Unsupported inventory resource');
}
$this->validateInventoryIdentity($hostId, $attemptId);
// The correlation ID is covered by the existing HMAC canonical target.
return $this->inventoryRequest('/internal/v1/hosts/' . $hostId . '/'
. $resource . '?inventory_attempt_id=' . $attemptId, $actor,
$this->timeoutSeconds);
}
/** Bounded local evidence lookup: never runs inventory or touches a hypervisor. */
public function inventoryDiagnostics(int $hostId, array $actor,
string $attemptId): array
{
$this->validateInventoryIdentity($hostId, $attemptId);
$value = $this->inventoryRequest('/internal/v1/hosts/' . $hostId
. '/host/inventory-diagnostics/' . $attemptId, $actor, 1.0);
return self::inventoryEvidence($value, $attemptId);
}
private function validateInventoryIdentity(int $hostId, string $attemptId): void
{
if ($hostId < 1 || preg_match('/^[0-9a-f]{32}$/D', $attemptId) !== 1) {
throw new \InvalidArgumentException('Invalid inventory identity');
}
}
private function inventoryRequest(string $path, array $actor, float $timeout): array
{
try {
$base = $this->base($path);
$headers = self::headers('GET', $path, '', $actor);
} catch (\Throwable $error) {
throw new CloudInventoryRequestError('engine_client_configuration_invalid', null, 503);
}
try {
$response = $this->client->request('GET', $base . $path, [
'headers' => $headers, 'body' => '', 'timeout' => $timeout,
'connect_timeout' => min(3.0, $timeout), 'allow_redirects' => false,
'http_errors' => false, 'proxy' => '',
]);
} catch (\GuzzleHttp\Exception\TransferException $error) {
$context = method_exists($error, 'getHandlerContext')
? $error->getHandlerContext() : [];
// cURL 28 proves a timeout, but not which remote stage caused it.
$code = (int) ($context['errno'] ?? 0) === 28
? 'engine_http_timeout'
: ($error instanceof \GuzzleHttp\Exception\ConnectException
? (in_array((int) ($context['errno'] ?? 0), [5, 6, 7], true)
? 'engine_connect_failed' : 'engine_connection_or_timeout')
: 'engine_transport_failed');
throw new CloudInventoryRequestError($code);
} catch (\Throwable $error) {
throw new CloudInventoryRequestError('engine_transport_failed');
}
$status = $response->getStatusCode();
try {
$body = $response->getBody()->read(2097153);
$value = strlen($body) <= 2097152
? json_decode($body, true, 64, JSON_THROW_ON_ERROR) : null;
} catch (\Throwable $error) {
$value = null;
}
if ($status >= 400) {
$code = match ($status) {
401, 403 => 'engine_auth_rejected',
404 => 'engine_resource_missing',
408, 504 => 'engine_http_timeout',
409 => 'engine_request_conflict',
423 => 'engine_configuration_locked',
429 => 'engine_rate_limited',
default => $status >= 500 ? 'engine_service_error' : 'engine_request_rejected',
};
// Preserve the old scheduling classification: a non-JSON error
// was an uncertain 502, including a non-JSON upstream 404.
$legacyStatus = $status === 404 && is_array($value) ? 404 : 502;
throw new CloudInventoryRequestError($code, $status, $legacyStatus);
}
if (!is_array($value)) {
throw new CloudInventoryRequestError('engine_response_invalid', $status);
}
return $value;
}
/** Allowlisted evidence only: never pass arbitrary engine strings to the admin UI. */
private static function inventoryEvidence(array $value, string $attemptId): array
{
$stages = ['request', 'registry_profile', 'update_lock_wait', 'update_check', 'iso_gc',
'template_gc', 'profile', 'libvirt_connect', 'host_info', 'domain_list',
'domain_scan', 'storage_scan', 'public_pool_catalog', 'public_ip_claims',
'public_ip_scan', 'response'];
$states = ['running', 'succeeded', 'failed', 'interrupted'];
$codes = array_merge(['', 'request_failed', 'request_interrupted'],
array_map(static fn (string $stage): string => $stage . '_failed', $stages));
if (($value['attempt_id'] ?? null) !== $attemptId
|| !is_array($value['traces'] ?? null) || count($value['traces']) > 2
|| (isset($value['diagnostics_available']) && !is_bool($value['diagnostics_available']))) {
throw new CloudInventoryRequestError('engine_response_invalid');
}
$number = static function ($input, bool $nullable = false): ?int {
if ($input === null && $nullable) {
return null;
}
if (!is_int($input) || $input < 0) {
throw new CloudInventoryRequestError('engine_response_invalid');
}
return $input;
};
$traces = [];
$seen = [];
foreach ($value['traces'] as $trace) {
if (!is_array($trace)
|| !in_array($trace['resource'] ?? null, ['capacity', 'network'], true)
|| isset($seen[$trace['resource']])
|| !in_array($trace['state'] ?? null, $states, true)
|| !in_array($trace['stage'] ?? null, $stages, true)
|| !is_array($trace['stages'] ?? null) || count($trace['stages']) > 64) {
throw new CloudInventoryRequestError('engine_response_invalid');
}
$seen[$trace['resource']] = true;
$code = $trace['error_code'] ?? '';
if (!is_string($code) || (!in_array($code, $codes, true)
&& preg_match('/^http_[45][0-9]{2}$/D', $code) !== 1)) {
$code = 'request_failed';
}
$http = $number($trace['http_status'] ?? null, true);
if ($http !== null && ($http < 100 || $http > 599)) {
throw new CloudInventoryRequestError('engine_response_invalid');
}
$version = $trace['engine_version'] ?? '';
$version = is_string($version)
&& preg_match('/^[0-9a-f]{7,64}$/D', $version) === 1 ? $version : 'unknown';
$rows = [];
foreach ($trace['stages'] as $row) {
if (!is_array($row) || !in_array($row['stage'] ?? null, $stages, true)
|| !in_array($row['state'] ?? null, $states, true)) {
throw new CloudInventoryRequestError('engine_response_invalid');
}
$rows[] = ['stage' => $row['stage'], 'state' => $row['state'],
'duration_ms' => $number($row['duration_ms'] ?? null)];
}
$traces[] = ['resource' => $trace['resource'], 'state' => $trace['state'],
'stage' => $trace['stage'], 'error_code' => $code, 'http_status' => $http,
'started_at' => $number($trace['started_at'] ?? null),
'finished_at' => $number($trace['finished_at'] ?? null, true),
'duration_ms' => $number($trace['duration_ms'] ?? null),
'stages' => $rows, 'engine_version' => $version];
}
return ['attempt_id' => $attemptId, 'traces' => $traces,
'diagnostics_available' => $value['diagnostics_available'] ?? true];
}
}

@ -0,0 +1,423 @@
<?php
namespace App\Services\Cloud;
use App\Models\CloudHost;
use App\Models\CloudHostInventoryAttempt;
/** Best-effort diagnostic history, never a capacity or health authority. */
final class CloudHostInventoryDiagnostics
{
// A compiled code-contract marker, deliberately not the on-disk Git HEAD:
// a daemon may still be executing an older loaded class after deployment.
private const PANEL_VERSION = 'inventory-diagnostics-v1';
private const RETENTION_SECONDS = 7 * 86400;
private const HOST_RECORD_LIMIT = 25000;
private const CLEANUP_BATCH = 256;
private const INTERRUPTED_SECONDS = 300;
private const OUTCOMES = [
'running' => '进行中', 'succeeded' => '成功', 'failed' => '失败',
'timeout' => '请求超时', 'skipped' => '跳过', 'superseded' => '已被更新的尝试取代',
'interrupted' => '中断或结果未知',
];
private const STAGES = [
'admission' => '面板准备盘点', 'capacity_request' => '请求引擎容量',
'capacity_validate' => '校验容量响应', 'network_request' => '请求公网 IP 信息',
'network_validate' => '校验公网 IP 响应', 'database_apply' => '提交容量与 IP 数据',
'failure_marker' => '记录同步失败状态',
];
private const ERRORS = [
'attempt_start_failed' => '面板无法创建容量同步代际',
'engine_http_timeout' => '等待引擎响应超时;内部阶段需查看引擎证据',
'engine_connect_failed' => '面板无法连接引擎',
'engine_connection_or_timeout' => '连接失败或请求超时;客户端未提供可确认的细分原因',
'engine_transport_failed' => '面板与引擎之间的传输失败',
'engine_response_invalid' => '引擎响应无法解析或不符合协议',
'engine_auth_rejected' => '引擎拒绝内部请求认证',
'engine_configuration_locked' => '引擎配置被锁定或正在更新',
'engine_request_conflict' => '引擎报告状态或配置冲突',
'engine_resource_missing' => '引擎报告请求资源不存在',
'engine_rate_limited' => '引擎请求受到频率或并发限制',
'engine_service_error' => '引擎处理请求失败;内部阶段需查看引擎证据',
'engine_request_rejected' => '引擎拒绝此次盘点请求',
'engine_client_configuration_invalid' => '面板内部引擎客户端配置无效',
'host_inventory_capacity_invalid' => '容量响应校验未通过',
'host_inventory_network_invalid' => '公网 IP 响应校验未通过',
'host_inventory_ip_conflict' => '公网 IP 或地址池记录冲突',
'host_inventory_write_failed' => '容量和 IP 数据事务提交失败',
'host_inventory_unavailable' => '本次盘点未能取得可用的引擎响应',
'host_inventory_engine_rejected' => '引擎未接受本次盘点',
'unexpected_error' => '发生未分类错误,未记录原始异常内容',
];
private array $record;
private array $stages = [];
private int $startedMonotonic;
private int $stageStartedMonotonic;
private bool $stored = false;
private bool $finished = false;
private bool $fallbackLogged = false;
private static array $lastCleanup = [];
private function __construct(int $hostId, string $source, int $startedAt,
float $timeoutSeconds)
{
$this->startedMonotonic = hrtime(true);
$this->stageStartedMonotonic = $this->startedMonotonic;
try {
$id = bin2hex(random_bytes(16));
} catch (\Throwable $ignored) {
// Correlation only; this identifier is never an authentication token.
$id = substr(hash('sha256', $hostId . ':' . $this->startedMonotonic
. ':' . getmypid() . ':' . uniqid('', true)), 0, 32);
}
$this->record = [
'attempt_id' => $id, 'host_id' => max(1, $hostId),
'source' => $source === 'manual' ? 'manual' : 'auto', 'generation' => 0,
'started_at' => max(1, $startedAt), 'finished_at' => null,
'updated_at' => max(1, $startedAt), 'duration_ms' => 0,
'outcome' => 'running', 'stage' => 'admission', 'error_code' => '',
'http_status' => null, 'applied' => false,
'timeout_ms' => is_finite($timeoutSeconds)
? max(1, min(86400000, (int) round($timeoutSeconds * 1000))) : 8000,
'panel_version' => self::PANEL_VERSION,
'php_version' => self::version(PHP_VERSION), 'stages_json' => '[]',
];
}
public static function start(int $hostId, string $source, int $startedAt,
float $timeoutSeconds): self
{
$attempt = new self($hostId, $source, $startedAt, $timeoutSeconds);
try {
CloudHostInventoryAttempt::query()->insert($attempt->record);
$attempt->stored = true;
self::cleanup($hostId, $attempt->record['started_at']);
} catch (\Throwable $ignored) {
$attempt->fallback();
}
return $attempt;
}
public function id(): string
{
return $this->record['attempt_id'];
}
public function generation(int $generation): void
{
if (!$this->finished) {
$this->record['generation'] = max(0, $generation);
$this->persist();
}
}
public function stage(string $stage): void
{
if ($this->finished || !isset(self::STAGES[$stage]) || $stage === $this->record['stage']) {
return;
}
$this->closeStage();
$this->record['stage'] = $stage;
$this->persist();
}
public function finish(string $outcome, string $errorCode = '',
?int $httpStatus = null, bool $applied = false): void
{
if ($this->finished) {
return;
}
$this->finished = true;
$this->closeStage();
$this->record['outcome'] = isset(self::OUTCOMES[$outcome]) && $outcome !== 'running'
? $outcome : 'failed';
$this->record['error_code'] = self::errorCode($errorCode);
$this->record['http_status'] = $httpStatus !== null && $httpStatus >= 100
&& $httpStatus <= 599 ? $httpStatus : null;
$this->record['applied'] = $outcome === 'succeeded' && $applied;
// Match the injected observation clock's wall-time origin, but measure
// elapsed time solely within this process using the monotonic clock.
$this->record['finished_at'] = $this->record['started_at']
+ (int) floor($this->record['duration_ms'] / 1000);
$this->persist();
}
private function closeStage(): void
{
$now = hrtime(true);
if (count($this->stages) < 32) {
$this->stages[] = ['stage' => $this->record['stage'],
'duration_ms' => max(0, (int) floor(($now - $this->stageStartedMonotonic) / 1000000))];
}
$this->stageStartedMonotonic = $now;
$this->record['duration_ms'] = max(0,
(int) floor(($now - $this->startedMonotonic) / 1000000));
}
private function persist(): void
{
try {
if (!$this->stored) {
$this->fallback();
return;
}
$this->record['updated_at'] = $this->record['started_at']
+ (int) floor($this->record['duration_ms'] / 1000);
$this->record['stages_json'] = json_encode($this->stages, JSON_THROW_ON_ERROR);
CloudHostInventoryAttempt::where('attempt_id', $this->id())
->where('outcome', 'running')->update($this->record);
} catch (\Throwable $ignored) {
$this->fallback();
}
}
private function fallback(): void
{
if ($this->fallbackLogged && !$this->finished) {
return;
}
$this->fallbackLogged = true;
try {
error_log('cloud_inventory_diagnostics_unavailable attempt_id=' . $this->id()
. ' host_id=' . $this->record['host_id'] . ' outcome=' . $this->record['outcome']
. ' stage=' . $this->record['stage'] . ' error_code=' . $this->record['error_code']);
} catch (\Throwable $ignored) {
// Failure of diagnostic storage or logging cannot change scheduling.
}
}
/** Bounded best-effort cleanup on attempts only, never a page-view side effect. */
private static function cleanup(int $hostId, int $now): void
{
if (($now - (self::$lastCleanup[$hostId] ?? 0)) < 60) {
return;
}
if (count(self::$lastCleanup) > 512) {
self::$lastCleanup = [];
}
self::$lastCleanup[$hostId] = $now;
try {
$expired = CloudHostInventoryAttempt::where('started_at', '<', $now - self::RETENTION_SECONDS)
->orderBy('started_at')->orderBy('sequence_id')->limit(self::CLEANUP_BATCH)
->pluck('attempt_id')->all();
if ($expired !== []) {
CloudHostInventoryAttempt::whereIn('attempt_id', $expired)->delete();
}
$overflow = CloudHostInventoryAttempt::where('host_id', $hostId)
->orderByDesc('started_at')->orderByDesc('sequence_id')->offset(self::HOST_RECORD_LIMIT)
->limit(self::CLEANUP_BATCH)->pluck('attempt_id')->all();
if ($overflow !== []) {
CloudHostInventoryAttempt::whereIn('attempt_id', $overflow)->delete();
}
} catch (\Throwable $ignored) {
// History cleanup must not fail the current observation.
}
}
public static function page(array $actor, array $input = []): array
{
self::authorize($actor);
$filters = self::filters($input);
try {
return self::readPage($filters);
} catch (\Throwable $ignored) {
throw new CloudError('盘点诊断历史不可读取,请确认数据库已初始化或升级', 503);
}
}
private static function readPage(array $filters): array
{
$now = time();
$base = CloudHostInventoryAttempt::query();
if ($filters['host_id'] !== null) {
$base->where('host_id', (int) $filters['host_id']);
}
$summaryRow = (clone $base)->selectRaw('COUNT(*) AS total, MAX(started_at) AS last_attempt_at, '
. "MAX(CASE WHEN outcome = 'succeeded' AND applied = 1 THEN finished_at ELSE NULL END) AS last_success_at, "
. "MAX(CASE WHEN outcome = 'succeeded' AND applied = 1 THEN sequence_id ELSE NULL END) AS last_success_sequence, "
. "MAX(CASE WHEN outcome IN ('failed', 'timeout') THEN finished_at ELSE NULL END) AS last_failure_at, "
. "COALESCE(SUM(CASE WHEN outcome = 'running' AND updated_at >= ? THEN 1 ELSE 0 END), 0) AS in_progress, "
. "COALESCE(SUM(CASE WHEN outcome = 'failed' THEN 1 ELSE 0 END), 0) AS failed, "
. "COALESCE(SUM(CASE WHEN outcome = 'timeout' THEN 1 ELSE 0 END), 0) AS timeout",
[$now - self::INTERRUPTED_SECONDS])->first();
$summary = [];
foreach (['total', 'in_progress', 'failed', 'timeout'] as $field) {
$summary[$field] = max(0, (int) ($summaryRow?->{$field} ?? 0));
}
foreach (['last_attempt_at', 'last_success_at', 'last_failure_at'] as $field) {
$summary[$field] = self::positiveTime($summaryRow?->{$field});
}
$summary['consecutive_failures_scope'] = $filters['host_id'] === null
? 'all_hosts' : 'selected_host';
$summary['consecutive_failures'] = null;
if ($filters['host_id'] !== null) {
$failed = (clone $base)->whereIn('outcome', ['failed', 'timeout']);
if ($summaryRow?->last_success_sequence !== null) {
$failed->where('sequence_id', '>', (int) $summaryRow->last_success_sequence);
}
// Interrupted or skipped attempts are not evidence of another failure.
$summary['consecutive_failures'] = $failed->count();
}
$query = (clone $base);
if ($filters['outcome'] === 'attention') {
$query->where(static function ($q) use ($now): void {
$q->whereIn('outcome', ['failed', 'timeout', 'interrupted'])
->orWhere(static function ($q) use ($now): void {
$q->where('outcome', 'running')->where('updated_at', '<', $now - self::INTERRUPTED_SECONDS);
});
});
} elseif ($filters['outcome'] === 'interrupted') {
$query->where(static function ($q) use ($now): void {
$q->where('outcome', 'interrupted')->orWhere(static function ($q) use ($now): void {
$q->where('outcome', 'running')->where('updated_at', '<', $now - self::INTERRUPTED_SECONDS);
});
});
} elseif ($filters['outcome'] === 'running') {
$query->where('outcome', 'running')->where('updated_at', '>=', $now - self::INTERRUPTED_SECONDS);
} elseif ($filters['outcome'] !== 'all') {
$query->where('outcome', $filters['outcome']);
}
$total = $query->count();
$pages = max(1, (int) ceil($total / $filters['per_page']));
$filters['page'] = min($filters['page'], $pages);
$records = $query->orderByDesc('started_at')->orderByDesc('sequence_id')
->offset(($filters['page'] - 1) * $filters['per_page'])->limit($filters['per_page'])->get();
$hosts = [];
$hostNames = [];
foreach (CloudHost::orderBy('name')->orderBy('id')->get(['id', 'name']) as $host) {
$name = self::text((string) $host->name, 80);
$hosts[] = ['id' => (string) $host->id, 'name' => $name];
$hostNames[(int) $host->id] = $name;
}
$items = [];
foreach ($records as $record) {
$items[] = self::dto($record, $hostNames[(int) $record->host_id] ?? '宿主机已删除', $now);
}
$from = $total === 0 ? 0 : ($filters['page'] - 1) * $filters['per_page'] + 1;
return ['items' => $items, 'filters' => $filters, 'hosts' => $hosts,
'summary' => $summary, 'pagination' => ['page' => $filters['page'],
'per_page' => $filters['per_page'], 'total' => $total, 'total_pages' => $pages,
'from' => $from, 'to' => $from === 0 ? 0 : $from + count($items) - 1],
'generated_at' => $now, 'retention_days' => 7];
}
public static function detail(array $actor, string $attemptId): array
{
self::authorize($actor);
if (preg_match('/^[a-f0-9]{32}$/D', $attemptId) !== 1) {
throw new CloudError('盘点请求编号无效', 400);
}
try {
$record = CloudHostInventoryAttempt::find($attemptId);
$host = $record === null ? null : CloudHost::find((int) $record->host_id, ['id', 'name']);
} catch (\Throwable $ignored) {
throw new CloudError('盘点诊断历史不可读取,请确认数据库已初始化或升级', 503);
}
if ($record === null) {
throw new CloudError('盘点记录不存在或已超过保留期限', 404);
}
return ['attempt' => self::dto($record, $host === null ? '宿主机已删除'
: self::text((string) $host->name, 80), time()), 'generated_at' => time(), 'notes' => [
'面板结果与引擎最终结果独立;引擎晚到成功不代表容量已经提交。',
'超时只确认面板未及时收到响应,不能据此认定物理宿主机离线。',
'最后确认阶段不是已证实的故障根因;超过 5 分钟未结束的记录按中断或未知展示。',
'版本字段为运行时代码的诊断协议标识,不是 Git 提交号;部署后应重启常驻进程。',
'历史保留约 7 天且每宿主上限 25000 条,分批清理;本页不执行任何盘点。',
]];
}
private static function dto(CloudHostInventoryAttempt $record, string $hostName, int $now): array
{
$outcome = isset(self::OUTCOMES[(string) $record->outcome])
? (string) $record->outcome : 'interrupted';
$interrupted = $outcome === 'running' && (int) $record->updated_at < $now - self::INTERRUPTED_SECONDS;
if ($interrupted) {
$outcome = 'interrupted';
}
$stage = isset(self::STAGES[(string) $record->stage]) ? (string) $record->stage : 'admission';
$error = self::errorCode((string) $record->error_code);
$stages = [];
$storedStages = strlen((string) $record->stages_json) <= 16384
? json_decode((string) $record->stages_json, true) : null;
foreach (is_array($storedStages) ? array_slice($storedStages, 0, 32) : [] as $item) {
if (is_array($item) && is_string($item['stage'] ?? null)
&& isset(self::STAGES[$item['stage']]) && is_int($item['duration_ms'] ?? null)) {
$stages[] = ['stage' => $item['stage'], 'label' => self::STAGES[$item['stage']],
'duration_ms' => max(0, min(86400000, $item['duration_ms']))];
}
}
$status = (int) $record->http_status;
return ['attempt_id' => (string) $record->attempt_id, 'host_id' => (int) $record->host_id,
'host_name' => $hostName, 'source' => $record->source === 'manual' ? 'manual' : 'auto',
'source_label' => $record->source === 'manual' ? '管理员手动同步' : '自动盘点',
'started_at' => (int) $record->started_at, 'finished_at' => self::positiveTime($record->finished_at),
'duration_ms' => max(0, (int) $record->duration_ms), 'duration_is_final' => $record->finished_at !== null,
'outcome' => $outcome, 'outcome_label' => self::OUTCOMES[$outcome],
'stage' => $stage, 'stage_label' => self::STAGES[$stage], 'last_confirmed_stage' => $stage,
'error_code' => $error, 'error_message' => $error === '' ? '' : self::ERRORS[$error],
'http_status' => $status >= 100 && $status <= 599 ? $status : null,
'applied' => $outcome === 'succeeded' && (bool) $record->applied,
'timeout_ms' => max(0, (int) $record->timeout_ms), 'panel_version' => self::version((string) $record->panel_version),
'php_version' => self::version((string) $record->php_version), 'generation' => max(0, (int) $record->generation),
'stages' => $stages, 'interrupted' => $interrupted || $outcome === 'interrupted'];
}
public static function filters(array $input): array
{
foreach ($input as $key => $value) {
if (!in_array($key, ['host_id', 'outcome', 'page', 'per_page'], true)
|| (!is_string($value) && !is_int($value))) {
throw new CloudError('盘点诊断筛选参数无效', 400);
}
}
$outcome = (string) ($input['outcome'] ?? 'all');
if (!in_array($outcome, array_merge(['all', 'attention'], array_keys(self::OUTCOMES)), true)) {
throw new CloudError('盘点结果筛选无效', 400);
}
$host = (string) ($input['host_id'] ?? '');
$page = self::positiveId((string) ($input['page'] ?? '1'));
$perPage = self::positiveId((string) ($input['per_page'] ?? '20'));
if (!in_array($perPage, [20, 50, 100], true)) {
throw new CloudError('每页条数只能为 20、50 或 100', 400);
}
return ['host_id' => $host === '' ? null : (string) self::positiveId($host),
'outcome' => $outcome, 'page' => $page, 'per_page' => $perPage];
}
private static function positiveId(string $value): int
{
if (preg_match('/^[1-9][0-9]{0,17}$/D', $value) !== 1) {
throw new CloudError('编号或页码无效', 400);
}
return (int) $value;
}
private static function authorize(array $actor): void
{
if (!($actor['is_admin'] ?? false)) {
throw new CloudError('需要管理员权限', 403);
}
}
private static function errorCode(string $value): string
{
return $value === '' || isset(self::ERRORS[$value]) ? $value : 'unexpected_error';
}
private static function version(string $value): string
{
return strlen($value) <= 64 && preg_match('/^[A-Za-z0-9][A-Za-z0-9._+-]*$/D', $value) === 1
? $value : 'unknown';
}
private static function positiveTime($value): ?int
{
return $value === null || (int) $value <= 0 ? null : (int) $value;
}
private static function text(string $value, int $limit): string
{
return mb_substr(preg_replace('/[\x00-\x1f\x7f]/u', '', $value) ?? '', 0, $limit, 'UTF-8');
}
}

@ -85,7 +85,7 @@ final class CloudHostInventorySyncService
foreach ($hosts as $host) {
$result['checked']++;
$this->pollHost((int) $host->id, $actor, $result);
$this->pollHost((int) $host->id, $actor, $result, 'auto');
}
$result['errors'] = array_values(array_unique($result['errors']));
return $result;
@ -118,67 +118,88 @@ final class CloudHostInventorySyncService
return $result;
}
$result['checked'] = 1;
$this->pollHost($hostId, $actor, $result);
$this->pollHost($hostId, $actor, $result, 'manual');
$result['errors'] = array_values(array_unique($result['errors']));
return $result;
}
private function pollHost(int $hostId, array $actor, array &$result): void
private function pollHost(int $hostId, array $actor, array &$result,
string $source): void
{
// This local lower bound is authoritative for comparing database
// reservations with the engine snapshot. The engine's observed_at is
// retained for diagnostics only because host clocks may legitimately
// differ from the PHP/database clock.
$attemptedAt = $this->now();
$diagnostics = CloudHostInventoryDiagnostics::start(
$hostId, $source, $attemptedAt, $this->engine->inventoryTimeoutSeconds()
);
try {
$attemptGeneration = $this->beginAttempt($hostId, $attemptedAt);
} catch (\Throwable $error) {
$result['errors'][] = 'host_inventory_sync_failed';
$diagnostics->finish('failed', 'attempt_start_failed');
return;
}
if ($attemptGeneration === null) {
$result['skipped']++;
$diagnostics->finish('skipped');
return;
}
$diagnostics->generation($attemptGeneration);
$diagnostics->stage('capacity_request');
try {
$capacityPayload = $this->engine->host(
$hostId, 'GET', 'host/capacity', $actor
$capacityPayload = $this->engine->hostInventory(
$hostId, 'host/capacity', $actor, $diagnostics->id()
);
} catch (CloudInventoryRequestError $error) {
$this->failed($hostId, $this->engineFailure($error), $result,
$attemptedAt, $attemptGeneration, $diagnostics,
$error->diagnosticCode, $error->upstreamHttpStatus);
return;
} catch (CloudError $error) {
$this->failed($hostId, $this->engineFailure($error), $result,
$attemptedAt, $attemptGeneration);
$attemptedAt, $attemptGeneration, $diagnostics);
return;
} catch (\Throwable $error) {
$this->failed($hostId,
['unavailable', self::ERROR_UNAVAILABLE], $result,
$attemptedAt, $attemptGeneration);
$attemptedAt, $attemptGeneration, $diagnostics);
return;
}
$diagnostics->stage('capacity_validate');
try {
$capacity = self::capacity($capacityPayload, $this->now());
} catch (\UnexpectedValueException $error) {
$this->failed($hostId,
['degraded', self::ERROR_INVALID_CAPACITY], $result,
$attemptedAt, $attemptGeneration);
$attemptedAt, $attemptGeneration, $diagnostics);
return;
}
$diagnostics->stage('network_request');
try {
$networkPayload = $this->engine->host(
$hostId, 'GET', 'host/network-options', $actor
$networkPayload = $this->engine->hostInventory(
$hostId, 'host/network-options', $actor, $diagnostics->id()
);
} catch (CloudInventoryRequestError $error) {
$this->failed($hostId, $this->engineFailure($error), $result,
$attemptedAt, $attemptGeneration, $diagnostics,
$error->diagnosticCode, $error->upstreamHttpStatus);
return;
} catch (CloudError $error) {
$this->failed($hostId, $this->engineFailure($error), $result,
$attemptedAt, $attemptGeneration);
$attemptedAt, $attemptGeneration, $diagnostics);
return;
} catch (\Throwable $error) {
$this->failed($hostId,
['unavailable', self::ERROR_UNAVAILABLE], $result,
$attemptedAt, $attemptGeneration);
$attemptedAt, $attemptGeneration, $diagnostics);
return;
}
$diagnostics->stage('network_validate');
try {
$network = self::network($networkPayload);
[$capacity, $pools] = self::reconcilePublicInventory(
@ -187,10 +208,11 @@ final class CloudHostInventorySyncService
} catch (\UnexpectedValueException $error) {
$this->failed($hostId,
['degraded', self::ERROR_INVALID_NETWORK], $result,
$attemptedAt, $attemptGeneration);
$attemptedAt, $attemptGeneration, $diagnostics);
return;
}
$diagnostics->stage('database_apply');
try {
$applied = $this->apply(
$hostId, $capacity, $pools, $attemptedAt,
@ -198,32 +220,39 @@ final class CloudHostInventorySyncService
);
if (!$applied) {
$result['skipped']++;
$diagnostics->finish('superseded');
return;
}
$result['succeeded']++;
$diagnostics->finish('succeeded', '', null, true);
} catch (\UnexpectedValueException $error) {
$this->failed($hostId,
['degraded', self::ERROR_INVALID_NETWORK], $result,
$attemptedAt, $attemptGeneration);
$attemptedAt, $attemptGeneration, $diagnostics);
} catch (\DomainException $error) {
$this->failed($hostId,
['degraded', self::ERROR_INVENTORY_CONFLICT], $result,
$attemptedAt, $attemptGeneration);
$attemptedAt, $attemptGeneration, $diagnostics);
} catch (\Throwable $error) {
$this->failed($hostId,
['degraded', self::ERROR_WRITE_FAILED], $result,
$attemptedAt, $attemptGeneration);
$attemptedAt, $attemptGeneration, $diagnostics);
}
}
private function failed(int $hostId, array $failure, array &$result,
int $attemptedAt, int $attemptGeneration): void
int $attemptedAt, int $attemptGeneration,
CloudHostInventoryDiagnostics $diagnostics, string $diagnosticCode = '',
?int $upstreamStatus = null): void
{
[$health, $code] = $failure;
$diagnosticCode = $diagnosticCode === '' ? $code : $diagnosticCode;
$outcome = $diagnosticCode === 'engine_http_timeout' ? 'timeout' : 'failed';
try {
if (!$this->markFailure($hostId, $health, $code, $attemptedAt,
$attemptGeneration)) {
$result['skipped']++;
$diagnostics->finish('superseded', $diagnosticCode, $upstreamStatus);
return;
}
$result[$health]++;
@ -234,8 +263,9 @@ final class CloudHostInventorySyncService
// The lifecycle pass must continue to other hosts and billing
// policy. Never log an upstream error body from this boundary.
error_log('cloud host inventory failure marker unavailable: '
. $code . ' host=' . $hostId);
. $code . ' host=' . $hostId . ' attempt=' . $diagnostics->id());
}
$diagnostics->finish($outcome, $diagnosticCode, $upstreamStatus);
}
/** Claim a monotonic per-host observation generation before network I/O. */
@ -265,7 +295,7 @@ final class CloudHostInventorySyncService
);
}
private function engineFailure(CloudError $error): array
private function engineFailure(CloudError|CloudInventoryRequestError $error): array
{
if (in_array($error->httpStatus, [408, 429, 502, 503, 504], true)) {
return ['unavailable', self::ERROR_UNAVAILABLE];

@ -0,0 +1,15 @@
<?php
namespace App\Services\Cloud;
/** Inventory-only transport evidence; never contains an upstream error body. */
final class CloudInventoryRequestError extends \RuntimeException
{
public function __construct(
public readonly string $diagnosticCode,
public readonly ?int $upstreamHttpStatus = null,
public readonly int $httpStatus = 502,
) {
parent::__construct('宿主盘点请求未成功');
}
}

@ -166,6 +166,7 @@ $app->group('/admin', function () {
$this->get('/resources', 'App\Controllers\Admin\CloudPageController:resources');
// Host preparation (admin only; backend integration is not enabled yet)
$this->get('/hosts', 'App\Controllers\Admin\CloudPageController:hosts');
$this->get('/host-inventory', 'App\Controllers\Admin\CloudPageController:hostInventory');
$this->get('/cloud/templates', 'App\Controllers\Admin\CloudPageController:templates');
$this->get('/cloud/rescue', 'App\Controllers\Admin\CloudPageController:rescue');
$this->get('/cloud/instances', 'App\Controllers\Admin\CloudPageController:instances');

@ -68,6 +68,11 @@ $app->group('/api/v1', function () {
$this->get('/admin/operations', 'App\Controllers\CloudAdminOperationsController:index');
$this->get('/admin/operations/health', 'App\Controllers\CloudAdminRuntimeController:index');
$this->get('/admin/operations/jobs/{id:[0-9]+}', 'App\Controllers\CloudAdminOperationsController:detail');
$this->get('/admin/host-inventory', 'App\Controllers\CloudAdminHostInventoryController:index');
$this->get('/admin/host-inventory/{id:[0-9a-f]{32}}',
'App\Controllers\CloudAdminHostInventoryController:detail');
$this->get('/admin/host-inventory/{id:[0-9a-f]{32}}/engine',
'App\Controllers\CloudAdminHostInventoryController:engine');
$this->get('/admin/billing/invoices',
'App\Controllers\CloudAdminBillingController:invoices');
$this->get('/admin/billing/invoices/{id:[0-9]+}',

@ -0,0 +1,312 @@
const root = document.querySelector('[data-admin-host-inventory]');
const byId = id => document.getElementById(id);
const filterNames = ['host_id', 'outcome', 'per_page'];
const columns = ['宿主机 / 来源', '开始 / 结束', '耗时 / 预算', '面板结果', '阶段 / 错误码', '容量生效', '操作'];
const stageLabels = {
request: '引擎接收请求', registry_profile: '解析宿主机配置',
update_lock_wait: '等待全局锁', update_check: '检查在线更新状态',
iso_gc: '清理 ISO 待删除资源', template_gc: '清理镜像待删除资源',
profile: '装载宿主机凭据', libvirt_connect: '连接 libvirt', host_info: '读取宿主机资源',
domain_list: '枚举虚拟机', domain_scan: '扫描虚拟机', storage_scan: '读取存储池',
public_pool_catalog: '读取公网池目录', public_ip_claims: '读取公网 IP 占用',
public_ip_scan: '盘点公网 IP', response: '返回响应',
};
const traceStates = { running: '进行中', succeeded: '采集成功', failed: '失败', interrupted: '中断 / 未知' };
const state = { page: null, url: null, request: null, loading: false,
detailId: null, detailRequest: null, engineRequest: null, closing: false, scroll: null, opener: null };
function el(tag, className = '', value) {
const node = document.createElement(tag); node.className = className;
if (value !== undefined) node.textContent = String(value);
return node;
}
function text(value, fallback = '未记录') {
return value === null || value === undefined || String(value).trim() === '' ? fallback : String(value);
}
function date(value) {
if (!value || !Number.isFinite(Number(value))) return '尚未记录';
const parsed = new Date(Number(value) * 1000);
if (!Number.isFinite(parsed.getTime())) return '时间无效';
// Match the server-rendered rows even when the administrator's browser uses
// another time zone. Elapsed durations still come from each process's clock.
return parsed.toLocaleString('zh-CN', { hour12: false, timeZone: root?.dataset.timezone || 'UTC' });
}
function duration(value) {
return value === null || value === undefined || !Number.isFinite(Number(value)) ? '未确认' : `${Number(value).toLocaleString('zh-CN')} ms`;
}
function applied(attempt) {
if (attempt.applied === true) return '容量已更新';
if (attempt.outcome === 'superseded') return '结果已被取代';
if (attempt.outcome === 'interrupted') return '是否生效待核对';
return attempt.outcome === 'running' ? '尚未确认' : '容量未更新';
}
function stack(primary, ...secondary) {
const node = el('span', 'admin-checkout-user');
node.append(primary instanceof Node ? primary : el('strong', '', primary));
for (const value of secondary.filter(Boolean)) node.append(value instanceof Node ? value : el('span', '', value));
return node;
}
function infoList(entries) {
const list = el('dl', 'admin-instance-details');
for (const [label, value] of entries) list.append(el('dt', '', label), el('dd', 'text-break', text(value)));
return list;
}
function notice(message, error = false) {
const node = byId('inventory-notice'); node.textContent = message;
node.className = `admin-config-status ${error ? 'is-error' : 'is-success'} mb-3`;
node.setAttribute('role', error ? 'alert' : 'status');
}
function detailNotice(message, error = false) {
const node = byId('inventory-detail-state'); node.textContent = message; node.hidden = !message;
node.className = `alert alert-${error ? 'danger' : 'info'}`;
node.setAttribute('role', error ? 'alert' : 'status');
}
async function api(path, controller, timeoutMs = 15000) {
const timer = setTimeout(() => controller.abort('timeout'), timeoutMs);
try {
const response = await fetch(path, { headers: { Accept: 'application/json' },
credentials: 'same-origin', cache: 'no-store', signal: controller.signal });
if (response.status === 401) throw new Error('登录已过期,请重新登录。');
if (response.status === 403) throw new Error('当前账号没有查看权限。');
let payload;
try { payload = await response.json(); } catch { throw new Error('服务器响应格式无效,请稍后重试。'); }
if (!response.ok) throw new Error(typeof payload?.detail === 'string' ? payload.detail : '诊断记录暂不可读取,请稍后重试。');
return payload;
} catch (error) {
if (controller.signal.reason === 'timeout') throw new Error('读取诊断超时;未触发新的盘点。');
throw error;
} finally { clearTimeout(timer); }
}
function listUrl(input = new URL(location.href)) {
const url = new URL('/admin/host-inventory', location.origin);
for (const key of [...filterNames, 'page']) {
const value = input.searchParams.get(key);
if (value) url.searchParams.set(key, value);
}
return url;
}
function readForm(url) {
const form = byId('inventory-filter');
for (const key of filterNames) {
const control = form.elements.namedItem(key);
const value = url.searchParams.get(key) || ({ outcome: 'all', per_page: '20' })[key] || '';
if (value && !Array.from(control.options).some(option => option.value === value)) control.add(new Option(value, value));
control.value = value;
}
}
function formUrl() {
const url = new URL('/admin/host-inventory', location.origin);
for (const key of filterNames) {
const value = byId('inventory-filter').elements.namedItem(key).value;
if (value) url.searchParams.set(key, value);
}
return url;
}
function pagination() {
const page = state.page?.pagination;
byId('inventory-count').textContent = page ? `共 ${page.total} 条` : '数据未获取';
byId('inventory-pagination').hidden = !page?.total;
byId('inventory-page-label').textContent = page ? `第 ${page.page} / ${page.total_pages} 页 · 共 ${page.total} 条` : '记录暂不可用';
byId('inventory-previous').disabled = state.loading || !page || page.page <= 1;
byId('inventory-next').disabled = state.loading || !page || page.page >= page.total_pages;
byId('inventory-refresh').disabled = state.loading;
byId('inventory-body').setAttribute('aria-busy', String(state.loading));
}
function renderRows() {
const body = byId('inventory-body'); body.replaceChildren();
for (const attempt of state.page.items) {
const row = el('tr'); const button = el('button', 'btn btn-primary btn-sm', '查看详情');
button.type = 'button'; button.dataset.inventoryDetail = String(attempt.attempt_id);
const statusClass = attempt.outcome === 'succeeded' ? 'text-success'
: (['failed', 'timeout'].includes(attempt.outcome) ? 'text-danger' : (attempt.outcome === 'interrupted' ? 'text-warning' : ''));
const cells = [
stack(`${text(attempt.host_name, '宿主机已移除')} · #${attempt.host_id}`, attempt.source_label),
stack(date(attempt.started_at), attempt.finished_at ? date(attempt.finished_at) : (attempt.outcome === 'running' ? '尚未结束' : '结束时间未确认')),
stack(`${attempt.duration_is_final === false ? '已记录 ' : ''}${duration(attempt.duration_ms)}`, `每请求 ${duration(attempt.timeout_ms)}`),
stack(el('strong', statusClass, text(attempt.outcome_label)), attempt.error_message ? el('span', 'text-danger', attempt.error_message) : ''),
stack(text(attempt.stage_label, '阶段未确认'), text(attempt.error_code, '未记录错误码')),
applied(attempt), button,
];
cells.forEach((value, index) => {
const cell = el('td'); cell.dataset.label = columns[index];
cell.append(value instanceof Node ? value : el('span', '', value)); row.append(cell);
});
body.append(row);
}
if (!state.page.items.length) {
const row = el('tr'); const cell = el('td', 'text-center text-muted py-4', '没有符合条件的记录;新功能只记录启用后的盘点尝试。');
cell.colSpan = columns.length; row.append(cell); body.append(row);
}
}
function applyPage(page, replaceRows = true) {
if (!Array.isArray(page?.items) || !page.pagination || !page.filters || !page.summary) throw new Error('盘点记录格式无效,请重试。');
state.page = page;
const url = new URL('/admin/host-inventory', location.origin);
for (const key of [...filterNames, 'page']) {
const value = ['page', 'per_page'].includes(key) ? page.pagination[key] : page.filters[key];
if (value !== null && value !== undefined && value !== '') url.searchParams.set(key, String(value));
}
state.url = url;
const host = byId('inventory-host'); host.replaceChildren(new Option('全部宿主机', ''));
for (const item of page.hosts || []) host.add(new Option(`${item.name} · #${item.id}`, item.id));
readForm(url);
for (const key of ['last_attempt_at', 'last_success_at', 'last_failure_at']) byId(`inventory-summary-${key}`).textContent = date(page.summary[key]);
byId('inventory-summary-state').textContent = `记录中的进行中:${page.summary.in_progress} 次。`
+ (page.filters.host_id ? `该宿主机连续失败:${page.summary.consecutive_failures} 次。` : '选择宿主机可查看连续失败次数。');
if (replaceRows) renderRows();
pagination();
}
async function loadPage(url, historyMode = 'push') {
state.request?.abort(); const controller = new AbortController(); state.request = controller;
state.loading = true; pagination(); notice('正在读取数据库记录…');
try {
const payload = await api(`/api/v1/admin/host-inventory${url.search}`, controller);
if (controller.signal.aborted || state.request !== controller) return;
applyPage(payload);
if (historyMode === 'push') history.pushState(history.state, '', state.url);
notice(`已读取 ${payload.pagination.total} 条盘点记录;未执行宿主机扫描。`);
} catch (error) {
if (state.request !== controller || (controller.signal.aborted && controller.signal.reason !== 'timeout')) return;
notice(`${error.message}${state.page ? ' 下方保留上次成功读取的记录。' : ''}`, true);
} finally {
if (state.request === controller) { state.request = null; state.loading = false; pagination(); }
}
}
function renderStages(stages, engine = false) {
const list = el('ul', 'list-unstyled mb-0');
for (const item of (Array.isArray(stages) ? stages : [])) {
const label = engine ? (stageLabels[item.stage] || text(item.stage, '阶段未确认')) : text(item.label, item.stage);
const status = engine ? ` · ${traceStates[item.state] || text(item.state)}` : '';
list.append(el('li', 'text-break py-1', `${label}${status} · ${duration(item.duration_ms)}`));
}
if (!list.children.length) list.append(el('li', 'text-muted', '未记录阶段耗时。'));
return list;
}
function renderDetail(payload) {
const attempt = payload?.attempt;
if (!attempt || String(attempt.attempt_id) !== state.detailId) throw new Error('记录与请求不一致,请重新打开。');
byId('inventory-detail-title').textContent = `${text(attempt.host_name, '宿主机已移除')} · ${text(attempt.outcome_label)}`;
const body = byId('inventory-detail-body');
body.replaceChildren(el('h3', 'h6', '面板同步结果'), infoList([
['尝试编号', attempt.attempt_id], ['宿主机', `${text(attempt.host_name)} · #${attempt.host_id}`],
['来源', attempt.source_label], ['面板结果', attempt.outcome_label], ['容量是否生效', applied(attempt)],
['开始时间', date(attempt.started_at)], ['结束时间', attempt.finished_at ? date(attempt.finished_at) : '尚未确认'],
[attempt.duration_is_final === false ? '已记录耗时(非最终)' : '总耗时', duration(attempt.duration_ms)], ['每个引擎请求预算', duration(attempt.timeout_ms)],
['面板记录阶段', attempt.stage_label], ['错误码', attempt.error_code],
['脱敏说明', text(attempt.error_message, '未记录错误')], ['引擎 HTTP 状态', attempt.http_status],
['面板诊断版本', attempt.panel_version], ['PHP 版本', attempt.php_version], ['同步代际', attempt.generation],
]), el('h3', 'h6 mt-4', 'PHP 阶段耗时'), renderStages(attempt.stages));
for (const note of payload.notes || []) body.append(el('p', 'text-muted fs-13 mt-3 mb-0', note));
byId('inventory-engine').hidden = false;
byId('inventory-engine-refresh').disabled = false;
}
function renderEvidence(payload) {
if (!payload || payload.attempt_id !== state.detailId || !Array.isArray(payload.traces)) throw new Error('引擎线索与请求不一致。');
const box = byId('inventory-engine-body'); box.replaceChildren();
byId('inventory-engine-state').textContent = text(payload.message,
payload.available ? '已读取引擎记录;不改变面板同步结果。' : '无法取得引擎线索,内部阶段尚未确认。');
if (!payload.available) {
if (payload.error_code) box.append(el('p', 'text-muted text-break', `诊断查询错误码:${payload.error_code}`));
return;
}
if (!payload.traces.length) { box.append(el('p', 'text-muted', '没有对应引擎记录:可能尚未接收、未启用记录或已过保留期;不能据此认定宿主机故障。')); return; }
for (const trace of payload.traces) {
const section = el('section', 'border-top pt-3 mt-3');
section.append(el('h4', 'h6', ({ capacity: '容量请求', network: '公网网络请求' })[trace.resource] || '引擎请求'), infoList([
['引擎结果(独立)', traceStates[trace.state] || text(trace.state)],
['最后确认阶段', stageLabels[trace.stage] || text(trace.stage, '尚未确认')],
['引擎开始时间', date(trace.started_at)], ['引擎结束时间', trace.finished_at ? date(trace.finished_at) : '尚未确认'],
['引擎耗时', duration(trace.duration_ms)], ['引擎 HTTP 状态', trace.http_status],
['引擎错误码', trace.error_code], ['引擎运行版本', trace.engine_version],
]), renderStages(trace.stages, true));
box.append(section);
}
box.append(el('p', 'text-muted fs-13 mt-3', '各进程独立计时,不能直接相减面板与引擎的时间。引擎父阶段包含子阶段耗时,不能相加计算总时长。只有阶段开始记录时,不能将该阶段直接认定为故障根因。'));
}
async function readEvidence() {
if (!state.detailId || state.engineRequest || state.closing) return;
const id = state.detailId; const controller = new AbortController(); state.engineRequest = controller;
byId('inventory-engine-refresh').disabled = true;
byId('inventory-engine-state').textContent = '正在读取引擎已有记录,不会扫描宿主机…';
byId('inventory-engine-body').replaceChildren();
try {
const payload = await api(`/api/v1/admin/host-inventory/${id}/engine`, controller, 5000);
if (controller.signal.aborted || state.detailId !== id || state.engineRequest !== controller) return;
renderEvidence(payload);
} catch (error) {
if (state.detailId !== id || state.engineRequest !== controller || (controller.signal.aborted && controller.signal.reason !== 'timeout')) return;
byId('inventory-engine-state').textContent = `${error.message} 引擎内部阶段尚未确认。`;
} finally {
if (state.engineRequest === controller) { state.engineRequest = null; byId('inventory-engine-refresh').disabled = false; }
}
}
async function openDetail(id, opener) {
if (!/^[0-9a-f]{32}$/.test(id) || state.closing) return;
state.detailRequest?.abort(); state.engineRequest?.abort(); state.engineRequest = null;
state.detailId = id; state.opener = opener;
const controller = new AbortController(); state.detailRequest = controller;
byId('inventory-detail-title').textContent = '盘点诊断'; byId('inventory-detail-body').replaceChildren();
byId('inventory-engine').hidden = true; byId('inventory-engine-body').replaceChildren();
byId('inventory-engine-state').textContent = '尚未查询。仅读取已存在的线索,不会发起盘点。';
detailNotice('正在读取数据库记录…');
if (!byId('inventory-detail').open) {
state.scroll = { x: window.scrollX, y: window.scrollY };
document.body.style.setProperty('--admin-instance-scroll-top', `${-state.scroll.y}px`);
document.documentElement.classList.add('admin-instance-detail-open');
window.VenixDrawerMotion.openDrawerWithMotion(byId('inventory-detail'));
}
try {
const payload = await api(`/api/v1/admin/host-inventory/${id}`, controller);
if (controller.signal.aborted || state.detailId !== id || state.detailRequest !== controller) return;
renderDetail(payload); detailNotice('');
} catch (error) {
if (state.detailRequest !== controller || (controller.signal.aborted && controller.signal.reason !== 'timeout')) return;
detailNotice(`${error.message} 请关闭后重试。`, true);
} finally { if (state.detailRequest === controller) state.detailRequest = null; }
}
function closeDetail() {
if (state.closing || !byId('inventory-detail').open) return;
state.detailRequest?.abort(); state.engineRequest?.abort();
state.detailRequest = null; state.engineRequest = null; state.detailId = null; state.closing = true;
void window.VenixDrawerMotion.closeDrawerWithMotion(byId('inventory-detail')).then(closed => {
state.closing = false; if (!closed) return;
document.documentElement.classList.remove('admin-instance-detail-open');
document.body.style.removeProperty('--admin-instance-scroll-top');
if (state.scroll) window.scrollTo({ left: state.scroll.x, top: state.scroll.y, behavior: 'instant' });
state.scroll = null; if (state.opener?.isConnected) state.opener.focus({ preventScroll: true }); state.opener = null;
});
}
if (root) {
try {
const payload = JSON.parse(byId('admin-inventory-bootstrap').textContent);
if (payload) applyPage(payload, false); else readForm(listUrl());
} catch { notice('首屏诊断记录无效,请点击刷新记录重试。', true); readForm(listUrl()); }
byId('inventory-filter').addEventListener('submit', event => { event.preventDefault(); void loadPage(formUrl()); });
byId('inventory-reset').addEventListener('click', event => {
if (event.button !== 0 || event.ctrlKey || event.metaKey || event.altKey || event.shiftKey) return;
event.preventDefault(); void loadPage(new URL('/admin/host-inventory', location.origin));
});
byId('inventory-refresh').addEventListener('click', () => { void loadPage(listUrl(), 'none'); });
for (const [id, delta] of [['inventory-previous', -1], ['inventory-next', 1]]) byId(id).addEventListener('click', () => {
if (!state.page || state.loading) return;
const url = new URL(state.url); url.searchParams.set('page', state.page.pagination.page + delta); void loadPage(url);
});
root.addEventListener('click', event => {
const button = event.target.closest('[data-inventory-detail]');
if (button) void openDetail(button.dataset.inventoryDetail, button);
});
byId('inventory-detail-close').addEventListener('click', closeDetail);
byId('inventory-engine-refresh').addEventListener('click', () => { void readEvidence(); });
const dialog = byId('inventory-detail');
dialog.addEventListener('cancel', event => { event.preventDefault(); closeDetail(); });
dialog.addEventListener('click', event => {
if (event.target !== dialog) return;
const rect = dialog.getBoundingClientRect();
if (event.clientX < rect.left || event.clientX > rect.right || event.clientY < rect.top || event.clientY > rect.bottom) closeDetail();
});
window.addEventListener('popstate', () => { closeDetail(); readForm(listUrl()); void loadPage(listUrl(), 'none'); });
window.addEventListener('pagehide', () => { state.request?.abort(); state.detailRequest?.abort(); state.engineRequest?.abort(); closeDetail(); });
pagination();
// Intentionally no initial fetch, timer, visibility refresh or automatic engine query.
}

@ -98,7 +98,9 @@ function renderHosts() {
scheduling.type = 'button'; scheduling.addEventListener('click', () => openHostScheduling(host.id, scheduling));
const networkPools = element('button', 'btn btn-outline-primary btn-sm', '公网地址池');
networkPools.type = 'button'; networkPools.addEventListener('click', () => openHostNetworkPools(host.id, networkPools));
actions.append(scheduling, networkPools); row.append(button, actions); list.append(row);
const inventory = element('a', 'btn btn-outline-secondary btn-sm', '盘点诊断');
inventory.href = `/admin/host-inventory?host_id=${encodeURIComponent(host.id)}`;
actions.append(scheduling, networkPools, inventory); row.append(button, actions); list.append(row);
}
}
function schedulerLabel(value) { return ({ active: '调度启用', draining: '正在排空', disabled: '调度停用' })[value] || '调度未配置'; }

@ -0,0 +1,42 @@
{assign var=admin_title value='宿主机盘点诊断'}
{assign var=admin_active value='host-inventory'}
{include file='admin/static_header.tpl'}
{assign var=inventory_filters value=$inventory_page.filters|default:[]}
{assign var=inventory_outcomes value=['all'=>'全部结果','attention'=>'失败 / 超时 / 中断','running'=>'进行中','succeeded'=>'成功','failed'=>'失败','timeout'=>'超时','interrupted'=>'中断 / 未知','skipped'=>'跳过','superseded'=>'已被更新尝试取代']}
<main data-admin-host-inventory data-timezone="{$inventory_timezone|default:'UTC'|escape}">
<div class="py-3 d-flex align-items-sm-center flex-sm-row flex-column gap-3">
<div class="flex-grow-1"><h1 class="fs-20 fw-semibold mb-1">宿主机盘点诊断</h1><p class="text-muted mb-0">逐次查看自动与手动盘点的结果、失败阶段和耗时。刷新此页只读取记录,不会扫描宿主机。</p></div>
<div class="admin-checkout-actions"><a class="btn btn-light" href="/admin/hosts">宿主机设置</a><button class="btn btn-light" id="inventory-refresh" type="button"><i class="mdi mdi-refresh me-1" aria-hidden="true"></i>刷新记录</button></div>
</div>
<p class="admin-config-status {if $inventory_error|default:''}is-error{else}is-success{/if} mb-3" id="inventory-notice" role="status" aria-live="polite">{if $inventory_error|default:''}{$inventory_error|escape}{else}已读取 {$inventory_page.pagination.total|default:0|escape} 条盘点记录{/if}</p>
<section class="row g-3 mb-3" aria-label="所选宿主机盘点摘要">
{foreach ['last_attempt_at'=>'最近尝试','last_success_at'=>'最近成功','last_failure_at'=>'最近失败'] as $key=>$label}
<div class="col-md-4"><article class="card h-100"><div class="card-body"><h2 class="text-muted fs-14 fw-normal mb-2">{$label}</h2><strong class="fs-14" id="inventory-summary-{$key}">{if !$inventory_page}—{elseif $inventory_page.summary[$key]}{$inventory_page.summary[$key]|date_format:'%Y-%m-%d %H:%M:%S'|escape}{else}尚未记录{/if}</strong></div></article></div>
{/foreach}
<div class="col-12"><p class="text-muted fs-13 mb-0" id="inventory-summary-state">{if !$inventory_page}盘点摘要暂不可用。{else}记录中的进行中:{$inventory_page.summary.in_progress|default:0|escape} 次。{if $inventory_filters.host_id|default:''}该宿主机连续失败:{$inventory_page.summary.consecutive_failures|default:0|escape} 次。{else}选择宿主机可查看连续失败次数。{/if}{/if}</p></div>
</section>
<section class="card mb-3" aria-labelledby="inventory-filter-title">
<div class="card-header"><h2 class="h5 card-title mb-1" id="inventory-filter-title">筛选记录</h2><p class="text-muted fs-13 mb-0">摘要按所选宿主机统计,不受结果筛选影响。时间采用面板时区 {$inventory_timezone|default:'UTC'|escape};历史记录有保留上限。</p></div>
<div class="card-body"><form class="row g-2 align-items-end" id="inventory-filter" method="get" action="/admin/host-inventory">
<label class="col-md-5" for="inventory-host"><span class="form-label d-block">宿主机</span><select class="form-select" id="inventory-host" name="host_id"><option value="">全部宿主机</option>{foreach $inventory_page.hosts|default:[] as $host}<option value="{$host.id|escape}" {if $inventory_filters.host_id|default:'' == $host.id}selected{/if}>{$host.name|escape} · #{$host.id|escape}</option>{/foreach}</select></label>
<label class="col-md-5 col-8" for="inventory-outcome"><span class="form-label d-block">面板同步结果</span><select class="form-select" id="inventory-outcome" name="outcome">{foreach $inventory_outcomes as $value=>$label}<option value="{$value|escape}" {if $inventory_filters.outcome|default:'all' == $value}selected{/if}>{$label|escape}</option>{/foreach}</select></label>
<label class="col-md-2 col-4" for="inventory-per-page"><span class="form-label d-block">每页</span><select class="form-select" id="inventory-per-page" name="per_page">{foreach [20,50,100] as $size}<option value="{$size}" {if $inventory_page.pagination.per_page|default:20 == $size}selected{/if}>{$size} 条</option>{/foreach}</select></label>
<div class="col-12 admin-form-actions mt-2"><button class="btn btn-primary" type="submit">应用筛选</button><a class="btn btn-light" href="/admin/host-inventory" id="inventory-reset">清除筛选</a></div>
</form></div>
</section>
<section class="card" aria-labelledby="inventory-list-title">
<div class="card-header d-flex justify-content-between align-items-center gap-2 flex-wrap"><div><h2 class="h5 card-title mb-1" id="inventory-list-title">盘点尝试</h2><p class="text-muted fs-13 mb-0">成功表示面板校验与容量提交已完成;引擎晚到的成功不等于容量已更新。</p></div><span class="badge text-bg-light" id="inventory-count">{if $inventory_page}共 {$inventory_page.pagination.total|escape} 条{else}数据未获取{/if}</span></div>
<div class="card-body"><div class="table-responsive admin-checkout-table-wrap"><table class="table table-hover align-middle mb-0 admin-checkout-table"><caption class="visually-hidden">管理员宿主机盘点历史</caption><thead><tr>{foreach ['宿主机 / 来源','开始 / 结束','耗时 / 预算','面板结果','阶段 / 错误码','容量生效','操作'] as $column}<th scope="col">{$column}</th>{/foreach}</tr></thead><tbody id="inventory-body">{include file='admin/cloud/host_inventory_rows.tpl'}</tbody></table></div>
<nav class="admin-user-pagination" id="inventory-pagination" aria-label="盘点列表分页" {if !$inventory_page || !$inventory_page.pagination.total}hidden{/if}><button class="btn btn-light" id="inventory-previous" type="button" {if !$inventory_page || $inventory_page.pagination.page <= 1}disabled{/if}>上一页</button><span id="inventory-page-label">第 {$inventory_page.pagination.page|default:1|escape} / {$inventory_page.pagination.total_pages|default:1|escape} 页 · 共 {$inventory_page.pagination.total|default:0|escape} 条</span><button class="btn btn-light" id="inventory-next" type="button" {if !$inventory_page || $inventory_page.pagination.page >= $inventory_page.pagination.total_pages}disabled{/if}>下一页</button></nav>
</div>
</section>
<noscript><p class="alert alert-info mt-3">当前已显示数据库记录。启用 JavaScript 后可翻页、查看阶段详情并手动读取引擎线索。</p></noscript>
</main>
<dialog data-admin-instance-drawer id="inventory-detail" aria-labelledby="inventory-detail-title">
<header class="admin-instance-drawer-header"><div><p class="text-muted fs-13 mb-1">盘点尝试详情</p><h2 class="h4 fw-semibold mb-0" id="inventory-detail-title">盘点诊断</h2></div><button class="admin-modal-close position-static flex-shrink-0" id="inventory-detail-close" type="button" aria-label="关闭盘点详情" autofocus><i class="mdi mdi-close" aria-hidden="true"></i></button></header>
<div class="admin-instance-drawer-body"><p class="alert alert-info" id="inventory-detail-state" role="status" aria-live="polite">正在读取记录…</p><div id="inventory-detail-body"></div><section class="mt-4" id="inventory-engine" hidden aria-labelledby="inventory-engine-title"><div class="d-flex justify-content-between align-items-center gap-2 flex-wrap"><h3 class="h6 mb-0" id="inventory-engine-title">引擎独立记录</h3><button class="btn btn-light btn-sm" id="inventory-engine-refresh" type="button">读取引擎线索</button></div><p class="text-muted fs-13 mt-2" id="inventory-engine-state" role="status" aria-live="polite">尚未查询。仅读取已存在的线索,不会发起盘点。</p><div id="inventory-engine-body"></div></section></div>
<footer class="admin-instance-drawer-footer"><span class="text-muted fs-12">最后确认阶段不是已证明的根因;引擎结果不会修改面板的同步结果。</span></footer>
</dialog>
<script id="admin-inventory-bootstrap" type="application/json">{$inventory_bootstrap_json|default:'null' nofilter}</script>
<script type="module" src="/cloud/admin-host-inventory.js?v={$venix_asset_version}"></script>
{include file='admin/static_footer.tpl'}

@ -0,0 +1,11 @@
{foreach $inventory_page.items|default:[] as $attempt}
<tr>
<td data-label="宿主机 / 来源"><span class="admin-checkout-user"><strong>{$attempt.host_name|default:'宿主机已移除'|escape} · #{$attempt.host_id|escape}</strong><span>{$attempt.source_label|escape}</span></span></td>
<td data-label="开始 / 结束"><span class="admin-checkout-user"><strong>{$attempt.started_at|date_format:'%Y-%m-%d %H:%M:%S'|escape}</strong><span>{if $attempt.finished_at}{$attempt.finished_at|date_format:'%Y-%m-%d %H:%M:%S'|escape}{elseif $attempt.outcome == 'running'}尚未结束{else}结束时间未确认{/if}</span></span></td>
<td data-label="耗时 / 预算"><span class="admin-checkout-user"><strong>{if !$attempt.duration_is_final|default:false}已记录 {/if}{if $attempt.duration_ms !== null}{$attempt.duration_ms|escape} ms{else}未确认{/if}</strong><span>每请求 {$attempt.timeout_ms|escape} ms</span></span></td>
<td data-label="面板结果"><span class="admin-checkout-user"><strong class="{if $attempt.outcome == 'succeeded'}text-success{elseif $attempt.outcome == 'failed' || $attempt.outcome == 'timeout'}text-danger{elseif $attempt.outcome == 'interrupted'}text-warning{/if}">{$attempt.outcome_label|escape}</strong>{if $attempt.error_message}<span class="text-danger">{$attempt.error_message|escape}</span>{/if}</span></td>
<td data-label="阶段 / 错误码"><span class="admin-checkout-user"><strong>{$attempt.stage_label|default:'阶段未确认'|escape}</strong><span>{$attempt.error_code|default:'未记录错误码'|escape}</span></span></td>
<td data-label="容量生效">{if $attempt.applied}容量已更新{elseif $attempt.outcome == 'superseded'}结果已被取代{elseif $attempt.outcome == 'interrupted'}是否生效待核对{elseif $attempt.outcome == 'running'}尚未确认{else}容量未更新{/if}</td>
<td data-label="操作"><button class="btn btn-primary btn-sm" type="button" data-inventory-detail="{$attempt.attempt_id|escape}">查看详情</button></td>
</tr>
{foreachelse}<tr><td colspan="7" class="text-center text-muted py-4">{if $inventory_error|default:''}盘点记录暂不可读取,请确认数据库已升级后重试。{else}没有符合条件的记录;新功能只记录启用后的盘点尝试。{/if}</td></tr>{/foreach}

@ -4,6 +4,7 @@
{assign var=operations_filters value=$operations_page.filters|default:[]}
{assign var=operations_statuses value=['attention'=>'需要关注','all'=>'全部状态','queued'=>'已排队','running'=>'执行中','uncertain'=>'待核对','failed'=>'失败','succeeded'=>'已完成']}
<main data-admin-operations>
<p class="text-sm-end mb-0"><a class="btn btn-light btn-sm" href="/admin/host-inventory">查看宿主机盘点诊断</a></p>
<div class="py-3 d-flex align-items-sm-center flex-sm-row flex-column gap-3">
<div class="flex-grow-1"><h1 class="fs-20 fw-semibold mb-1">系统运行与异常任务</h1><p class="text-muted mb-0">查看后台运行状态与全局任务记录;仅供核对,不会重试、取消任务或触发退款。</p></div>
<div class="admin-checkout-actions"><span class="admin-config-status {if $operations_error|default:''}is-error{else}is-success{/if}" id="operations-notice" role="status" aria-live="polite">{if $operations_error|default:''}{$operations_error|escape}{else}已读取 {$operations_page.pagination.total|default:0|escape} 条任务{/if}</span><button class="btn btn-light" id="operations-refresh" type="button"><i class="mdi mdi-refresh me-1" aria-hidden="true"></i>刷新任务</button></div>

@ -11,6 +11,7 @@
<p class="text-muted mb-0">管理计算节点连接、存储目录和公网网络配置。</p>
</div>
<div class="text-sm-end mt-3 mt-sm-0 cloud-admin-actions">
<a class="btn btn-light" href="/admin/host-inventory">盘点诊断</a>
<p id="cloud-admin-status" class="alert alert-info cloud-admin-notice" role="status" aria-live="polite">正在加载…</p>
</div>
</div>

@ -163,6 +163,7 @@
<li><a class="{if $admin_active|default:'' == 'cloud-rescue'}active{/if}" href="/admin/cloud/rescue"><i class="mdi mdi-lifebuoy"></i><span>救援模版</span></a></li>
<li><a class="{if $admin_active|default:'' == 'cloud-instances'}active{/if}" href="/admin/cloud/instances"><i class="mdi mdi-server"></i><span>虚拟机管理</span></a></li>
<li><a class="{if $admin_active|default:'' == 'cloud-operations'}active{/if}" href="/admin/operations"><i class="mdi mdi-clipboard-pulse-outline"></i><span>系统运行与异常任务</span></a></li>
<li><a class="{if $admin_active|default:'' == 'host-inventory'}active{/if}" href="/admin/host-inventory"><i class="mdi mdi-clipboard-search-outline"></i><span>宿主机盘点诊断</span></a></li>
{if $admin_can_manage|default:false}<li><a class="{if $admin_active|default:'' == 'update'}active{/if}" href="/admin/update"><i class="mdi mdi-update"></i><span>在线更新</span></a></li>{/if}

@ -1,2 +1,2 @@
{* Shared Venix CSS/JS cache key. Increment whenever a shared browser asset changes. *}
{assign var=venix_asset_version value='2026100401' scope='global'}
{assign var=venix_asset_version value='2026100402' scope='global'}

@ -247,6 +247,32 @@ CREATE TABLE IF NOT EXISTS cloud_host_capacity_snapshots (
KEY cloud_host_capacity_sync (sync_state, last_attempt_at, host_id)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
-- Independent, bounded diagnostic history. These rows never authorize placement
-- and an engine finishing after HTTP timeout cannot rewrite the panel outcome.
CREATE TABLE IF NOT EXISTS cloud_host_inventory_attempts (
attempt_id CHAR(32) CHARACTER SET ascii COLLATE ascii_bin NOT NULL PRIMARY KEY,
sequence_id BIGINT UNSIGNED NOT NULL AUTO_INCREMENT UNIQUE,
host_id BIGINT UNSIGNED NOT NULL,
source VARCHAR(16) CHARACTER SET ascii COLLATE ascii_bin NOT NULL,
generation BIGINT UNSIGNED NOT NULL DEFAULT 0,
started_at BIGINT UNSIGNED NOT NULL,
finished_at BIGINT UNSIGNED DEFAULT NULL,
updated_at BIGINT UNSIGNED NOT NULL,
duration_ms BIGINT UNSIGNED NOT NULL DEFAULT 0,
outcome VARCHAR(16) CHARACTER SET ascii COLLATE ascii_bin NOT NULL DEFAULT 'running',
stage VARCHAR(32) CHARACTER SET ascii COLLATE ascii_bin NOT NULL DEFAULT 'admission',
error_code VARCHAR(64) CHARACTER SET ascii COLLATE ascii_bin NOT NULL DEFAULT '',
http_status SMALLINT UNSIGNED DEFAULT NULL,
applied TINYINT(1) NOT NULL DEFAULT 0,
timeout_ms INT UNSIGNED NOT NULL,
panel_version VARCHAR(64) CHARACTER SET ascii COLLATE ascii_bin NOT NULL,
php_version VARCHAR(32) CHARACTER SET ascii COLLATE ascii_bin NOT NULL,
stages_json TEXT NOT NULL,
KEY cloud_inventory_host_time (host_id, started_at, sequence_id),
KEY cloud_inventory_outcome_time (outcome, started_at, sequence_id),
KEY cloud_inventory_retention (started_at, sequence_id)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
-- PHP reserves one concrete address before a paid provisioning job is queued.
-- The private engine still validates the claim on the selected host. An
-- uncertain operation keeps the address unavailable until reconciliation.

@ -0,0 +1,171 @@
import assert from 'node:assert/strict';
import { readFile } from 'node:fs/promises';
import vm from 'node:vm';
const source = await readFile(new URL('../public/cloud/admin-host-inventory.js', import.meta.url), 'utf8');
class Node {
constructor(tag = 'div') {
this.tagName = tag.toUpperCase(); this.children = []; this.dataset = {}; this.attributes = new Map();
this.listeners = new Map(); this.hidden = false; this.disabled = false; this.isConnected = true;
this.open = false; this.value = ''; this._text = ''; this.style = { setProperty() {}, removeProperty() {} };
const classes = new Set(); this.classList = { add: x => classes.add(x), remove: x => classes.delete(x) };
}
set textContent(value) { this._text = String(value); this.children = []; }
get textContent() { return this._text + this.children.map(item => item.textContent).join(''); }
set innerHTML(_) { throw new Error('Never interpret diagnostic content as HTML.'); }
get options() { return this.children; }
append(...values) { this.children.push(...values); }
add(option) { this.children.push(option); }
replaceChildren(...values) { this._text = ''; this.children = values; }
setAttribute(key, value) { this.attributes.set(key, String(value)); }
addEventListener(key, handler) { const listeners = this.listeners.get(key) || []; listeners.push(handler); this.listeners.set(key, listeners); }
dispatch(key, fields = {}) {
const event = { target: this, button: 0, defaultPrevented: false, preventDefault() { this.defaultPrevented = true; }, ...fields };
for (const listener of this.listeners.get(key) || []) listener(event);
}
closest(selector) { return selector === '[data-inventory-detail]' && this.dataset.inventoryDetail ? this : null; }
focus() {}
getBoundingClientRect() { return { left: 0, right: 500, top: 0, bottom: 800 }; }
}
class Option extends Node { constructor(label, value) { super('option'); this.textContent = label; this.value = String(value); } }
const flush = async () => { for (let i = 0; i < 20; i += 1) await Promise.resolve(); };
const firstId = 'a'.repeat(32);
const secondId = 'b'.repeat(32);
function attempt(id = firstId, changes = {}) {
return { attempt_id: id, host_id: 7, host_name: '示例宿主机', source: 'manual', source_label: '管理员手动同步',
started_at: 1234, finished_at: 1242, duration_ms: 8000, duration_is_final: true,
outcome: 'timeout', outcome_label: '超时', stage: 'capacity_request', stage_label: '请求容量数据',
error_code: 'engine_http_timeout', error_message: '超时;引擎阶段未确认', applied: false,
timeout_ms: 8000, http_status: null, panel_version: 'inventory-v1', php_version: '8.5.0', generation: 1, stages: [], ...changes };
}
function page(items = [attempt()], pageNumber = 1) {
return { items, pagination: { page: pageNumber, per_page: 20, total: 21, total_pages: 2 },
filters: { host_id: '7', outcome: 'all', page: pageNumber, per_page: 20 },
hosts: [{ id: '7', name: '示例宿主机' }],
summary: { last_attempt_at: 1234, last_success_at: 1230, last_failure_at: 1242, in_progress: 0, consecutive_failures: 1 } };
}
function fixture(initial = page()) {
const ids = [...source.matchAll(/byId\('([^']+)'\)/g)].map(match => match[1]);
ids.push('inventory-previous', 'inventory-next', 'inventory-outcome', 'inventory-per-page', ...['last_attempt_at', 'last_success_at', 'last_failure_at'].map(key => `inventory-summary-${key}`));
const nodes = Object.fromEntries([...new Set(ids)].map(id => [id, new Node()]));
nodes['admin-inventory-bootstrap'].textContent = JSON.stringify(initial);
nodes['inventory-body'].textContent = 'SSR records';
nodes['inventory-filter'].elements = { namedItem: key => nodes[({ host_id: 'inventory-host', outcome: 'inventory-outcome', per_page: 'inventory-per-page' })[key]] };
const root = new Node(); const document = new Node(); document.documentElement = new Node(); document.body = new Node();
document.getElementById = id => nodes[id]; document.createElement = tag => new Node(tag);
document.querySelector = selector => selector === '[data-admin-host-inventory]' ? root : null;
const window = new Node(); window.scrollX = 0; window.scrollY = 100; window.scrollTo = () => {};
window.VenixDrawerMotion = {
openDrawerWithMotion: dialog => { dialog.open = true; },
closeDrawerWithMotion: dialog => { dialog.open = false; return Promise.resolve(true); },
};
const location = { origin: 'https://example.test', href: 'https://example.test/admin/host-inventory?host_id=7' };
const history = { state: null, pushState(value, _, url) { this.state = value; location.href = String(url); } };
let now = 0; let lastTimer = 0; const timers = new Map(); const requests = [];
const setTimeout = (callback, delay) => { const id = ++lastTimer; timers.set(id, { callback, due: now + delay }); return id; };
const fetch = (url, options) => new Promise((resolve, reject) => { requests.push({ url, options, resolve, reject, settled: false }); });
vm.runInNewContext(source, { document, window, location, history, URL, Node, Option, Date, AbortController,
fetch, setTimeout, clearTimeout: id => timers.delete(id), console }, { filename: 'admin-host-inventory.js' });
return {
nodes, root, window, requests, location,
async click(id) { if (!nodes[id].disabled) nodes[id].dispatch('click'); await flush(); },
async open(id = firstId) { const button = new Node('button'); button.dataset.inventoryDetail = id; root.dispatch('click', { target: button }); await flush(); },
async respond(payload, request = requests.find(item => !item.settled), status = 200) {
assert.ok(request); request.settled = true; request.resolve({ ok: status < 300, status, json: async () => payload }); await flush();
},
async advance(ms) {
const end = now + ms;
while (true) {
const next = [...timers.entries()].filter(([, timer]) => timer.due <= end).sort((a, b) => a[1].due - b[1].due)[0];
if (!next) break;
now = next[1].due; timers.delete(next[0]); next[1].callback(); await flush();
}
now = end; await flush();
},
};
}
// SSR and idle pages never trigger requests. Refresh reads only history, and
// opening a detail does not retrieve engine evidence until the explicit click.
{
const test = fixture(); await test.advance(120000);
assert.equal(test.requests.length, 0, 'The SSR page must not refetch or poll.');
assert.equal(test.nodes['inventory-body'].textContent, 'SSR records');
await test.click('inventory-refresh'); assert.equal(test.requests.length, 1);
assert.match(test.requests[0].url, /^\/api\/v1\/admin\/host-inventory\?host_id=7$/);
const unsafe = attempt(firstId, { host_name: '<img src=x onerror=alert(1)>', error_message: '<script>bad()</script>' });
await test.respond(page([unsafe]));
assert.match(test.nodes['inventory-body'].textContent, /<script>bad\(\)<\/script>/);
assert.match(test.nodes['inventory-body'].textContent, /容量未更新/);
await test.open(); assert.equal(test.requests.length, 2);
await test.respond({ attempt: unsafe, notes: [] }); await test.advance(100000);
assert.equal(test.requests.length, 2, 'Details must not auto-fetch or poll the engine.');
assert.match(test.nodes['inventory-detail-body'].textContent, /超时/);
await test.click('inventory-engine-refresh'); await test.click('inventory-engine-refresh');
assert.equal(test.requests.length, 3, 'Repeated evidence clicks must share a single read.');
assert.equal(test.requests[2].url, `/api/v1/admin/host-inventory/${firstId}/engine`);
await test.respond({ available: true, attempt_id: firstId, traces: [{ resource: 'capacity', state: 'succeeded', stage: 'response',
started_at: 1234, finished_at: 1245, duration_ms: 11000, error_code: '', http_status: 200, engine_version: 'a'.repeat(40), stages: [] }] });
assert.match(test.nodes['inventory-engine-body'].textContent, /采集成功/);
assert.match(test.nodes['inventory-engine-body'].textContent, /父阶段包含子阶段耗时/);
assert.match(test.nodes['inventory-detail-body'].textContent, /超时/);
assert.match(test.nodes['inventory-detail-body'].textContent, /容量未更新/, 'Late engine success must not rewrite panel application state.');
assert.equal(test.requests.every(request => request.options.method === undefined), true, 'Diagnostics only uses GET.');
}
// Late detail/evidence responses cannot populate a different or closed drawer.
{
const test = fixture(); await test.open(firstId); const oldDetail = test.requests[0];
await test.open(secondId); await test.respond({ attempt: attempt(secondId), notes: [] }, test.requests[1]);
await test.respond({ attempt: attempt(firstId, { host_name: 'obsolete' }), notes: [] }, oldDetail);
assert.doesNotMatch(test.nodes['inventory-detail-body'].textContent, /obsolete/);
await test.click('inventory-engine-refresh'); const evidence = test.requests[2];
await test.click('inventory-detail-close'); assert.equal(evidence.options.signal.aborted, true);
await test.open(firstId); await test.respond({ attempt: attempt(), notes: [] }, test.requests[3]);
await test.respond({ available: false, attempt_id: secondId, traces: [], message: 'stale evidence' }, evidence);
assert.doesNotMatch(test.nodes['inventory-engine-state'].textContent, /stale evidence/);
}
// Empty evidence and failed reads must remain honest, with no implicit rescan.
{
const test = fixture(); await test.open(); await test.respond({ attempt: attempt(), notes: [] });
await test.click('inventory-engine-refresh');
await test.respond({ available: true, attempt_id: firstId, traces: [] });
assert.match(test.nodes['inventory-engine-body'].textContent, /不能据此认定宿主机故障/);
await test.click('inventory-engine-refresh');
await test.respond({ available: false, attempt_id: firstId, traces: [], error_code: 'engine_connect_failed', message: '诊断存储不可用' });
assert.match(test.nodes['inventory-engine-state'].textContent, /诊断存储不可用/);
assert.doesNotMatch(test.nodes['inventory-engine-body'].textContent, /没有对应引擎记录/);
await test.advance(120000); assert.equal(test.requests.length, 3);
}
// Pagination and browser history requests stay DB-only; older list responses
// cannot overwrite a newer filter result, even if abort is not honored.
{
const test = fixture(); await test.click('inventory-refresh'); const oldList = test.requests[0];
test.nodes['inventory-filter'].dispatch('submit'); await flush();
await test.respond(page([attempt(secondId)]), test.requests[1]);
await test.respond(page([attempt(firstId, { host_name: 'obsolete list' })]), oldList);
assert.doesNotMatch(test.nodes['inventory-body'].textContent, /obsolete list/);
await test.click('inventory-next'); assert.match(test.requests[2].url, /page=2/);
await test.respond(page([attempt(secondId)], 2));
assert.match(test.nodes['inventory-page-label'].textContent, /第 2 \/ 2 页/);
assert.equal(test.nodes['inventory-next'].disabled, true);
test.window.dispatch('popstate'); await flush();
assert.equal(test.requests[3].url.includes('/engine'), false);
}
// A bounded diagnostic lookup timeout remains an unknown diagnostic read, not
// a newly failed inventory, and does not retry itself.
{
const test = fixture(); await test.open(); await test.respond({ attempt: attempt(), notes: [] });
await test.click('inventory-engine-refresh'); const request = test.requests[1];
await test.advance(5000); assert.equal(request.options.signal.aborted, true);
request.reject(new Error('aborted')); await flush();
assert.match(test.nodes['inventory-engine-state'].textContent, /读取诊断超时/);
assert.match(test.nodes['inventory-engine-state'].textContent, /内部阶段尚未确认/);
assert.match(test.nodes['inventory-detail-body'].textContent, /容量未更新/);
await test.advance(120000); assert.equal(test.requests.length, 2);
}
console.log('Inventory UI SSR/no-scan, escaping, manual evidence, late-result isolation and pagination passed.');

@ -0,0 +1,123 @@
<?php
require dirname(__DIR__) . '/vendor/autoload.php';
set_error_handler(static function (int $severity, string $message, string $file, int $line): bool {
if ((error_reporting() & $severity) && ($severity & (E_WARNING | E_USER_WARNING | E_DEPRECATED))) {
throw new ErrorException($message, 0, $severity, $file, $line);
}
return false;
});
$assert = static function (bool $ok, string $message): void {
if (!$ok) throw new RuntimeException($message);
};
$now = time();
$attempt = [
'attempt_id' => str_repeat('a', 32), 'host_id' => 7, 'host_name' => '示例宿主机 A',
'source' => 'manual', 'source_label' => '管理员手动同步',
'started_at' => $now - 20, 'finished_at' => $now - 12, 'duration_ms' => 8010,
'duration_is_final' => true, 'outcome' => 'timeout', 'outcome_label' => '超时',
'stage' => 'capacity_request', 'stage_label' => '请求容量数据',
'error_code' => 'engine_http_timeout', 'error_message' => '请求超时;引擎内部阶段尚未确认',
'applied' => false, 'timeout_ms' => 8000, 'panel_version' => 'inventory-diagnostics-v1',
'php_version' => '8.5.0', 'http_status' => null, 'generation' => 12,
'stages' => [['stage' => 'capacity_request', 'label' => '请求容量数据', 'duration_ms' => 8000]],
];
$page = [
'items' => [$attempt],
'pagination' => ['page' => 1, 'per_page' => 20, 'total' => 21, 'total_pages' => 2, 'from' => 1, 'to' => 1],
'filters' => ['host_id' => '7', 'outcome' => 'timeout', 'page' => 1, 'per_page' => 20],
'hosts' => [['id' => '7', 'name' => '示例宿主机 A']],
'summary' => ['last_attempt_at' => $now - 20, 'last_success_at' => $now - 100,
'last_failure_at' => $now - 12, 'in_progress' => 0, 'consecutive_failures' => 1],
];
$smarty = new Smarty();
$compileDir = sys_get_temp_dir() . '/admin-inventory-view-' . bin2hex(random_bytes(6));
mkdir($compileDir);
$smarty->setCompileDir($compileDir);
$smarty->setTemplateDir(dirname(__DIR__) . '/resources/views/venix');
$smarty->assign('config', ['appName' => 'Venix Test']);
$smarty->assign('cdn', '');
$smarty->assign('user', (object) ['isLogin' => false]);
$smarty->assign('admin_can_manage', false);
$smarty->assign('admin_csrf_token', 'fixture-not-a-real-token');
$render = static function (?array $payload, string $error = '') use ($smarty): string {
$smarty->assign('inventory_page', $payload);
$smarty->assign('inventory_error', $error);
$smarty->assign('inventory_bootstrap_json', json_encode($payload,
JSON_HEX_TAG | JSON_HEX_AMP | JSON_HEX_APOS | JSON_HEX_QUOT
| JSON_UNESCAPED_UNICODE | JSON_INVALID_UTF8_SUBSTITUTE));
return $smarty->fetch('admin/cloud/host_inventory.tpl');
};
$html = $render($page);
if (in_array('--fixture', $argv ?? [], true)) {
echo json_encode(['html' => $html, 'page' => $page,
'detail' => ['attempt' => $attempt, 'notes' => []]], JSON_UNESCAPED_UNICODE);
exit;
}
$dom = new DOMDocument();
libxml_use_internal_errors(true);
$dom->loadHTML('<?xml encoding="UTF-8">' . $html);
$xpath = new DOMXPath($dom);
$assert($xpath->query('//tbody[@id="inventory-body"]/tr')->length === 1,
'Inventory history must render on the server.');
$assert(str_contains($dom->textContent, 'engine_http_timeout')
&& str_contains($dom->textContent, '容量未更新') && str_contains($dom->textContent, '管理员手动同步'),
'Initial rows must expose diagnostic results without JavaScript.');
$assert($xpath->query('//select[@name="host_id"]/option[@value="7" and @selected]')->length === 1
&& $xpath->query('//select[@name="outcome"]/option[@value="timeout" and @selected]')->length === 1,
'Server-rendered filters must preserve the requested host and outcome.');
$assert($xpath->query('//dialog[@data-admin-instance-drawer and @id="inventory-detail"]')->length === 1,
'Diagnostic details must reuse the Venix drawer.');
$assert($xpath->query('//section[@id="inventory-engine" and @hidden]')->length === 1,
'Engine evidence must not pretend to be loaded during server rendering.');
$assert($xpath->query('//main[@data-timezone="UTC"]')->length === 1
&& str_contains($html, '时间采用面板时区 UTC'),
'SSR and JavaScript timestamps must share an explicit server time zone.');
$assert($xpath->query('//button[contains(@id,"sync") or contains(@id,"retry") or contains(@id,"delete")]')->length === 0,
'The diagnostic page must not expose mutating controls.');
$unsafe = $page;
$unsafe['items'][0]['host_name'] = '</script><img src=x onerror=alert(1)>';
$unsafe['items'][0]['error_message'] = '<script>window.unsafe=1</script>';
$unsafe['hosts'][0]['name'] = '" onmouseover="alert(2)';
$unsafeHtml = $render($unsafe);
$dom->loadHTML('<?xml encoding="UTF-8">' . $unsafeHtml);
$xpath = new DOMXPath($dom);
$assert($xpath->query('//*[@onerror or @onmouseover]')->length === 0
&& !str_contains($unsafeHtml, '<script>window.unsafe=1</script>')
&& str_contains($dom->textContent, '</script><img src=x onerror=alert(1)>'),
'Database labels must be escaped in both templates and JSON bootstraps.');
$empty = $page;
$empty['items'] = [];
$empty['pagination']['total'] = 0;
$assert(str_contains($render($empty), '新功能只记录启用后的盘点尝试'), 'Empty history requires an honest explanation.');
$failedHtml = $render(null, '数据库暂不可读');
$assert(str_contains($failedHtml, '数据库暂不可读') && !str_contains($failedHtml, '已读取 0 条'),
'Unavailable records must not be rendered as an empty successful list.');
$unknown = $page;
$unknown['items'][0] = array_replace($attempt, ['outcome' => 'interrupted',
'outcome_label' => '中断 / 未知', 'finished_at' => null, 'duration_is_final' => false]);
$assert(str_contains($render($unknown), '是否生效待核对') && str_contains($render($unknown), '已记录 8010 ms'),
'Interrupted attempts must not claim final duration or definite non-application.');
$root = dirname(__DIR__);
$controller = file_get_contents($root . '/app/Controllers/CloudAdminHostInventoryController.php');
$pageController = file_get_contents($root . '/app/Controllers/Admin/CloudPageController.php');
$routes = file_get_contents($root . '/config/routes_cloud.php');
$middleware = file_get_contents($root . '/app/Middleware/CloudApi.php');
$js = file_get_contents($root . '/public/cloud/admin-host-inventory.js');
$assert(str_contains($routes, "'/admin/host-inventory/{id:[0-9a-f]{32}}/engine'")
&& str_contains($middleware, '^/api/v1/admin/host-inventory(?:/|$)')
&& str_contains($controller, 'CloudAccess::actor(false)'),
'Diagnostic APIs need signed-in admin identity without creating engine sessions.');
$assert(str_contains($pageController, "'inventory_bootstrap_json'")
&& str_contains($controller, 'CloudHostInventoryDiagnostics::detail($actor, $attemptId)'),
'Engine evidence must authorize and bind host identity through a persisted attempt.');
$assert(str_contains($controller, 'new CloudEngineClient(null, 1.0)')
&& str_contains($controller, '->inventoryDiagnostics(')
&& !str_contains($controller, '->inventory(') && !str_contains($controller, 'syncOne('),
'Evidence retrieval must be a bounded read-only lookup, never another inventory.');
$assert(!str_contains($js, 'innerHTML') && !str_contains($js, "method: 'POST'")
&& !str_contains($js, '/capacity-sync') && !str_contains($js, 'setInterval('),
'UI must be text-only and must not poll or run inventory.');
echo "Administrator inventory SSR, escaping, read-only routes and diagnostic boundaries passed.\n";

@ -0,0 +1,31 @@
<?php
use Illuminate\Database\Schema\Blueprint;
/** SQLite-only test fixture; production uses the MariaDB source schema. */
function createCloudHostInventoryAttemptTable($schema): void
{
$schema->create('cloud_host_inventory_attempts', static function (Blueprint $table): void {
$table->increments('sequence_id');
$table->string('attempt_id', 32)->unique();
$table->integer('host_id');
$table->string('source', 16);
$table->integer('generation')->default(0);
$table->integer('started_at');
$table->integer('finished_at')->nullable();
$table->integer('updated_at');
$table->integer('duration_ms')->default(0);
$table->string('outcome', 16)->default('running');
$table->string('stage', 32)->default('admission');
$table->string('error_code', 64)->default('');
$table->integer('http_status')->nullable();
$table->boolean('applied')->default(false);
$table->integer('timeout_ms');
$table->string('panel_version', 64);
$table->string('php_version', 32);
$table->text('stages_json');
$table->index(['host_id', 'started_at', 'sequence_id']);
$table->index(['outcome', 'started_at', 'sequence_id']);
$table->index(['started_at', 'sequence_id']);
});
}

@ -0,0 +1,210 @@
<?php
namespace App\Services\Cloud {
final class CloudEngineClient
{
public function __construct()
{
throw new \RuntimeException('History reads must never contact the engine.');
}
}
}
namespace {
require dirname(__DIR__) . '/vendor/autoload.php';
require __DIR__ . '/CloudHostInventoryDiagnosticsFixture.php';
use App\Models\CloudHostInventoryAttempt;
use App\Services\Cloud\CloudError;
use App\Services\Cloud\CloudHostInventoryDiagnostics as Diagnostics;
use Illuminate\Database\Capsule\Manager as Capsule;
use Illuminate\Database\Schema\Blueprint;
$assert = static function ($condition, string $message): void {
if (!$condition) {
throw new \RuntimeException($message);
}
};
$reject = static function (callable $call, int $status) use ($assert): void {
try {
$call();
} catch (CloudError $error) {
$assert($error->httpStatus === $status, 'Unexpected error status.');
return;
}
throw new \RuntimeException('Expected request rejection.');
};
$capsule = new Capsule();
$capsule->addConnection(['driver' => 'sqlite', 'database' => ':memory:'], 'default');
$capsule->setAsGlobal();
$capsule->bootEloquent();
$schema = $capsule->schema();
$schema->create('cloud_hosts', static function (Blueprint $table): void {
$table->increments('id');
$table->string('name');
$table->string('credential_cipher')->default('never-read-this-secret');
});
createCloudHostInventoryAttemptTable($schema);
Capsule::table('cloud_hosts')->insert([
['id' => 1, 'name' => 'Host One'], ['id' => 2, 'name' => 'Host Two'],
]);
$actor = ['is_admin' => true];
$now = time();
$success = Diagnostics::start(1, 'auto', $now, 8.0);
$assert(preg_match('/^[a-f0-9]{32}$/D', $success->id()) === 1, 'Attempt IDs must be stable opaque identifiers.');
$success->generation(10);
$success->stage('capacity_request');
$success->stage('capacity_validate');
$success->stage('network_request');
$success->stage('network_validate');
$success->stage('database_apply');
$success->finish('succeeded', '', null, true);
$beforeLateFinish = CloudHostInventoryAttempt::find($success->id())->toArray();
$success->finish('failed', 'unexpected_error');
$success->stage('capacity_request');
$success->generation(500);
$assert(CloudHostInventoryAttempt::find($success->id())->toArray() === $beforeLateFinish,
'Terminal outcomes must not change on a second finish or a late phase.');
$dto = Diagnostics::detail($actor, $success->id())['attempt'];
$assert($dto['outcome'] === 'succeeded' && $dto['applied'] && $dto['generation'] === 10,
'Successful commit metadata was lost.');
$assert($dto['timeout_ms'] === 8000 && $dto['finished_at'] >= $dto['started_at']
&& $dto['duration_ms'] >= 0 && count($dto['stages']) === 6,
'Timeout and monotonic phase durations must be recorded.');
$assert($dto['panel_version'] === 'inventory-diagnostics-v1', 'Never pretend runtime version is current disk Git HEAD.');
$timeout = Diagnostics::start(1, 'manual', $now, 8.0);
$timeout->generation(11);
$timeout->stage('capacity_request');
$timeout->finish('timeout', 'engine_http_timeout', null, false);
$failed = Diagnostics::start(1, 'auto', $now, 8.0);
$failed->generation(12);
$failed->stage('network_request');
$failed->finish('failed', 'engine_resource_missing', 404, false);
$unrelated = Diagnostics::start(2, 'auto', $now, 8.0);
$unrelated->finish('succeeded', '', null, true);
$summary = Diagnostics::page($actor, ['host_id' => '1'])['summary'];
$assert($summary['consecutive_failures'] === 2,
'Same-second failures must not disappear behind the earlier success timestamp.');
$assert($summary['last_success_at'] !== null && $summary['last_failure_at'] !== null,
'The last success/failure times must survive subsequent attempts.');
$assert(Diagnostics::page($actor)['summary']['consecutive_failures'] === null,
'A global failure streak across unrelated hosts is misleading.');
$superseded = Diagnostics::start(1, 'manual', $now, 8.0);
$superseded->finish('superseded');
$assert(Diagnostics::page($actor, ['host_id' => 1])['summary']['consecutive_failures'] === 2,
'Superseded/skipped attempts cannot be counted as confirmed failures.');
$running = Diagnostics::start(1, 'auto', $now, 8.0);
$running->stage('capacity_request');
$assert(Diagnostics::page($actor, ['host_id' => 1])['summary']['in_progress'] === 1,
'Recent running attempt should be visible.');
CloudHostInventoryAttempt::where('attempt_id', $running->id())->update(['updated_at' => $now - 301]);
$interrupted = Diagnostics::detail($actor, $running->id())['attempt'];
$assert($interrupted['outcome'] === 'interrupted' && $interrupted['finished_at'] === null
&& $interrupted['last_confirmed_stage'] === 'capacity_request',
'Abandoned attempts must display unknown without inventing a completion or failure cause.');
$assert(CloudHostInventoryAttempt::find($running->id())->outcome === 'running',
'Read-side interpretation must not mutate diagnostic history.');
$assert(Diagnostics::page($actor, ['host_id' => 1, 'outcome' => 'attention'])['pagination']['total'] === 3,
'Attention filter should include failed, timed-out, and interrupted records.');
$assert(Diagnostics::page($actor, ['outcome' => 'running'])['pagination']['total'] === 0,
'Interrupted records must not appear as actively running.');
$assert(Diagnostics::page($actor, ['outcome' => 'interrupted'])['pagination']['total'] === 1,
'Interrupted filter must match stale ongoing records.');
$malicious = Diagnostics::start(1, 'Bearer never-store-source-secret', $now, 8.0);
$malicious->stage('password=never-store-stage-secret');
$malicious->finish('failed', 'token=never-store-error-secret', 999, true);
$maliciousDto = Diagnostics::detail($actor, $malicious->id())['attempt'];
$assert($maliciousDto['error_code'] === 'unexpected_error' && $maliciousDto['stage'] === 'admission'
&& $maliciousDto['source'] === 'auto' && $maliciousDto['http_status'] === null && !$maliciousDto['applied'],
'Unknown raw diagnostics must be reduced to fixed safe values.');
CloudHostInventoryAttempt::where('attempt_id', $malicious->id())->update([
'stages_json' => '[{"stage":"password=never-display-stage-secret","duration_ms":4},'
. '{"stage":"capacity_request","duration_ms":5,"password":"never-display-extra-secret"}]',
'error_code' => 'token=never-display-error-secret',
]);
$clean = json_encode(Diagnostics::detail($actor, $malicious->id()), JSON_THROW_ON_ERROR);
$assert(!str_contains($clean, 'never-') && !str_contains($clean, 'credential'),
'Only allowlisted diagnostic keys may be exposed by the DTO.');
$assert(count(Diagnostics::detail($actor, $malicious->id())['attempt']['stages']) === 1,
'Unsafe stage entries must be discarded, extra keys stripped.');
$reject(static fn () => Diagnostics::page(['is_admin' => false]), 403);
$reject(static fn () => Diagnostics::detail([], $success->id()), 403);
$reject(static fn () => Diagnostics::detail($actor, '../secret'), 400);
$reject(static fn () => Diagnostics::detail($actor, str_repeat('0', 32)), 404);
foreach ([['host_id' => '-1'], ['outcome' => 'injected'], ['page' => '0'],
['per_page' => '10000'], ['host_id' => []], ['q' => 'not-supported']] as $input) {
$reject(static fn () => Diagnostics::page($actor, $input), 400);
}
$lastPage = Diagnostics::page($actor, ['page' => '900']);
$assert($lastPage['pagination']['page'] === $lastPage['pagination']['total_pages'],
'Out-of-range pages should clamp to the last page.');
// Capture only page/detail queries: they must never write or load host secrets.
$connection = $capsule->getConnection('default');
$connection->enableQueryLog();
Diagnostics::page($actor, ['host_id' => '1']);
Diagnostics::detail($actor, $timeout->id());
foreach ($connection->getQueryLog() as $query) {
$assert(str_starts_with(strtolower(ltrim($query['query'])), 'select'),
'Viewing diagnostic records must be database-read-only.');
$assert(!str_contains($query['query'], 'credential_cipher'), 'History must not select host credentials.');
}
$connection->disableQueryLog();
// Old history is removed in a bounded batch on a new attempt, not on a read.
$seed = CloudHostInventoryAttempt::find($timeout->id())->getAttributes();
unset($seed['sequence_id']);
for ($i = 1; $i <= 300; $i++) {
$row = $seed;
$row['attempt_id'] = sprintf('%032x', $i);
$row['started_at'] = $now - 8 * 86400;
$row['updated_at'] = $row['started_at'];
$row['finished_at'] = $row['started_at'];
CloudHostInventoryAttempt::query()->insert($row);
}
Diagnostics::start(3, 'auto', $now, 8.0)->finish('skipped');
$assert(CloudHostInventoryAttempt::where('started_at', '<', $now - 7 * 86400)->count() === 44,
'Retention cleanup must be bounded rather than deleting unbounded history per sync.');
// The per-host cap is independent of the age cutoff, including a burst of
// same-second attempts. Bulk fixtures avoid one query per synthetic row.
$rows = [];
for ($i = 10000; $i < 35020; $i++) {
$row = $seed;
$row['attempt_id'] = sprintf('%032x', $i);
$row['host_id'] = 4;
$row['started_at'] = $now - 1;
$row['updated_at'] = $now - 1;
$row['finished_at'] = $now - 1;
$rows[] = $row;
if (count($rows) === 100) {
CloudHostInventoryAttempt::query()->insert($rows);
$rows = [];
}
}
if ($rows !== []) {
CloudHostInventoryAttempt::query()->insert($rows);
}
$capAttempt = Diagnostics::start(4, 'auto', $now, 8.0);
$capAttempt->finish('skipped');
$assert(CloudHostInventoryAttempt::where('host_id', 4)->count() === 25000
&& CloudHostInventoryAttempt::find($capAttempt->id()) !== null,
'The per-host cap must prune older rows while preserving the current attempt.');
// Diagnostic storage failure is fail-open and cannot abort the actual inventory.
$schema->drop('cloud_host_inventory_attempts');
$fallback = Diagnostics::start(1, 'manual', $now, 8.0);
$fallback->stage('capacity_request');
$fallback->generation(20);
$fallback->finish('timeout', 'engine_http_timeout');
$assert(strlen($fallback->id()) === 32, 'An ID is still needed for engine correlation when DB logging fails.');
$reject(static fn () => Diagnostics::page($actor), 503);
$reject(static fn () => Diagnostics::detail($actor, $fallback->id()), 503);
echo "Cloud inventory diagnostic history tests passed.\n";
}

@ -18,8 +18,8 @@ $assert = static function ($condition, string $message): void {
}
};
$assert(str_contains($service, "'GET', 'host/capacity'")
&& str_contains($service, "'GET', 'host/network-options'")
$assert(str_contains($service, "'host/capacity', \$actor, \$diagnostics->id()")
&& str_contains($service, "'host/network-options', \$actor, \$diagnostics->id()")
&& str_contains($service, 'private const ENGINE_TIMEOUT_SECONDS = 8.0;')
&& str_contains($service, 'public_ip_available_count')
&& str_contains($service, "['available', 'disabled']")

@ -27,9 +27,11 @@ namespace App\Services\Cloud {
namespace {
require dirname(__DIR__) . '/vendor/autoload.php';
require __DIR__ . '/CloudHostInventoryDiagnosticsFixture.php';
use App\Models\CloudHost;
use App\Models\CloudHostCapacitySnapshot;
use App\Models\CloudHostInventoryAttempt;
use App\Models\CloudHostNetworkAttachment;
use App\Models\CloudHostResourcePoolMembership;
use App\Models\CloudPublicIp;
@ -163,6 +165,7 @@ namespace {
$capsule->setAsGlobal();
$capsule->bootEloquent();
$schema = $capsule->schema('default');
createCloudHostInventoryAttemptTable($schema);
$schema->create('user', function (Blueprint $table): void {
$table->increments('id');
$table->integer('enable')->default(1);
@ -447,14 +450,27 @@ namespace {
->public_ip_pool_id === (int) $publicPools[$poolB]->id,
'A new address was written without its public-pool association.');
$assert(count($history) === 2
&& (string) $history[0]['request']->getUri()
=== 'http://127.0.0.1:8000/internal/v1/hosts/1/host/capacity'
&& (string) $history[1]['request']->getUri()
=== 'http://127.0.0.1:8000/internal/v1/hosts/1/host/network-options',
&& $history[0]['request']->getUri()->getPath()
=== '/internal/v1/hosts/1/host/capacity'
&& $history[1]['request']->getUri()->getPath()
=== '/internal/v1/hosts/1/host/network-options'
&& preg_match('/^inventory_attempt_id=[0-9a-f]{32}$/D',
$history[0]['request']->getUri()->getQuery()) === 1
&& $history[0]['request']->getUri()->getQuery()
=== $history[1]['request']->getUri()->getQuery(),
'Lifecycle inventory did not use both bounded host endpoints.');
$actorHeader = $history[0]['request']->getHeaderLine('X-Cloud-Actor');
$actorHeader .= str_repeat('=', (4 - strlen($actorHeader) % 4) % 4);
$actor = json_decode(base64_decode(strtr($actorHeader, '-_', '+/'), true), true);
$record = CloudHostInventoryAttempt::orderByDesc('sequence_id')->firstOrFail();
$assert($record->outcome === 'succeeded' && $record->applied
&& $record->source === 'auto' && $record->stage === 'database_apply'
&& (int) $record->timeout_ms === 3000
&& (int) $record->started_at === 1000
&& (int) $record->finished_at >= 1000
&& $history[0]['request']->getUri()->getQuery()
=== 'inventory_attempt_id=' . $record->attempt_id,
'A full committed inventory needs its own successful correlated history record.');
$assert(($actor['user_id'] ?? null) === 99
&& ($actor['session_id'] ?? null) === 'cloud-host-inventory-sync'
&& ($actor['is_super_admin'] ?? null) === true,
@ -500,6 +516,11 @@ namespace {
&& $snapshot->last_error_code === 'host_inventory_network_invalid'
&& $states === $beforeStates && $afterPools === $beforePools,
'Failure cleared capacity, pool metadata or concrete IP state.');
$record = CloudHostInventoryAttempt::orderByDesc('sequence_id')->firstOrFail();
$assert($record->outcome === 'failed' && !$record->applied
&& $record->source === 'manual' && $record->stage === 'network_validate'
&& $record->error_code === 'host_inventory_network_invalid',
'Network validation failure must retain its layer and outcome.');
$now = 1020;
$unavailableClient = $client([
@ -520,6 +541,11 @@ namespace {
&& $host->last_seen_at === 1000 && $host->health_checked_at === 1020
&& !str_contains($persisted, 'private engine path'),
'Engine failure cleared last success or persisted an upstream error.');
$record = CloudHostInventoryAttempt::orderByDesc('sequence_id')->firstOrFail();
$assert($record->error_code === 'engine_service_error' && (int) $record->http_status === 503
&& $record->stage === 'capacity_request' && !$record->applied
&& !str_contains(json_encode($record->toArray()), 'private engine path'),
'Transport and upstream HTTP evidence must survive safe classification.');
$reflection = new ReflectionClass(CloudHostInventorySyncService::class);
$beginAttempt = $reflection->getMethod('beginAttempt');
@ -736,5 +762,37 @@ namespace {
&& $host->scheduler_state === 'disabled',
'Clearing the authoritative membership recreated a stale scalar link.');
echo "Cloud host multi-pool inventory and membership checks passed.\n";
$now = 1060;
$timeoutClient = $client([new \GuzzleHttp\Exception\ConnectException('SECRET timeout',
new \GuzzleHttp\Psr7\Request('GET', 'http://127.0.0.1'), null, ['errno' => 28])]);
$timedOut = (new CloudHostInventorySyncService($timeoutClient,
static function () use (&$now): int { return $now; }))->syncOne(1);
$record = CloudHostInventoryAttempt::orderByDesc('sequence_id')->firstOrFail();
$assert($timedOut['unavailable'] === 1 && $record->outcome === 'timeout'
&& $record->error_code === 'engine_http_timeout' && !$record->applied
&& $record->stage === 'capacity_request' && $record->http_status === null,
'A timeout needs durable evidence without inventing an engine failure phase.');
$now = 1070;
$supersededClient = $client([
static function () use ($capsule, $capacity, $json, &$now) {
$capsule->getConnection()->table('cloud_host_capacity_snapshots')
->where('host_id', 1)->increment('attempt_generation');
return new Response(200, [], $json($capacity($now)));
},
new Response(200, [], $json($network())),
]);
$superseded = (new CloudHostInventorySyncService($supersededClient,
static function () use (&$now): int { return $now; }))->syncOne(1);
$record = CloudHostInventoryAttempt::orderByDesc('sequence_id')->firstOrFail();
$assert($superseded['skipped'] === 1 && $record->outcome === 'superseded' && !$record->applied,
'A stale generation must be retained as superseded, never successful.');
$schema->drop('cloud_host_inventory_attempts');
$withoutHistory = (new CloudHostInventorySyncService($client([
new Response(503, [], '{"error":"SECRET"}'),
]), static function () use (&$now): int { return $now; }))->syncOne(1);
$assert($withoutHistory['unavailable'] === 1,
'Unavailable diagnostic storage must not prevent ordinary failure handling.');
echo "Cloud host multi-pool inventory, diagnostic history and membership checks passed.\n";
}

@ -0,0 +1,122 @@
<?php
namespace App\Services {
final class Config
{
public static function get($key, $default = null) { return $default; }
}
}
namespace App\Services\Cloud {
final class CloudSecrets
{
public static function key(string $purpose): string { return str_repeat('k', 32); }
}
}
namespace {
require dirname(__DIR__) . '/vendor/autoload.php';
use App\Services\Cloud\CloudEngineClient;
use App\Services\Cloud\CloudError;
use App\Services\Cloud\CloudInventoryRequestError;
use GuzzleHttp\Client;
use GuzzleHttp\Exception\ConnectException;
use GuzzleHttp\Handler\MockHandler;
use GuzzleHttp\HandlerStack;
use GuzzleHttp\Middleware;
use GuzzleHttp\Psr7\Request;
use GuzzleHttp\Psr7\Response;
$assert = static function ($ok, string $message): void {
if (!$ok) { throw new \RuntimeException($message); }
};
$client = static function (array $replies, array &$history): CloudEngineClient {
$history = [];
$handler = HandlerStack::create(new MockHandler($replies));
$handler->push(Middleware::history($history));
return new CloudEngineClient(new Client(['handler' => $handler]), 8.0);
};
$actor = ['user_id' => 1, 'is_admin' => true, 'is_super_admin' => true];
$id = str_repeat('a', 32);
$history = [];
$engine = $client([new Response(200, [], '{"health":"healthy"}')], $history);
$assert($engine->hostInventory(1, 'host/capacity', $actor, $id)['health'] === 'healthy',
'Inventory response changed');
$request = $history[0]['request'];
$target = '/internal/v1/hosts/1/host/capacity?inventory_attempt_id=' . $id;
$expected = hash_hmac('sha256', CloudEngineClient::canonical('GET', $target, '',
$request->getHeaderLine('X-Cloud-Timestamp'), $request->getHeaderLine('X-Cloud-Nonce'),
$request->getHeaderLine('X-Cloud-Actor')), str_repeat('k', 32));
$assert($request->getRequestTarget() === $target
&& hash_equals($expected, $request->getHeaderLine('X-Cloud-Signature')),
'Correlation ID must be authenticated with the complete query');
$assert($history[0]['options']['timeout'] === 8.0
&& $history[0]['options']['connect_timeout'] === 3.0
&& $history[0]['options']['allow_redirects'] === false,
'Effective bounded inventory request options missing');
$cases = [
[new ConnectException('SECRET', new Request('GET', 'http://localhost'), null,
['errno' => 28]), 'engine_http_timeout', null, 502],
[new ConnectException('SECRET', new Request('GET', 'http://localhost'), null,
['errno' => 7]), 'engine_connect_failed', null, 502],
[new ConnectException('SECRET timed out', new Request('GET', 'http://localhost')),
'engine_connection_or_timeout', null, 502],
[new Response(401, [], 'SECRET'), 'engine_auth_rejected', 401, 502],
[new Response(403, [], 'SECRET'), 'engine_auth_rejected', 403, 502],
[new Response(404, [], 'SECRET'), 'engine_resource_missing', 404, 502],
[new Response(404, [], '{"error":"SECRET"}'), 'engine_resource_missing', 404, 404],
[new Response(409, [], 'SECRET'), 'engine_request_conflict', 409, 502],
[new Response(423, [], 'SECRET'), 'engine_configuration_locked', 423, 502],
[new Response(429, [], 'SECRET'), 'engine_rate_limited', 429, 502],
[new Response(500, [], 'SECRET'), 'engine_service_error', 500, 502],
[new Response(504, [], 'SECRET'), 'engine_http_timeout', 504, 502],
[new Response(200, [], 'SECRET'), 'engine_response_invalid', 200, 502],
];
foreach ($cases as [$reply, $code, $upstream, $legacy]) {
$engine = $client([$reply], $history);
try {
$engine->hostInventory(1, 'host/network-options', $actor, $id);
throw new \RuntimeException('Expected inventory failure');
} catch (CloudInventoryRequestError $error) {
$assert($error->diagnosticCode === $code && $error->upstreamHttpStatus === $upstream
&& $error->httpStatus === $legacy && !str_contains($error->getMessage(), 'SECRET'),
'Incorrect safe inventory error classification: ' . $code);
$assert(count($history) === 1, 'Errors must not trigger fallback scans');
}
}
$trace = ['resource' => 'capacity', 'state' => 'running', 'stage' => 'update_lock_wait',
'error_code' => '', 'http_status' => null, 'started_at' => time(), 'finished_at' => null,
'duration_ms' => 8001, 'engine_version' => 'SECRET', 'raw_error' => 'SECRET',
'stages' => [['stage' => 'update_lock_wait', 'state' => 'running', 'duration_ms' => 8000]]];
$engine = $client([new Response(200, [], json_encode(['attempt_id' => $id,
'traces' => [$trace], 'password' => 'SECRET']))], $history);
$evidence = $engine->inventoryDiagnostics(1, $actor, $id);
$assert($evidence['traces'][0]['stage'] === 'update_lock_wait'
&& !str_contains(json_encode($evidence), 'SECRET')
&& $history[0]['options']['timeout'] === 1.0
&& $history[0]['options']['connect_timeout'] === 1.0
&& $history[0]['request']->getUri()->getPath()
=== '/internal/v1/hosts/1/host/inventory-diagnostics/' . $id,
'Evidence reads must be bounded, sanitized and separate from inventory');
foreach ([['attempt_id' => str_repeat('b', 32), 'traces' => [$trace]],
['attempt_id' => $id, 'traces' => [array_replace($trace, ['stage' => 'SECRET'])]]] as $invalid) {
$engine = $client([new Response(200, [], json_encode($invalid))], $history);
try {
$engine->inventoryDiagnostics(1, $actor, $id);
throw new \RuntimeException('Invalid evidence accepted');
} catch (CloudInventoryRequestError $error) {
$assert($error->diagnosticCode === 'engine_response_invalid', 'Evidence contract failed');
}
}
// General VM mutation handling must retain its original error boundary.
$engine = $client([new Response(423, [], '{"error":"SECRET"}')], $history);
try {
$engine->host(1, 'POST', 'vms/example/start', $actor, []);
throw new \RuntimeException('Expected mutation error');
} catch (CloudError $error) {
$assert($error->httpStatus === 502 && !str_contains($error->getMessage(), 'SECRET'),
'Inventory diagnostics changed mutation error semantics');
}
echo "Inventory engine diagnostics client checks passed.\n";
}
Loading…
Cancel
Save