diff --git a/docs/DEPLOYMENT.md b/docs/DEPLOYMENT.md index f799609a..25046233 100644 --- a/docs/DEPLOYMENT.md +++ b/docs/DEPLOYMENT.md @@ -178,7 +178,11 @@ location = /engine-console { 1. 管理员添加宿主机:填写现有 SSH/root/私钥口令、从可信控制台核验指纹,并填写仅用于展示的位置标签、`default_network`、存储/ISO/救援目录后测试连接。位置标签不决定销售地区或 Fabric/VPC 归属;宿主机基础表单不再写入旧版单 CIDR、网关或 DNS 字段。`Fabric 传输 IPv4` 可留空:当 SSH 地址是字面 IPv4 时 Engine 会将它作为当前公网 underlay;SSH 使用域名或希望以后改走同地区内网时,应填写宿主机真实持有、所有地区成员都能访问的 IPv4。该地址只用于 WireGuard,不是 VM 公网网桥或地址池;修改后必须重新检测,不会自动迁移已固定的 Fabric 计划。 2. 在“资源调度”中把宿主机加入一个资源池,并设置容量、安全余量和调度状态。宿主机的权威销售地区由该成员关系沿资源池、线路到地区唯一确定,而不是读取宿主机位置标签。当前每台宿主机只有一个权威资源池成员关系,因此也只有该资源池对应的一条权威线路;一台宿主机挂多条线路、多个 bridge/VLAN 或多个路由表属于后续阶段。 3. 打开宿主机“公网地址池”,为同一个 `default_network` 添加一个或多个互不重叠的实际公网 IPv4 CIDR,分别填写网关、DNS、客户机前缀、保留地址和状态。面板先在事务中保存期望配置,再向引擎提交同一版本;引擎确认后标记为 `applied`,并由面板把 CIDR 物化为具体 IP 账本。网络超时或响应不确定时保留同一 `request_key` 和版本重试,不能创建第二个池。调度器只使用面板策略、reservation 和 IP 账本计算的剩余资源。完整顺序和失败/回滚边界见 [单宿主机多公网 IPv4 地址池](MULTI_PUBLIC_IPV4_POOLS.md)。 -4. 打开“地区私网”,选择一个地区后按页面向导执行“检测宿主机 → 自动配置地区私网 → 重新核对”。检测通过后先核对服务端只读方案摘要;不要准备或提交 Fabric JSON,也不要直接写地区映射。只有服务端阶段允许时相应按钮才会启用,自动配置或重新核对只有在最新权威投影到达 `active` 后才算成功。全新且无 desired state 的地区才使用“自动配置”;页面发现任何固定方案、历史 disabled 映射或确定性部分成功时,应按提示使用“重新核对”,不能重复创建。未绑定的自定义 Fabric 不会自动认领或兼容,会保持 `migration_required`。宿主机防火墙必须落入且只落入两种受支持模式之一:native `nftables.service` active/enabled,且 firewalld unit 不存在或 inactive+masked;或 firewalld active/enabled、`FirewallBackend=nftables`、`NftablesTableOwner=yes`,且 native `nftables.service` 不存在或 inactive+disabled/masked。仅 disabled 的 firewalld 仍可被 D-Bus 激活,不能进入 native 模式;混合、未知、source-bound zone 或运行态/持久 zone 不一致均 fail-closed。firewalld 模式只在 endpoint 是本机真实地址、普通路由和带该 endpoint 源地址的路由均选择同一地址归属接口、且该物理接口不在 `trusted` zone 时,自动添加目标为本机 endpoint `/32` 和固定 UDP 端口的精确 rich rule。endpoint 还必须位于全局租户 VPC CIDR 池之外;当前默认池为 `10.0.0.0/8`,后续内网 underlay 应使用与它隔离的地址规划。平台不会把所选 endpoint 的物理接口设为 `trusted`,也不会代替外部 NAT、云安全组或上游防火墙。两种模式都要求先完整回读原始 nft ACL,再激活平台创建的 WireGuard/私网接口;仅 firewalld 模式使用临时 runtime `trusted`,native 模式既不使用也不要求 firewalld zone/trusted。不要手工把这些接口永久加入 `trusted`。失败事务留下的 emergency guard 只能由相同 owner token 和完整规则快照证明后恢复或清理。只有没有 VPC state、panel manifest 或未完成 journal 的空宿主,才能在把一次失败的 firewalld 前置尝试纠正为 native 后,由“重新核对”接管并清理 guard;已有 VPC 的宿主切换防火墙 owner 是尚未自动化的维护迁移,必须先恢复原模式并在维护窗口迁出/清空 VPC,不能手工删除同名 nft table、owner intent、snapshot 或 manifest。按 [Fabric 宿主机验收矩阵](FABRIC_HOST_ACCEPTANCE.md) 验证 MTU、SELinux、firewalld/nftables 与网络管理器重启、libvirt 启动门禁、握手和跨宿主机流量。64 台探针的引擎窗口为 180 秒,浏览器为 240 秒;Web 服务器、反代和 PHP-FPM 的请求终止/读超时应至少配置为 270 秒。HTTP 超时、连接中断或 HTTP 5xx 不能视为失败或成功;不要重复点击自动配置,只使用“重新核对”读取并收敛固定 desired state。 +4. 打开“地区私网”,选择一个地区后按页面向导执行“检测宿主机 → 自动配置地区私网 → 重新核对”。检测通过后先核对服务端只读方案摘要;不要准备或提交 Fabric JSON,也不要直接写地区映射。只有服务端阶段允许时相应按钮才会启用,自动配置或重新核对只有在最新权威投影到达 `active` 后才算成功。全新且无 desired state 的地区才使用“自动配置”;页面发现任何固定方案、历史 disabled 映射或确定性部分成功时,应按提示使用“重新核对”,不能重复创建。未绑定的自定义 Fabric 不会自动认领或兼容,会保持 `migration_required`。 + + 宿主机防火墙只支持 native nftables:`nftables.service` 必须 active/enabled,且 firewalld unit 不存在或 inactive+masked。仅 disabled 的 firewalld 仍可被 D-Bus 激活,不得通过检测;实时规则中也不得存在 `inet firewalld` table。旧 `firewalld_nft` desired state、manifest、journal 或 zone/rich-rule 所有权一律 fail-closed;即使旧 journal 标记为 native,只要仍携带已退役的 `firewalld_*`/`underlay_*` schema 字段,也不会被普通 apply/reconcile 恢复或删除。测试环境需要转换时,应先停止租户流量并保存规则/journal 快照,在维护窗口清理旧所有权状态,然后以全新 native 方案重建;不要直接删除未完成事务的 nft table、owner intent、snapshot 或 manifest。 + + Endpoint 必须位于全局租户 VPC CIDR 池之外;当前默认池为 `10.0.0.0/8`,后续内网 underlay 应使用与它隔离的地址规划。panel-owned Fabric nft table 不负责放行 underlay UDP;运维必须在宿主持久 native nftables 规则或上游云安全组/防火墙中,以最窄可证明的 peer 源地址、本机 endpoint `/32`、UDP 和固定 Fabric 端口显式放行,外部 NAT 也由运维管理。平台先完整回读受管 nft ACL,再激活 WireGuard/私网接口;接口 active 或 unit green 不能代替双向握手和实际流量验收。按 [Fabric 宿主机验收矩阵](FABRIC_HOST_ACCEPTANCE.md) 验证 MTU、SELinux、native nftables 持久化与网络管理器重启、libvirt 启动门禁、握手和跨宿主机流量。64 台探针的引擎窗口为 180 秒,浏览器为 240 秒;Web 服务器、反代和 PHP-FPM 的请求终止/读超时应至少配置为 270 秒。HTTP 超时、连接中断或 HTTP 5xx 不能视为失败或成功;不要重复点击自动配置,只使用“重新核对”读取并收敛固定 desired state。 5. 导入云镜像与救援模板;无已有模板不能开通云镜像 VM。云镜像是面板全局目录,验证后的源文件持久保存在 `runtime/engine/cloud-templates/`;首次在某宿主机创建或重装时才同步一份缓存,之后复用,所以切换地区不会更换或刷新镜像列表。支持 AlmaLinux 8/9/10、Debian 11/12/13、CentOS Stream 9/10、Ubuntu 20.04/22.04/24.04/25.04/25.10/26.04 x86_64 云镜像。官方镜像可填写官方镜像站的 HTTPS 文件地址,引擎只跟随由官方入口返回且通过公网地址安全校验的跳转链;例如 AlmaLinux 8 可使用 `https://repo.almalinux.org/almalinux/8/cloud/x86_64/images/AlmaLinux-8-GenericCloud-latest.x86_64.qcow2`,Debian 12 可使用 `https://cloud.debian.org/images/cloud/bookworm/latest/debian-12-generic-amd64.qcow2`,CentOS Stream 9 可使用 `https://cloud.centos.org/centos/9-stream/x86_64/images/CentOS-Stream-GenericCloud-9-latest.x86_64.qcow2`,Ubuntu 24.04 可使用 `https://cloud-images.ubuntu.com/releases/noble/release/ubuntu-24.04-server-cloudimg-amd64.img`。自制或修复镜像使用管理员页面的本地 QCOW2 上传入口,并明确选择与内容一致的系统和版本;文件必须是 1 MiB 至 4 GiB 的独立、未加密 QCOW2,不能包含 backing file 或外部 data file,ISO 和 raw 镜像均不接受。面板磁盘需为上传临时文件、全局源镜像及 256 MiB 安全余量预留空间,各宿主机的镜像存储池也需容纳本地缓存。远程导入的 SHA-256 可留空;本地上传始终由面板计算、签名并保存实际摘要。 6. 套餐设置规格、每期价格、账期天数、每期流量额度和库存,并绑定可销售的资源池。用户提交部署后先生成 30 分钟有效的开通账单,支付成功才创建实例;实例到期前 7 天再生成续费账单。两种账单均可用余额支付,也可由启用的易支付、Paytaro 或 USDT 渠道生成支付订单。在线支付回调会自动结清对应账单;失效账单的晚到款只保留在账户余额中,不会创建实例或恢复已进入删除流程的实例。 7. 测试用户注册验证码、密码重置、充值、生成开通账单和账单支付,确认未支付时没有引擎任务,支付后只创建一个实例,并收到一次开通成功邮件;再验证到期前 7 天只生成一张续费账单并发送提醒、到期仅断网、宽限期内续费恢复网络,以及到期满 3 天后永久删除实例和系统盘;同时验收任务、VM 控制台、SSH、重装、救援、手动删除与图表。 diff --git a/docs/FABRIC_HOST_ACCEPTANCE.md b/docs/FABRIC_HOST_ACCEPTANCE.md index d83ace4a..e5aafab2 100644 --- a/docs/FABRIC_HOST_ACCEPTANCE.md +++ b/docs/FABRIC_HOST_ACCEPTANCE.md @@ -22,74 +22,55 @@ 两种来源字段都必须等于本机 endpoint,地址 owner 接口也必须与出口接口一致;loopback、dummy、 `wg-fabric0`、VPC bridge、多路径、错误源地址或不同出口均应阻断自动放行。 -## 防火墙所有权模式 - -每台宿主机必须精确落入下列一种模式,不能混用: - -1. `native_nftables`:`nftables.service` active/enabled,firewalld unit 不存在或 - inactive+masked。仅 disabled 不够,因为 D-Bus 仍可激活 firewalld。发行版 nftables - 主配置只有一个 panel 受管 include。 -2. `firewalld_nft`:firewalld active/enabled,`FirewallBackend=nftables`、 - `NftablesTableOwner=yes`、`firewall-cmd` 可用,native `nftables.service` unit 不存在或 - inactive+disabled/masked。`/etc/virt-cloud/nftables.conf` 是 root-only 的 panel - manifest,发行版 nftables 主配置不能再 include panel table。 - -active/enabled/masked 状态无法确定、两个服务可能同时接管、firewalld 使用非 nft 后端、 -存在 source-bound zone、panel manifest 多出未知 include,或模式与已保存状态漂移时都应 -fail-closed。两种模式都支持全新部署及同模式恢复;已有 VPC state、panel manifest 或未完成 -journal 时切换防火墙 owner 属于维护窗口迁移,当前向导不会在线转换这些宿主级持久化对象, -也不能把一次普通 reconcile 当成迁移。 - -### firewalld underlay 放行 - -对所选公网或内网 endpoint 核验其 effective runtime zone 和持久 zone 完全一致且不为 `trusted`。 -平台只能在该 zone 建立一条目标为本机 endpoint `/32`、协议 UDP、端口为 Fabric listen -port 的精确 runtime+permanent rich rule。核对以下所有权边界: - -- 规则原本两处都不存在时,apply 创建两处并记录 panel-owned;回滚和 endpoint 迁移只 - 删除该精确旧规则。 -- 两处原本都存在且内容完全一致时,视为外部规则并只复用;panel 绝不能删除。 -- 只有 runtime 或只有 permanent、zone 漂移或内容不一致时拒绝继续,不能猜测所有权。 -- 所选 endpoint 的物理接口、默认 route 接口和 VM 公网桥永远不能被加入 `trusted`。外部 DNAT、云 - 安全组和上游防火墙仍由运维显式管理;公布的 NAT 地址不是本机地址时不支持自动放行。 - -正常执行 `firewall-cmd --reload` 和 `systemctl restart firewalld`,确认精确 rich rule、 -panel raw nft table、WireGuard/VPC 隔离在 repair timer 一个周期内恢复。不要把 -`firewall-cmd --complete-reload` 当日常测试;它可能丢失运行状态并中断连接,只能在已 -隔离或迁走 VM 的维护窗口单独演练。 +## 防火墙所有权 + +每台宿主机只支持 `native_nftables`:`nftables.service` 必须 +active/enabled,发行版主配置只能有一个 panel 受管 include。firewalld +unit 必须不存在或 inactive+masked,且实时规则中不得存在其所有的 +`inet firewalld` table。仅 disabled 不够,因为 D-Bus 仍可重新激活它。 + +能力探测发现 firewalld 服务、旧 `firewalld_nft` desired state、manifest、 +journal、zone/rich-rule 所有权或其他管理器冲突时必须 fail-closed。普通 +apply/reconcile 不得把旧模式静默映射为 native nftables,也不得删除旧对象。 +测试环境的一次性转换应先停止租户流量,备份规则与 journal,在维护窗口 +清理 firewalld 及旧 panel 所有权记录,再以全新 native 方案重建。 + +panel-owned Fabric nft table 不负责放行 underlay UDP。管理员必须在宿主的持久 +native nftables 规则或上游防火墙/云安全组中,以可证明的最窄 peer 源地址、 +本机 endpoint `/32`、UDP 和 Fabric listen port 显式放行。外部 DNAT 也由运维 +显式管理。执行 `systemctl reload nftables` 后必须重新核对这条 underlay 路径、 +panel owner/revision/digest、Fabric/VPC table、WireGuard 双向握手及允许/拒绝流量。 +单纯看到 `wg-fabric0` active 或 systemd unit green 不能证明 underlay 可达。 ## AlmaLinux 10 / NetworkManager 确认 NetworkManager 是唯一网络管理器。panel 的 `virt-cloud-wg-fabric0` 和 VPC bridge keyfile 必须位于 `/etc/NetworkManager/system-connections/`、owner 为 root、mode 为 -`0600`,并设置 `connection.autoconnect=no`、不含持久 `connection.zone`;`/run`、 -`/usr/lib`、in-memory/external profile 或其他 mode 不作为可持久恢复证据。两种模式都只 -能在 ACL 精确回读后通过已验证的 UUID 激活接口;仅 `firewalld_nft` 模式在激活前使用 -带 `--temporary` 的 `nmcli connection modify` 将 `connection.zone` 设为 `trusted`,并 -精确回读 runtime zone。`native_nftables` 模式不查询、设置或要求 firewalld -zone/trusted。执行 `nmcli connection reload` 和 `systemctl restart NetworkManager` -时,接口不得先于 repair helper 自动连接;firewalld 模式不得永久进入 `trusted`, +`0600`,并设置 `connection.autoconnect=no`、不含 `connection.zone`;`/run`、 +`/usr/lib`、in-memory/external profile 或其他 mode 不作为可持久恢复证据。接口只能在 +nft ACL 精确回读后通过已验证的 UUID 激活,不查询、设置或要求任何 +firewalld zone/trusted 状态。执行 `nmcli connection reload` 和 +`systemctl restart NetworkManager` 时,接口不得先于 repair helper 自动连接, `br0`、管理接口和默认路由不得改变。 AlmaLinux 10 的 `nmcli` 把 `FILENAME` 作为无 ID 的 `connection show` 列表字段, 不提供 `connection show uuid ` 下的 `GENERAL.FILENAME`。验收必须确认引擎 使用 `nmcli -t --escape no -f UUID,FILENAME connection show` 并按 UUID 精确、唯一 -解析 keyfile;不得因为无效的 detail 字段把合法 endpoint 误报为 -`firewalld_endpoint_zone_unverified`。与 endpoint 无关的 `/run`/external profile 可以单独 +解析 keyfile;不得因为无效的 detail 字段拒绝合法 endpoint。与 endpoint 无关的 +`/run`/external profile 可以单独 fail-closed,但不得污染已有唯一持久 keyfile 的 endpoint 接口。 -分别在 native 与 firewalld 模式执行相应防火墙 reload,然后重启宿主机。记录 +执行 native nftables reload,然后重启宿主机。记录 `wg show wg-fabric0`、`ip -4 route show table all`、受管 nft table/marker、默认路由和 管理地址;确认 helper 先恢复并回读 Fabric 隔离及全部 VPC inet/bridge/arp ACL,随后才 -激活接口。native 模式不应出现 firewalld zone/trusted 依赖;firewalld 模式还应记录接口 -runtime zone,并确认接口只在 ACL 精确回读后加入 runtime `trusted`。 +激活接口。过程中不应出现 firewalld zone/trusted 依赖。 ## Debian 13 / ifupdown 确认 `/etc/network/interfaces` 或 `interfaces.d` 由经典 ifupdown 管理,探测到的 `ifup`、`ifdown`、`ifquery` 路径与面板记录一致,systemd-networkd 和 NetworkManager 没有 active/enabled/configured 接管证据。panel fragment 不得包含 `auto`,同名 ifcfg -不得存在;仅 `firewalld_nft` 模式要求 permanent firewalld zone 为空。 +不得存在。 `ifup --force wg-fabric0` 只能在 ACL 精确回读后运行,不能触碰 `br0`、默认路由或管理 地址。 @@ -100,10 +81,9 @@ Debian 13 的 iproute2 6.15 在 WireGuard 内核模块尚未加载时, 仍必须 fail-closed。此兼容不代替 `modinfo`/模块可用性证据,也不允许引擎在只读 检测中执行 `modprobe`。 -分别在 native 与 firewalld 模式执行相应 reload,并执行 `systemctl restart networking` +执行 native nftables reload,并执行 `systemctl restart networking` 和宿主机重启。确认 fragment、peer、地址、路由和受管 nft table 都由 helper 按 -ACL-first 顺序恢复;native 模式不应出现 firewalld zone/trusted 依赖,firewalld 模式还 -应确认 runtime `trusted` 只在 ACL 精确回读后恢复。 +ACL-first 顺序恢复,且不应出现 firewalld zone/trusted 依赖。 ## libvirt 启动门禁 @@ -111,7 +91,7 @@ ACL-first 顺序恢复;native 模式不应出现 firewalld zone/trusted 依赖 `virtqemud.service`、`virtproxyd.service` 以及 `libvirt-guests.service`。受管 daemon drop-in 必须同时包含对 Fabric service 的 `Requires=` 与 `After=`;外部同名冲突 drop-in 应阻断 apply。分别测试普通 service 启动、socket activation 和 VM autostart,均不得在 -Fabric/VPC ACL、bridge,以及 firewalld 模式的 runtime zone 恢复前开放来宾私网。 +Fabric/VPC ACL 和 bridge 恢复前开放来宾私网。 socket 保留发行版默认启动顺序,其激活的 daemon 仍受上述门禁约束。不得给默认 `Before=sockets.target` 的 socket 再添加 `After=Fabric`,也不得让 Fabric `Before=socket`; Fabric 是依赖网络的普通 service,这种顺序会通过 `basic.target` 形成启动循环。 @@ -135,14 +115,15 @@ repair timer 有两个互斥且可核验的分支:主 Fabric service 明确 fa 且有效门禁完整时才可自动排队。每 daemon 每 boot 只消费一次有界、root-owned 的证据, 修复后再次回读并用 `--no-block --job-mode=fail start` 排队;缺日志、未知对象、后续手动 stop/新 job/成功/自身错误均跳过。不得用 service `Result=dependency` 判断:dependency -是 job 的结果,不是 service 的 Result。主 service 为 active、firewalld 也为 running,但受管 Fabric -或 VPC 接口缺少 runtime `trusted` 时,reconcile service 直接执行同一份带 flock 的完整 -ACL-first helper。后一分支不得 restart 主 Fabric unit 或任何 libvirt unit,健康状态也 +是 job 的结果,不是 service 的 Result。主 service 为 active 但受管 nft table、 +Fabric 或 VPC 接口漂移时,reconcile service 直接执行同一份带 flock 的完整 +ACL-first helper。该分支不得 restart 主 Fabric unit 或任何 libvirt unit,健康状态也 不得进入 helper。 在全新宿主执行 apply 后、不重启宿主机,确认事务 `committed` 后 apply 释放 flock、同步 -启动独立 dispatcher,主 oneshot 最终进入 active,完整 helper 清理双 guard 后 apply 才返回 -成功;再 reload/restart firewalld,确认 timer 能在首次 reboot 之前修复 runtime 漂移。 +启动独立 dispatcher,主 oneshot 最终进入 active,完整 helper 证明无旧 +emergency owner/table 并完成 ACL/WireGuard 回读后 apply 才返回 +成功;再 reload nftables,确认 timer 能在首次 reboot 之前修复 runtime 漂移。 人为让主 helper 持有 Fabric flock,确认 dispatcher 等待期间 apply 不会持锁死锁;让同步 启动失败时,确认 committed journal 仍保留且旧进程不会删除并发新事务的 ready journal。 @@ -180,7 +161,7 @@ ACL-first helper。后一分支不得 restart 主 Fabric unit 或任何 libvirt 出现一个同 ID 任务,已接纳节点不得重派,也不得生成新 ID;在终态未证明前,新 apply 必须 被拒绝、Fabric gate 必须关闭。 5. 人为卡住宿主 helper 超过 operation 运行上限,systemd 必须终止该 cgroup,状态只记录 - 固定的 interrupted/uncertain 原因,保留 journal/emergency guard 并禁止自动重放。 + 固定的 interrupted/uncertain 原因,保留 journal 与当前现场并禁止自动重放。 6. 在 Debian 13 把 `/run` 挂载为 `noexec`,确认 transient oneshot 的命令行为绝对路径 `/bin/sh /run/virt-cloud/fabric-operations//runner`,任务能进入 `host_lock` 而不是 `203/EXEC`;确认 unit 使用 `TimeoutStartSec=840s` 且没有无效的 `RuntimeMaxSec=`。 @@ -192,34 +173,31 @@ ACL-first helper。后一分支不得 restart 主 Fabric unit 或任何 libvirt ## 失败回滚和安全边界 -保留同计划的 `ready` journal 和历史 `helper.present`:即使历史 helper 不支持新版 -NetworkManager runtime shadow,也只能在旧 helper 与 journal pre/post-image 精确一致、 -meta 中 digest/UUID/manager/firewall/underlay 全部绑定当前方案、当前临时 helper 的权限与 -计划头核验后,交由当前实现恢复。恢复完成的 recovery-only 调用不得再次执行已恢复的 -旧 helper。跨计划仍沿历史恢复合同,不得把旧计划的权限移交给新模板;外部文件/密钥/nft +保留同计划的 `native_nftables` `ready` journal 和历史 `helper.present`:只能在旧 helper +与 journal pre/post-image 精确一致、meta 中 digest/UUID/manager/firewall 全部绑定 +当前方案,且 underlay 由当前 plan/helper digest 间接绑定时,才能在核验 +当前临时 helper 的权限与计划头后,交由当前实现恢复。恢复完成的 recovery-only +调用不得再次执行已恢复的旧 helper。任何 `firewalld_nft` journal 及其 snapshot 不进入此 +恢复路径;标记为 native 但仍携带已退役 `firewalld_*`/`underlay_*` 字段的旧 schema +也只保留现场并进入维护清理。跨计划不得把旧计划的权限移交给新模板。外部文件/密钥/nft 漂移必须保留现场并拒绝。新增 timer 必须在 `state=committed` 同步落盘之后才启动, timer 启动失败只能保留 committed journal,不能回滚已提交的数据面。 -在隔离环境中分别让 `nft -c/-f`、`firewall-cmd`、`wg`、`ip route`、 +在隔离环境中分别让 `nft -c/-f`、`wg`、`ip route`、 `nmcli/ifup`、systemd gate 和 timer 返回失败。每次都应看到 apply 失败并保持 -`uncertain/degraded`;firewalld 模式的回滚先撤销 panel 接口的 runtime `trusted`,再以 -单次 nft batch 恢复旧 Fabric/VPC table,精确回读后才能重加原 trusted 状态。 -panel-owned peer、地址、 -路由、profile/fragment、rich rule、manifest/include 和 systemd 状态必须按快照恢复; +`uncertain/degraded`。panel-owned peer、地址、路由、profile/fragment、 +manifest/include 和 systemd 状态必须按快照恢复; 无法恢复时不得报告 active。 -分别在 emergency guard owner intent 写入后、nft 安装前、安装后、manifest refresh 前后 -中断进程并重跑;每次只能在随机 token、完整 shape 和 snapshot 全部匹配时继续。随后在没有 -任何 VPC state、manifest 或未完成 journal 的空宿主上,把一次前置门禁失败的 -`firewalld_nft` 尝试纠正为 `native_nftables`:native 必须接管遗留 guard、保持隔离直到 -Fabric/WireGuard/route/path 全部回读,再清理 guard;owner-only 可以在证明 table 不存在后 -清理,table-only、token 漂移或伪造的同名 table 必须拒绝且不得删除。若宿主已有任一 VPC -state,即使它仍是合法 native state,也必须保留 guard 并进入维护迁移,不能用 owner comment -数量代替三族 VPC ACL 的完整语义证明后解除隔离。 +分别在 snapshot/journal 写入后、nft 预检前、原子 batch 提交前后和 manifest refresh +前后中断进程并重跑;每次只能在 owner、完整 shape、snapshot 和现场回读全部匹配时继续。 +预先放置旧 emergency owner/table、`firewalld_nft` owner/manifest/journal 或 `inet firewalld` +table,确认普通 +apply/reconcile 拒绝并保留现场;不得用新 native desired state 覆盖或清理它们。 让 `bridge link show master` 失败或返回畸形、重复、超过上限的端口记录,并模拟 `ip link set down` 返回成功但 NetworkManager 立即把 bridge 恢复为 `UP`。boot/apply 均 -必须 fail-closed、保留 bridge/tap 端口关系且不得移除 emergency guard 或提前恢复 trusted。 +必须 fail-closed、保留 bridge/tap 端口关系,不得继续改写 nft 或提前恢复流量。 预先放置非 panel-owned 同名 nft table、宽前缀路由、策略表路由、同名接口或冲突 systemd drop-in,apply 必须拒绝并保留原状态。最后从其他宿主机核验 WireGuard UDP、 diff --git a/docs/INTEGRATION_PLAN.md b/docs/INTEGRATION_PLAN.md index 0e1582d1..70cf2f16 100644 --- a/docs/INTEGRATION_PLAN.md +++ b/docs/INTEGRATION_PLAN.md @@ -53,12 +53,12 @@ Debian 13 只接受 `apt`、经典 ifupdown/ifupdown2 共有的 NetworkManager、systemd-networkd、ifupdown 同时运行、开机启用或持有配置时 一律标为冲突并拒绝私网变更;持有配置包括 NetworkManager keyfile、 systemd-networkd 的 `.network/.netdev` 文件以及 ifupdown 的 interfaces 文件。 -防火墙只接受两个互斥且可证明的所有权模式:native `nftables.service` -active/enabled,且 firewalld unit 不存在或 inactive+masked;或 firewalld -active/enabled、`FirewallBackend=nftables`、`NftablesTableOwner=yes`,且 native -`nftables.service` unit 不存在或 inactive+disabled/masked。仅 disabled 的 firewalld -仍可能被 D-Bus 激活,不能归入 native 模式。混合、未知、source-bound firewalld zone -和当前/持久 zone 漂移均 fail-closed。探测只识别现状,不在租户请求中安装包、加载 +防火墙只接受 `native_nftables` 所有权:`nftables.service` 必须 +active/enabled,firewalld unit 必须不存在或 inactive+masked,且实时规则中 +不得存在 `inet firewalld` table。仅 disabled 的 firewalld 仍可能被 D-Bus +激活,不能归入 native 模式。任何旧 `firewalld_nft` desired state、manifest、 +journal、zone/rich-rule 所有权或其他管理器冲突都 fail-closed;普通 +apply/reconcile 不做静默映射或自动接管。探测只识别现状,不在租户请求中安装包、加载 模块或修改规则。缺少能力时旧公网 VM 路径仍可保存宿主机,但 VPC 控制器 必须要求 `vpc_ready=true`,否则拒绝创建或变更私网。 @@ -91,7 +91,7 @@ active/enabled、`FirewallBackend=nftables`、`NftablesTableOwner=yes`,且 nat VPC reconciler 单独管理。私网 libvirt 网卡在该阶段只接受已核验的独立 bridge/tap/device 来源,不接受无法确认底层桥接的 `network=` 间接来源;公网 nwfilter 与私网 ACL 永远是两套策略上下文。WireGuard 能力探测通过不改变系统的命令完成, -真正的 AlmaLinux/Debian 部署、SELinux/firewalld 共存和跨宿主机连通性仍须 +真正的 AlmaLinux/Debian 部署、SELinux/native nftables 持久化和跨宿主机连通性仍须 在验收环境验证。 当前阶段已增加引擎全局 Fabric 控制面。管理员页面按地区展示宿主机 readiness,正常 @@ -153,8 +153,8 @@ systemd 以绝对路径 `/bin/sh` 解释 runner,runner/update/apply 的内部 `/run` 文件,从而兼容宿主把 `/run` 挂载为 `noexec`;oneshot 用 `TimeoutStartSec` 约束完整 任务,不使用对 `Type=oneshot` 无效的 `RuntimeMaxSec`。 SSH 只负责精确 ID 的 dispatch/status 协议;任务 stdout/stderr 不返回面板,journal 只记录 -operation ID、固定 state、checkpoint 和 reason。运行时间有上限,中断保留事务日志与 -emergency guard;远端 operation 目录有严格 owner/mode/hash 校验和有界终态清理,不删除 +operation ID、固定 state、checkpoint 和 reason。运行时间有上限,中断保留事务日志和当前现场 +并禁止自动重放;远端 operation 目录有严格 owner/mode/hash 校验和有界终态清理,不删除 queued/running 或无法证明归属的对象。 Engine 重启或 dispatch 回包丢失时,页面 GET 与周期 reconcile 只使用原 operation ID @@ -231,44 +231,38 @@ apply 会先保存 panel-owned 配置、WireGuard peer/地址/路由、systemd nftables 规则保存为 root-only 的 `/etc/nftables.d/virt-cloud-wg-fabric.nft`。native 模式要求 Debian 的 `/etc/nftables.conf` 或 AlmaLinux 的 `/etc/sysconfig/nftables.conf` 存在唯一受管 -include;firewalld+nft 模式不修改发行版 nftables 主配置,而使用 -`/etc/virt-cloud/nftables.conf` 的受限 manifest,由 Fabric helper/timer 在 firewalld -之后原子恢复受管 Fabric/VPC table。两种模式的规则均带 owner 和 digest;helper 校验 -归属、结构、revision 和完整 include 集合,不把 raw nft 规则写入 firewalld direct -接口,也不执行会重载所有 zone 的全局 firewalld reload。 - -firewalld 模式的 underlay 只允许本机真实 endpoint:普通路由及显式 `from endpoint` -路由都必须返回唯一且相同的地址归属接口,并选择该 endpoint 作为源地址;运行态与持久 -effective zone 必须一致且不得为 `trusted`。 -平台只维护该 zone 内目标为本机 endpoint `/32`、协议 UDP、固定监听端口的精确 -runtime+permanent rich rule;已有的完全相同外部规则只复用不删除,半存在或发生漂移则 -fail-closed。平台不会信任所选 endpoint 的物理接口,也不代替云安全组、上游防火墙或外部 DNAT; -NAT 后公布的非本机地址仍需外部显式管理,不能走自动 firewalld 放行。 - -两种模式的 NetworkManager profile 都不写持久 zone、禁止 autoconnect,ifupdown -fragment 都不写 `auto`。`native_nftables` 不查询、设置或要求 firewalld zone/trusted; -仅 `firewalld_nft` 要求永久 zone 为空,并让 `wg-fabric0` 与 VPC 私网 bridge 使用可恢复的 -runtime `trusted`,激活时按已核验 UUID 使用 temporary zone。helper 必须先在单次原子 -nft transaction 恢复并回读 Fabric 隔离和全部 VPC inet/bridge/arp ACL,再激活接口;仅 -firewalld 模式随后加入 runtime `trusted`,异常与回滚先撤销 trust。panel-owned systemd +include。规则均带 owner 和 digest;helper 校验归属、结构、revision 和完整 include +集合。发现旧 firewalld manifest、journal、zone/rich-rule 所有权或 `inet firewalld` +table 时保留现场并拒绝普通 apply/reconcile,不会继续写规则。 + +underlay 只允许本机真实 endpoint:普通路由及显式 `from endpoint` 路由都必须 +返回唯一且相同的地址归属接口,并选择该 endpoint 作为源地址。 +panel-owned Fabric nft table 不放行 underlay UDP;管理员必须在宿主持久 native nftables +规则或上游云安全组/防火墙中,以最窄可证明的 peer 源地址、本机 endpoint +`/32`、UDP 和固定监听端口显式放行。外部 DNAT 也由运维管理;本地接口 active +或 unit green 不能代替双向 WireGuard 握手和实际流量验收。 + +NetworkManager profile 不写 zone、禁止 autoconnect,ifupdown fragment 不写 `auto`。 +helper 必须先在单次原子 nft transaction 恢复并回读 Fabric 隔离和全部 VPC +inet/bridge/arp ACL,再激活接口,不查询、设置或要求任何 firewalld +zone/trusted 状态。panel-owned systemd gate 同时约束已安装的 libvirt/ virtqemud/virtproxyd service 与 socket,避免 socket activation 或 VM autostart 在 ACL -恢复前开放私网。公网/内网 endpoint 实际可达、NAT、MTU、SELinux、正常 firewalld/ +恢复前开放私网。公网/内网 endpoint 实际可达、NAT、MTU、SELinux、native nftables/ NetworkManager/ifupdown 重启以及宿主机重启后的恢复仍属于真实宿主机验收项。Fabric 只使用独立的 `wg-fabric0`;管理器适配不得修改 `br0`、默认路由或宿主机管理接口。 repair timer 在主 Fabric service 明确 failed 时只重新排队该主 service,并只附带此前 -确因 dependency 失败的 libvirt 单元;若主 service 为 active 且核验到 firewalld 重启 -造成的受管 runtime trusted 漂移,则 reconcile service 直接执行同一份带 flock 的完整 +确因 dependency 失败的 libvirt 单元;若主 service 为 active 且核验到受管 nft table +或接口漂移,则 reconcile service 直接执行同一份带 flock 的完整 ACL-first helper,不 restart 主 Fabric unit 或 libvirt。两条分支都保证完整 helper 每轮 最多执行一次,健康状态不触发修复。 -firewalld 事务在读取可变 profile、zone、manifest 或 VPC state 前先建立独立 emergency -nft guard。guard 使用每次事务随机 token,并由 `/run/virt-cloud` 下 root-only owner -intent 与完整 stateless snapshot 共同证明所有权;同名 table 缺少 intent、token/规则 -漂移或回读不完整时一律保留现场且拒绝删除。空宿主把一次失败的 firewalld 前置尝试纠正为 -native 后,native helper 只能接管并最终清理由上述证据完整证明的遗留 guard,不会在正常 -native 路径创建额外 table;一旦存在任何 VPC state、firewalld manifest 或未完成 journal, -跨 owner 切换必须 fail-closed 并进入单独的维护迁移,当前 reconcile 不改写这些持久状态。 +受管 nft 事务使用完整 stateless snapshot、阶段 journal、`nft -c` 预检、最后一次 +precommit 回读和单次原子 `nft -f` batch 保护现有数据面;任何所有权、snapshot +或回读不完整都保留现场并 fail-closed。旧 emergency owner/table 以及 +firewalld owner/manifest/journal 都只是遗留检测证据,不是当前事务机制;它们 +只能在事先备份且已停止租户流量的维护窗口由运维显式清理;当前 reconcile +不改写这些持久状态。 nft journal 明确记录 mutation/install/refresh 阶段,崩溃点恢复会重新核验 live table,而 不是根据上一条命令返回值推断提交完成。 @@ -277,8 +271,8 @@ bridge 命令失败、畸形/重复/超量输出或端口集合变化,并在 ` `UP` flag 已消失。这样 NetworkManager 竞态不能越过 hard quarantine,同时不需要停用 bridge profile 或拆除正在运行 VM 的 tap。首次 apply 在 durable transaction commit 后释放 flock,再同步启动独立 reconcile dispatcher;dispatcher 等待 `RemainAfterExit` oneshot, -或在主 unit 已 active 时直接执行同一份完整 helper。apply 重新持锁并确认 helper 已清理 -双 guard、主 service/timer active 及当前 digest/ACL/WireGuard 均匹配后才返回成功;提交态 +或在主 unit 已 active 时直接执行同一份完整 helper。apply 重新持锁并确认不存在旧 +emergency owner/table,且主 service/timer active 及当前 digest/ACL/WireGuard 均匹配后才返回成功;提交态 旧进程不删除共享 journal,使 repair timer 在首次重启前也有可恢复、可核验的状态。 受管 nft 事务以完整 stateless snapshot、`nft -c` 后的最后 precommit 回读、单次原子 @@ -389,9 +383,9 @@ fail closed,不能回退接受仅有 XML 的旧 succeeded receipt。 且完成资源不存在性证明后才清理 journal。PHP 只在每台宿主机的 XML、路由、ACL、反欺骗、代理 ARP、rp_filter 和完整 peer AllowedIPs 证据都匹配当前 revision/digest 后才把 VPC 标为 `active`。 -在 firewalld 模式中,VPC apply/remove/rollback/启动恢复先隔离 runtime trusted,再以 -单次 nft batch 建立或恢复受管 table;只有全部 ACL marker/digest 精确回读后才能重新 -激活 bridge 和 trust。NetworkManager 或 ifupdown 重启不会依赖 profile 自动连接来绕过 +VPC apply/remove/rollback/启动恢复以单次 nft batch 建立或恢复受管 table; +只有全部 ACL marker/digest 精确回读后才能重新激活 bridge。NetworkManager 或 +ifupdown 重启不会依赖 profile 自动连接来绕过 该顺序;libvirt daemon service 受 Fabric gate 约束,socket activation 触发的仍是 同一个受门禁的 daemon。socket 保留默认早期启动顺序,避免与网络依赖形成循环。 @@ -441,7 +435,7 @@ VPC,以及未释放或仍有 claim 的私网 reservation,避免独立 VPC wo 租户计划。 AlmaLinux 10 与 Debian 13 的 WireGuard/NetworkManager/ifupdown 能力、真实宿主机 -路由、SELinux/firewalld、MTU 和跨机房连通性尚未在本地环境宣称通过;必须完成 +路由、SELinux/native nftables 持久化、MTU 和跨机房连通性尚未在本地环境宣称通过;必须完成 上述 Fabric 与 VPC 宿主机验收矩阵后,才能把环境状态推进为可上线。 PHP 不直接解密 Python 记录或执行任意 virsh 命令。主库与日志库可同实例或分别配置地址/端口。敏感业务数据加密;部署数据库连接配置仍是权限受限配置文件,不代表配置文件本身加密。 diff --git a/docs/VALIDATION.md b/docs/VALIDATION.md index 9f5c0c7b..bd697610 100644 --- a/docs/VALIDATION.md +++ b/docs/VALIDATION.md @@ -1,5 +1,24 @@ # 本版验证记录 +## 2026-10-10:firewalld 支持退役 + +- 当前 Fabric/VPC 宿主机只支持 `native_nftables`。能力探测只在 + `firewall_manager=nftables` 且 `firewall_mode=native_nftables` 时允许 readiness。 + 旧 `firewalld_nft` 输入、manifest、journal、zone/rich-rule 所有权或实时 + `inet firewalld` table 必须 fail-closed,不得在普通 apply/reconcile 中静默 + 映射、接管或删除。 +- PHP readiness 边界会再次核验原生 nftables 模式;即使旧 Engine 将 + `firewalld_nft` 回报为 ready,页面也必须关闭写入并显示维护迁移阻断。 +- 本次本地已通过 `admin-fabric.js` 语法检查、 + `AdminFabricWizardBehaviorTest.mjs` 前端行为回归及 `git diff --check`。 + 当前工作区没有 PHP CLI 且没有 `web/vendor/autoload.php`,因此本地未能 + 执行 PHP lint 和 PHP 合同/readiness 测试;发布前必须在完整 PHP 依赖环境补跑。 +- 本节之后仍出现的 firewalld 段落只是 2026-10-08/09 的历史实测与旧实现 + 记录,不表示当前仍支持该模式,也不能作为新部署验收依据。 +- 仍需在 AlmaLinux 10 和 Debian 13 验证 native nftables apply/readback、reload、 + restart/reboot、WireGuard 握手、VPC 允许/拒绝流量及失败回滚;本地静态 + 回归不代替这些真机证据。 + ## 2026-10-09:严格 NM shadow 恢复与丢失回执只读查询 - Fabric 的共享 NM 归属检查只新增对固定 runtime shadow 的支持:持久/临时 @@ -217,7 +236,7 @@ 因而 `AdminFabricReadinessWorkflowTest.php` 仅完成语法检查,未执行数据库流程;该测试必须在 部署端完整 Composer dev 依赖与 PDO SQLite 环境复跑。 - 尚未连接真实 Debian 13 / AlmaLinux 10 宿主机验证 SSH 断线、transient unit、journal、重启、 - NetworkManager/ifupdown 及 firewalld/native nftables。部署后必须用页面显示的同一 ID 查询 + NetworkManager/ifupdown 及 native nftables。部署后必须用页面显示的同一 ID 查询 `journalctl --utc -u virt-cloud-fabric-apply-.service --no-pager -o short-iso-precise`, 并完成双宿主 WireGuard 握手、跨宿主 VPC 流量和重启恢复后,才能认定生产组网验收完成。 @@ -246,7 +265,7 @@ 不涉及本次探针代码。 - 上述现场输出是修复输入,不代表新版本已经在宿主机验收。部署本提交后仍须重新点击 “检测宿主机”,确认两个旧阻断消失;随后执行地区自动配置、WireGuard 握手、跨宿主流量、 - firewalld/NetworkManager reload 与重启恢复验收,才能认定生产组网完成。 + native nftables/NetworkManager reload 与重启恢复验收,才能认定生产组网完成。 ## 2026-10-09:Fabric canonical nft 回读与首装事务恢复 @@ -276,11 +295,11 @@ 3 个公网池 fixture,均不在本次 Fabric 改动范围。`git diff --check`、生产/测试 Python `py_compile` 与最终只读源码审计通过。上述静态与本地回归不替代真实宿主机验收;部署后 仍须证明 journal/emergency guard 收敛、两台主机 WireGuard 握手、跨宿主 VPC 流量 - 以及 firewalld/native reload 与重启恢复。 + 以及 native nftables reload 与重启恢复。 -## 2026-10-08:公网优先 underlay 与 firewalld+nft 兼容 +## 2026-10-08:公网优先 underlay 与已退役 firewalld+nft 实验 -- Fabric/VPC 现在只接受两个互斥模式:native nftables 要求 +- 当时的 Fabric/VPC 实验曾接受两个互斥模式:native nftables 要求 `nftables.service` active/enabled 且 firewalld 不存在或 inactive+masked; firewalld 模式要求 active/enabled、nft backend、`NftablesTableOwner=yes`,native nftables unit 则只能不存在或 inactive+disabled/masked。能力探针不会查询 inactive @@ -342,7 +361,7 @@ VPC 创建和 CIDR allocator 回归、Fabric Node 行为测试、两份 JavaScript 语法检查及 `git diff --check` 通过。 -- 这些结果没有替代真实宿主机验收。AlmaLinux 10、Debian 13、firewalld/nftables reload、 +- 这些结果没有替代真实宿主机验收。AlmaLinux 10、Debian 13、native nftables reload、 NetworkManager/ifupdown 重启、libvirt socket activation、宿主重启、跨宿主 WireGuard 握手、MTU、SELinux、外部 NAT/安全组和允许/拒绝流量仍须按验收矩阵执行。 - nft CLI 不提供跨进程 CAS。本轮以完整 stateless snapshot、`nft -c` 后的最后 @@ -378,7 +397,7 @@ 关系相同”的正常路径,并覆盖实例 pool pin 不一致、多条 live membership、 pool/line/region 断链与跨地区拒绝。本轮 Engine Fabric/VPC/边界回归 `268 passed`, 21 个聚焦 PHP 测试脚本、7 个变更 PHP 文件语法检查和管理员 Fabric Node.js 行为测试 - 通过。开发环境仍未替代真实多宿主 WireGuard、nftables/firewalld、libvirt 和重启恢复验收。 + 通过。开发环境仍未替代真实多宿主 WireGuard、native nftables、libvirt 和重启恢复验收。 - 扩大探测仍命中未修改的既有测试问题:`SourceSyntaxTest` 被 `AdminSystemLogsContractTest.php` 的双引号插值语法错误阻断; `CloudEngineReadinessTest` 保留了已过时的静态次数断言;`CloudProtocolTest` 未导入 @@ -418,7 +437,7 @@ ID/lease/phase。映射提交前已在长网络操作后重读拓扑,并在事务中锁地区、映射和 live VPC;跨全部库存写路径的 topology generation/CAS 仍是后续并发强化项。 - 开发环境没有连接真实多宿主 SSH、 - WireGuard、nftables/firewalld 或 libvirt,生产部署仍须执行 Fabric 宿主机验收矩阵。 + WireGuard、native nftables 或 libvirt,生产部署仍须执行 Fabric 宿主机验收矩阵。 ## 2026-10-08:管理员地区宿主机 readiness 与自动组网 @@ -452,9 +471,9 @@ detail envelope 和 3 个公网池 fixture;本轮 Fabric/readiness 与 Engine boundary 没有 新增失败。 - 当前未连接真实 PHP-FPM、MariaDB、浏览器或 SSH/libvirt 多宿主机。生产验收仍须覆盖 - WireGuard 握手、UDP/51820 underlay、MTU、SELinux、受支持的 native nftables 或 - firewalld+nft 持久化、网络/防火墙重启修复和跨宿主机租户流量。两种防火墙模式必须 - 互斥;不受支持或所有权不明确的组合继续 fail-closed,不能直接清空生产防火墙。 + WireGuard 握手、UDP/51820 underlay、MTU、SELinux、native nftables 持久化、 + 网络/防火墙重启修复和跨宿主机租户流量。不受支持或所有权不明确的组合 + 继续 fail-closed,不能直接清空生产防火墙。 反代与 PHP-FPM 请求终止/读超时建议至少 270 秒。 ## 2026-10-08:租户单自动子网与创建页简化 @@ -529,7 +548,7 @@ HAVING live_subnets <> 1 OR deleting_subnets <> 0; - SQLite 不实现 MariaDB 的 `SELECT ... FOR UPDATE` 并发语义;上线前仍须在隔离 MariaDB 上并发争抢账户最后一个名额、并发同 key 丢响应重试,并核对最终仅有一条 root、network、 claim 和 create operation。当前未连接真实数据库、PHP-FPM、浏览器或宿主机,也未改变 - WireGuard/libvirt/firewalld 数据面。额外全仓 `SourceSyntaxTest` 仍被未修改的 + WireGuard/libvirt 宿主数据面。额外全仓 `SourceSyntaxTest` 仍被未修改的 `AdminSystemLogsContractTest.php:108` 既有解析错误阻断,不归因于本轮 VPC 修改。 ## 2026-10-08:VPC 来宾网络启动代次门禁 diff --git a/docs/VPC_DATAPLANE_ACCEPTANCE.md b/docs/VPC_DATAPLANE_ACCEPTANCE.md index 2ff30cfd..2357d0a4 100644 --- a/docs/VPC_DATAPLANE_ACCEPTANCE.md +++ b/docs/VPC_DATAPLANE_ACCEPTANCE.md @@ -18,12 +18,13 @@ - AlmaLinux 10 只有 NetworkManager/nmcli 处于 active/enabled,Debian 13 只有 ifupdown 的 `ifup`、`ifdown`、`ifquery` 和 `/etc/network/interfaces`(含已加载 的 `interfaces.d`)作为网络管理来源; -- 防火墙精确处于 Fabric 已确认的 `native_nftables` 或 `firewalld_nft` 模式,不能混合; - native 模式的持久主配置只有一个受管 include,firewalld 模式则只有 root-only panel - manifest,且精确 underlay rich rule 的 panel/external 所有权已回读; -- NetworkManager profile 不 autoconnect、不写持久 zone;ifupdown fragment 不含 `auto`。 - native 模式不查询、设置或要求 firewalld zone/trusted;仅 firewalld 模式让 - `wg-fabric0` 和 VPC bridge 使用 runtime `trusted`,endpoint/管理接口绝不 trusted; +- 防火墙精确处于 Fabric 已确认的 `native_nftables` 模式; + `nftables.service` active/enabled,持久主配置只有一个受管 include, + firewalld 不存在或 inactive+masked,且没有遗留 `inet firewalld` table。任何旧 + `firewalld_nft` desired/manifest/journal 都必须阻断普通 reconcile,不得自动迁移; +- NetworkManager profile 不 autoconnect、不写 zone;ifupdown fragment 不含 `auto`。 + `wg-fabric0` 和 VPC bridge 只在受管 nft ACL 精确回读后激活,不依赖任何 + firewalld zone/trusted 状态; - Fabric 公网或内网 endpoint 均不位于全局租户 VPC CIDR 池,且当前 VPC CIDR 不包含 任一 Fabric endpoint;否则目的策略路由会吸走 WireGuard underlay,必须在 apply 前阻断; `/etc/virt-cloud`、`/run/virt-cloud`、`/etc/nftables.d` 和 `sysctl.d` 不是符号链; @@ -76,19 +77,18 @@ ## 管理器和重启矩阵 -AlmaLinux 10:执行 `nmcli connection reload`、`systemctl restart NetworkManager` 和 -宿主机重启;native 模式另执行 `systemctl reload nftables`,firewalld 模式执行正常 -`firewall-cmd --reload` 与 `systemctl restart firewalld`。每一步都要确认 private bridge +AlmaLinux 10:执行 `nmcli connection reload`、`systemctl restart NetworkManager`、 +`systemctl reload nftables` 和宿主机重启。每一步都要确认 private bridge 没有靠 autoconnect 抢先启动,helper 先恢复并回读 nft inet/bridge/arp ACL,再按 UUID -激活 profile;仅 firewalld 模式随后加入 runtime `trusted`,native 模式不应出现 -firewalld zone/trusted 依赖;随后 route/rule、proxy ARP、WireGuard peer、 +激活 profile;过程不应出现 firewalld zone/trusted 依赖;随后 +route/rule、proxy ARP、WireGuard peer、 `ua-private0` XML 和默认公网路径恢复,NetworkManager 没有接管 `br0` 或管理接口。 Debian 13:执行 `ifquery `、`systemctl restart networking` 和宿主机 -重启;native 模式另执行 `systemctl reload nftables`,firewalld 模式执行正常 reload/ -restart。确认不含 `auto` 的 `interfaces.d` fragment 不能让 bridge 绕过 helper 抢先启动, -ACL 精确回读后才执行 `ifup --force`;仅 firewalld 模式随后恢复 runtime `trusted`, -native 模式不应出现 firewalld zone/trusted 依赖;随后 route/rule、proxy ARP、WireGuard +重启,并执行 `systemctl reload nftables`。确认不含 `auto` 的 `interfaces.d` +fragment 不能让 bridge 绕过 helper 抢先启动,ACL 精确回读后才执行 +`ifup --force`,且不应出现 firewalld zone/trusted 依赖;随后 route/rule、 +proxy ARP、WireGuard peer 和 `ua-private0` XML 恢复。严禁用 ifdown 操作 `br0` 或管理接口。 两种发行版都要分别测试 libvirt service、socket activation 和 VM autostart。实际存在的 @@ -104,12 +104,12 @@ peer 和 `ua-private0` XML 恢复。严禁用 ifdown 操作 `br0` 或管理接 - apply、实例接入和停机 libvirt 变更失败时,原 panel-owned bridge、route/rule、 peer AllowedIPs、nft 表/include、proxy 项、sysctl、XML 和 seed 按快照恢复; -- firewalld 模式的 apply/remove/rollback/journal 恢复必须先撤销 panel 接口的 runtime - trusted,再以单次 nft batch 建立或恢复受管 table;只有 marker/digest 精确回读后才 - 能重新激活接口并恢复 trust,不能用分离的 table delete/load 暴露无 ACL 窗口; +- apply/remove/rollback/journal 恢复必须以单次 nft batch 建立或恢复受管 + table;只有 marker/digest 精确回读后才能重新激活接口,不能用分离的 + table delete/load 暴露无 ACL 窗口; - boot 对 panel-owned bridge 执行 hard quarantine 时,必须在 down 前后严格核对有界端口 快照并回读确认 `UP` flag 消失;bridge 命令/解析失败、端口变化或 NetworkManager 竞态 - 重新拉起接口都必须保留 emergency guard 并停止恢复,不能靠停用 profile 拆除 tap; + 重新拉起接口都必须停止恢复且不得继续改写 nft,不能靠停用 profile 拆除 tap; - 删除/撤销路径使用阶段 journal 和 fail-closed 收敛:未提交的 journal 在回读前 返回 `uncertain/manual_reconcile` 并保留现场,不重复执行破坏性删除;只有已经 提交且能证明面板资源全部不存在的 journal 才会被清理。因此删除路径的“回滚” @@ -122,9 +122,10 @@ peer 和 `ua-private0` XML 恢复。严禁用 ifdown 操作 `br0` 或管理接 - 重试只依据新的完整宿主机回读,不对不确定的 libvirt 或网络变更盲目反向执行。 跨宿主机最后还要单独验证 underlay 公网防火墙、NAT、MTU、SELinux 和 UDP 监听。 -firewalld 的本机精确 rich rule 也不能替代外部 DNAT/云安全组;Fabric/VPC 的本地 nft -表不等于端到端 underlay 已放行。`firewall-cmd --complete-reload` 只在隔离或迁走 VM 的 -维护窗口演练,不能作为普通在线 reload 验收。 +Fabric/VPC 的 panel-owned nft 表不负责放行 underlay UDP;运维必须在宿主 +持久 native nftables 规则或上游云安全组/防火墙中,以精确 peer 源地址、 +本机 endpoint `/32`、UDP 和 Fabric 监听端口放行。还需单独核对外部 DNAT, +并用双向握手和实际流量验收;接口 active 或 unit green 不能证明端到端可达。 ## 发布和规模边界 diff --git a/engine/tests/test_fabric.py b/engine/tests/test_fabric.py index a699dc2b..911a5f21 100644 --- a/engine/tests/test_fabric.py +++ b/engine/tests/test_fabric.py @@ -19,7 +19,6 @@ from types import SimpleNamespace import pytest from virsh_panel.fabric import ( - FABRIC_NFT_MANIFEST, FABRIC_OPERATION_CHECKPOINTS, FABRIC_OPERATION_ROOT, FABRIC_OPERATION_REASONS, @@ -32,7 +31,6 @@ from virsh_panel.fabric import ( FABRIC_REMOTE_APPLY_TIMEOUT_SECONDS, FABRIC_REMOTE_READ_TIMEOUT_SECONDS, FABRIC_SYSCTL, - FIREWALL_MODE_FIREWALLD, FIREWALL_MODE_NATIVE, MAX_VPC_STATE_FILES, MAX_FABRIC_OPERATION_DIRECTORIES, @@ -44,7 +42,6 @@ from virsh_panel.fabric import ( _FIREWALL_MODE_GUARD_SH, _FABRIC_SYSCTL_VALIDATION_SH, _NMCLI_FILENAME_RESOLVER_SH, - _NFT_MANIFEST_VALIDATION_SH, _REPAIR_UNIT, _UNIT, _apply_script, @@ -52,7 +49,6 @@ from virsh_panel.fabric import ( _fabric_operation_dispatch_script, _fabric_operation_runner_script, _fabric_operation_status_script, - _firewalld_endpoint_zone_evidence, _identity_script, _new_operation_id, _nm_profile_template, @@ -242,7 +238,7 @@ def test_generated_helpers_are_posix_safe_and_do_not_embed_private_keys(): identity_check = '[ "$($WG pubkey < "$KEY")" = "$EXPECTED_PUBLIC_KEY" ]' assert identity_check in apply assert apply.index(identity_check) < apply.index( - "vc_fabric_quarantine down") + "vc_fabric_nm_quarantine || exit 71") assert f"SYSCTL_FILE={FABRIC_SYSCTL}" in apply assert _sysctl_text(current).rstrip() in apply assert "wg-fabric0" in apply @@ -327,79 +323,21 @@ def test_reconcile_identity_probe_recreates_only_volatile_runtime_lock(): assert '[ -f "$KEY" ]' in script -def test_firewalld_scripts_use_only_panel_manifest_and_trusted_overlay_devices(): - current = plan() - apply = _apply_script( - current, current.nodes[0], {"host-a": KEY1, "host-b": KEY2}, - firewall_mode=FIREWALL_MODE_FIREWALLD, - ) - observe = _observe_script( - ("100.64.1.2",), "100.64.1.1", ("203.0.113.2",), - firewall_mode=FIREWALL_MODE_FIREWALLD, - connection_uuid="12345678-1234-5678-1234-567812345678", - host_id="host-a", fabric_id=current.fabric_id, - overlay_cidr=current.overlay_cidr, - ) - ifup = _apply_script( - current, current.nodes[0], {"host-a": KEY1, "host-b": KEY2}, - deployment_mode="ifupdown", ifup_path="/usr/bin/ifup", - firewall_mode=FIREWALL_MODE_FIREWALLD, - ) - for script in (apply, observe, ifup): - result = subprocess.run( - ["/bin/sh", "-n", "-c", script], - check=False, capture_output=True, text=True, - ) - assert result.returncode == 0, result.stderr - assert f"NFT_MANIFEST={FABRIC_NFT_MANIFEST}" in script - assert "runtime-to-permanent" not in script - assert "firewall-cmd --reload" not in script - assert "--complete-reload" not in script - assert "--permanent --get-default-zone" not in script - - assert "# virt-cloud-nft-manifest-v1" in apply - assert 'NFT_MAIN="$NFT_MANIFEST"' in apply - assert 'mktemp "$BASE/.nftables.conf.XXXXXX"' in apply - assert 'chmod 600 "$_tmp_nft_main"' in apply - assert ( - 'publish_desired_file "$_tmp_nft_main" "$NFT_MANIFEST" nft_main' - in apply - ) - # A manifest can include already-active VPC tables, so checking the whole - # manifest with nft -c would produce false conflicts. Rules are checked - # and loaded through their separately ownership-validated files. - assert '"$NFT" -c -f "$NFT_MANIFEST"' not in apply - assert "zone=trusted" in apply - assert '--zone=trusted --add-interface="$IFACE"' in ifup - assert '--zone=trusted --add-interface="$vpc_bridge"' in ifup - assert '--zone=trusted --add-interface="$endpoint_dev"' not in apply - assert "__VC_FABRIC_FIREWALL_MODE__" in observe - assert "__VC_FABRIC_FIREWALL_STATE__" in observe - assert "firewalld_runtime_zone_before" in apply - assert "firewalld_permanent_zone_before" in apply - assert 'firewall|firewalld_nft' in apply - - -def test_networkmanager_trust_is_bound_only_to_the_fabric_profile(): + +def test_networkmanager_profile_has_no_firewall_zone(): current = plan() profile = _nm_profile_template( - current, current.nodes[0], - "12345678-1234-5678-1234-567812345678", - FIREWALL_MODE_FIREWALLD, - ) - native_profile = _nm_profile_template( current, current.nodes[0], "12345678-1234-5678-1234-567812345678", FIREWALL_MODE_NATIVE, ) - assert "autoconnect=false" in profile assert "autoconnect-retries=0" in profile assert "zone=" not in profile assert "interface-name=wg-fabric0" in profile assert "203.0.113." not in profile - assert "zone=trusted" not in native_profile + assert "zone=trusted" not in profile def test_endpoint_rp_filter_evidence_uses_linux_max_without_blocking_strict(): @@ -1012,7 +950,7 @@ def test_fixed_artifacts_publish_desired_snapshot_before_atomic_install(): current = plan() apply = _apply_script( current, current.nodes[0], {"host-a": KEY1, "host-b": KEY2}, - firewall_mode=FIREWALL_MODE_FIREWALLD, + firewall_mode=FIREWALL_MODE_NATIVE, ) assert "record_desired_file() {" in apply assert "record_desired_absent() {" in apply @@ -1023,7 +961,7 @@ def test_fixed_artifacts_publish_desired_snapshot_before_atomic_install(): for source, target, name in ( ("$_tmp_owner", "$OWNER", "owner"), ("$_tmp_nft", "$NFT_RULES", "nft_rules"), - ("$_tmp_nft_main", "$NFT_MANIFEST", "nft_main"), + ("$_tmp_nft_main", "$NFT_MAIN", "nft_main"), ("$_tmp_helper", "$HELPER", "helper"), ("$_tmp_nm", "$NM_PROFILE", "nm_profile"), ("$_tmp_conf", "$CONF", "conf"), @@ -1044,18 +982,16 @@ def test_rollback_keeps_recovery_helper_until_every_other_restore_succeeds(): current = plan() apply = _apply_script( current, current.nodes[0], {"host-a": KEY1, "host-b": KEY2}, - firewall_mode=FIREWALL_MODE_FIREWALLD, + firewall_mode=FIREWALL_MODE_NATIVE, ) rollback = apply.index("rollback() {") first_fixed_restore = apply.index( 'restore_file "$UNIT" unit', rollback) verify_runtime = apply.index( "\n verify_rollback_rp_filter\n", first_fixed_restore) - guarded_handoff = apply.index( - "ROLLBACK_GUARDED=yes", verify_runtime) helper_comment = apply.index( "# Keep the transaction's complete recovery helper installed", - guarded_handoff, + verify_runtime, ) cleanup_state = apply.index( "rollback_publish_cleanup_helper_state || ROLLBACK_FAILED=yes", @@ -1066,8 +1002,7 @@ def test_rollback_keeps_recovery_helper_until_every_other_restore_succeeds(): final_decision = apply.index( 'if [ "$ROLLBACK_FAILED" = yes ]; then', helper_restore) - assert first_fixed_restore < verify_runtime < guarded_handoff - assert guarded_handoff < helper_comment < cleanup_state + assert first_fixed_restore < verify_runtime < helper_comment < cleanup_state assert cleanup_state < helper_restore < final_decision assert apply.count('restore_file "$HELPER" helper') == 1 @@ -1279,8 +1214,7 @@ def test_ifupdown_transition_hook_executes_without_reconciling_stale_config( start = _BOOT_HELPER.index( 'if [ "$DEPLOYMENT_MODE" = ifupdown ] &&\n' ' [ "${1:-}" = --ifupdown-configure ]; then') - end = _BOOT_HELPER.index( - 'if [ "$FIREWALL_MODE" = firewalld_nft ]', start) + end = _BOOT_HELPER.index("vc_firewall_mode_ready || exit 71", start) block = _BOOT_HELPER[start:end].replace( '"/proc/sys/net/ipv4/conf/$IFACE/rp_filter"', '"$VC_TEST_RPF"', @@ -1424,17 +1358,12 @@ def test_recovery_and_rollback_restore_rp_filter_before_reactivation(): ) recovery_ifup = recovery.index( '"$IFUP_PATH" --force "$IFACE"', recovery_restore) - recovery_trusted = recovery.index( - '"$FIREWALL_CMD" --zone=trusted --add-interface="$IFACE"', - recovery_restore, - ) recovery_verify = recovery.index( - "if ! journal_verify_rp_filter;", recovery_trusted) + "if ! journal_verify_rp_filter;", recovery_restore) assert recovery_phase_load < recovery_phase_adopt < recovery_recreating assert recovery_recreating < recovery_recreate < recovery_recreated assert recovery_recreated < recovery_restore < recovery_nm_up < recovery_verify assert recovery_restore < recovery_ifup < recovery_verify - assert recovery_restore < recovery_trusted < recovery_verify assert ( 'VC_FABRIC_APPLYING=1 \\\n' ' VC_FABRIC_RESTORE_RP_FILTER="$journal_rp_filter_before"' @@ -1458,16 +1387,11 @@ def test_recovery_and_rollback_restore_rp_filter_before_reactivation(): '"$IP" link set dev "$IFACE" up', rollback_restore) rollback_nm_up = apply.index( '"$NMCLI" connection up uuid', rollback_restore) - rollback_trusted = apply.index( - '"$FIREWALL_CMD" --zone=trusted --add-interface="$IFACE"', - rollback_restore, - ) rollback_verify = apply.index( - "\n verify_rollback_rp_filter\n", rollback_trusted) + "\n verify_rollback_rp_filter\n", rollback_restore) assert rollback_recreating < rollback_recreate < rollback_recreated assert rollback_recreated < rollback_restore < rollback_link_up < rollback_verify assert rollback_restore < rollback_nm_up < rollback_verify - assert rollback_restore < rollback_trusted < rollback_verify def test_first_deploy_recovery_and_rollback_delete_link_before_owner(): @@ -1857,9 +1781,32 @@ def test_transaction_journal_preparing_state_is_atomically_published(): assert existing < state_required < existing_case < publish_start assert publish_start < preparing_state < preparing_meta < absence_proof assert absence_proof < publish < readback < first_snapshot + assert "firewall_mode=$FIREWALL_MODE" in apply[ + preparing_meta:absence_proof + ] assert 'mkdir "$TXN"' not in apply +def test_observe_marks_any_fabric_transaction_journal_as_mismatch(): + current = plan() + observe = _observe_script( + ("100.64.1.2",), "100.64.1.1", ("203.0.113.2",), + firewall_mode=FIREWALL_MODE_NATIVE, + connection_uuid="12345678-1234-5678-1234-567812345678", + host_id="host-a", fabric_id=current.fabric_id, + overlay_cidr=current.overlay_cidr, + ) + state = observe.index("firewall_mode_state=owned") + journal_path = observe.index( + "[ ! -e /etc/virt-cloud/.wg-fabric-txn ]", state) + journal_link = observe.index( + "[ ! -L /etc/virt-cloud/.wg-fabric-txn ]", journal_path) + mismatch = observe.index( + "|| firewall_mode_state=mismatch", journal_link) + marker = observe.index("__VC_FABRIC_FIREWALL_STATE__", mismatch) + assert state < journal_path < journal_link < mismatch < marker + + def test_networkmanager_filename_resolution_uses_supported_unique_list_rows(): assert "GENERAL.FILENAME" not in _NMCLI_FILENAME_RESOLVER_SH assert "GENERAL.FILENAME" not in _FIREWALL_MODE_GUARD_SH @@ -1868,27 +1815,23 @@ def test_networkmanager_filename_resolution_uses_supported_unique_list_rows(): 'LC_ALL=C "$NMCLI" -t --escape no -f UUID,FILENAME' in _NMCLI_FILENAME_RESOLVER_SH ) - # Resolve once before semantic ownership checks and once after them to - # reject a runtime filename switch during the readback transaction. + # Resolve before and after semantic ownership checks to reject a filename + # switch during the readback transaction. assert ( _FIREWALL_MODE_GUARD_SH.count( 'vc_nm_filename_for_uuid "$CONNECTION_UUID"') == 2 ) assert ( - _FIREWALL_MODE_GUARD_SH.count( - 'vc_nm_filename_for_uuid "$vc_nm_uuid"') == 1 - ) - assert ( - _BOOT_HELPER.count('vc_nm_filename_for_uuid "$vpc_nm_uuid"') == 2 + _BOOT_HELPER.count('vc_nm_filename_for_uuid "$vpc_nm_uuid"') == 1 ) current = plan() apply = _apply_script( current, current.nodes[0], {"host-a": KEY1, "host-b": KEY2}, - firewall_mode=FIREWALL_MODE_FIREWALLD, + firewall_mode=FIREWALL_MODE_NATIVE, ) observe = _observe_script( ("100.64.1.2",), "100.64.1.1", ("203.0.113.2",), - firewall_mode=FIREWALL_MODE_FIREWALLD, + firewall_mode=FIREWALL_MODE_NATIVE, connection_uuid="12345678-1234-5678-1234-567812345678", host_id="host-a", fabric_id=current.fabric_id, overlay_cidr=current.overlay_cidr, @@ -1973,108 +1916,27 @@ def test_networkmanager_quarantine_keeps_owned_device_until_hard_link_down(): current = plan() helper = _apply_script( current, current.nodes[0], {"host-a": KEY1, "host-b": KEY2}, - firewall_mode=FIREWALL_MODE_FIREWALLD, - ) - minimum_owner = helper.index( - '[ "$(cat "$OWNER" 2>/dev/null)" = virt-cloud-wg-fabric-v1 ]') - emergency = helper.index( - "vc_fabric_emergency_install || return 1", minimum_owner) - profile_read = helper.index( - "vc_fabric_nm_profile_owned any || return 1", emergency) - unmanaged = helper.index( + firewall_mode=FIREWALL_MODE_NATIVE, + ) + start = helper.index("vc_fabric_nm_quarantine() {") + end = helper.index("\n}", start) + block = helper[start:end] + profile_read = block.index("vc_fabric_nm_profile_owned any || return 1") + unmanaged = block.index( '"$NMCLI" device set "$IFACE" managed no', profile_read) - hard_down = helper.index( + hard_down = block.index( '"$IP" link set dev "$IFACE" down', unmanaged) - runtime_zone = helper.index( - 'vc_fabric_runtime=$(vc_firewall_zone runtime)', hard_down) - clear_zone = helper.index( - 'connection modify --temporary uuid "$CONNECTION_UUID" ' - 'connection.zone ""', runtime_zone) - remove_trust = helper.index( - '--zone=trusted --remove-interface="$IFACE"', clear_zone) - assert ( - minimum_owner < emergency < profile_read < unmanaged < hard_down - < runtime_zone < clear_zone < remove_trust - ) - assert 'device reapply "$IFACE"' not in helper[unmanaged:remove_trust] - assert 'connection down uuid "$CONNECTION_UUID"' not in helper - assert 'device disconnect "$IFACE"' not in helper + assert profile_read < unmanaged < hard_down + assert "firewall-cmd" not in block + assert "connection.zone" not in block + assert 'device reapply "$IFACE"' not in block + assert 'connection down uuid "$CONNECTION_UUID"' not in block + assert 'device disconnect "$IFACE"' not in block -def test_networkmanager_zone_query_failure_occurs_only_after_hard_isolation( - tmp_path): - """Inject a zone-read failure and prove the owned link is already down.""" - owner = tmp_path / "owner" - owner.write_text("virt-cloud-wg-fabric-v1\n", encoding="utf-8") - profile = tmp_path / "profile" - profile.write_text("owned\n", encoding="utf-8") - log = tmp_path / "events.log" - script = tmp_path / "quarantine.sh" - script.write_text( - "#!/bin/sh\nset -eu\n" - + _FIREWALL_MODE_GUARD_SH - + f'''\nFIREWALL_MODE=firewalld_nft -DEPLOYMENT_MODE=networkmanager_nmcli -IFACE=wg-fabric0 -OWNER={owner} -NM_PROFILE={profile} -CONNECTION_UUID=12345678-1234-5678-1234-567812345678 -VC_TEST_LOG={log} -IP=vc_test_ip -WG=vc_test_wg -STAT=vc_test_stat -NMCLI=vc_test_nmcli -FIREWALL_CMD=vc_test_firewall -vc_test_ip() {{ - case " $* " in - *" wg-fabric0 "*) : ;; - *) printf '%s\\n' foreign-interface >> "$VC_TEST_LOG"; return 99 ;; - esac - case "$*" in - "link show dev wg-fabric0") return 0 ;; - "-o link show dev wg-fabric0") - printf '%s\\n' '7: wg-fabric0: mtu 1420'; return 0 ;; - *) return 0 ;; - esac -}} -vc_test_wg() {{ return 0; }} -vc_test_stat() {{ printf '%s\\n' '0:600'; }} -vc_test_nmcli() {{ - case "$*" in - *"connection.zone connection show"*) printf '%s\\n' trusted ;; - *) return 0 ;; - esac -}} -vc_test_firewall() {{ return 0; }} -vc_fabric_emergency_install() {{ - printf '%s\\n' emergency-guard >> "$VC_TEST_LOG" -}} -vc_fabric_nm_profile_owned() {{ return 0; }} -vc_fabric_nm_managed_state() {{ printf '%s' yes; }} -vc_fabric_nm_quarantine() {{ - printf '%s\\n' unmanaged >> "$VC_TEST_LOG" - printf '%s\\n' link-down >> "$VC_TEST_LOG" -}} -vc_firewall_zone() {{ - printf '%s\\n' "zone-query:$1" >> "$VC_TEST_LOG" - return 1 -}} -vc_fabric_quarantine down -''', - encoding="utf-8", - ) - script.chmod(0o700) - - result = subprocess.run( - ["/bin/sh", str(script)], check=False, capture_output=True, text=True, - ) - assert result.returncode != 0 - assert log.read_text(encoding="utf-8").splitlines() == [ - "emergency-guard", "unmanaged", "link-down", "zone-query:runtime", - ] -def _write_firewall_guard_tools(tmp_path: Path) -> tuple[Path, Path, Path]: +def _write_firewall_guard_tools(tmp_path: Path) -> Path: systemctl = tmp_path / "systemctl" systemctl.write_text("""#!/bin/sh case "$1:$2" in @@ -2092,46 +1954,22 @@ case "$1:$2" in esac """, encoding="utf-8") systemctl.chmod(0o700) - firewall_cmd = tmp_path / "firewall-cmd" - firewall_cmd.write_text("""#!/bin/sh -[ "$1" = --state ] || exit 2 -printf '%s\\n' "${VC_FIREWALLD_STATE:-running}" -""", encoding="utf-8") - firewall_cmd.chmod(0o700) - nft = tmp_path / "nft" - nft.write_text("""#!/bin/sh -[ "$1:$2:$3:$4:$5:$6" = "-j:list:table:inet:firewalld:" ] || exit 2 -if [ "${VC_FIREWALLD_NFT_JSON+x}" = x ]; then - printf '%s\n' "$VC_FIREWALLD_NFT_JSON" -else - printf '%s\n' '{"nftables":[{"table":{"family":"inet","name":"firewalld","flags":["owner","persist"]}}]}' -fi -""", encoding="utf-8") - nft.chmod(0o700) - return systemctl, firewall_cmd, nft + return systemctl def _run_firewall_guard( tmp_path: Path, mode: str, *, nft_active: str, nft_enabled: str, nft_load: str, firewalld_active: str, firewalld_enabled: str, - firewalld_load: str, config: str | None = None, - runtime_nft_json: str | None = None, + firewalld_load: str, ) -> subprocess.CompletedProcess[str]: - systemctl, firewall_cmd, nft = _write_firewall_guard_tools(tmp_path) - config_path = tmp_path / "firewalld.conf" - if config is not None: - config_path.write_text(config, encoding="utf-8") - guard = _FIREWALL_MODE_GUARD_SH.replace( - "/etc/firewalld/firewalld.conf", str(config_path), - ) + systemctl = _write_firewall_guard_tools(tmp_path) script = "\n".join(( "set -eu", f"SYSTEMCTL='{systemctl}'", - f"FIREWALL_CMD='{firewall_cmd}'", - f"NFT='{nft}'", + "NFT=/bin/false", f"FIREWALL_MODE='{mode}'", "find_tool() { for p in \"$@\"; do [ -x \"$p\" ] && { printf '%s' \"$p\"; return 0; }; done; return 1; }", - guard, + _FIREWALL_MODE_GUARD_SH, "vc_firewall_mode_ready", )) env = { @@ -2143,58 +1981,17 @@ def _run_firewall_guard( "VC_FIREWALLD_ENABLED": firewalld_enabled, "VC_FIREWALLD_LOAD": firewalld_load, } - if runtime_nft_json is not None: - env["VC_FIREWALLD_NFT_JSON"] = runtime_nft_json return subprocess.run( ["/bin/sh", "-c", script], env=env, check=False, capture_output=True, text=True, ) -@pytest.mark.parametrize(("nft_active", "nft_enabled", "nft_load"), [ - ("inactive", "disabled", "loaded"), - ("inactive", "masked", "masked"), - ("unknown", "unknown", "not-found"), -]) -def test_firewalld_runtime_guard_accepts_only_proven_safe_competitor_states( - tmp_path, nft_active, nft_enabled, nft_load, -): - result = _run_firewall_guard( - tmp_path, FIREWALL_MODE_FIREWALLD, - nft_active=nft_active, nft_enabled=nft_enabled, nft_load=nft_load, - firewalld_active="active", firewalld_enabled="enabled", - firewalld_load="loaded", - ) - assert result.returncode == 0, result.stderr -@pytest.mark.parametrize(("nft_active", "nft_enabled", "nft_load"), [ - ("active", "disabled", "loaded"), - ("failed", "disabled", "loaded"), - ("unknown", "unknown", "unknown"), - ("inactive", "static", "loaded"), -]) -def test_firewalld_runtime_guard_fails_closed_on_competing_or_unknown_nft( - tmp_path, nft_active, nft_enabled, nft_load, -): - result = _run_firewall_guard( - tmp_path, FIREWALL_MODE_FIREWALLD, - nft_active=nft_active, nft_enabled=nft_enabled, nft_load=nft_load, - firewalld_active="active", firewalld_enabled="enabled", - firewalld_load="loaded", - ) - assert result.returncode != 0 -def test_firewall_runtime_guard_requires_persistent_service_enablement(tmp_path): - firewalld = _run_firewall_guard( - tmp_path, FIREWALL_MODE_FIREWALLD, - nft_active="inactive", nft_enabled="masked", nft_load="masked", - firewalld_active="active", firewalld_enabled="enabled-runtime", - firewalld_load="loaded", - ) - assert firewalld.returncode != 0 - +def test_native_runtime_guard_requires_persistent_service_enablement(tmp_path): native = _run_firewall_guard( tmp_path, FIREWALL_MODE_NATIVE, nft_active="active", nft_enabled="enabled-runtime", nft_load="loaded", @@ -2204,44 +2001,8 @@ def test_firewall_runtime_guard_requires_persistent_service_enablement(tmp_path) assert native.returncode != 0 -@pytest.mark.parametrize("runtime_nft_json", [ - "", - '{"nftables":[]}', - '{"nftables":[{"table":{"family":"inet","name":"firewalld",' - '"flags":["persist"]}}]}', - '{"nftables":[{"table":{"family":"inet","name":"firewalld",' - '"flags":["owner"]}}]}', - '{"nftables":[{"table":{"family":"ip","name":"firewalld",' - '"flags":["owner","persist"]}}]}', -]) -def test_firewalld_runtime_guard_requires_live_owner_and_persist_flags( - tmp_path, runtime_nft_json, -): - result = _run_firewall_guard( - tmp_path, FIREWALL_MODE_FIREWALLD, - nft_active="inactive", nft_enabled="masked", nft_load="masked", - firewalld_active="active", firewalld_enabled="enabled", - firewalld_load="loaded", runtime_nft_json=runtime_nft_json, - ) - assert result.returncode != 0 -@pytest.mark.parametrize("config", [ - "FirewallBackend=iptables\n", - "NftablesTableOwner=no\n", - "FirewallBackend=nftables\nFirewallBackend=nftables\n", - "FirewallBackend=unknown\n", -]) -def test_firewalld_runtime_guard_rejects_unsupported_or_ambiguous_config( - tmp_path, config, -): - result = _run_firewall_guard( - tmp_path, FIREWALL_MODE_FIREWALLD, - nft_active="inactive", nft_enabled="masked", nft_load="masked", - firewalld_active="active", firewalld_enabled="enabled", - firewalld_load="loaded", config=config, - ) - assert result.returncode != 0 def test_native_runtime_guard_accepts_a_not_found_firewalld_unit(tmp_path): @@ -2461,7 +2222,6 @@ def _run_native_apply_underlay_preflight( _FIREWALL_MODE_GUARD_SH, rendered[preflight_start:preflight_end], rendered[endpoint_start:endpoint_end], - '[ -z "$UNDERLAY_ENDPOINT$UNDERLAY_PORT$UNDERLAY_INTERFACE" ]', )) return subprocess.run( ["/bin/sh", "-c", shell], @@ -2491,8 +2251,6 @@ def test_native_apply_uses_local_endpoint_without_firewalld_metadata(tmp_path): assert 'vc_underlay_route_path 203.0.113.2 "$LOCAL_ENDPOINT"' in rendered assert "underlay|local|" not in rendered assert "firewall_rule|" not in rendered - assert "underlay_endpoint=$LOCAL_ENDPOINT" not in rendered - assert "underlay_endpoint=$UNDERLAY_ENDPOINT" in rendered @pytest.mark.parametrize(("kwargs", "expected_status"), [ @@ -2522,117 +2280,14 @@ def test_native_apply_rejects_guest_or_tenant_underlay_interface( assert result.returncode == 71, result.stderr -def test_firewalld_endpoint_zone_ignores_unrelated_unverified_nm_profile(): - manager = FakeManager(KEY1, firewall_mode=FIREWALL_MODE_FIREWALLD) - capabilities = manager.profile["capabilities"] - network = capabilities["network"] - network["networkmanager_persistent_interface_zones"]["eth9"] = "?" - - evidence, error = _firewalld_endpoint_zone_evidence( - capabilities, "203.0.113.1") - assert error is None - assert evidence == { - "fabric_endpoint_interface": "eth0", - "fabric_endpoint_runtime_zone": "public", - "fabric_endpoint_permanent_zone": "public", - } - - network["networkmanager_persistent_interface_zones"]["eth0"] = "?" - evidence, error = _firewalld_endpoint_zone_evidence( - capabilities, "203.0.113.1") - assert evidence is None - assert error == "firewalld_endpoint_zone_unverified" - - -def test_firewalld_endpoint_zone_ignores_bounded_high_density_guest_profiles(): - manager = FakeManager(KEY1, firewall_mode=FIREWALL_MODE_FIREWALLD) - capabilities = manager.profile["capabilities"] - network = capabilities["network"] - guest_zones = {f"vnet{index}": "public" for index in range(300)} - network["firewalld_runtime_interface_zones"].update(guest_zones) - network["firewalld_permanent_interface_zones"].update(guest_zones) - network["networkmanager_persistent_interface_zones"].update( - {name: "" for name in guest_zones}) - - evidence, error = _firewalld_endpoint_zone_evidence( - capabilities, "203.0.113.1") - - assert error is None - assert evidence["fabric_endpoint_interface"] == "eth0" - assert evidence["fabric_endpoint_runtime_zone"] == "public" - - -@pytest.mark.parametrize( - "interface", ["vnet3", "vcv-123", "tap3", "macvtap3", "fwpr3", "fwln3"], -) -def test_firewalld_endpoint_zone_rejects_guest_or_tenant_endpoint(interface): - manager = FakeManager(KEY1, firewall_mode=FIREWALL_MODE_FIREWALLD) - network = manager.profile["capabilities"]["network"] - network["local_ipv4_interfaces"] = {"203.0.113.1": [interface]} - - evidence, error = _firewalld_endpoint_zone_evidence( - manager.profile["capabilities"], "203.0.113.1") - assert evidence is None - assert error == "firewalld_endpoint_zone_unsafe_or_drifted" -def test_firewalld_endpoint_zone_rejects_inventory_above_4096_interfaces(): - manager = FakeManager(KEY1, firewall_mode=FIREWALL_MODE_FIREWALLD) - capabilities = manager.profile["capabilities"] - capabilities["network"][ - "networkmanager_persistent_interface_zones" - ] = {f"vnet{index}": "" for index in range(4096)} - capabilities["network"][ - "networkmanager_persistent_interface_zones" - ]["eth0"] = "public" - evidence, error = _firewalld_endpoint_zone_evidence( - capabilities, "203.0.113.1") - assert evidence is None - assert error == "firewalld_endpoint_zone_unverified" -def test_panel_manifest_validator_is_strict_and_allowlisted(tmp_path): - fake_stat = tmp_path / "stat" - fake_stat.write_text("#!/bin/sh\nprintf '%s\\n' '0:600'\n", encoding="utf-8") - fake_stat.chmod(0o700) - manifest = tmp_path / "nftables.conf" - script = "\n".join(( - "set -eu", - f"STAT='{fake_stat}'", - _NFT_MANIFEST_VALIDATION_SH, - f"vc_nft_manifest_validate '{manifest}'", - )) - manifest.write_text( - '# virt-cloud-nft-manifest-v1\n' - 'include "/etc/nftables.d/virt-cloud-wg-fabric.nft"\n' - 'include "/etc/nftables.d/virt-cloud-vpc-0123456789a.nft"\n', - encoding="utf-8", - ) - valid = subprocess.run( - ["/bin/sh", "-c", script], check=False, - capture_output=True, text=True, - ) - assert valid.returncode == 0, valid.stderr - - for invalid in ( - 'include "/etc/nftables.d/*.nft"\n', - 'include "/etc/nftables.d/virt-cloud-wg-fabric.nft"\n' - 'include "/etc/nftables.d/virt-cloud-wg-fabric.nft"\n', - 'include "/etc/nftables.d/virt-cloud-vpc-0123456789.nft"\n', - ): - manifest.write_text( - "# virt-cloud-nft-manifest-v1\n" + invalid, - encoding="utf-8", - ) - rejected = subprocess.run( - ["/bin/sh", "-c", script], check=False, - capture_output=True, text=True, - ) - assert rejected.returncode != 0 def test_fabric_nft_validator_rejects_unowned_rules_and_extra_objects(): @@ -2762,77 +2417,6 @@ def test_boot_recovery_accepts_alma_output_dstnat_priority(): assert result.returncode == 0, result.stderr -def test_firewalld_boot_contract_is_fail_closed_and_gates_libvirt(): - helper = _BOOT_HELPER - emergency = helper.index("vc_fabric_emergency_install || exit 71") - vpc_emergency = helper.index("vc_vpc_emergency_down || exit 71", emergency) - manager_quarantine = helper.index( - "vc_fabric_quarantine down || exit 71", vpc_emergency) - early_quarantine = manager_quarantine - mode_guard = helper.index("vc_firewall_mode_ready || exit 71") - config_read = helper.index('done < "$CONF"') - nft_readback = helper.index( - 'nft_table=$("$NFT" -s list table inet virt_cloud_wg_fabric') - trust = helper.index( - 'connection modify --temporary uuid "$CONNECTION_UUID" ' - 'connection.zone trusted') - - assert emergency < vpc_emergency < manager_quarantine - assert early_quarantine < mode_guard < config_read < nft_readback < trust - assert "vc_fabric_nft_strict_shape" in helper - assert '"$NFT" -s list table' in helper - assert "nft_snapshot" in helper - assert '"$CMP" -s "$vpc_live_snapshot" "$vpc_snapshot_file"' in helper - native_refresh = helper.index( - 'vc_vpc_native_live_digest "$vpc_nft_name" ' - '$vpc_existing_families || exit 71') - native_check = helper.index( - '"$NFT" -c -f "$vpc_nft_batch"', native_refresh) - native_recheck = helper.index( - 'vc_vpc_native_live_digest "$vpc_nft_name" ' - '$vpc_existing_families || exit 71', native_check) - native_compare = helper.index( - '[ "$VC_VPC_NATIVE_DIGEST" = "$vpc_native_precommit" ]', - native_recheck) - native_apply = helper.index( - '"$NFT" -f "$vpc_nft_batch"', native_compare) - assert native_refresh < native_check < native_recheck < native_compare < native_apply - fabric_refresh = helper.index("vpc_state_refresh_nft || exit 71") - fabric_readback = helper.index("fabric_acl_ready || exit 71", fabric_refresh) - fabric_trust = helper.index( - 'connection modify --temporary uuid "$CONNECTION_UUID" ' - 'connection.zone trusted', fabric_readback) - assert fabric_refresh < fabric_readback < fabric_trust - vpc_pretrust = helper.index( - "vpc_boot_nft_snapshots_match || exit 71") - vpc_trust = helper.index( - 'connection modify --temporary uuid "$vpc_nm_uuid" ' - 'connection.zone trusted', vpc_pretrust) - assert vpc_pretrust < vpc_trust - # The boot quarantine must cover every VPC state entry accepted by the - # authoritative loader. In particular, a host with 65 VPCs must not leave - # the 65th bridge up while nft ACL recovery is still in progress. - assert MAX_VPC_STATE_FILES > 64 - assert ( - f'[ "$vc_vpc_emergency_count" -gt {MAX_VPC_STATE_FILES} ]' - in helper - ) - assert "__VC_MAX_VPC_STATE_FILES__" not in helper - topology_before = helper.index("vpc_bridge_ports_before_activation=") - activation = helper.index( - 'connection up uuid "$vpc_nm_uuid"', topology_before) - topology_after = helper.index( - 'vpc_bridge_ports_after_activation=', activation) - topology_compare = helper.index( - '[ "$vpc_bridge_ports_after_activation" = ' - '"$vpc_bridge_ports_before_activation" ]', topology_after) - assert topology_before < activation < topology_after < topology_compare - assert "TimeoutStartSec=5min" in _UNIT - assert "TimeoutStartSec=5min" in _REPAIR_UNIT - for unit in ("libvirtd.service", "virtqemud.service", "virtproxyd.service", - "libvirt-guests.service"): - assert unit in _UNIT - assert ".socket" not in _UNIT def _bridge_port_snapshot_runner(tmp_path, mode): @@ -2957,704 +2541,62 @@ esac '[ "$nm_failed" = yes ]', )) else: - body.append("vc_vpc_bridge_force_down vcv-0123456789a") - result = subprocess.run( - ["/bin/sh", "-c", "\n".join(body)], - env={ - "PATH": "/usr/bin:/bin", - "VC_IP_LOG": str(log), - "VC_LINK_UP": "yes" if link_up else "no", - }, - check=False, capture_output=True, text=True, - ) - return result, log - - -def test_vpc_force_down_rejects_nm_race_that_restores_up_flag(tmp_path): - result, log = _vpc_force_down_runner(tmp_path, link_up=True) - - assert result.returncode == 1, result.stderr - assert log.read_text(encoding="utf-8").splitlines() == [ - "link set dev vcv-0123456789a down", - "-o link show dev vcv-0123456789a", - ] - - -def test_vpc_force_down_still_runs_and_proves_isolation_after_nm_failure( - tmp_path, -): - result, log = _vpc_force_down_runner( - tmp_path, link_up=False, nm_fails=True, - ) - - assert result.returncode == 0, result.stderr - assert log.read_text(encoding="utf-8").splitlines() == [ - "link set dev vcv-0123456789a down", - "-o link show dev vcv-0123456789a", - ] - early = _BOOT_HELPER.index("vc_vpc_emergency_managed=") - early_down = _BOOT_HELPER.index( - 'vc_vpc_bridge_force_down "$vc_vpc_emergency_bridge"', early, - ) - early_after = _BOOT_HELPER.index( - "vc_vpc_emergency_ports_after=", early_down, - ) - bulk = _BOOT_HELPER.index("vpc_quarantine_managed=") - bulk_down = _BOOT_HELPER.index( - 'vc_vpc_bridge_force_down "$vpc_quarantine_bridge"', bulk, - ) - bulk_zone = _BOOT_HELPER.index("vpc_quarantine_permanent=", bulk_down) - assert early < early_down < early_after - assert bulk < bulk_down < bulk_zone - - -def test_firewalld_emergency_guard_is_retained_until_full_reconcile(): - current = plan() - apply = _apply_script( - current, current.nodes[0], {"host-a": KEY1, "host-b": KEY2}, - firewall_mode=FIREWALL_MODE_FIREWALLD, - ) - assert "table inet virt_cloud_fabric_emergency" in apply - assert "table bridge virt_cloud_fabric_emergency" in apply - assert ( - 'iifname "wg-fabric0" counter drop comment ' - '"virt-cloud-fabric-emergency-v2:$vc_emergency_token"' - ) in apply - assert ( - 'oifname "wg-fabric0" counter drop comment ' - '"virt-cloud-fabric-emergency-v2:$vc_emergency_token"' - ) in apply - assert 'iifname "vcv-*" counter drop comment ' in apply - assert 'oifname "vcv-*" counter drop comment ' in apply - assert 'ibrname "vcv-*" counter drop comment ' in apply - assert 'obrname "vcv-*" counter drop comment ' in apply - assert "virt-cloud-fabric-emergency-v2" in apply - assert "wg-fabric-emergency.owner" in apply - install = apply.index("vc_fabric_emergency_install || return 1") - nm_profile = apply.index("vc_fabric_nm_profile_owned any || return 1", install) - final_acl = apply.index( - 'nft_plan_owned "$final_fabric_acl" "$EXPECTED_DIGEST"') - final_set = apply.index("vpc_state_nft_set_matches || exit 71", final_acl) - final_public_key = apply.index( - 'final_public_key=$($WG pubkey < "$KEY")', final_set) - final_peers = apply.index( - 'final_actual_peers=$($WG show "$IFACE" peers', final_public_key) - final_routes = apply.index( - 'final_fabric_routes=$($IP -o route show table all dev "$IFACE"', - final_peers, - ) - final_path = apply.index('fabric_path=$($IP route get ', final_routes) - committed = apply.index('committed > "$_txn_state_tmp"', final_path) - queue = apply.index( - '"$SYSTEMCTL" start ' - 'virt-cloud-wg-fabric-reconcile.service', - committed, - ) - assert ( - install < nm_profile < final_acl < final_set < final_public_key - < final_peers < final_routes < final_path < committed < queue - ) - assert "vc_fabric_emergency_remove || exit 79" not in apply[final_path:] - assert ( - '"$CMP" -s "$TXN/route.verify" "$TXN/route.save"' - in apply - ) - - -def _emergency_nft_table(token: str) -> str: - return f'''table inet virt_cloud_fabric_emergency {{ - chain input {{ - type filter hook input priority 300; policy accept; - iifname "wg-fabric0" counter drop comment "virt-cloud-fabric-emergency-v2:{token}" - iifname "vcv-*" counter drop comment "virt-cloud-fabric-emergency-v2:{token}" - }} - chain output {{ - type filter hook output priority 300; policy accept; - oifname "wg-fabric0" counter drop comment "virt-cloud-fabric-emergency-v2:{token}" - oifname "vcv-*" counter drop comment "virt-cloud-fabric-emergency-v2:{token}" - }} - chain forward {{ - type filter hook forward priority 300; policy accept; - iifname "wg-fabric0" counter drop comment "virt-cloud-fabric-emergency-v2:{token}" - oifname "wg-fabric0" counter drop comment "virt-cloud-fabric-emergency-v2:{token}" - iifname "vcv-*" counter drop comment "virt-cloud-fabric-emergency-v2:{token}" - oifname "vcv-*" counter drop comment "virt-cloud-fabric-emergency-v2:{token}" - }} -}} -''' - - -def _emergency_bridge_nft_table( - token: str, *, canonical_meta: bool = True, - priority: str = "300", -) -> str: - meta = "meta " if canonical_meta else "" - return f'''table bridge virt_cloud_fabric_emergency {{ - chain input {{ - type filter hook input priority {priority}; policy accept; - {meta}ibrname "vcv-*" counter drop comment "virt-cloud-fabric-emergency-v2:{token}" - }} - chain output {{ - type filter hook output priority {priority}; policy accept; - {meta}obrname "vcv-*" counter drop comment "virt-cloud-fabric-emergency-v2:{token}" - }} - chain forward {{ - type filter hook forward priority {priority}; policy accept; - {meta}ibrname "vcv-*" counter drop comment "virt-cloud-fabric-emergency-v2:{token}" - {meta}obrname "vcv-*" counter drop comment "virt-cloud-fabric-emergency-v2:{token}" - }} -}} -''' - - -def _run_emergency_bridge_shape( - table: str, token: str, -) -> subprocess.CompletedProcess[str]: - script = "\n".join(( - "set -eu", - _FIREWALL_MODE_GUARD_SH, - 'vc_fabric_emergency_bridge_shape "$1" "$2"', - )) - return subprocess.run( - ["/bin/sh", "-c", script, "bridge-shape", table, token], - env={"PATH": "/usr/bin:/bin"}, - check=False, capture_output=True, text=True, - ) - - -@pytest.mark.parametrize("platform", [ - pytest.param("debian-13-nftables-1.1.3", id="debian-13-nftables-1.1.3"), - pytest.param("almalinux-10-nftables-1.1.5", - id="almalinux-10-nftables-1.1.5"), -]) -def test_emergency_bridge_shape_accepts_canonical_host_output(platform): - token = "K" * 24 - result = _run_emergency_bridge_shape( - _emergency_bridge_nft_table(token), token, - ) - - assert result.returncode == 0, f"{platform}: {result.stderr}" - - -@pytest.mark.parametrize(("canonical_meta", "priority"), [ - pytest.param(False, "300", id="legacy-unprefixed-meta-expression"), - pytest.param(True, "filter + 300", id="canonical-symbolic-priority"), -]) -def test_emergency_bridge_shape_keeps_exact_supported_serializations( - canonical_meta, priority, -): - token = "L" * 24 - result = _run_emergency_bridge_shape( - _emergency_bridge_nft_table( - token, canonical_meta=canonical_meta, priority=priority, - ), - token, - ) - - assert result.returncode == 0, result.stderr - - -@pytest.mark.parametrize("case", [ - "wrong-ingress-meta-key", - "wrong-egress-meta-key", - "wrong-ingress-interface", - "wrong-egress-interface", - "wrong-verdict", - "wrong-token", - "double-meta", - "missing-rule", - "extra-rule", - "extra-statement", - "wrong-chain", - "wrong-priority", - "wrong-policy", -]) -def test_emergency_bridge_shape_rejects_noncanonical_or_drifted_rules(case): - token = "M" * 24 - table = _emergency_bridge_nft_table(token) - first_rule = ( - f' meta ibrname "vcv-*" counter drop comment ' - f'"virt-cloud-fabric-emergency-v2:{token}"\n' - ) - replacements = { - "wrong-ingress-meta-key": ("meta ibrname", "meta iifname"), - "wrong-egress-meta-key": ("meta obrname", "meta oifname"), - "wrong-ingress-interface": ( - 'meta ibrname "vcv-*"', 'meta ibrname "vcx-*"', - ), - "wrong-egress-interface": ( - 'meta obrname "vcv-*"', 'meta obrname "vcx-*"', - ), - "wrong-verdict": ("counter drop", "counter accept"), - "wrong-token": (token, "N" * 24), - "double-meta": ("meta ibrname", "meta meta ibrname"), - "missing-rule": (first_rule, ""), - "extra-rule": (first_rule, first_rule + first_rule), - "extra-statement": (first_rule, first_rule + " counter drop\n"), - "wrong-chain": ("chain input {", "chain ingress {"), - "wrong-priority": ("priority 300", "priority 301"), - "wrong-policy": ("policy accept", "policy drop"), - } - old, new = replacements[case] - drifted = table.replace(old, new, 1) - assert drifted != table - - result = _run_emergency_bridge_shape(drifted, token) - - assert result.returncode != 0 - - -def _write_emergency_nft_mock( - tmp_path: Path, table: str, *, table_present: bool = True, - families: tuple[str, ...] = ("inet", "bridge"), -) -> tuple[Path, Path, Path]: - live = tmp_path / "emergency.nft" - live.write_text(table, encoding="utf-8") - bridge_live = tmp_path / "emergency-bridge.nft" - token_match = re.search( - r"virt-cloud-fabric-emergency-v2:([A-Za-z0-9]{24})", table, - ) - bridge_live.write_text( - _emergency_bridge_nft_table(token_match.group(1)) - if token_match else "", - encoding="utf-8", - ) - state = tmp_path / "table-present" - tables = tmp_path / "tables" - if table_present: - state.write_text("yes\n", encoding="utf-8") - tables.write_text( - "".join( - f"table {family} virt_cloud_fabric_emergency\n" - for family in families - ) if table_present else "", - encoding="utf-8", - ) - log = tmp_path / "nft.log" - nft = tmp_path / "nft" - nft.write_text( - "#!/bin/sh\n" - "extract_table() {\n" - " awk -v wanted=\"$1\" '\n" - " $0 == \"table \" wanted \" virt_cloud_fabric_emergency {\" {on=1}\n" - " on {print; opens=gsub(/{/, \"&\"); closes=gsub(/}/, \"&\"); " - "depth+=opens-closes; if (depth==0) exit}\n" - " ' \"$2\" > \"$3\"\n" - "}\n" - "printf '%s\\n' \"$*\" >> \"$VC_NFT_LOG\"\n" - "case \"$*\" in\n" - " 'list tables')\n" - " [ ! -f \"$VC_NFT_STATE\" ] || cat \"$VC_NFT_TABLES\" ;;\n" - " '-s list table inet virt_cloud_fabric_emergency')\n" - " [ -f \"$VC_NFT_STATE\" ] || exit 1\n" - " grep -Fqx 'table inet virt_cloud_fabric_emergency' \"$VC_NFT_TABLES\" || exit 1\n" - " cat \"$VC_NFT_LIVE\" ;;\n" - " '-s list table bridge virt_cloud_fabric_emergency')\n" - " [ -f \"$VC_NFT_STATE\" ] || exit 1\n" - " grep -Fqx 'table bridge virt_cloud_fabric_emergency' \"$VC_NFT_TABLES\" || exit 1\n" - " cat \"$VC_NFT_BRIDGE_LIVE\" ;;\n" - " -c\\ -f\\ *) exit 0 ;;\n" - " -f\\ *)\n" - " if grep -Fqx 'delete table inet virt_cloud_fabric_emergency' \"$2\" && " - "grep -Fqx 'delete table bridge virt_cloud_fabric_emergency' \"$2\"; then " - "rm -f \"$VC_NFT_STATE\"; : > \"$VC_NFT_TABLES\"; else " - "touch \"$VC_NFT_STATE\"; printf '%s\\n' " - "'table inet virt_cloud_fabric_emergency' " - "'table bridge virt_cloud_fabric_emergency' > \"$VC_NFT_TABLES\"; " - "extract_table inet \"$2\" \"$VC_NFT_LIVE\"; " - "extract_table bridge \"$2\" \"$VC_NFT_BRIDGE_LIVE.raw\"; " - "sed -e 's/ ibrname / meta ibrname /' " - "-e 's/ obrname / meta obrname /' " - "\"$VC_NFT_BRIDGE_LIVE.raw\" > \"$VC_NFT_BRIDGE_LIVE\"; " - "rm -f \"$VC_NFT_BRIDGE_LIVE.raw\"; fi ;;\n" - " *) exit 2 ;;\n" - "esac\n", - encoding="utf-8", - ) - nft.chmod(0o700) - return nft, state, log - - -def _run_emergency_guard( - tmp_path: Path, nft: Path, state: Path, log: Path, body: str, *, - firewall_mode: str = FIREWALL_MODE_FIREWALLD, -) -> subprocess.CompletedProcess[str]: - runtime = tmp_path / "run" - runtime.mkdir(exist_ok=True) - owner = runtime / "wg-fabric-emergency.owner" - guard = _FIREWALL_MODE_GUARD_SH.replace("/run/virt-cloud", str(runtime)) - script = "\n".join(( - "set -eu", - f"FIREWALL_MODE={firewall_mode}", - f"NFT='{nft}'", - "CMP=/usr/bin/cmp", - "STAT=vc_test_stat", - f"EMERGENCY_OWNER='{owner}'", - "vc_test_stat() { printf '%s\\n' '0:600'; }", - guard, - body, - )) - return subprocess.run( - ["/bin/sh", "-c", script], - env={ - "PATH": "/usr/bin:/bin", - "VC_NFT_LOG": str(log), - "VC_NFT_STATE": str(state), - "VC_NFT_TABLES": str(tmp_path / "tables"), - "VC_NFT_LIVE": str(tmp_path / "emergency.nft"), - "VC_NFT_BRIDGE_LIVE": str(tmp_path / "emergency-bridge.nft"), - "VC_NFT_BRIDGE_DRIFT": str( - tmp_path / "emergency-bridge-drift.nft"), - "VC_NFT_EXPECTED": str(tmp_path / "expected.nft"), - }, - check=False, capture_output=True, text=True, - ) - - -def _write_emergency_owner(tmp_path: Path, token: str) -> Path: - owner = tmp_path / "run" / "wg-fabric-emergency.owner" - owner.parent.mkdir(exist_ok=True) - owner.write_text( - f"virt-cloud-fabric-emergency-v2:{token}\n", encoding="utf-8", - ) - owner.chmod(0o600) - return owner - - -def test_emergency_guard_never_adopts_preexisting_same_shape_table(tmp_path): - token = "A" * 24 - nft, state, log = _write_emergency_nft_mock( - tmp_path, _emergency_nft_table(token), - ) - result = _run_emergency_guard( - tmp_path, nft, state, log, - "if vc_fabric_emergency_install; then exit 90; fi", - ) - - assert result.returncode == 0, result.stderr - assert state.exists() - assert not (tmp_path / "run" / "wg-fabric-emergency.owner").exists() - assert log.read_text(encoding="utf-8").splitlines() == ["list tables"] - - -def test_emergency_guard_rejects_partial_owned_two_table_batch(tmp_path): - token = "G" * 24 - nft, state, log = _write_emergency_nft_mock( - tmp_path, _emergency_nft_table(token), families=("inet",), - ) - owner = _write_emergency_owner(tmp_path, token) - result = _run_emergency_guard( - tmp_path, nft, state, log, - "if vc_fabric_emergency_install; then exit 90; fi", - ) - - assert result.returncode == 0, result.stderr - assert state.exists() and owner.exists() - assert not any( - line.startswith("-f ") - for line in log.read_text(encoding="utf-8").splitlines() - ) - - -def test_emergency_guard_fresh_install_is_owned_and_reversible(tmp_path): - nft, state, log = _write_emergency_nft_mock( - tmp_path, "", table_present=False, - ) - result = _run_emergency_guard( - tmp_path, nft, state, log, - "vc_fabric_emergency_install\n" - "[ -n \"$VC_FABRIC_EMERGENCY_TOKEN\" ]\n" - "vc_fabric_emergency_remove", - ) - - assert result.returncode == 0, result.stderr - assert not state.exists() - assert not (tmp_path / "run" / "wg-fabric-emergency.owner").exists() - writes = [line for line in log.read_text(encoding="utf-8").splitlines() - if line.startswith("-f ")] - assert len(writes) == 2 - - -def test_emergency_guard_reuses_only_matching_random_owner_intent(tmp_path): - token = "B" * 24 - table = _emergency_nft_table(token) - nft, state, log = _write_emergency_nft_mock(tmp_path, table) - owner = _write_emergency_owner(tmp_path, token) - result = _run_emergency_guard( - tmp_path, nft, state, log, - "vc_fabric_emergency_install\n" - "[ \"$VC_FABRIC_EMERGENCY_TOKEN\" = 'BBBBBBBBBBBBBBBBBBBBBBBB' ]\n" - "vc_fabric_emergency_remove", - ) - - assert result.returncode == 0, result.stderr - assert not state.exists() - assert not owner.exists() - writes = [line for line in log.read_text(encoding="utf-8").splitlines() - if line.startswith("-f ")] - assert len(writes) == 1 - - -def test_emergency_guard_drift_refuses_delete_and_preserves_owner(tmp_path): - token = "C" * 24 - original = _emergency_nft_table(token) - (tmp_path / "expected.nft").write_text(original, encoding="utf-8") - nft, state, log = _write_emergency_nft_mock( - tmp_path, original.replace(" counter drop ", " counter accept ", 1), - ) - owner = _write_emergency_owner(tmp_path, token) - result = _run_emergency_guard( - tmp_path, nft, state, log, - f"VC_FABRIC_EMERGENCY_TOKEN='{token}'\n" - "VC_FABRIC_EMERGENCY_SNAPSHOT=$(cat \"$VC_NFT_EXPECTED\")\n" - "VC_FABRIC_EMERGENCY_BRIDGE_SNAPSHOT=$(cat \"$VC_NFT_BRIDGE_LIVE\")\n" - "if vc_fabric_emergency_remove; then exit 90; fi", - ) - - assert result.returncode == 0, result.stderr - assert state.exists() - assert owner.exists() - assert not any( - line.startswith("-f ") - for line in log.read_text(encoding="utf-8").splitlines() - ) - - -def test_native_adopts_guard_left_by_failed_firewalld_gate_and_removes_it( - tmp_path, -): - nft, state, log = _write_emergency_nft_mock( - tmp_path, "", table_present=False, - ) - failed = _run_emergency_guard( - tmp_path, nft, state, log, - "vc_fabric_emergency_install\nexit 71", - ) - - assert failed.returncode == 71 - owner = tmp_path / "run" / "wg-fabric-emergency.owner" - assert state.exists() and owner.exists() - - recovered = _run_emergency_guard( - tmp_path, nft, state, log, - "vc_fabric_emergency_adopt\n" - "[ -n \"$VC_FABRIC_EMERGENCY_SNAPSHOT\" ]\n" - "vc_fabric_native_adopt_vpc_state_empty\n" - "vc_fabric_emergency_remove", - firewall_mode=FIREWALL_MODE_NATIVE, - ) - - assert recovered.returncode == 0, recovered.stderr - assert not state.exists() - assert not owner.exists() - - -@pytest.mark.parametrize("residual_name", [ - "0123456789a.state", - "0123456789a.owner", - ".migration-residual", -]) -def test_native_adopt_with_any_vpc_residual_keeps_guard_for_maintenance( - tmp_path, residual_name, -): - token = "J" * 24 - nft, state, log = _write_emergency_nft_mock( - tmp_path, _emergency_nft_table(token), - ) - owner = _write_emergency_owner(tmp_path, token) - vpc_state_dir = tmp_path / "vpc-dataplane" - vpc_state_dir.mkdir(mode=0o700) - (vpc_state_dir / residual_name).write_text( - "owner|virt-cloud-vpc-dataplane-v1\n" - "firewall_mode|native_nftables\n", - encoding="utf-8", - ) - result = _run_emergency_guard( - tmp_path, nft, state, log, - f"VPC_STATE_DIR='{vpc_state_dir}'\n" - "vc_fabric_emergency_adopt\n" - "vc_test_vpc_stat() { printf '%s\\n' '0:700'; }\n" - "STAT=vc_test_vpc_stat\n" - "if vc_fabric_native_adopt_vpc_state_empty; then exit 90; fi", - firewall_mode=FIREWALL_MODE_NATIVE, - ) - - assert result.returncode == 0, result.stderr - assert state.exists() and owner.exists() - assert not any( - line.startswith("-f ") - for line in log.read_text(encoding="utf-8").splitlines() + body.append("vc_vpc_bridge_force_down vcv-0123456789a") + result = subprocess.run( + ["/bin/sh", "-c", "\n".join(body)], + env={ + "PATH": "/usr/bin:/bin", + "VC_IP_LOG": str(log), + "VC_LINK_UP": "yes" if link_up else "no", + }, + check=False, capture_output=True, text=True, ) + return result, log -def test_native_failure_after_adoption_keeps_guard_and_owner(tmp_path): - token = "D" * 24 - nft, state, log = _write_emergency_nft_mock( - tmp_path, _emergency_nft_table(token), - ) - owner = _write_emergency_owner(tmp_path, token) - result = _run_emergency_guard( - tmp_path, nft, state, log, - "vc_fabric_emergency_adopt\nexit 71", - firewall_mode=FIREWALL_MODE_NATIVE, - ) +def test_vpc_force_down_rejects_nm_race_that_restores_up_flag(tmp_path): + result, log = _vpc_force_down_runner(tmp_path, link_up=True) - assert result.returncode == 71 - assert state.exists() and owner.exists() - assert not any( - line.startswith("-f ") - for line in log.read_text(encoding="utf-8").splitlines() - ) + assert result.returncode == 1, result.stderr + assert log.read_text(encoding="utf-8").splitlines() == [ + "link set dev vcv-0123456789a down", + "-o link show dev vcv-0123456789a", + ] -def test_native_rejects_external_emergency_table_without_owner(tmp_path): - token = "E" * 24 - nft, state, log = _write_emergency_nft_mock( - tmp_path, _emergency_nft_table(token), - ) - result = _run_emergency_guard( - tmp_path, nft, state, log, - "if vc_fabric_emergency_adopt; then exit 90; fi", - firewall_mode=FIREWALL_MODE_NATIVE, +def test_vpc_force_down_still_runs_and_proves_isolation_after_nm_failure( + tmp_path, +): + result, log = _vpc_force_down_runner( + tmp_path, link_up=False, nm_fails=True, ) assert result.returncode == 0, result.stderr - assert state.exists() - assert not (tmp_path / "run" / "wg-fabric-emergency.owner").exists() - assert log.read_text(encoding="utf-8").splitlines() == ["list tables"] - - -def test_native_clears_owner_only_crash_intent_without_nft_write(tmp_path): - token = "F" * 24 - nft, state, log = _write_emergency_nft_mock( - tmp_path, "", table_present=False, - ) - owner = _write_emergency_owner(tmp_path, token) - result = _run_emergency_guard( - tmp_path, nft, state, log, - "vc_fabric_emergency_adopt", - firewall_mode=FIREWALL_MODE_NATIVE, + assert log.read_text(encoding="utf-8").splitlines() == [ + "link set dev vcv-0123456789a down", + "-o link show dev vcv-0123456789a", + ] + early = _BOOT_HELPER.index("vc_vpc_emergency_managed=") + early_down = _BOOT_HELPER.index( + 'vc_vpc_bridge_force_down "$vc_vpc_emergency_bridge"', early, ) - - assert result.returncode == 0, result.stderr - assert not state.exists() and not owner.exists() - assert not any( - line.startswith("-f ") - for line in log.read_text(encoding="utf-8").splitlines() - ) - - -def test_native_second_adopt_clears_all_stale_in_memory_guard_state(tmp_path): - token = "H" * 24 - nft, state, log = _write_emergency_nft_mock( - tmp_path, _emergency_nft_table(token), - ) - owner = _write_emergency_owner(tmp_path, token) - result = _run_emergency_guard( - tmp_path, nft, state, log, - "vc_fabric_emergency_adopt\n" - "vc_fabric_emergency_remove\n" - "vc_fabric_emergency_adopt\n" - "[ -z \"${VC_FABRIC_EMERGENCY_SNAPSHOT:-}\" ]\n" - "[ -z \"${VC_FABRIC_EMERGENCY_BRIDGE_SNAPSHOT:-}\" ]\n" - "[ -z \"${VC_FABRIC_EMERGENCY_TOKEN:-}\" ]", - firewall_mode=FIREWALL_MODE_NATIVE, + early_after = _BOOT_HELPER.index( + "vc_vpc_emergency_ports_after=", early_down, ) + assert early < early_down < early_after - assert result.returncode == 0, result.stderr - assert not state.exists() and not owner.exists() - - -def test_emergency_bridge_drift_refuses_atomic_two_table_delete(tmp_path): - token = "I" * 24 - nft, state, log = _write_emergency_nft_mock( - tmp_path, _emergency_nft_table(token), - ) - owner = _write_emergency_owner(tmp_path, token) - bridge_drift = _emergency_bridge_nft_table(token).replace( - "counter drop", "counter accept", 1, - ) - (tmp_path / "emergency-bridge-drift.nft").write_text( - bridge_drift, encoding="utf-8", - ) - result = _run_emergency_guard( - tmp_path, nft, state, log, - "vc_fabric_emergency_adopt\n" - "cp \"$VC_NFT_BRIDGE_DRIFT\" \"$VC_NFT_BRIDGE_LIVE\"\n" - "if vc_fabric_emergency_remove; then exit 90; fi", - firewall_mode=FIREWALL_MODE_NATIVE, - ) - assert result.returncode == 0, result.stderr - assert state.exists() and owner.exists() - assert not any( - line.startswith("-f ") - for line in log.read_text(encoding="utf-8").splitlines() - ) -def test_native_apply_keeps_adopted_guard_for_full_boot_helper_readback(): - current = plan() - apply = _apply_script( - current, current.nodes[0], {"host-a": KEY1, "host-b": KEY2}, - firewall_mode=FIREWALL_MODE_NATIVE, - ) - apply_adopt = apply.index("vc_fabric_emergency_adopt || exit 71") - apply_vpc_empty = apply.index( - "vc_fabric_native_adopt_vpc_state_empty || exit 71", - apply_adopt, - ) - mode_gate = apply.index("vc_firewall_mode_ready || exit 71") - final_acl = apply.rindex( - "final_fabric_acl=$($NFT -s list table inet virt_cloud_wg_fabric" - ) - committed = apply.index("committed > \"$_txn_state_tmp\"") - queue = apply.index( - '"$SYSTEMCTL" start ' - 'virt-cloud-wg-fabric-reconcile.service', - committed, - ) - transaction = apply.index('TXN="$BASE/.wg-fabric-txn"') - assert apply_adopt < apply_vpc_empty < mode_gate < transaction - assert mode_gate < final_acl < committed < queue - assert "vc_fabric_emergency_remove || exit 79" not in apply[final_acl:] - - boot_adopt = _BOOT_HELPER.index("vc_fabric_emergency_adopt || exit 71") - boot_vpc_empty = _BOOT_HELPER.index( - "vc_fabric_native_adopt_vpc_state_empty || exit 71", - boot_adopt, - ) - vpc_down = _BOOT_HELPER.index("vc_vpc_emergency_down || exit 71") - final_acl = _BOOT_HELPER.rindex("fabric_acl_ready || exit 71") - boot_remove = _BOOT_HELPER.rindex("vc_fabric_emergency_remove || exit 79") - assert boot_adopt < boot_vpc_empty < vpc_down < final_acl < boot_remove - assert ( - 'if [ "$FIREWALL_MODE" = firewalld_nft ] ||\n' - ' [ -n "${VC_FABRIC_EMERGENCY_SNAPSHOT:-}" ]; then' - in _BOOT_HELPER - ) -def test_firewalld_apply_verifies_gate_dependencies_before_commit(): - current = plan() - script = _apply_script( - current, current.nodes[0], {"host-a": KEY1, "host-b": KEY2}, - firewall_mode=FIREWALL_MODE_FIREWALLD, - ) - daemon_reload = script.index('"$SYSTEMCTL" daemon-reload') - dependency_readback = script.index( - 'gate_requires=$($SYSTEMCTL show -p Requires --value') - committed = script.index('committed > "$_txn_state_tmp"') - assert daemon_reload < dependency_readback < committed - assert 'case " $gate_requires " in' in script - assert 'case " $gate_after " in' in script def test_apply_queues_reconcile_dispatcher_only_after_durable_commit(): current = plan() script = _apply_script( current, current.nodes[0], {"host-a": KEY1, "host-b": KEY2}, - firewall_mode=FIREWALL_MODE_FIREWALLD, + firewall_mode=FIREWALL_MODE_NATIVE, ) committed_marker = script.rindex('committed > "$_txn_state_tmp"') committed_flag = script.rindex("COMMITTED=yes") @@ -3916,12 +2858,12 @@ def test_old_committed_trap_never_deletes_concurrent_ready_journal(tmp_path): def test_inactive_primary_retry_requires_complete_managed_evidence(tmp_path): start = _BOOT_HELPER.index("vc_primary_inactive_retry_ready() {") - end = _BOOT_HELPER.index("\n}\nvc_firewalld_runtime_repair_needed()", start) + 2 + end = _BOOT_HELPER.index("\n}\nvc_fabric_rpf_repair_needed()", start) + 2 function = _BOOT_HELPER[start:end] digest = "a" * 64 conf = tmp_path / "wg-fabric0.conf" conf.write_text( - f"digest|{digest}\nfirewall|firewalld_nft\n", encoding="utf-8", + f"digest|{digest}\nfirewall|native_nftables\n", encoding="utf-8", ) owner = tmp_path / "wg-fabric0.owner" owner.write_text("virt-cloud-wg-fabric-v1\n", encoding="utf-8") @@ -3980,7 +2922,7 @@ esac f"HELPER='{helper}'", f"UNIT='{unit}'", f"PLAN_DIGEST='{digest}'", - "FIREWALL_MODE=firewalld_nft", + "FIREWALL_MODE=native_nftables", "IFACE=wg-fabric0", function, "vc_primary_inactive_retry_ready", @@ -4001,32 +2943,13 @@ esac assert rejected.returncode == 1 -def test_firewalld_apply_rechecks_exact_fabric_acl_immediately_before_trust(): - current = plan() - script = _apply_script( - current, current.nodes[0], {"host-a": KEY1, "host-b": KEY2}, - firewall_mode=FIREWALL_MODE_FIREWALLD, - ) - first_shape = script.index( - 'nft_plan_owned "$fabric_acl_applied" "$EXPECTED_DIGEST"') - refresh = script.rfind( - "vpc_state_refresh_nft || exit 71", 0, first_shape) - assert refresh >= 0 - first_set = script.index("vpc_state_nft_set_matches || exit 71", first_shape) - pretrust_shape = script.index( - 'nft_plan_owned "$fabric_acl_pretrust" "$EXPECTED_DIGEST"', first_set) - pretrust_set = script.index( - "vpc_state_nft_set_matches || exit 71", pretrust_shape) - trust = script.index( - 'connection modify --temporary uuid ', pretrust_set) - assert refresh < first_shape < first_set < pretrust_shape < pretrust_set < trust def test_fabric_nft_journal_records_only_post_refresh_as_applied(): current = plan() script = _apply_script( current, current.nodes[0], {"host-a": KEY1, "host-b": KEY2}, - firewall_mode=FIREWALL_MODE_FIREWALLD, + firewall_mode=FIREWALL_MODE_NATIVE, ) installing = script.index("write_nft_phase installing") installed_snapshot = script.index( @@ -4099,7 +3022,7 @@ def test_first_deploy_recovery_uses_only_fenced_temporary_helper(): current = plan() script = _apply_script( current, current.nodes[0], {"host-a": KEY1, "host-b": KEY2}, - firewall_mode=FIREWALL_MODE_FIREWALLD, + firewall_mode=FIREWALL_MODE_NATIVE, ) helper_created = script.index( @@ -4166,7 +3089,7 @@ def _run_transaction_recovery_helper_owned_harness( current = plan() apply = _apply_script( current, current.nodes[0], {"host-a": KEY1, "host-b": KEY2}, - firewall_mode=FIREWALL_MODE_FIREWALLD, + firewall_mode=FIREWALL_MODE_NATIVE, ) function_start = apply.index("transaction_recovery_helper_owned() {") function_end = apply.index( @@ -4266,7 +3189,7 @@ def _run_ready_recovery_dispatch_harness( current = plan() apply = _apply_script( current, current.nodes[0], {"host-a": KEY1, "host-b": KEY2}, - firewall_mode=FIREWALL_MODE_FIREWALLD, + firewall_mode=FIREWALL_MODE_NATIVE, ) function_start = apply.index("transaction_recovery_helper_owned() {") function_end = apply.index( @@ -4440,7 +3363,7 @@ def test_installing_recovery_requires_exact_stable_first_deploy_evidence(): current = plan() script = _apply_script( current, current.nodes[0], {"host-a": KEY1, "host-b": KEY2}, - firewall_mode=FIREWALL_MODE_FIREWALLD, + firewall_mode=FIREWALL_MODE_NATIVE, ) recover_start = script.index("recover_journal() {") recover_end = script.index( @@ -4464,21 +3387,8 @@ def test_installing_recovery_requires_exact_stable_first_deploy_evidence(): '[ "$journal_firewall_mode" = "$FIREWALL_MODE" ]', deployment_match, ) - underlay_fence = recover.index( - 'if [ "${RECOVER_ONLY:-no}" != yes ] ||\n' - ' [ "${RECOVER_CURRENT:-no}" = yes ]; then', - firewall_match, - ) - endpoint_match = recover.index( - '[ "$journal_underlay_endpoint" = "$UNDERLAY_ENDPOINT" ]', - underlay_fence, - ) - port_match = recover.index( - '[ "$journal_underlay_port" = "$UNDERLAY_PORT" ]', - endpoint_match, - ) special = recover.index( - 'if [ "${RECOVER_INSTALLING:-no}" = yes ]; then', port_match) + 'if [ "${RECOVER_INSTALLING:-no}" = yes ]; then', firewall_match) phase_gate = recover.index( '[ "$journal_nft_phase" = installing ]', special) @@ -4530,16 +3440,8 @@ def test_installing_recovery_requires_exact_stable_first_deploy_evidence(): '|| exit 79', sysctl_artifact, ) - mode_case = recover.index( - 'case "$journal_firewall_mode" in', nft_rules_artifact) - firewalld_baseline = recover.index( - 'journal_marker_secure nft_main.absent || exit 79', mode_case) - nft_main_artifact = recover.index( - '"$journal_nft_main" nft_main || exit 79', - firewalld_baseline, - ) native_baseline = recover.index( - 'journal_native_nft_main_safe || exit 79', nft_main_artifact, + 'journal_native_nft_main_safe || exit 79', nft_rules_artifact, ) current_owner = recover.index( 'journal_current_owner_state=absent', native_baseline) @@ -4614,12 +3516,10 @@ def test_installing_recovery_requires_exact_stable_first_deploy_evidence(): assert ( digest_fence < digest_match < connection_match < deployment_match - < firewall_match < underlay_fence < endpoint_match < port_match - < special < phase_gate < absent_baseline < clean_first + < firewall_match < special < phase_gate < absent_baseline < clean_first < clean_markers < desired_rules < desired_mode < desired_sha < expected_sha < desired_shape < desired_empty < owner_artifact < sysctl_artifact - < nft_rules_artifact < mode_case < firewalld_baseline - < nft_main_artifact < native_baseline < current_owner + < nft_rules_artifact < native_baseline < current_owner < installing_owner_gate < first_read < live_shape < live_empty < second_read < stable_compare < atomic_snapshot < installed_compare < expected_file < allow_replace < verify_read @@ -4671,7 +3571,7 @@ def _installing_recovery_fragments(): def _run_installing_recovery_harness( tmp_path, *, failure="none", artifact_state="post_trap_absent", - firewall_mode=FIREWALL_MODE_FIREWALLD, + firewall_mode=FIREWALL_MODE_NATIVE, native_state="baseline_current_no_desired", ): digest = "a" * 64 @@ -5064,6 +3964,7 @@ exit 1 native_snapshot_target=native_snapshot_target, native_desired_target=native_desired_target, native_live_target=native_live_target, + native_baseline=native_baseline, ) @@ -5091,14 +3992,17 @@ def test_installing_recovery_execution_snapshots_stable_alma_live_table( assert fixture.call_count.read_text(encoding="utf-8") == "2\n" assert not list(fixture.journal.glob(".nft-installed.*")) expected_present = { - "post_trap_absent": set(), + "post_trap_absent": {"nft_main"}, "hard_crash_exact_desired": set(fixture.artifact_values), - "mixed": {"owner", "nft_rules"}, + "mixed": {"owner", "nft_rules", "nft_main"}, }[artifact_state] for name, value in fixture.artifact_values.items(): target = fixture.targets[name] if name in expected_present: - assert target.read_bytes() == value + expected = ( + fixture.native_baseline if name == "nft_main" else value + ) + assert target.read_bytes() == expected assert stat.S_IMODE(target.stat().st_mode) == 0o600 else: assert not target.exists() @@ -5155,7 +4059,7 @@ def test_installing_recovery_execution_rejects_ambiguous_evidence( elif failure == "artifact_symlink": assert fixture.targets["nft_main"].is_symlink() assert fixture.artifact_symlink_target.read_bytes() == ( - fixture.artifact_values["nft_main"] + fixture.native_baseline ) elif failure == "artifact_desired_absent": assert (fixture.journal / "owner.desired.absent").is_file() @@ -5257,19 +4161,6 @@ def test_native_installing_recovery_rejects_unproven_nft_main_images( ) -def test_firewalld_apply_renders_installing_recovery_helper_syntax(): - current = plan() - script = _apply_script( - current, current.nodes[0], {"host-a": KEY1, "host-b": KEY2}, - firewall_mode=FIREWALL_MODE_FIREWALLD, - ) - - assert "__VC_FABRIC_NFT_RULES_SHA256__" not in script - syntax = subprocess.run( - ["/bin/sh", "-n"], input=script, - check=False, capture_output=True, text=True, - ) - assert syntax.returncode == 0, syntax.stderr def _timer_activity_recovery_runner( @@ -5380,7 +4271,7 @@ def test_apply_rollback_recovers_refresh_snapshot_before_phase_marker(): current = plan() script = _apply_script( current, current.nodes[0], {"host-a": KEY1, "host-b": KEY2}, - firewall_mode=FIREWALL_MODE_FIREWALLD, + firewall_mode=FIREWALL_MODE_NATIVE, ) snapshot_publish = script.index( 'mv -f "$TXN/nft.applied.tmp" "$TXN/nft.applied"') @@ -5407,147 +4298,15 @@ def test_apply_rollback_recovers_refresh_snapshot_before_phase_marker(): assert rollback_window < exact_snapshot < exact_set < stable_snapshot -def test_ifdown_removes_guard_but_apply_rollback_hands_it_to_reconcile(): - helper_install = _BOOT_HELPER.index( - "vc_fabric_emergency_install || exit 71") - helper_vpc_down = _BOOT_HELPER.index( - "vc_vpc_emergency_down || exit 71", helper_install) - ifdown = _BOOT_HELPER.index('if [ "${1:-}" = --ifdown ]; then') - link_delete = _BOOT_HELPER.index( - '"$IP" link del dev "$IFACE" || exit 72', ifdown) - helper_remove = _BOOT_HELPER.index( - "vc_fabric_emergency_remove || exit 79", link_delete) - assert helper_install < helper_vpc_down < ifdown < link_delete < helper_remove - - current = plan() - script = _apply_script( - current, current.nodes[0], {"host-a": KEY1, "host-b": KEY2}, - firewall_mode=FIREWALL_MODE_FIREWALLD, - ) - rollback = script.index("rollback() {") - quarantine = script.index( - "vc_fabric_quarantine || ROLLBACK_FAILED=yes", rollback) - first_zone_read = script.index( - "rollback_runtime_zone=$(vc_firewall_zone runtime", quarantine) - wg_readback = script.index( - '"$WG" showconf "$IFACE" > "$TXN/wg.verify"', first_zone_read) - address_readback = script.index( - '"$IP" address save dev "$IFACE" > "$TXN/address.verify"', - wg_readback, - ) - route_readback = script.index( - '"$IP" route save table all dev "$IFACE" > "$TXN/route.verify"', - address_readback, - ) - guarded_publish = script.index( - "printf '%s\\n' guarded > \"$rollback_state_tmp\"", - route_readback, - ) - guarded_failure = script.index("ROLLBACK_GUARDED=yes", guarded_publish) - assert ( - rollback < quarantine < first_zone_read < wg_readback - < address_readback < route_readback < guarded_publish - < guarded_failure - ) - assert "vc_fabric_emergency_remove || ROLLBACK_FAILED=yes" not in script - assert "__VC_FABRIC_ROLLBACK__=guarded" in script - assert 'guarded)\n # A synchronous apply rollback' in _BOOT_HELPER - - -def test_guarded_journal_is_consumed_only_inside_full_guarded_helper(tmp_path): - guard = _BOOT_HELPER.index("vc_fabric_emergency_install || exit 71") - recover_call = _BOOT_HELPER.index('recover_journal "$@"') - vpc_recovery = _BOOT_HELPER.index("recover_vpc_dataplanes", recover_call) - guard_remove = _BOOT_HELPER.index( - "vc_fabric_emergency_remove || exit 79", vpc_recovery, - ) - assert guard < recover_call < vpc_recovery < guard_remove - - function_start = _BOOT_HELPER.index("recover_journal() {") - function_end = _BOOT_HELPER.index( - '\n}\nif [ "${VC_FABRIC_APPLYING:-0}" != 1 ]; then', - function_start, - ) + 2 - recover_function = _BOOT_HELPER[function_start:function_end] - journal = tmp_path / "txn" - journal.mkdir() - (journal / "state").write_text("guarded\n", encoding="utf-8") - script = "\n".join(( - "set -eu", - f"JOURNAL='{journal}'", - "FIREWALL_MODE=native_nftables", - "IP=/bin/false", - "IFACE=wg-fabric0", - "journal_secure() { :; }", - recover_function, - "recover_journal", - "[ ! -e \"$JOURNAL\" ]", - )) - - result = subprocess.run( - ["/bin/sh", "-c", script], check=False, - capture_output=True, text=True, - ) - - assert result.returncode == 0, result.stderr - assert not journal.exists() - - -def test_new_apply_consumes_guarded_journal_without_old_helper_dependency(): - current = plan() - script = _apply_script( - current, current.nodes[0], {"host-a": KEY1, "host-b": KEY2}, - firewall_mode=FIREWALL_MODE_FIREWALLD, - ) - existing = script.index('case "$journal_existing_state" in') - guarded = script.index("guarded)", existing) - ready = script.index("ready)", guarded) - missing_snapshot = script.index( - 'transaction_missing_installing_snapshot=no', ready) - special_first = script.index( - 'if [ "$transaction_missing_installing_snapshot" = yes ]; then', - missing_snapshot, - ) - temporary_recovery = script.index( - 'VC_FABRIC_LOCK_HELD=1 /bin/sh "$_tmp_helper" ' - '--recover-installing || exit 79', - special_first, - ) - fixed_helper_owned = script.index( - 'transaction_recovery_helper_owned || exit 79', - temporary_recovery, - ) - fixed_recovery = script.index( - 'VC_FABRIC_LOCK_HELD=1 /bin/sh "$HELPER" --recover-only || exit 79', - fixed_helper_owned, - ) - assert ( - existing < guarded < ready < missing_snapshot < special_first - < temporary_recovery < fixed_helper_owned < fixed_recovery - ) - assert 'rm -rf "$BASE/.wg-fabric-txn"' in script[guarded:ready] - assert "TRANSACTION_RECOVERED=yes" in script[guarded:ready] - assert "$HELPER" not in script[guarded:ready] -def test_firewalld_crash_recovery_restores_removed_old_underlay_rule(): - helper = _BOOT_HELPER - metadata_read = helper.index( - "journal_old_underlay_remove=$(journal_value old_underlay_rule_remove") - restore_runtime = helper.index( - '"$FIREWALL_CMD" --zone="$journal_old_underlay_zone" ' - '--add-rich-rule="$journal_old_rule"') - restore_permanent = helper.index( - '"$FIREWALL_CMD" --permanent --zone="$journal_old_underlay_zone" ' - '--add-rich-rule="$journal_old_rule"') - assert metadata_read < restore_runtime < restore_permanent def test_repair_unit_contains_only_the_retry_dispatcher(): current = plan() script = _apply_script( current, current.nodes[0], {"host-a": KEY1, "host-b": KEY2}, - firewall_mode=FIREWALL_MODE_FIREWALLD, + firewall_mode=FIREWALL_MODE_NATIVE, ) rendered = _REPAIR_UNIT.replace( "__VC_FABRIC_DIGEST__", current.digest) @@ -5674,218 +4433,8 @@ def test_repair_retry_dispatches_before_lock_and_only_requeues_failures(): ) in _REPAIR_UNIT -def test_firewalld_runtime_drift_dispatches_helper_without_restarting_primary( - tmp_path): - generic = tmp_path / "generic" - generic.write_text("#!/bin/sh\nexit 0\n", encoding="utf-8") - generic.chmod(0o700) - stat = tmp_path / "stat" - stat.write_text( - "#!/bin/sh\n" - "if [ -d \"$3\" ]; then printf '%s\\n' '0:700'; " - "elif [ \"$3\" = \"$VC_TEST_SYSCTL\" ]; then " - "printf '%s\\n' '0:644'; " - "else printf '%s\\n' '0:600'; fi\n", - encoding="utf-8", - ) - stat.chmod(0o700) - ip = tmp_path / "ip" - ip.write_text("#!/bin/sh\nexit 0\n", encoding="utf-8") - ip.chmod(0o700) - firewall = tmp_path / "firewall-cmd" - firewall.write_text( - """#!/bin/sh -if [ "$1" = --state ]; then printf '%s\n' running; exit 0; fi -if [ "$1" = --zone=trusted ]; then - [ "${VC_TEST_DRIFT:-yes}" = yes ] && exit 1 - [ "${VC_TEST_DRIFT:-yes}" = vpc ] && \ - [ "$2" = --query-interface=vcv-0123456789a ] && exit 1 - exit 0 -fi -exit 2 -""", - encoding="utf-8", - ) - firewall.chmod(0o700) - systemctl = tmp_path / "systemctl" - systemctl.write_text( - """#!/bin/sh -printf '%s\n' "$*" >> "$VC_TEST_SYSTEMCTL_LOG" -case "$1:$2:$3" in - is-failed:--quiet:virt-cloud-wg-fabric.service) exit 1 ;; - is-active:virt-cloud-wg-fabric.service:) \ - printf '%s\n' "${VC_TEST_PRIMARY_STATE:-active}"; exit 0 ;; - start:virt-cloud-wg-fabric.service:) exit 0 ;; - *) exit 2 ;; -esac -""", - encoding="utf-8", - ) - systemctl.chmod(0o700) - recovery = tmp_path / "recovery" - recovery.write_text( - "#!/bin/sh\nprintf '%s\\n' \"$*\" > \"$VC_TEST_RECOVERY_LOG\"\n", - encoding="utf-8", - ) - recovery.chmod(0o700) - - test_digest = "0" * 64 - helper = _BOOT_HELPER.replace( - "__VC_FIREWALL_MODE__", FIREWALL_MODE_FIREWALLD, - ).replace( - "__VC_FABRIC_DIGEST__", test_digest, - ).replace( - "__VC_FABRIC_SYSCTL_VALIDATION__", - _FABRIC_SYSCTL_VALIDATION_SH, - ) - helper_lines = [] - generic_tools = {"WG", "NFT", "SHA256", "CMP", "AWK", "BRIDGE_CMD"} - for line in helper.splitlines(): - if line == "BASE=/etc/virt-cloud": - line = 'BASE="$VC_TEST_BASE"' - elif line == "SYSCTL_FILE=/etc/sysctl.d/90-virt-cloud-wg-fabric.conf": - line = 'SYSCTL_FILE="$VC_TEST_SYSCTL"' - elif line == ' vc_repair_rpf_path="/proc/sys/net/ipv4/conf/$IFACE/rp_filter"': - line = ' vc_repair_rpf_path="$VC_TEST_RPF"' - elif line == "EMERGENCY_OWNER=/run/virt-cloud/wg-fabric-emergency.owner": - line = 'EMERGENCY_OWNER="$VC_TEST_EMERGENCY_OWNER"' - elif line == "HELPER=/usr/local/libexec/virt-cloud-wg-fabric": - line = 'HELPER="$VC_TEST_RECOVERY"' - elif line.startswith("SYSTEMCTL=$(find_tool "): - line = 'SYSTEMCTL="$VC_TEST_SYSTEMCTL"' - elif line.startswith("FIREWALL_CMD=$(find_tool "): - line = 'FIREWALL_CMD="$VC_TEST_FIREWALL"' - elif line.startswith("STAT=$(find_tool "): - line = 'STAT="$VC_TEST_STAT"' - elif line.startswith("IP=$(find_tool "): - line = 'IP="$VC_TEST_IP"' - elif any(line.startswith(f"{name}=$(find_tool ") - for name in generic_tools): - name = line.split("=", 1)[0] - line = f'{name}="$VC_TEST_GENERIC"' - helper_lines.append(line) - - helper_path = tmp_path / "virt-cloud-wg-fabric" - helper_path.write_text("\n".join(helper_lines) + "\n", encoding="utf-8") - helper_path.chmod(0o700) - base = tmp_path / "state" - base.mkdir() - (base / "wg-fabric0.conf").write_text("owned\n", encoding="utf-8") - (base / "wg-fabric0.owner").write_text( - "virt-cloud-wg-fabric-v1\n", encoding="utf-8") - log = tmp_path / "recovery.log" - systemctl_log = tmp_path / "systemctl.log" - emergency_owner = tmp_path / "emergency.owner" - sysctl_file = tmp_path / "fabric-sysctl.conf" - sysctl_file.write_text( - "# virt-cloud-wg-fabric-v1\n" - f"# fabric-plan-digest={test_digest}\n" - "net.ipv4.conf.wg-fabric0.rp_filter = 2\n", - encoding="utf-8", - ) - rpf_file = tmp_path / "rp_filter" - rpf_file.write_text("2\n", encoding="utf-8") - env = { - "VC_TEST_BASE": str(base), - "VC_TEST_RECOVERY": str(recovery), - "VC_TEST_RECOVERY_LOG": str(log), - "VC_TEST_SYSTEMCTL": str(systemctl), - "VC_TEST_FIREWALL": str(firewall), - "VC_TEST_STAT": str(stat), - "VC_TEST_IP": str(ip), - "VC_TEST_GENERIC": str(generic), - "VC_TEST_EMERGENCY_OWNER": str(emergency_owner), - "VC_TEST_SYSTEMCTL_LOG": str(systemctl_log), - "VC_TEST_SYSCTL": str(sysctl_file), - "VC_TEST_RPF": str(rpf_file), - "VC_TEST_DRIFT": "yes", - "VC_TEST_PRIMARY_STATE": "active", - } - - condition = subprocess.run( - ["/bin/sh", str(helper_path), "--needs-repair"], - check=False, capture_output=True, text=True, env=env, - ) - assert condition.returncode == 0, condition.stderr - dispatcher = subprocess.run( - ["/bin/sh", str(helper_path), "--retry-gated-units"], - check=False, capture_output=True, text=True, env=env, - ) - assert dispatcher.returncode == 0, dispatcher.stderr - assert log.read_text(encoding="utf-8") == "\n" - - env["VC_TEST_DRIFT"] = "no" - healthy = subprocess.run( - ["/bin/sh", str(helper_path), "--needs-repair"], - check=False, capture_output=True, text=True, env=env, - ) - assert healthy.returncode == 1, healthy.stderr - - emergency_owner.write_text("foreign-or-owned-evidence\n", encoding="utf-8") - guarded_condition = subprocess.run( - ["/bin/sh", str(helper_path), "--needs-repair"], - check=False, capture_output=True, text=True, env=env, - ) - assert guarded_condition.returncode == 0, guarded_condition.stderr - log.unlink(missing_ok=True) - guarded_dispatch = subprocess.run( - ["/bin/sh", str(helper_path), "--retry-gated-units"], - check=False, capture_output=True, text=True, env=env, - ) - assert guarded_dispatch.returncode == 0, guarded_dispatch.stderr - assert log.read_text(encoding="utf-8") == "\n" - emergency_owner.unlink() - - env["VC_TEST_DRIFT"] = "yes" - env["VC_TEST_PRIMARY_STATE"] = "activating" - activating = subprocess.run( - ["/bin/sh", str(helper_path), "--needs-repair"], - check=False, capture_output=True, text=True, env=env, - ) - assert activating.returncode == 1, activating.stderr - - emergency_owner.write_text("owned-guard-evidence\n", encoding="utf-8") - activating_guard = subprocess.run( - ["/bin/sh", str(helper_path), "--retry-gated-units"], - check=False, capture_output=True, text=True, env=env, - ) - assert activating_guard.returncode == 0, activating_guard.stderr - assert "start virt-cloud-wg-fabric.service" in systemctl_log.read_text( - encoding="utf-8", - ) - emergency_owner.unlink() - env["VC_TEST_PRIMARY_STATE"] = "active" - - vpc_dir = base / "vpc-dataplane" - vpc_dir.mkdir(mode=0o700) - (vpc_dir / "0123456789a.owner").write_text( - "virt-cloud-vpc-dataplane-v1\n", encoding="utf-8") - env["VC_TEST_DRIFT"] = "vpc" - vpc_drift = subprocess.run( - ["/bin/sh", str(helper_path), "--needs-repair"], - check=False, capture_output=True, text=True, env=env, - ) - assert vpc_drift.returncode == 0, vpc_drift.stderr -def test_firewalld_apply_retry_accepts_only_proven_quarantine(): - current = plan() - script = _apply_script( - current, current.nodes[0], {"host-a": KEY1, "host-b": KEY2}, - firewall_mode=FIREWALL_MODE_FIREWALLD, - ) - assert "vc_fabric_retry_quarantine_ready()" in script - assert "vc_fabric_nm_profile_owned any || return 1" in script - assert "vc_fabric_nm_expect_managed no || return 1" in script - assert '[ "$vc_fabric_memory_before" = trusted ]' in script - assert 'vc_fabric_ifup_fragment_owned any || return 1' in script - assert ( - 'PREFLIGHT_FIREWALL_RUNTIME_ZONE_BEFORE=' - '$VC_FABRIC_QUARANTINE_RUNTIME_BEFORE' - ) in script - assert 'quarantined) vc_fabric_retry_quarantine_ready || exit 71 ;;' in script - assert "none:none|trusted:none|quarantined:none" in _BOOT_HELPER - assert '[ "$FIREWALL_RUNTIME_ZONE_BEFORE" = quarantined ]' in script def _operation_status_output( @@ -6382,7 +4931,7 @@ def test_fabric_journal_recovery_reports_fixed_safe_subcheckpoints(): assert nft_read < runtime < runtime_restore < finalize < helper_cleanup checkpoint_function = helper[ helper.index("vc_fabric_recovery_checkpoint()"): - helper.index("vc_fabric_emergency_reconcile_needed()") + helper.index("vc_legacy_firewalld_state_detected()") ] assert "/run/virt-cloud/fabric-operations/$VC_FABRIC_OPERATION_ID" in ( checkpoint_function) @@ -6428,10 +4977,6 @@ class FakeManager: self.sysctl_persistence = "owned" self.emergency_state = "clear" self.firewall_mode = firewall_mode - firewall_manager = ( - "firewalld" if firewall_mode == FIREWALL_MODE_FIREWALLD - else "nftables" - ) effective_host = host or ( "203.0.113.1" if public_key == KEY1 else "203.0.113.2") self.effective_host = effective_host @@ -6452,7 +4997,7 @@ class FakeManager: "network": { "network_manager": "networkmanager", "network_manager_supported": True, - "firewall_manager": firewall_manager, + "firewall_manager": "nftables", "firewall_mode": firewall_mode, "ip_probe_valid": True, "local_ipv4_addresses": [ @@ -6468,18 +5013,10 @@ class FakeManager: "rp_filter_interfaces": { endpoint_device: 1, }, - "firewalld_runtime_interface_zones": { - endpoint_device: "public", - }, - "firewalld_permanent_interface_zones": {}, "networkmanager_persistent_interface_zones": { endpoint_device: "", }, "ifcfg_zone_interfaces": [], - "firewall_manager_evidence": { - "firewalld_default_runtime": "public", - "firewalld_default_permanent": "public", - }, }, "nftables": { "ready": True, @@ -6518,11 +5055,6 @@ class FakeManager: f"__VC_FABRIC_ROUTE__={address}|local|{FABRIC_INTERFACE}\n" if self.self_route else "" ) - firewall_markers = ( - f"__VC_FABRIC_CONFIG_UNDERLAY__={self.underlay_endpoint}|{self.endpoint_device}|public\n" - "__VC_FABRIC_FIREWALL_RULE__=panel|yes|yes\n" - if self.firewall_mode == FIREWALL_MODE_FIREWALLD else "" - ) output = ( f"__VC_FABRIC_OWNER__={FABRIC_OWNER_MARKER}\n" "__VC_FABRIC_CONFIG_FABRIC__=hk-fabric\n" @@ -6531,7 +5063,6 @@ class FakeManager: f"__VC_FABRIC_CONFIG_FIREWALL__={self.firewall_mode}\n" "__VC_FABRIC_CONFIG_NETWORK__=100.64.1.0/24\n" f"__VC_FABRIC_CONFIG_OVERLAY__={address}\n" - f"{firewall_markers}" "__VC_FABRIC_CONFIG_OK__=yes\n" f"__VC_FABRIC_CONFIG_PEER__={peer}|{endpoint}|{target}/32\n" f"__VC_FABRIC_CONFIG_ROUTE__={target}/32\n" @@ -6811,20 +5342,6 @@ def test_adapter_remote_failure_exposes_only_whitelisted_phase(failure_phase): assert "private-key-material" not in safe_error -def test_adapter_pins_firewalld_mode_through_apply_and_observe(): - current = plan() - manager = FakeManager(KEY1, firewall_mode=FIREWALL_MODE_FIREWALLD) - adapter = FabricHostAdapter(manager) - - adapter.apply(current, current.nodes[0], {"host-a": KEY1, "host-b": KEY2}) - observed = adapter.observe( - current, current.nodes[0], {"host-a": KEY1, "host-b": KEY2}, - ) - - assert observed["config_ok"] is True - assert observed["observed"]["firewall_mode"] == FIREWALL_MODE_FIREWALLD - assert "FIREWALL_MODE=firewalld_nft" in manager.scripts[-2] - assert "FIREWALL_MODE=firewalld_nft" in manager.scripts[-1] def test_profile_firewall_mode_rejects_missing_or_crossed_capability_evidence(): @@ -6833,11 +5350,13 @@ def test_profile_firewall_mode_rejects_missing_or_crossed_capability_evidence(): manager.profile["capabilities"]["network"].update({ "firewall_manager": "firewalld", - "firewall_mode": FIREWALL_MODE_FIREWALLD, + "firewall_mode": "firewalld_nft", }) manager.profile["capabilities"]["nftables"]["firewall_mode"] = ( - FIREWALL_MODE_FIREWALLD) - assert _profile_firewall_mode(manager.profile) == FIREWALL_MODE_FIREWALLD + "firewalld_nft") + with pytest.raises(FabricError) as error: + _profile_firewall_mode(manager.profile) + assert error.value.error_code == "fabric_host_firewall_mode_conflict" manager.profile["capabilities"]["nftables"]["firewall_mode"] = ( FIREWALL_MODE_NATIVE) @@ -7311,8 +5830,6 @@ def test_readiness_probe_is_bounded_isolated_and_secret_free(): "firewall_manager": "nftables", "firewall_mode": "native_nftables", "underlay_interface": "br0", - "underlay_runtime_zone": None, - "underlay_permanent_zone": None, "ip_forward": 1, "rp_filter": 2, "rp_filter_all": 2, @@ -7346,7 +5863,7 @@ def test_readiness_probe_is_bounded_isolated_and_secret_free(): @pytest.mark.parametrize(("raw", "projected"), [ (FIREWALL_MODE_NATIVE, FIREWALL_MODE_NATIVE), - (FIREWALL_MODE_FIREWALLD, FIREWALL_MODE_FIREWALLD), + ("firewalld_nft", "unsupported"), ("../../operator-data", "unsupported"), (None, "unsupported"), ]) diff --git a/engine/tests/test_fabric_apply_final_readback.py b/engine/tests/test_fabric_apply_final_readback.py index c7500f3f..00c63f97 100644 --- a/engine/tests/test_fabric_apply_final_readback.py +++ b/engine/tests/test_fabric_apply_final_readback.py @@ -1,7 +1,7 @@ """Execute the production-rendered final readback with real CLI serialization. -Only external tools are replaced. The final shell body, nft owner checks, -tenant-set matcher and firewall-zone enumerator are the production code. +Only external tools are replaced. The final shell body, nft owner checks and +tenant-set matcher are the production code. """ from __future__ import annotations @@ -15,10 +15,10 @@ import sys import pytest from virsh_panel.fabric import ( + FIREWALL_MODE_NATIVE, _FABRIC_NFT_VALIDATION_SH, _FABRIC_WG_READBACK_SH, _BOOT_HELPER, - _FIREWALL_MODE_GUARD_SH, _TRANSACTION_SCRIPT, _VPC_STATE_LOADER, _apply_script, @@ -79,11 +79,11 @@ class FinalFixture: tool = tmp_path / "tool" tool.write_text(f"#!{sys.executable}\n" + _TOOLS, encoding="utf-8") tool.chmod(0o700) - for name in ("ip", "wg", "nft", "firewall-cmd"): + for name in ("ip", "wg", "nft"): (tmp_path / name).symlink_to(tool) (tmp_path / "private").write_text("test-only-private-identity\n") (tmp_path / "config").write_text(_config_text( - self.plan, self.plan.nodes[0], self.keys, "firewalld_nft")) + self.plan, self.plan.nodes[0], self.keys, FIREWALL_MODE_NATIVE)) for filename in ("vpc-peers", "vpc-routes", "vpc-tenants"): (tmp_path / filename).write_text("") @@ -129,15 +129,6 @@ class FinalFixture: "from", "100.64.1.1"], node.overlay_ip + " from 100.64.1.1 dev wg-fabric0 uid 0\n" " cache\n") - for permanent in (False, True): - prefix = ["--permanent"] if permanent else [] - self.respond("firewall-cmd", prefix + ["--get-zones"], "public trusted\n") - for zone in ("public", "trusted"): - active = not permanent and zone == "trusted" - self.respond("firewall-cmd", prefix + [f"--zone={zone}", - "--query-interface=wg-fabric0"], - "yes\n" if active else "no\n", 0 if active else 1) - def respond(self, tool: str, args: list[str], stdout: str, status: int = 0): self.responses[json.dumps([tool, *args])] = { "stdout": stdout, "status": status, @@ -162,7 +153,7 @@ class FinalFixture: def run(self): rendered = _apply_script( self.plan, self.plan.nodes[0], self.keys, - deployment_mode=self.manager, firewall_mode="firewalld_nft") + deployment_mode=self.manager, firewall_mode=FIREWALL_MODE_NATIVE) final = _between(rendered, "vc_fabric_operation_checkpoint final_readback\n", "_txn_state_tmp=$(mktemp") final = final.split("\n", 1)[1] @@ -173,12 +164,10 @@ class FinalFixture: _between(_VPC_STATE_LOADER, "vpc_state_allowed_for_peer() {", "vpc_state_route_owned() {"), _VPC_STATE_LOADER[_VPC_STATE_LOADER.index("vpc_state_nft_set_matches() {"):], - _between(_FIREWALL_MODE_GUARD_SH, "vc_firewall_zone() {", - "vc_firewall_zone_exists() {"), )) env = {**os.environ, "VC_FINAL_ROOT": str(self.root), "IFACE": "wg-fabric0", "EXPECTED_DIGEST": self.plan.digest, - "FIREWALL_MODE": "firewalld_nft", + "FIREWALL_MODE": FIREWALL_MODE_NATIVE, "MANAGED_PEERS": " ".join(self.keys[n.host_id] for n in self.plan.nodes[1:]), "MANAGED_ROUTES": " ".join(n.allowed_ip for n in self.plan.nodes[1:]), "KEY": str(self.root / "private"), "CONF": str(self.root / "config"), @@ -186,8 +175,7 @@ class FinalFixture: "VPC_ROUTE_FILE": str(self.root / "vpc-routes"), "VPC_TENANT_FILE": str(self.root / "vpc-tenants")} env.update({name: str(self.root / tool) for name, tool in ( - ("WG", "wg"), ("IP", "ip"), ("NFT", "nft"), - ("FIREWALL_CMD", "firewall-cmd"))}) + ("WG", "wg"), ("IP", "ip"), ("NFT", "nft"))}) (self.root / "responses").write_text(json.dumps(self.responses)) result = subprocess.run(["/bin/sh", "-eu", "-c", functions + "\n" + final], capture_output=True, text=True, env=env, check=False) @@ -210,7 +198,7 @@ def test_actual_final_slice_accepts_debian_and_alma_serialization(tmp_path, mana @pytest.mark.parametrize("drift", ["nft", "public-key", "listen-port", "down", "address", - "extra-peer", "endpoint", "allowed", "route", "path", "zone"]) + "extra-peer", "endpoint", "allowed", "route", "path"]) def test_actual_final_slice_rejects_mutable_readback_drift(tmp_path, drift): fixture = FinalFixture(tmp_path, "ifupdown") key = fixture.keys["2"] @@ -237,8 +225,6 @@ def test_actual_final_slice_rejects_mutable_readback_drift(tmp_path, drift): elif drift == "path": fixture.respond("ip", ["route", "get", "100.64.1.2", "from", "100.64.1.1"], "100.64.1.2 from 100.64.1.1 dev br0 uid 0\n cache\n") - else: - fixture.respond("firewall-cmd", ["--zone=trusted", "--query-interface=wg-fabric0"], "no\n", 1) result = fixture.run() assert result.returncode in (71, 78), result.stderr diff --git a/engine/tests/test_fabric_apply_safe_log.py b/engine/tests/test_fabric_apply_safe_log.py index 55ba0adb..a1f6e17d 100644 --- a/engine/tests/test_fabric_apply_safe_log.py +++ b/engine/tests/test_fabric_apply_safe_log.py @@ -36,7 +36,7 @@ def _write(path: Path, body: str, mode: int = 0o700): path.chmod(mode) -def _render(manager="ifupdown", firewall="firewalld_nft"): +def _render(manager="ifupdown", firewall="native_nftables"): plan = parse_fabric_plan({ "fabric_id": "safe-log-test", "region": "hk", "overlay_cidr": "100.64.1.0/24", "revision": 1, @@ -82,7 +82,7 @@ def _journal_wrapper(body): @pytest.mark.parametrize("detail", [ "entry", "final_acl", "final_tenants", "final_identity", "final_link", "final_addresses", "final_peers", "final_endpoints", "final_allowed_ips", - "final_routes", "final_overlay_paths", "final_zone", "commit", "convergence", + "final_routes", "final_overlay_paths", "commit", "convergence", "complete", ]) def test_rendered_apply_logger_has_one_fixed_fd3_record(detail): @@ -217,9 +217,8 @@ def test_first_apply_failure_precedes_secondary_rollback_cleanup_record(tmp_path @pytest.mark.parametrize("manager", ["ifupdown", "networkmanager_nmcli"]) -@pytest.mark.parametrize("firewall", ["native_nftables", "firewalld_nft"]) -def test_rendered_timer_start_strictly_follows_durable_commit(manager, firewall): - rendered = _render(manager, firewall) +def test_rendered_timer_start_strictly_follows_durable_commit(manager): + rendered = _render(manager) timer_command = '"$SYSTEMCTL" start virt-cloud-wg-fabric-reconcile.timer >/dev/null' assert rendered.count(timer_command) == 1 commit_marker = rendered.rindex('committed > "$_txn_state_tmp"') diff --git a/engine/tests/test_fabric_first_deploy_entry.py b/engine/tests/test_fabric_first_deploy_entry.py index 1d2a03b9..a70c77eb 100644 --- a/engine/tests/test_fabric_first_deploy_entry.py +++ b/engine/tests/test_fabric_first_deploy_entry.py @@ -12,7 +12,7 @@ from pathlib import Path import pytest from virsh_panel.fabric import ( - FIREWALL_MODE_FIREWALLD, + FIREWALL_MODE_NATIVE, _apply_script, _ifupdown_fragment, _nm_profile_template, @@ -28,34 +28,6 @@ def _write(path: Path, content: str, mode: int = 0o600) -> None: path.chmod(mode) -def _emergency_table(family: str, token: str) -> str: - rows = [f"table {family} virt_cloud_fabric_emergency {{"] - for chain in ("input", "output", "forward"): - rows.extend((f" chain {chain} {{", ( - f" type filter hook {chain} priority 300; policy accept;" - ))) - if family == "inet": - matches = [] - if chain in ("input", "forward"): - matches.extend(('iifname "wg-fabric0"', 'iifname "vcv-*"')) - if chain in ("output", "forward"): - matches.extend(('oifname "wg-fabric0"', 'oifname "vcv-*"')) - else: - matches = [] - if chain in ("input", "forward"): - matches.append('meta ibrname "vcv-*"') - if chain in ("output", "forward"): - matches.append('meta obrname "vcv-*"') - for match in matches: - rows.append( - f' {match} counter drop comment ' - f'"virt-cloud-fabric-emergency-v2:{token}"' - ) - rows.append(" }") - rows.append("}") - return "\n".join(rows) + "\n" - - _TOOLS = r'''import hashlib import json import os @@ -112,10 +84,7 @@ elif name == "ip": raise SystemExit(1) elif name == "nft": table_root = root / "tables" - if args == ["-j", "list", "table", "inet", "firewalld"]: - print('{"nftables":[{"table":{"family":"inet",' - '"name":"firewalld","flags":["owner","persist"]}}]}') - elif args == ["list", "tables"]: + if args == ["list", "tables"]: for path in sorted(table_root.glob("*")): family, table = path.name.split("--", 1) print("table", family, table) @@ -127,7 +96,7 @@ elif name == "nft": elif len(args) in (2, 3) and args[-2] == "-f": batch = Path(args[-1]).read_text(encoding="utf-8") # Recovery may delete only the completely proved interrupted Fabric - # table. Releasing emergency isolation is a test failure. + # table. if batch != "delete table inet virt_cloud_wg_fabric\n": raise SystemExit(2) if "-c" not in args: @@ -137,37 +106,20 @@ elif name == "nft": elif name == "systemctl": if args[:2] == ["show", "-p"]: if args[2] == "LoadState": - print("loaded" if args[-1] in ("firewalld", "nftables") - else "not-found") + print("masked" if args[-1] == "firewalld" else + "loaded" if args[-1] == "nftables" else "not-found") elif args[2] == "ActiveState": print("inactive") else: raise SystemExit(2) elif args[0] == "is-active" and args[1] in ("firewalld", "nftables"): - print("active" if args[1] == "firewalld" else "inactive") + print("inactive" if args[1] == "firewalld" else "active") elif args[0] == "is-enabled" and args[1] in ("firewalld", "nftables"): - print("enabled" if args[1] == "firewalld" else "disabled") + print("masked" if args[1] == "firewalld" else "enabled") elif args and args[0] in ("daemon-reload", "disable"): pass else: raise SystemExit(2) -elif name == "firewall-cmd": - if args == ["--state"]: - print("running") - elif "--get-zones" in args: - print("public trusted") - elif any(arg.startswith("--query-interface=") for arg in args): - zone = root / "runtime-zone" - wanted = next(arg.split("=", 1)[1] for arg in args - if arg.startswith("--zone=")) - raise SystemExit(0 if "--permanent" not in args and zone.is_file() - and zone.read_text().strip() == wanted else 1) - elif args == ["--zone=trusted", "--remove-interface=wg-fabric0"]: - (root / "runtime-zone").unlink(missing_ok=True) - elif any(arg.startswith("--query-rich-rule=") for arg in args): - raise SystemExit(1) - else: - raise SystemExit(2) elif name == "nmcli": profile = (root / "NetworkManager" / "system-connections" / "virt-cloud-wg-fabric0.nmconnection") @@ -238,7 +190,7 @@ def _fixture(tmp_path: Path, mode: str): } apply = _apply_script( plan, plan.nodes[0], public_keys, deployment_mode=mode, - firewall_mode=FIREWALL_MODE_FIREWALLD, + firewall_mode=FIREWALL_MODE_NATIVE, ) start = apply.index("#!/bin/sh\n# virt-cloud-wg-fabric-v1\n") end = re.search(r"\n__VC_FABRIC_[0-9a-f]{16}\n", apply[start:]).start() + start @@ -251,7 +203,7 @@ def _fixture(tmp_path: Path, mode: str): tool_path = tools / "external-tool" _write(tool_path, f"#!{sys.executable}\n" + _TOOLS, 0o700) for name in ("stat", "sha256sum", "ip", "nft", "systemctl", - "firewall-cmd", "nmcli", "wg", "bridge", "flock", "sync", + "nmcli", "wg", "bridge", "flock", "sync", "ifup", "ifquery", "ifdown"): (tools / name).symlink_to(tool_path) @@ -266,13 +218,15 @@ def _fixture(tmp_path: Path, mode: str): "/etc/network": tmp_path / "network", "/etc/sysctl.d": tmp_path / "sysctl.d", "/etc/nftables.d": tmp_path / "nftables.d", + "/etc/nftables.conf": tmp_path / "nftables.conf", "/proc/sys/net/ipv4": tmp_path / "proc" / "ipv4", } # Only redirect host resources/external tools. Keep all production shell # control flow and variable declarations, notably TIMER_UNIT, intact. for source, destination in roots.items(): helper = helper.replace(source, str(destination)) - destination.mkdir(parents=True, exist_ok=True) + if source != "/etc/nftables.conf": + destination.mkdir(parents=True, exist_ok=True) # The secure shadow ancestor check also reads /run itself; redirect only # that exact loop token, not unrelated paths or production control flow. helper = helper.replace("in /run ", "in " + shlex.quote(str(tmp_path / "run")) + " ") @@ -289,7 +243,7 @@ def _fixture(tmp_path: Path, mode: str): for name in ("ifup", "ifquery", "ifdown"): helper = helper.replace(f"/usr/sbin/{name}", str(tools / name)) for name in ("wg", "ip", "nft", "stat", "sha256sum", "systemctl", - "firewall-cmd", "nmcli", "bridge", "flock"): + "nmcli", "bridge", "flock"): helper = re.sub( rf"find_tool (?:/(?:usr/)?s?bin/{re.escape(name)} ?)+", f"find_tool {shlex.quote(str(tools / name))} ", @@ -300,14 +254,10 @@ def _fixture(tmp_path: Path, mode: str): tables = tmp_path / "tables" tables.mkdir() - token = "A" * 24 - _write(runtime / "wg-fabric-emergency.owner", ( - f"virt-cloud-fabric-emergency-v2:{token}\n" - )) - for family in ("inet", "bridge"): - _write(tables / f"{family}--virt_cloud_fabric_emergency", ( - _emergency_table(family, token) - )) + _write(roots["/etc/nftables.conf"], ( + '# distribution-owned main file\n' + f'include "{roots["/etc/nftables.d"]}/*.nft"\n' + ), 0o644) return plan, helper_path, base, runtime, roots @@ -322,22 +272,14 @@ def _journal(plan, base: Path, mode: str, helper: Path, root: Path, *, ).group(1) metadata = { "schema": "1", "deployment_mode": mode, - "firewall_mode": FIREWALL_MODE_FIREWALLD, + "firewall_mode": FIREWALL_MODE_NATIVE, "plan_digest": plan.digest, "connection_uuid": uuid, - "nft_main": str(base / "nftables.conf"), + "nft_main": str(root / "nftables.conf"), "iface_before": "no", "iface_up_before": "no", "rp_filter_before": "missing", "nft_before": "no", "service_enabled_before": "not-found", "service_active_before": "inactive", "timer_enabled_before": "not-found", "timer_active_before": "inactive", - "firewalld_runtime_zone_before": "none", - "firewalld_permanent_zone_before": "none", - "underlay_endpoint": "203.0.113.1", "underlay_port": "51820", - "underlay_zone": "public", "underlay_rule_owner": "panel", - "underlay_rule_runtime_before": "no", - "underlay_rule_permanent_before": "no", - "old_underlay_rule_remove": "no", "old_underlay_endpoint": "", - "old_underlay_port": "", "old_underlay_zone": "", } _write(journal / "meta", "".join( f"{key}={value}\n" for key, value in metadata.items() @@ -347,8 +289,13 @@ def _journal(plan, base: Path, mode: str, helper: Path, root: Path, *, _write(journal / "nft.table", "") for name in ("conf", "owner", "helper", "unit", "repair", "timer", "gate_generator", "nm_profile", "ifupdown_fragment", - "sysctl", "nft_rules", "nft_main"): + "sysctl", "nft_rules"): _write(journal / f"{name}.absent", "") + _write(journal / "nft_main.present", "") + _write(journal / "nft_main", ( + '# distribution-owned main file\n' + f'include "{root / "nftables.d"}/*.nft"\n' + ), 0o644) _write(journal / "key.present", "") _write(journal / "key", "test-private-key-snapshot\n") _write(base / "wg-fabric0.key", "test-private-key-snapshot\n") @@ -385,12 +332,7 @@ def test_first_deploy_complete_entry_recovers_absent_generator_and_timer( assert not (roots["/etc/systemd/system"] / "virt-cloud-wg-fabric-reconcile.timer").exists() assert not (tmp_path / "tables" / "inet--virt_cloud_wg_fabric").exists() - # Recovery keeps independent isolation; only a complete normal apply may - # activate interfaces or release these tables. - assert (runtime / "wg-fabric-emergency.owner").is_file() - for family in ("inet", "bridge"): - assert (tmp_path / "tables" / - f"{family}--virt_cloud_fabric_emergency").is_file() + assert not (runtime / "wg-fabric-emergency.owner").exists() calls = [json.loads(line) for line in (tmp_path / "calls").read_text(encoding="utf-8").splitlines()] assert ["systemctl", "show", "-p", "LoadState", "--value", @@ -424,10 +366,7 @@ def test_ready_refreshed_complete_entry_recovers_exact_first_deploy_snapshots( assert not (tmp_path / "tables" / "inet--virt_cloud_wg_fabric").exists() assert (base / "wg-fabric0.key").read_text(encoding="utf-8") == ( "test-private-key-snapshot\n") - assert (runtime / "wg-fabric-emergency.owner").is_file() - for family in ("inet", "bridge"): - assert (tmp_path / "tables" / - f"{family}--virt_cloud_fabric_emergency").is_file() + assert not (runtime / "wg-fabric-emergency.owner").exists() calls = [json.loads(line) for line in (tmp_path / "calls").read_text(encoding="utf-8").splitlines()] assert any(call[0] == "nft" and call[1] == "-f" for call in calls) @@ -437,11 +376,8 @@ def test_ready_refreshed_complete_entry_recovers_exact_first_deploy_snapshots( and call[-1] == "up" for call in calls) -@pytest.mark.parametrize("mode,runtime_shadow", [ - ("networkmanager_nmcli", False), ("networkmanager_nmcli", True), ("ifupdown", False), -]) -def test_ready_refreshed_persistent_entry_recovers_created_trusted_link( - tmp_path, mode, runtime_shadow): +@pytest.mark.parametrize("mode", ["networkmanager_nmcli", "ifupdown"]) +def test_ready_refreshed_persistent_entry_recovers_created_link(tmp_path, mode): plan, helper, base, runtime, roots = _fixture(tmp_path, mode) journal = _journal(plan, base, mode, helper, tmp_path, nft_phase="refreshed") @@ -459,26 +395,14 @@ def test_ready_refreshed_persistent_entry_recovers_created_trusted_link( uuid = re.search(r'^CONNECTION_UUID="([^"]+)"$', content, re.MULTILINE).group(1) profile = _nm_profile_template( - plan, plan.nodes[0], uuid, FIREWALL_MODE_FIREWALLD, + plan, plan.nodes[0], uuid, FIREWALL_MODE_NATIVE, ).replace("private-key=\n", "private-key=" + base64.b64encode(bytes([7]) * 32).decode() + "\n") persistent = (roots["/etc/NetworkManager"] / "system-connections" / "virt-cloud-wg-fabric0.nmconnection") publish(persistent, "nm_profile", profile) - if runtime_shadow: - shadow = (roots["/run/NetworkManager"] / "system-connections" / - "virt-cloud-wg-fabric0.nmconnection") - shadow_content = "\n".join(line for line in profile.splitlines() - if not line.startswith("#")) - shadow_content = shadow_content.replace( - "[connection]", "[connection]\nzone=trusted") - shadow_content += ("\n[proxy]\n[.nmmeta]\n" - f"shadowed-storage={persistent}\nshadowed-owned=true\n") - _write(shadow, shadow_content) - for directory in (tmp_path / "run", shadow.parent.parent, shadow.parent): - directory.chmod(0o755) _write(tmp_path / "nm-loaded-uuid", uuid + "\n") - _write(tmp_path / "nm-zone", "trusted\n") + _write(tmp_path / "nm-zone", "\n") else: fragment = _ifupdown_fragment(plan, plan.nodes[0]).replace( "/usr/local/libexec", str(roots["/usr/local/libexec"]), @@ -487,7 +411,6 @@ def test_ready_refreshed_persistent_entry_recovers_created_trusted_link( "virt-cloud-wg-fabric0", "ifupdown_fragment", fragment) _write(tmp_path / "link-present", "") _write(tmp_path / "link-up", "") - _write(tmp_path / "runtime-zone", "trusted\n") rpf = roots["/proc/sys/net/ipv4"] / "conf" / "wg-fabric0" / "rp_filter" _write(rpf, "2\n") @@ -497,19 +420,14 @@ def test_ready_refreshed_persistent_entry_recovers_created_trusted_link( assert not helper.exists() assert not (tmp_path / "link-present").exists() assert not (tmp_path / "link-up").exists() - assert not (tmp_path / "runtime-zone").exists() assert not rpf.exists() assert not (base / "wg-fabric0.owner").exists() assert not (tmp_path / "tables" / "inet--virt_cloud_wg_fabric").exists() - assert (runtime / "wg-fabric-emergency.owner").is_file() - for family in ("inet", "bridge"): - assert (tmp_path / "tables" / - f"{family}--virt_cloud_fabric_emergency").is_file() + assert not (runtime / "wg-fabric-emergency.owner").exists() calls = [json.loads(line) for line in (tmp_path / "calls").read_text(encoding="utf-8").splitlines()] assert ["ip", "link", "del", "dev", "wg-fabric0"] in calls - assert ["firewall-cmd", "--zone=trusted", - "--remove-interface=wg-fabric0"] in calls + assert not any(call[0] == "firewall-cmd" for call in calls) if mode == "networkmanager_nmcli": assert ["nmcli", "device", "set", "wg-fabric0", "managed", "no"] in calls assert ["nmcli", "connection", "delete", "uuid", uuid] in calls @@ -546,10 +464,7 @@ def test_ready_refreshed_persistent_entry_rejects_unproved_post_images( assert journal.is_dir() assert helper.read_text(encoding="utf-8") == content assert live.read_bytes() == before - assert (runtime / "wg-fabric-emergency.owner").is_file() - for family in ("inet", "bridge"): - assert (tmp_path / "tables" / - f"{family}--virt_cloud_fabric_emergency").is_file() + assert not (runtime / "wg-fabric-emergency.owner").exists() calls = [json.loads(line) for line in (tmp_path / "calls").read_text(encoding="utf-8").splitlines()] assert not any(call[0] == "nft" and call[1] == "-f" for call in calls) @@ -598,12 +513,9 @@ def test_full_ifupdown_transition_requires_transaction_published_gate( assert result.returncode == (0 if gate_state == "valid" else 71), result.stderr # A first deployment has no config yet; nested hooks only operate on the # transaction-owned link and rp_filter. They must not parse stale desired - # config, release the emergency guard, or start any systemd/libvirt unit. + # config or start any systemd/libvirt unit. assert not (base / "wg-fabric0.conf").exists() - assert (runtime / "wg-fabric-emergency.owner").is_file() - for family in ("inet", "bridge"): - assert (tmp_path / "tables" / - f"{family}--virt_cloud_fabric_emergency").is_file() + assert not (runtime / "wg-fabric-emergency.owner").exists() assert (tmp_path / "link-up").exists() == ( gate_state == "valid" and entry == "--ifupdown-configure") rpf = (roots["/proc/sys/net/ipv4"] / "conf" / @@ -620,7 +532,7 @@ def test_ifupdown_configure_without_transition_flags_cannot_activate(tmp_path): result = _run(helper, tmp_path, "--ifupdown-configure") assert result.returncode != 0, result.stderr assert not (tmp_path / "link-up").exists() - assert (runtime / "wg-fabric-emergency.owner").is_file() + assert not (runtime / "wg-fabric-emergency.owner").exists() @pytest.mark.parametrize("mode", ["ifupdown", "networkmanager_nmcli"]) @@ -638,7 +550,7 @@ def test_transaction_publishes_generator_before_network_activation(mode): keys = {host.host_id: base64.b64encode(bytes([index]) * 32).decode() for index, host in enumerate(plan.nodes, start=1)} script = _apply_script(plan, plan.nodes[0], keys, deployment_mode=mode, - firewall_mode=FIREWALL_MODE_FIREWALLD) + firewall_mode=FIREWALL_MODE_NATIVE) # Inspect the real transaction after the embedded helper here-document. transaction = script.index("COMMITTED=no\nROLLBACK_FAILED=no") snapshot = script.index('snapshot_file "$GATE_GENERATOR" gate_generator', @@ -680,8 +592,7 @@ def test_normal_complete_entry_rejects_config_outside_rendered_plan(tmp_path, fi key = base64.b64encode(bytes([2]) * 32).decode() _write(base / "wg-fabric0.conf", ( f"schema|1\nfabric|{fabric}\nrevision|1\ndigest|{plan.digest}\n" - "firewall|firewalld_nft\nunderlay|local|203.0.113.1|eth0|public\n" - "firewall_rule|51820|panel\n" + "firewall|native_nftables\n" f"network|{network}\noverlay|{overlay}\nlisten|51820\n" f"peer|b|{key}|203.0.113.2:51820|{peer_ip}/32\nroute|{peer_ip}/32\n" )) @@ -689,11 +600,11 @@ def test_normal_complete_entry_rejects_config_outside_rendered_plan(tmp_path, fi assert result.returncode == 72, result.stderr calls = [json.loads(line) for line in (tmp_path / "calls").read_text(encoding="utf-8").splitlines()] - assert ["nft", "-j", "list", "table", "inet", "firewalld"] in calls + assert not any(call[-3:] == ["table", "inet", "firewalld"] for call in calls) # Both otherwise syntactically valid configurations fail at their exact # identity/network assertion, before any route or link activation. assert not any(call[0] == "ip" and "route" in call for call in calls) - assert (runtime / "wg-fabric-emergency.owner").is_file() + assert not (runtime / "wg-fabric-emergency.owner").exists() assert not any(call[0] == "systemctl" and call[1] in ("start", "enable") for call in calls) @@ -717,7 +628,7 @@ def test_first_deploy_complete_entry_rejects_unproved_artifacts( assert journal.is_dir() assert (tmp_path / "tables" / "inet--virt_cloud_wg_fabric").read_bytes() == before - assert (runtime / "wg-fabric-emergency.owner").is_file() + assert not (runtime / "wg-fabric-emergency.owner").exists() @pytest.mark.parametrize("gate_state", ["absent", "insecure", "wrong_digest", "valid"]) @@ -740,4 +651,4 @@ def test_normal_complete_entry_still_validates_generator(tmp_path, gate_state): sysctl_dir_checked = ["stat", "-c", "%u:%a", str(roots["/etc/sysctl.d"])] in calls assert sysctl_dir_checked == (gate_state == "valid") - assert (runtime / "wg-fabric-emergency.owner").is_file() + assert not (runtime / "wg-fabric-emergency.owner").exists() diff --git a/engine/tests/test_fabric_gate_retry.py b/engine/tests/test_fabric_gate_retry.py index 5e771313..3db35c40 100644 --- a/engine/tests/test_fabric_gate_retry.py +++ b/engine/tests/test_fabric_gate_retry.py @@ -159,9 +159,8 @@ find_tool() {{ esac }} vc_primary_inactive_retry_ready() {{ return 1; }} -vc_fabric_emergency_reconcile_needed() {{ return 1; }} +vc_legacy_firewalld_state_detected() {{ return 1; }} vc_fabric_rpf_repair_needed() {{ return 1; }} -vc_firewalld_runtime_repair_needed() {{ return 1; }} {branch} ''' diff --git a/engine/tests/test_fabric_nft_transaction.py b/engine/tests/test_fabric_nft_transaction.py index 839c9d16..de0015d3 100644 --- a/engine/tests/test_fabric_nft_transaction.py +++ b/engine/tests/test_fabric_nft_transaction.py @@ -19,9 +19,7 @@ from virsh_panel.fabric import ( @pytest.fixture(params=[ ("ifupdown", "native_nftables"), - ("ifupdown", "firewalld_nft"), ("networkmanager_nmcli", "native_nftables"), - ("networkmanager_nmcli", "firewalld_nft"), ]) def rendered_transaction(request): plan = parse_fabric_plan({ diff --git a/engine/tests/test_fabric_nm_profile_directory.py b/engine/tests/test_fabric_nm_profile_directory.py index fca322da..48dc7ac6 100644 --- a/engine/tests/test_fabric_nm_profile_directory.py +++ b/engine/tests/test_fabric_nm_profile_directory.py @@ -13,7 +13,7 @@ import pytest from virsh_panel.fabric import _apply_script, parse_fabric_plan -@pytest.fixture(params=["native_nftables", "firewalld_nft"]) +@pytest.fixture(params=["native_nftables"]) def rendered_nm_publication(request): plan = parse_fabric_plan({ "fabric_id": "nm-dir-test", "region": "test", diff --git a/engine/tests/test_fabric_nm_runtime_shadow.py b/engine/tests/test_fabric_nm_runtime_shadow.py deleted file mode 100644 index 9d0373e2..00000000 --- a/engine/tests/test_fabric_nm_runtime_shadow.py +++ /dev/null @@ -1,304 +0,0 @@ -"""Strict ownership for NetworkManager's normal in-memory keyfile shadow.""" -from __future__ import annotations - -import base64 -import json -import re -import shlex -import subprocess -import sys -from pathlib import Path - -import pytest - -from virsh_panel.fabric import ( - _BOOT_HELPER, - _FIREWALL_MODE_GUARD_SH, - _NMCLI_FILENAME_RESOLVER_SH, - _apply_script, - _fabric_connection_uuid, - _nm_profile_template, - _observe_script, - parse_fabric_plan, -) - - -KEY = base64.b64encode(bytes([7]) * 32).decode() -UUID = "aaaaaaaa-bbbb-cccc-dddd-eeeeeeeeeeee" -DIGEST = "a" * 64 -NAME = "virt-cloud-wg-fabric0" - - -def _write(path: Path, text: str, mode=0o600): - path.parent.mkdir(parents=True, exist_ok=True) - path.write_text(text, encoding="utf-8") - path.chmod(mode) - - -def _persistent(): - return f'''# virt-cloud-wg-fabric-v1 -# fabric-plan-digest={DIGEST} -[connection] -id={NAME} -uuid={UUID} -type=wireguard -interface-name=wg-fabric0 -autoconnect=false -autoconnect-retries=0 - -[wireguard] -private-key={KEY} -listen-port=51820 - -[ipv4] -method=manual -address1=100.64.1.1/32 -never-default=true -route-metric=50 -route1=100.64.1.2/32 - -[ipv6] -method=disabled -''' - - -def _runtime(persistent: Path, zone="trusted"): - # Real NM serialization loses comments, changes section/key order and adds - # only zone, an empty proxy group and these exact two shadow metadata keys. - return f'''[wireguard] -listen-port=51820 -private-key={KEY} - -[connection] -autoconnect-retries=0 -interface-name=wg-fabric0 -uuid={UUID} -autoconnect=false -type=wireguard -id={NAME} -zone={zone} - -[ipv6] -method=disabled - -[ipv4] -route1=100.64.1.2/32 -route-metric=50 -never-default=true -address1=100.64.1.1/32 -method=manual - -[proxy] - -[.nmmeta] -shadowed-storage={persistent} -shadowed-owned=true -''' - - -_TOOL = r'''import hashlib, json, os, stat, sys -from pathlib import Path -name = Path(sys.argv[0]).name -args = sys.argv[1:] -root = Path(os.environ["VC_SHADOW_ROOT"]) -if name == "stat": - path = Path(args[-1]) - value = os.stat(path) - override = json.loads((root / "owners").read_text()) - uid, gid = override.get(str(path), [0, 0]) - mode = format(stat.S_IMODE(value.st_mode), "o") - values = {"%u": str(uid), "%a": mode, "%u:%a": f"{uid}:{mode}", - "%u:%g:%a": f"{uid}:{gid}:{mode}", "%s": str(value.st_size)} - print(values[args[1]]) -elif name == "sha256sum": - print(hashlib.sha256(Path(args[0]).read_bytes()).hexdigest(), args[0]) -elif name == "nmcli": - if args[:5] == ["-t", "--escape", "no", "-f", "NAME,UUID"]: - print((root / "names").read_text(), end="") - elif args[:5] == ["-t", "--escape", "no", "-f", "UUID,FILENAME"]: - print((root / "filenames").read_text(), end="") - else: - raise SystemExit(2) -else: - raise SystemExit(2) -''' - - -def _fixture(tmp_path): - roots = { - "/etc/NetworkManager": tmp_path / "etc" / "NetworkManager", - "/run": tmp_path / "run", - } - persistent = roots["/etc/NetworkManager"] / "system-connections" / (NAME + ".nmconnection") - runtime = roots["/run"] / "NetworkManager" / "system-connections" / (NAME + ".nmconnection") - _write(persistent, _persistent()) - _write(runtime, _runtime(persistent)) - for path in (roots["/run"], runtime.parent.parent, runtime.parent): - path.chmod(0o755) - _write(tmp_path / "names", f"{NAME}:{UUID}\n") - _write(tmp_path / "filenames", f"{UUID}:{runtime}\n") - _write(tmp_path / "owners", "{}") - tools = tmp_path / "tools" - tools.mkdir() - actual = tools / "tool" - _write(actual, f"#!{sys.executable}\n" + _TOOL, 0o700) - for name in ("stat", "sha256sum", "nmcli"): - (tools / name).symlink_to(actual) - start = _FIREWALL_MODE_GUARD_SH.index("vc_fabric_nm_profile_secure_hash() {") - end = _FIREWALL_MODE_GUARD_SH.index("\nvc_fabric_nm_active()", start) - body = _NMCLI_FILENAME_RESOLVER_SH + "\n" + _FIREWALL_MODE_GUARD_SH[start:end] - for source, target in roots.items(): - body = body.replace(source, str(target)) - prefix = ( - f"STAT={shlex.quote(str(tools / 'stat'))}\n" - f"NMCLI={shlex.quote(str(tools / 'nmcli'))}\n" - f"NM_PROFILE={shlex.quote(str(persistent))}\n" - f"CONNECTION={shlex.quote(NAME)}\nCONNECTION_UUID={shlex.quote(UUID)}\n" - f"EXPECTED_DIGEST={shlex.quote(DIGEST)}\nFIREWALL_MODE=firewalld_nft\n" - "find_tool() { printf '%s' " + shlex.quote(str(tools / "sha256sum")) + "; }\n" - ) - script = prefix + body + "\nvc_fabric_nm_profile_owned expected\n" - return persistent, runtime, script - - -def _run(tmp_path, script): - return subprocess.run(["/bin/sh", "-eu", "-c", script], capture_output=True, - text=True, check=False, - env={"VC_SHADOW_ROOT": str(tmp_path)}, timeout=15) - - -@pytest.mark.parametrize("location", ["persistent", "runtime"]) -@pytest.mark.parametrize("zone", ["trusted", "", "absent"]) -def test_same_owned_profile_accepts_only_exact_normal_shadow(tmp_path, location, zone): - persistent, runtime, script = _fixture(tmp_path) - text = _runtime(persistent, zone if zone != "absent" else "") - if zone == "absent": - text = text.replace("zone=\n", "") - _write(runtime, text) - if location == "persistent": - _write(tmp_path / "filenames", f"{UUID}:{persistent}\n") - result = _run(tmp_path, script) - assert result.returncode == 0, result.stderr - assert result.stdout == result.stderr == "" - - -@pytest.mark.parametrize("drift", [ - "key", "port", "address", "route", "route_metric", "ipv4_method", "ipv6_method", - "autoconnect", "uuid", "type", "zone", "persistent_zone", "metadata_path", - "metadata_owner", "metadata_extra", "unknown_key", "unknown_section", "duplicate_key", - "duplicate_section", "extra_route", "proxy_key", "metadata_missing", -]) -def test_shadow_semantic_drift_or_unknown_shape_fails_without_secret_output(tmp_path, drift): - persistent, runtime, script = _fixture(tmp_path) - text = runtime.read_text(encoding="utf-8") - substitutions = { - "key": (KEY, base64.b64encode(bytes([8]) * 32).decode()), - "port": ("listen-port=51820", "listen-port=51821"), - "address": ("address1=100.64.1.1/32", "address1=100.64.1.3/32"), - "route": ("route1=100.64.1.2/32", "route1=100.64.1.3/32"), - "route_metric": ("route-metric=50", "route-metric=51"), - "ipv4_method": ("method=manual", "method=auto"), - "ipv6_method": ("method=disabled", "method=auto"), - "autoconnect": ("autoconnect=false", "autoconnect=true"), - "uuid": (UUID, "f" * 36), - "type": ("type=wireguard", "type=bridge"), - "zone": ("zone=trusted", "zone=public"), - "metadata_path": (f"shadowed-storage={persistent}", "shadowed-storage=/etc/external"), - "metadata_owner": ("shadowed-owned=true", "shadowed-owned=false"), - "metadata_extra": ("shadowed-owned=true", "shadowed-owned=true\nexternal=true"), - "unknown_key": ("listen-port=51820", "listen-port=51820\nfwmark=1"), - "unknown_section": ("[proxy]", "[unmanaged]"), - "duplicate_key": ("listen-port=51820", "listen-port=51820\nlisten-port=51820"), - "duplicate_section": ("[proxy]", "[ipv4]\nmethod=manual\n[proxy]"), - "extra_route": ("route1=100.64.1.2/32", "route1=100.64.1.2/32\nroute2=10.1.0.1/32"), - "proxy_key": ("[proxy]", "[proxy]\nmethod=auto"), - "metadata_missing": ("shadowed-owned=true\n", ""), - } - if drift == "persistent_zone": - _write(persistent, persistent.read_text().replace("[connection]", "[connection]\nzone=trusted")) - else: - old, new = substitutions[drift] - _write(runtime, text.replace(old, new)) - result = _run(tmp_path, script) - assert result.returncode != 0 - assert result.stdout == result.stderr == "" - - -@pytest.mark.parametrize("drift", [ - "filename", "basename", "runtime_symlink", "parent_symlink", "parent_writable", - "runtime_wide", "runtime_owner", "persistent_owner", "duplicate_uuid", "duplicate_name", - "runtime_empty", "runtime_large", -]) -def test_runtime_path_and_ownership_are_not_widened(tmp_path, drift): - persistent, runtime, script = _fixture(tmp_path) - if drift in ("filename", "basename"): - path = tmp_path / "external" if drift == "filename" else runtime.with_name("external.nmconnection") - _write(path, runtime.read_text()) - _write(tmp_path / "filenames", f"{UUID}:{path}\n") - elif drift == "runtime_symlink": - external = tmp_path / "external" - _write(external, runtime.read_text()) - runtime.unlink() - runtime.symlink_to(external) - elif drift == "parent_symlink": - external = runtime.parent.with_name("external") - runtime.parent.rename(external) - runtime.parent.symlink_to(external) - elif drift == "parent_writable": - runtime.parent.chmod(0o775) - elif drift == "runtime_wide": - runtime.chmod(0o644) - elif drift in ("runtime_owner", "persistent_owner"): - path = runtime if drift == "runtime_owner" else persistent - _write(tmp_path / "owners", json.dumps({str(path): [1000, 0]})) - elif drift == "duplicate_uuid": - _write(tmp_path / "names", f"{NAME}:{UUID}\nother:{UUID}\n") - elif drift == "duplicate_name": - _write(tmp_path / "names", f"{NAME}:{UUID}\n{NAME}:other\n") - elif drift == "runtime_empty": - _write(runtime, "") - else: - _write(runtime, runtime.read_text() + "#" * 65536) - result = _run(tmp_path, script) - assert result.returncode != 0 - assert result.stdout == result.stderr == "" - - -def test_shared_shadow_contract_is_rendered_in_apply_observe_and_boot(): - plan = parse_fabric_plan({ - "fabric_id": "shadow-fabric", "region": "hk", "overlay_cidr": "100.64.1.0/24", - "revision": 1, "nodes": [ - {"host_id": "a", "overlay_ip": "100.64.1.1", "endpoint": "203.0.113.1:51820"}, - {"host_id": "b", "overlay_ip": "100.64.1.2", "endpoint": "203.0.113.2:51820"}, - ], - }) - keys = {node.host_id: KEY for node in plan.nodes} - apply = _apply_script(plan, plan.nodes[0], keys, firewall_mode="firewalld_nft") - observe = _observe_script(("100.64.1.2",), "100.64.1.1", firewall_mode="firewalld_nft") - for script in (apply, observe): - assert "vc_fabric_nm_profile_semantics_match() {" in script - assert "shadowed-storage" in script - subprocess.run(["/bin/sh", "-n"], input=script, text=True, check=True) - assert "vc_fabric_nm_profile_semantics_match" in _BOOT_HELPER or "__VC_FIREWALL_MODE_GUARD__" in _BOOT_HELPER - - -def test_first_deploy_recovery_dispatch_requires_exact_published_old_helper(): - plan = parse_fabric_plan({ - "fabric_id": "shadow-fabric", "region": "hk", "overlay_cidr": "100.64.1.0/24", - "revision": 1, "nodes": [ - {"host_id": "a", "overlay_ip": "100.64.1.1", "endpoint": "203.0.113.1:51820"}, - {"host_id": "b", "overlay_ip": "100.64.1.2", "endpoint": "203.0.113.2:51820"}, - ], - }) - script = _apply_script(plan, plan.nodes[0], {node.host_id: KEY for node in plan.nodes}, - firewall_mode="firewalld_nft") - start = script.index("transaction_first_deploy_helper_absent=no") - end = script.index("transaction_missing_installing_snapshot=no", start) - guard = script[start:end] - assert 'transaction_recovery_helper_owned || exit 79' in guard - assert 'fabric-plan-digest=$EXPECTED_DIGEST' in guard - assert 'PLAN_DIGEST=\\"$EXPECTED_DIGEST\\"' in guard - assert guard.index('helper.present') < guard.index('transaction_recovery_helper_owned') - current = script.index('/bin/sh "$_tmp_helper" --recover-current', end) - ordinary = script.index('/bin/sh "$HELPER" --recover-only', current) - assert current < ordinary diff --git a/engine/tests/test_fabric_nm_runtime_shadow_recovery.py b/engine/tests/test_fabric_nm_runtime_shadow_recovery.py deleted file mode 100644 index 53429b09..00000000 --- a/engine/tests/test_fabric_nm_runtime_shadow_recovery.py +++ /dev/null @@ -1,121 +0,0 @@ -"""Replay a failed first-deploy post-image through the current recovery helper.""" -from __future__ import annotations - -import base64 -import json -import re -import shlex -import subprocess -from pathlib import Path - -import pytest - -from test_fabric_first_deploy_entry import _fixture, _journal, _write -from virsh_panel.fabric import ( - FIREWALL_MODE_FIREWALLD, - _FABRIC_NFT_VALIDATION_SH, - _FABRIC_SYSCTL_VALIDATION_SH, - _FIREWALL_MODE_GUARD_SH, - _NMCLI_FILENAME_RESOLVER_SH, - _TRANSACTION_SCRIPT, - _nm_profile_template, -) - - -def _failed_first_deploy(tmp_path): - plan, current, base, runtime, roots = _fixture(tmp_path, "networkmanager_nmcli") - journal = _journal(plan, base, "networkmanager_nmcli", current, tmp_path, - nft_phase="refreshed") - content = current.read_text(encoding="utf-8") - legacy = "\n".join(content.splitlines()[:4]) + ( - "\n# --recover-only\nprintf 'LEGACY_HELPER_EXECUTED\\n' >&2\nexit 99\n") - installed = roots["/usr/local/libexec"] / "virt-cloud-wg-fabric" - _write(installed, legacy, 0o700) - _write(journal / "helper.desired", legacy, 0o700) - _write(base / "wg-fabric0.owner", "virt-cloud-wg-fabric-v1\n") - _write(journal / "owner.desired", "virt-cloud-wg-fabric-v1\n") - uuid = re.search(r'^CONNECTION_UUID="([^"]+)"$', content, re.MULTILINE)[1] - persistent = (roots["/etc/NetworkManager"] / "system-connections" / - "virt-cloud-wg-fabric0.nmconnection") - profile = _nm_profile_template(plan, plan.nodes[0], uuid, FIREWALL_MODE_FIREWALLD).replace( - "private-key=\n", "private-key=" + base64.b64encode(bytes([7]) * 32).decode() + "\n") - _write(persistent, profile) - _write(journal / "nm_profile.desired", profile) - shadow = roots["/run/NetworkManager"] / "system-connections" / persistent.name - shadow_text = "\n".join(line for line in profile.splitlines() if not line.startswith("#")) - _write(shadow, shadow_text.replace("[connection]", "[connection]\nzone=trusted") + ( - f"\n[proxy]\n[.nmmeta]\nshadowed-storage={persistent}\nshadowed-owned=true\n")) - for directory in (tmp_path / "run", shadow.parent.parent, shadow.parent): - directory.chmod(0o755) - _write(tmp_path / "nm-loaded-uuid", uuid + "\n") - _write(tmp_path / "nm-zone", "trusted\n") - for name in ("link-present", "link-up"): - _write(tmp_path / name, "") - _write(tmp_path / "runtime-zone", "trusted\n") - _write(roots["/proc/sys/net/ipv4"] / "conf" / "wg-fabric0" / "rp_filter", "2\n") - return plan, current, installed, base, runtime, roots, journal, shadow - - -def _driver(plan, current, installed, base, runtime, roots, tmp_path): - boundary = "# A fixed-path sysctl file may be reused only when the authoritative owner\n" - recovery = _TRANSACTION_SCRIPT.split(boundary, 1)[0] - body = (_NMCLI_FILENAME_RESOLVER_SH + "\n" + _FABRIC_NFT_VALIDATION_SH + "\n" - + _FABRIC_SYSCTL_VALIDATION_SH + "\n" + _FIREWALL_MODE_GUARD_SH + "\n" + recovery) - for source, destination in roots.items(): - body = body.replace(source, str(destination)) - body = body.replace("in /run ", "in " + shlex.quote(str(tmp_path / "run")) + " ") - tools = tmp_path / "tools" - values = {"BASE": base, "HELPER": installed, "_tmp_helper": current, - "OWNER": base / "wg-fabric0.owner", "CONF": base / "wg-fabric0.conf", - "NM_PROFILE": roots["/etc/NetworkManager"] / "system-connections" / - "virt-cloud-wg-fabric0.nmconnection", - "EMERGENCY_OWNER": runtime / "wg-fabric-emergency.owner", - "EXPECTED_DIGEST": plan.digest, "DEPLOYMENT_MODE": "networkmanager_nmcli", - "FIREWALL_MODE": "firewalld_nft", "IFACE": "wg-fabric0", - "CONNECTION": "virt-cloud-wg-fabric0"} - uuid = re.search(r'^CONNECTION_UUID="([^"]+)"$', current.read_text(), re.MULTILINE)[1] - values["CONNECTION_UUID"] = uuid - for variable, name in (("STAT", "stat"), ("NFT", "nft"), ("IP", "ip"), - ("WG", "wg"), ("SYSTEMCTL", "systemctl"), - ("NMCLI", "nmcli"), ("FIREWALL_CMD", "firewall-cmd")): - values[variable] = tools / name - values["CMP"] = "/usr/bin/cmp" - prefix = "\n".join(key + "=" + shlex.quote(str(value)) for key, value in values.items()) - prefix += ("\nfind_tool() { printf '%s' " + shlex.quote(str(tools / "sha256sum")) + "; }\n" - "vc_fabric_operation_checkpoint() { :; }\n" - "PATH=" + shlex.quote(str(tools)) + ":/usr/bin:/bin; export PATH\n") - return "set -eu\n" + prefix + body - - -@pytest.mark.parametrize("failure", [None, "foreign_helper", "wrong_digest", "runtime_key"]) -def test_current_first_deploy_recovery_replays_only_exact_same_plan_post_image(tmp_path, failure): - plan, current, installed, base, runtime, roots, journal, shadow = _failed_first_deploy(tmp_path) - if failure == "foreign_helper": - _write(installed, installed.read_text() + "# external replacement\n", 0o700) - elif failure == "wrong_digest": - changed = installed.read_text().replace(plan.digest, "b" * 64) - _write(installed, changed, 0o700) - _write(journal / "helper.desired", changed, 0o700) - elif failure == "runtime_key": - value = base64.b64encode(bytes([7]) * 32).decode() - _write(shadow, shadow.read_text().replace(value, base64.b64encode(bytes([8]) * 32).decode())) - result = subprocess.run(["/bin/sh", "-c", _driver( - plan, current, installed, base, runtime, roots, tmp_path)], - capture_output=True, text=True, check=False, - env={"VC_ENTRY_ROOT": str(tmp_path), "VC_FABRIC_LOCK_HELD": "1"}, timeout=45) - assert result.returncode == (0 if failure is None else 79), result.stderr - assert "LEGACY_HELPER_EXECUTED" not in result.stdout + result.stderr - assert (runtime / "wg-fabric-emergency.owner").is_file() - for family in ("inet", "bridge"): - assert (tmp_path / "tables" / f"{family}--virt_cloud_fabric_emergency").is_file() - if failure is None: - assert not journal.exists() - assert not installed.exists() - assert not shadow.exists() - assert not (tmp_path / "link-present").exists() - assert not (tmp_path / "tables" / "inet--virt_cloud_wg_fabric").exists() - calls = [json.loads(line) for line in (tmp_path / "calls").read_text().splitlines()] - assert not any(call[0] == "systemctl" and call[1] in ("start", "enable") for call in calls) - else: - assert journal.is_dir() - assert installed.is_file() diff --git a/engine/tests/test_fabric_ready_current_recovery.py b/engine/tests/test_fabric_ready_current_recovery.py index 89ede63e..43e8d56d 100644 --- a/engine/tests/test_fabric_ready_current_recovery.py +++ b/engine/tests/test_fabric_ready_current_recovery.py @@ -1,272 +1,187 @@ -"""Recover a same-plan ready journal without redispatching its old helper.""" +"""Native journal recovery and retired firewalld fail-closed contracts.""" from __future__ import annotations import base64 import json +from pathlib import Path import re -import shlex +import stat import subprocess +import sys import pytest -from test_fabric_first_deploy_entry import _write -from test_fabric_nm_runtime_shadow_recovery import _failed_first_deploy +from test_fabric_first_deploy_entry import _fixture, _journal, _run, _write from virsh_panel.fabric import ( - FIREWALL_MODE_FIREWALLD, FIREWALL_MODE_NATIVE, - _FABRIC_NFT_VALIDATION_SH, - _FABRIC_SYSCTL_VALIDATION_SH, _FIREWALL_MODE_GUARD_SH, - _NMCLI_FILENAME_RESOLVER_SH, _apply_script, ) -def _baseline_fixture(tmp_path): - values = _failed_first_deploy(tmp_path) - _, _, installed, _, _, _, journal, _ = values - # The real Alma host now has a journaled old helper baseline, not the - # helper.absent case covered by the first-deployment recovery path. - (journal / "helper.absent").unlink() - _write(journal / "helper.present", "") - _write(journal / "helper", installed.read_text(encoding="utf-8"), 0o700) - return values - - -def _rendered_driver(values, root, firewall_mode=FIREWALL_MODE_FIREWALLD): - plan, current, installed, base, runtime, roots, _, _ = values - keys = {host.host_id: base64.b64encode(bytes([index]) * 32).decode() - for index, host in enumerate(plan.nodes, start=1)} - rendered = _apply_script( - plan, plan.nodes[0], keys, deployment_mode="networkmanager_nmcli", - firewall_mode=firewall_mode, - ) - start = rendered.index("COMMITTED=no\nROLLBACK_FAILED=no") - end = rendered.index("vc_fabric_operation_checkpoint preflight\n", start) - # Execute the actual rendered outer recovery functions and dispatch, then - # the complete current helper. Only host resources and CLI paths redirect - # to fixture doubles; no ownership/recovery shell function is replaced. - recovery = rendered[start:end] - body = (_NMCLI_FILENAME_RESOLVER_SH + "\n" + _FABRIC_NFT_VALIDATION_SH + "\n" - + _FABRIC_SYSCTL_VALIDATION_SH + "\n" + _FIREWALL_MODE_GUARD_SH + "\n" - + recovery) - for source, destination in roots.items(): - body = body.replace(source, str(destination)) - body = body.replace("in /run ", "in " + shlex.quote(str(root / "run")) + " ") - tools = root / "tools" - identity = re.search(r'^CONNECTION_UUID="([^"]+)"$', current.read_text(), re.MULTILINE)[1] - variables = { - "BASE": base, "HELPER": installed, "_tmp_helper": current, - "OWNER": base / "wg-fabric0.owner", "CONF": base / "wg-fabric0.conf", - "NM_PROFILE": roots["/etc/NetworkManager"] / "system-connections" / - "virt-cloud-wg-fabric0.nmconnection", - "EMERGENCY_OWNER": runtime / "wg-fabric-emergency.owner", - "EXPECTED_DIGEST": plan.digest, "DEPLOYMENT_MODE": "networkmanager_nmcli", - "FIREWALL_MODE": firewall_mode, "IFACE": "wg-fabric0", - "CONNECTION": "virt-cloud-wg-fabric0", "CONNECTION_UUID": identity, - "CMP": "/usr/bin/cmp", +def _journal_image(journal: Path) -> dict[str, tuple[bytes, int]]: + return { + str(path.relative_to(journal)): ( + path.read_bytes(), stat.S_IMODE(path.stat().st_mode), + ) + for path in journal.rglob("*") + if path.is_file() and not path.is_symlink() } - for variable, name in (("STAT", "stat"), ("NFT", "nft"), ("IP", "ip"), - ("WG", "wg"), ("SYSTEMCTL", "systemctl"), - ("NMCLI", "nmcli"), ("FIREWALL_CMD", "firewall-cmd")): - variables[variable] = tools / name - prefix = "\n".join(key + "=" + shlex.quote(str(value)) for key, value in variables.items()) - prefix += ("\nfind_tool() { printf '%s' " + shlex.quote(str(tools / "sha256sum")) + "; }\n" - "vc_fabric_operation_checkpoint() { :; }\n" - "PATH=" + shlex.quote(str(tools)) + ":/usr/bin:/bin; export PATH\n") - return "set -eu\n" + prefix + body -def _run(values, root, firewall_mode=FIREWALL_MODE_FIREWALLD): - return subprocess.run( - ["/bin/sh", "-c", _rendered_driver(values, root, firewall_mode)], check=False, - capture_output=True, text=True, - env={"VC_ENTRY_ROOT": str(root), "VC_FABRIC_LOCK_HELD": "1"}, timeout=45, +def _assert_no_recovery_mutation(root: Path) -> None: + calls_path = root / "calls" + calls = ( + [json.loads(line) for line in calls_path.read_text().splitlines()] + if calls_path.is_file() else [] + ) + assert not any(call[0] == "nft" and "-f" in call for call in calls) + assert not any( + call[:3] == ["ip", "link", "set"] or + call[:3] == ["ip", "link", "del"] + for call in calls + ) + assert not any( + call[0] == "nmcli" and call[1:3] in ( + ["connection", "delete"], ["connection", "modify"], + ) + for call in calls + ) + assert not any( + call[0] == "systemctl" and call[1] in ("start", "enable") + for call in calls ) -def _native_baseline_fixture(tmp_path, *, empty_header=False): - values = _baseline_fixture(tmp_path) - _, current, _, _, _, roots, journal, shadow = values - # Rendered helper implementation is identical for both owners; the - # explicit literal backend selects the native branch. No shell function - # or recovery conditional is replaced in this fixture. - content = current.read_text().replace( - 'FIREWALL_MODE="firewalld_nft"', 'FIREWALL_MODE="native_nftables"', 1, +@pytest.mark.parametrize("state", ("ready", "committed")) +def test_legacy_firewalld_journal_is_rejected_and_preserved(tmp_path, state): + plan, helper, base, _, _ = _fixture(tmp_path, "networkmanager_nmcli") + journal = _journal( + plan, base, "networkmanager_nmcli", helper, tmp_path, + nft_phase="refreshed", ) - native_main = tmp_path / "nftables.conf" - roots["/etc/nftables.conf"] = native_main - content = content.replace("/etc/nftables.conf", str(native_main)) - if empty_header: - content = content.replace('UNDERLAY_ENDPOINT="203.0.113.1"', 'UNDERLAY_ENDPOINT=""') - content = content.replace('UNDERLAY_PORT="51820"', 'UNDERLAY_PORT=""') - _write(current, content, 0o700) - metadata = dict(line.split("=", 1) for line in (journal / "meta").read_text().splitlines()) - metadata.update({ - "firewall_mode": FIREWALL_MODE_NATIVE, "nft_main": str(native_main), - "firewalld_runtime_zone_before": "na", "firewalld_permanent_zone_before": "na", - "underlay_endpoint": "", "underlay_port": "", "underlay_zone": "", - "underlay_rule_owner": "none", "underlay_rule_runtime_before": "na", - "underlay_rule_permanent_before": "na", - }) - _write(journal / "meta", "".join(f"{key}={value}\n" for key, value in metadata.items())) - distribution_rules = '# distribution-owned main file\ninclude "/etc/nftables.d/*.nft"\n' - _write(native_main, distribution_rules, 0o644) - (journal / "nft_main.absent").unlink() - _write(journal / "nft_main.present", "") - _write(journal / "nft_main", distribution_rules, 0o644) - _write(shadow, shadow.read_text().replace("zone=trusted", "zone=")) - _write(tmp_path / "nm-zone", "") - (tmp_path / "runtime-zone").unlink() - # Native owner fixture: nftables enabled/running, firewalld masked and - # inactive. Only external systemctl evidence differs from firewalld. - tool = tmp_path / "tools" / "external-tool" - external = tool.read_text().replace( - 'print("active" if args[1] == "firewalld" else "inactive")', - 'print("inactive" if args[1] == "firewalld" else "active")', - ).replace( - 'print("enabled" if args[1] == "firewalld" else "disabled")', - 'print("masked" if args[1] == "firewalld" else "enabled")', + _write(journal / "state", state + "\n") + _write( + journal / "meta", + (journal / "meta").read_text(encoding="utf-8").replace( + "firewall_mode=native_nftables", + "firewall_mode=firewalld_nft", + ), ) - _write(tool, external, 0o700) - return values + before = _journal_image(journal) + result = _run(helper, tmp_path, "--recover-current") -@pytest.mark.parametrize("empty_header", [False, True]) -@pytest.mark.parametrize("failure", [None, "endpoint", "port"]) -def test_native_same_plan_handoff_requires_empty_underlay_metadata_not_unused_header( - tmp_path, empty_header, failure): - values = _native_baseline_fixture(tmp_path, empty_header=empty_header) - _, _, installed, _, runtime, _, journal, shadow = values - historical = installed.read_bytes() - if failure: - _write(journal / "meta", (journal / "meta").read_text().replace( - f"underlay_{failure}=\n", - f"underlay_{failure}=" + ("203.0.113.1" if failure == "endpoint" else "51820") + "\n", - 1, - )) - live = tmp_path / "tables" / "inet--virt_cloud_wg_fabric" - original = live.read_bytes() - result = _run(values, tmp_path, FIREWALL_MODE_NATIVE) - assert result.returncode == (0 if failure is None else 79), result.stderr - assert "LEGACY_HELPER_EXECUTED" not in result.stdout + result.stderr - assert installed.read_bytes() == historical - assert (runtime / "wg-fabric-emergency.owner").is_file() - if failure is None: - assert not journal.exists() - assert not live.exists() - assert not shadow.exists() - assert not (tmp_path / "link-present").exists() - else: - assert journal.is_dir() - assert live.read_bytes() == original - assert shadow.is_file() + assert result.returncode == 79, result.stderr + assert journal.is_dir() + assert _journal_image(journal) == before + _assert_no_recovery_mutation(tmp_path) + + +@pytest.mark.parametrize("state", ("preparing", "committed")) +def test_malformed_native_journal_is_rejected_and_preserved(tmp_path, state): + plan, helper, base, _, _ = _fixture(tmp_path, "ifupdown") + journal = _journal(plan, base, "ifupdown", helper, tmp_path) + _write(journal / "state", state + "\n") + _write( + journal / "meta", + (journal / "meta").read_text(encoding="utf-8") + + "firewall_mode=native_nftables\n", + ) + before = _journal_image(journal) + result = _run(helper, tmp_path, "--recover-current") -def test_rendered_outer_replays_same_plan_old_baseline_with_current_helper(tmp_path): - values = _baseline_fixture(tmp_path) - _, current, installed, base, runtime, _, journal, shadow = values - historical = installed.read_bytes() - assert current.read_bytes() != historical - assert "vc_fabric_nm_profile_semantics_match" not in historical.decode() - result = _run(values, tmp_path) - assert result.returncode == 0, result.stderr - assert "LEGACY_HELPER_EXECUTED" not in result.stdout + result.stderr - assert not journal.exists() - # Complete recovery preserves its pre-existing, precisely journaled old - # helper but must not execute it again under --recover-current. - assert installed.read_bytes() == historical - assert not shadow.exists() - assert not (tmp_path / "link-present").exists() - assert not (base / "wg-fabric0.owner").exists() - assert not (tmp_path / "tables" / "inet--virt_cloud_wg_fabric").exists() - assert (runtime / "wg-fabric-emergency.owner").is_file() - for family in ("inet", "bridge"): - assert (tmp_path / "tables" / f"{family}--virt_cloud_fabric_emergency").is_file() - calls = [json.loads(line) for line in (tmp_path / "calls").read_text().splitlines()] - assert not any(call[0] == "systemctl" and call[1] in ("start", "enable") for call in calls) - assert not any(call[:5] == ["ip", "link", "set", "dev", "wg-fabric0"] - and call[-1] == "up" for call in calls) + assert result.returncode == 79, result.stderr + assert journal.is_dir() + assert _journal_image(journal) == before + _assert_no_recovery_mutation(tmp_path) -def test_normal_boot_recovery_still_reenters_verified_historical_helper(tmp_path): - values = _baseline_fixture(tmp_path) - _, current, installed, _, runtime, _, journal, _ = values - historical = installed.read_bytes() - result = subprocess.run( - ["/bin/sh", str(current)], check=False, capture_output=True, text=True, - env={"VC_ENTRY_ROOT": str(tmp_path), "VC_FABRIC_LOCK_HELD": "1"}, timeout=45, +@pytest.mark.parametrize("state", ("ready", "committed")) +def test_needs_repair_reports_every_native_transaction_journal(tmp_path, state): + plan, helper, base, _, _ = _fixture(tmp_path, "ifupdown") + journal = _journal( + plan, base, "ifupdown", helper, tmp_path, nft_phase="refreshed", ) - # This complete non-recovery-only entry still uses the restored old - # implementation. Its deliberate exit99 proves the normal exec contract - # was preserved instead of silently treating a regular boot as finished. - assert result.returncode == 99, result.stderr - assert "LEGACY_HELPER_EXECUTED" in result.stderr - assert not journal.exists() - assert installed.read_bytes() == historical - assert (runtime / "wg-fabric-emergency.owner").is_file() + _write(journal / "state", state + "\n") + before = _journal_image(journal) + result = _run(helper, tmp_path, "--needs-repair") -@pytest.mark.parametrize("failure", [ - "foreign_helper", "baseline_and_desired_drift", "temporary_mode", "temporary_digest", - "endpoint", "port", "runtime_key", "live_nft_verdict", -]) -def test_rendered_same_plan_handoff_rejects_unproved_ownership_or_plan(tmp_path, failure): - values = _baseline_fixture(tmp_path) - plan, current, installed, _, runtime, _, journal, shadow = values - if failure == "foreign_helper": - _write(installed, installed.read_text() + "# external replacement\n", 0o700) - elif failure == "baseline_and_desired_drift": - for name in ("helper", "helper.desired"): - _write(journal / name, (journal / name).read_text() + "# changed snapshot\n", 0o700) - elif failure == "temporary_mode": - current.chmod(0o600) - elif failure == "temporary_digest": - _write(current, current.read_text().replace(plan.digest, "a" * 64), 0o700) - elif failure in ("endpoint", "port"): - old, new = (("underlay_endpoint=203.0.113.1", "underlay_endpoint=203.0.113.9") - if failure == "endpoint" else ("underlay_port=51820", "underlay_port=51821")) - _write(journal / "meta", (journal / "meta").read_text().replace(old, new)) - elif failure == "runtime_key": - old = base64.b64encode(bytes([7]) * 32).decode() - _write(shadow, shadow.read_text().replace(old, base64.b64encode(bytes([8]) * 32).decode())) - else: - live = tmp_path / "tables" / "inet--virt_cloud_wg_fabric" - _write(live, live.read_text().replace("counter drop comment", "counter accept comment", 1)) - live = tmp_path / "tables" / "inet--virt_cloud_wg_fabric" - original = live.read_bytes() - result = _run(values, tmp_path) - assert result.returncode == 79, result.stderr - assert "LEGACY_HELPER_EXECUTED" not in result.stdout + result.stderr + assert result.returncode == 0, result.stderr assert journal.is_dir() - assert installed.is_file() - assert live.read_bytes() == original - assert (runtime / "wg-fabric-emergency.owner").is_file() - calls = [json.loads(line) for line in (tmp_path / "calls").read_text().splitlines()] - assert not any(call[0] == "nft" and call[1] == "-f" for call in calls) + assert _journal_image(journal) == before + _assert_no_recovery_mutation(tmp_path) + + +def _stat_tool(tmp_path: Path) -> Path: + tool = tmp_path / "stat" + tool.write_text( + f"#!{sys.executable}\n" + "import os, stat, sys\n" + "value = os.lstat(sys.argv[3])\n" + "mode = format(stat.S_IMODE(value.st_mode), 'o')\n" + "fields = {\n" + " '%u:%g:%a': '0:0:' + mode,\n" + " '%u:%a': '0:' + mode,\n" + " '%s': str(value.st_size),\n" + "}\n" + "print(fields[sys.argv[2]])\n", + encoding="utf-8", + ) + tool.chmod(0o700) + return tool -@pytest.mark.parametrize("field,value", [ - ("plan_digest", "f" * 64), - ("connection_uuid", "ffffffff-ffff-ffff-ffff-ffffffffffff"), - ("deployment_mode", "ifupdown"), - ("firewall_mode", "native_nftables"), -]) -def test_different_plan_keeps_historical_recovery_contract_without_current_handoff( - tmp_path, field, value): - values = _baseline_fixture(tmp_path) - _, _, _, _, runtime, _, journal, _ = values - text = (journal / "meta").read_text() - _write(journal / "meta", re.sub(rf"^{field}=.*$", f"{field}={value}", text, - flags=re.MULTILINE)) - before = (tmp_path / "tables" / "inet--virt_cloud_wg_fabric").read_bytes() - result = _run(values, tmp_path) - assert result.returncode == 79, result.stderr - # This controlled historical helper rejects at exit99. Existing cross- - # revision dispatch stays intact, and the current template never assumes - # authority over the other journal identity. - assert "LEGACY_HELPER_EXECUTED" in result.stderr - assert "helper_state=failed" not in result.stderr - assert journal.is_dir() - assert (tmp_path / "tables" / "inet--virt_cloud_wg_fabric").read_bytes() == before - assert (runtime / "wg-fabric-emergency.owner").is_file() +def test_rendered_apply_initializes_awk_before_running_native_guard(tmp_path): + plan, _, _, _, _ = _fixture(tmp_path, "networkmanager_nmcli") + keys = { + node.host_id: base64.b64encode(bytes([index]) * 32).decode() + for index, node in enumerate(plan.nodes, start=1) + } + rendered = _apply_script( + plan, plan.nodes[0], keys, + deployment_mode="networkmanager_nmcli", + firewall_mode=FIREWALL_MODE_NATIVE, + ) + assignment = re.search( + r"^AWK=\$\(find_tool [^\n]+\) \|\| exit 70$", + rendered, + re.MULTILINE, + ) + assert assignment is not None + guard_call = rendered.index("vc_legacy_firewalld_state_absent || exit 79") + assert assignment.start() < guard_call + + etc = tmp_path / "etc" / "virt-cloud" + state_dir = etc / "vpc-dataplane" + runtime = tmp_path / "run" / "virt-cloud" + state_dir.mkdir(parents=True) + runtime.mkdir(parents=True, exist_ok=True) + config = etc / "wg-fabric0.conf" + config.write_text("firewall|native_nftables\n", encoding="utf-8") + config.chmod(0o600) + guard = _FIREWALL_MODE_GUARD_SH.replace( + "/etc/virt-cloud", str(etc), + ).replace( + "/run/virt-cloud", str(runtime), + ) + result = subprocess.run( + ["/bin/sh", "-eu", "-c", "\n".join(( + "find_tool() { for p in \"$@\"; do [ -x \"$p\" ] && " + "{ printf '%s' \"$p\"; return 0; }; done; return 1; }", + assignment.group(0), + f"STAT={_stat_tool(tmp_path)}", + "vc_test_nft() {", + " [ \"$*\" = \"list tables\" ] || return 2", + " printf '%s\\n' 'table inet filter'", + "}", + "NFT=vc_test_nft", + guard, + "vc_legacy_firewalld_state_absent", + ))], + capture_output=True, text=True, check=False, + ) + + assert result.returncode == 0, result.stderr diff --git a/engine/tests/test_fabric_route_readback.py b/engine/tests/test_fabric_route_readback.py index e3e0f092..0c5f3092 100644 --- a/engine/tests/test_fabric_route_readback.py +++ b/engine/tests/test_fabric_route_readback.py @@ -8,7 +8,7 @@ import subprocess import pytest from virsh_panel.fabric import ( - FIREWALL_MODE_FIREWALLD, + FIREWALL_MODE_NATIVE, _BOOT_HELPER, _FABRIC_NFT_VALIDATION_SH, _apply_script, @@ -36,7 +36,7 @@ def _render_apply(mode="ifupdown"): keys = {node.host_id: base64.b64encode(bytes([index]) * 32).decode() for index, node in enumerate(plan.nodes, 1)} return _apply_script(plan, plan.nodes[0], keys, deployment_mode=mode, - firewall_mode=FIREWALL_MODE_FIREWALLD) + firewall_mode=FIREWALL_MODE_NATIVE) def _assignment(script, name, start=0): @@ -209,7 +209,7 @@ def _observe_route_program(): script = _observe_script(("100.64.1.2",), "100.64.1.1", endpoint_ips=("203.0.113.2",), deployment_mode="ifupdown", - firewall_mode=FIREWALL_MODE_FIREWALLD) + firewall_mode=FIREWALL_MODE_NATIVE) start = script.index('awk -v vpc_file="${VPC_ROUTE_FILE:-}"') program = script[start:script.index("\n }'", start) + len("\n }'")] return program.replace('"${VPC_ROUTE_FILE:-}"', '"$VPC_ROUTE_FILE"') diff --git a/engine/tests/test_remote_host.py b/engine/tests/test_remote_host.py index d6d96350..92fabe05 100644 --- a/engine/tests/test_remote_host.py +++ b/engine/tests/test_remote_host.py @@ -26,8 +26,6 @@ from virsh_panel.remote_host import ( HostProfile, RemoteHostError, RemoteHostManager, - _parse_firewall_interface_zones, - _parse_networkmanager_interface_zones, parse_host_capabilities, ) from virsh_panel.ssh_agent import SSHAgentError @@ -59,7 +57,6 @@ __VCAP_TOOL__wg=/usr/bin/wg __VCAP_TOOL__wg_quick=/usr/bin/wg-quick __VCAP_TOOL__ip=/usr/sbin/ip __VCAP_TOOL__nft=/usr/sbin/nft -__VCAP_TOOL__firewall_cmd= __VCAP_TOOL__nmcli=/usr/bin/nmcli __VCAP_TOOL__networkctl= __VCAP_TOOL__ifup= @@ -77,6 +74,13 @@ __VCAP_MODULE_FILE_BEGIN__ __VCAP_MODULE_FILE_END__ __VCAP_FABRIC_OWNER_BEGIN__ __VCAP_FABRIC_OWNER_END__ +__VCAP_LEGACY_FIREWALL__manifest=absent +__VCAP_LEGACY_FIREWALL__fabric_transaction=absent +__VCAP_LEGACY_FIREWALL__emergency_owner=absent +__VCAP_LEGACY_FIREWALL__vpc_transaction=absent +__VCAP_LEGACY_FIREWALL__vpc_snapshot=absent +__VCAP_LEGACY_FIREWALL__fabric_desired_state=absent +__VCAP_LEGACY_FIREWALL__vpc_state=absent __VCAP_SYSCTL__ip_forward=1 __VCAP_SYSCTL__rp_filter=2 __VCAP_SYSCTL__rp_filter_default=2 @@ -93,24 +97,6 @@ __VCAP_SERVICE_ENABLED__firewall=masked __VCAP_SERVICE_ENABLED__nftables=enabled __VCAP_SERVICE_LOAD__firewall=masked __VCAP_SERVICE_LOAD__nftables=loaded -__VCAP_FIREWALL_CONF__backend=nftables -__VCAP_FIREWALL_CONF__table_owner=yes -__VCAP_FIREWALL_CONF__source_zones=absent -__VCAP_FIREWALL_CONF__default_runtime=public -__VCAP_FIREWALL_CONF__default_permanent=public -__VCAP_FIREWALL_CONF__runtime_nft=ready -__VCAP_FIREWALL_IFACE_BEGIN__runtime -br0|public -eth1|public -__VCAP_FIREWALL_IFACE_END__runtime=0 -__VCAP_FIREWALL_IFACE_BEGIN__permanent -__VCAP_FIREWALL_IFACE_END__permanent=0 -__VCAP_NM_ZONE_BEGIN__ -br0| -eth1| -__VCAP_NM_ZONE_END__=0 -__VCAP_IFCFG_ZONE_BEGIN__ -__VCAP_IFCFG_ZONE_END__=0 __VCAP_FILE__interfaces=absent __VCAP_FILE__interfaces_d=absent __VCAP_FILE__networkmanager_profiles=absent @@ -128,7 +114,7 @@ __VCAP_IP_BEGIN__wireguard [] __VCAP_IP_END__wireguard=0 __VCAP_NFT_BEGIN__ -[{{"family":"inet","name":"filter"}}] +{{"nftables":[{{"table":{{"family":"inet","name":"filter"}}}}]}} __VCAP_NFT_END__0 __VCAP_END__ """ @@ -142,27 +128,27 @@ def test_capability_probe_command_is_parseable_by_posix_sh(): assert result.returncode == 0, result.stderr assert "probe_ip_json addresses -j -4 address show up scope global" in ( _CAPABILITY_PROBE_COMMAND) - assert "-g GENERAL.FILENAME" not in _CAPABILITY_PROBE_COMMAND - assert ( - "$firewall_nmcli -t --escape no -f UUID,FILENAME connection show" - in _CAPABILITY_PROBE_COMMAND - ) - assert 'test "${#firewall_nm_profiles}" -le 262144' in ( - _CAPABILITY_PROBE_COMMAND) assert "systemctl show -p LoadState --value" in _CAPABILITY_PROBE_COMMAND - assert "FirewallBackend" in _CAPABILITY_PROBE_COMMAND - assert "NftablesTableOwner" in _CAPABILITY_PROBE_COMMAND - assert "__VCAP_FIREWALL_CONF__source_zones=" in _CAPABILITY_PROBE_COMMAND - assert "--permanent --get-default-zone" not in _CAPABILITY_PROBE_COMMAND - # firewall-cmd is a D-Bus client and can activate an inactive unit. Every - # use must therefore be dominated by the read-only systemd active check. - assert _CAPABILITY_PROBE_COMMAND.count( - 'test "$firewall_service_state" = active') == 4 - assert _CAPABILITY_PROBE_COMMAND.count( - '$firewall_cmd --state') == 4 - assert _CAPABILITY_PROBE_COMMAND.index( - 'firewall_service_state="$status"') < _CAPABILITY_PROBE_COMMAND.index( - '$firewall_cmd --state') + assert "probe_service firewall firewalld" in _CAPABILITY_PROBE_COMMAND + assert "probe_service_enabled firewall firewalld" in _CAPABILITY_PROBE_COMMAND + assert "probe_service_load firewall firewalld" in _CAPABILITY_PROBE_COMMAND + assert "firewall-cmd" not in _CAPABILITY_PROBE_COMMAND + assert "FirewallBackend" not in _CAPABILITY_PROBE_COMMAND + assert "NftablesTableOwner" not in _CAPABILITY_PROBE_COMMAND + assert '"$capability_nft_path" -j list tables' in _CAPABILITY_PROBE_COMMAND + assert "probe_legacy_firewall_path manifest /etc/virt-cloud/nftables.conf" in ( + _CAPABILITY_PROBE_COMMAND + ) + assert "probe_legacy_firewall_path fabric_transaction " in ( + _CAPABILITY_PROBE_COMMAND + ) + assert "probe_legacy_firewall_path emergency_owner " in ( + _CAPABILITY_PROBE_COMMAND + ) + for marker in ( + "vpc_transaction", "vpc_snapshot", "fabric_desired_state", "vpc_state", + ): + assert f"__VCAP_LEGACY_FIREWALL__{marker}=" in _CAPABILITY_PROBE_COMMAND def test_capability_probe_executes_the_discovered_ip_binary(): @@ -227,105 +213,6 @@ def test_capability_probe_executes_discovered_nft_and_modinfo_paths(tmp_path): "-F filename wireguard\n") -def test_capability_probe_resolves_nm_filename_from_bounded_list_mode(tmp_path): - connection_uuid = "00000000-0000-0000-0000-000000000001" - profile = tmp_path / "br0:primary.nmconnection" - profile.write_text("[connection]\nzone=public\n", encoding="utf-8") - profile.chmod(0o600) - calls = tmp_path / "nmcli.calls" - - nmcli = tmp_path / "nmcli" - nmcli.write_text( - "#!/bin/sh\n" - f'printf "%s\\n" "$*" >> "{calls}"\n' - 'case "$*" in\n' - ' "-t --escape no -f UUID,DEVICE connection show --active")\n' - f' printf "%s\\n" "{connection_uuid}:br0" ' - '"22222222-2222-4222-8222-222222222222:vnet3"; exit 0 ;;\n' - ' "-t --escape no -f UUID,FILENAME connection show")\n' - f' printf "%s\\n" "{connection_uuid}:{profile}" ' - '"22222222-2222-4222-8222-222222222222:' - '/run/NetworkManager/system-connections/vnet3.nmconnection"; exit 0 ;;\n' - ' "-g GENERAL.FILENAME connection show uuid ' - f'{connection_uuid}") exit 2 ;;\n' - "esac\n" - "exit 64\n", - encoding="utf-8", - ) - nmcli.chmod(0o700) - - systemctl = tmp_path / "systemctl" - systemctl.write_text( - "#!/bin/sh\n" - 'if test "$*" = "is-active NetworkManager"; then\n' - ' printf "%s\\n" active\n' - " exit 0\n" - "fi\n" - "exit 3\n", - encoding="utf-8", - ) - systemctl.chmod(0o700) - - stat_command = tmp_path / "stat" - stat_command.write_text( - "#!/bin/sh\n" - 'if test "$1" = -c && test "$2" = %u:%a; then\n' - ' printf "%s\\n" 0:600\n' - " exit 0\n" - "fi\n" - 'if test "$1" = -c && test "$2" = %s; then\n' - ' printf "%s\\n" 25\n' - " exit 0\n" - "fi\n" - "exit 64\n", - encoding="utf-8", - ) - stat_command.chmod(0o700) - - detail = subprocess.run( - [ - str(nmcli), "-g", "GENERAL.FILENAME", "connection", "show", - "uuid", connection_uuid, - ], - capture_output=True, - text=True, - check=False, - ) - assert detail.returncode == 2 - - command = _CAPABILITY_PROBE_COMMAND.replace( - "for firewall_candidate in /usr/bin/nmcli /usr/sbin/nmcli " - "/bin/nmcli /sbin/nmcli; do", - f"for firewall_candidate in {nmcli}; do", - ).replace( - "for firewall_candidate in /usr/bin/stat /bin/stat /usr/sbin/stat " - "/sbin/stat; do", - f"for firewall_candidate in {stat_command}; do", - ).replace( - "for firewall_candidate in /usr/bin/systemctl /bin/systemctl " - "/usr/sbin/systemctl /sbin/systemctl; do", - f"for firewall_candidate in {systemctl}; do", - ).replace( - "/etc/NetworkManager/system-connections/", - f"{tmp_path}/", - ) - result = subprocess.run( - shlex.split(command), - capture_output=True, - text=True, - check=False, - timeout=15, - ) - - assert result.returncode == 0, result.stderr - assert ( - "__VCAP_NM_ZONE_BEGIN__\nbr0|public\nvnet3|?\n" - "__VCAP_NM_ZONE_END__=0" - ) in result.stdout - assert calls.read_text(encoding="utf-8").splitlines()[-2:] == [ - "-t --escape no -f UUID,DEVICE connection show --active", - "-t --escape no -f UUID,FILENAME connection show", - ] def test_capability_probe_command_executes_the_complete_wrapped_script(): @@ -347,33 +234,8 @@ def test_capability_probe_command_executes_the_complete_wrapped_script(): assert result.stdout.rstrip().endswith("__VCAP_END__") -def test_zone_inventory_parsers_allow_bounded_guest_density(): - firewalld = "\n".join( - ["br0|public"] - + [f"vnet{index}|public" for index in range(300)]) - networkmanager = "\n".join( - ["br0|public"] - + [f"vnet{index}|" for index in range(300)]) - firewall_ok, firewall_values = _parse_firewall_interface_zones( - firewalld, "0") - nm_ok, nm_values = _parse_networkmanager_interface_zones( - networkmanager, "0") - assert firewall_ok is True - assert firewall_values["br0"] == "public" - assert len(firewall_values) == 301 - assert nm_ok is True - assert nm_values["br0"] == "public" - assert len(nm_values) == 301 - - -def test_zone_inventory_parsers_reject_more_than_4096_interfaces(): - oversized = "\n".join( - f"vnet{index}|public" for index in range(4097)) - assert _parse_firewall_interface_zones(oversized, "0") == (False, {}) - assert _parse_networkmanager_interface_zones( - oversized, "0") == (False, {}) class Runner: @@ -793,14 +655,14 @@ def test_stage_test_commit_writes_private_pinned_ssh_config(tmp_path): "firewalld_active": False, "firewalld_enabled": False, "firewalld_load_state": "masked", - "firewalld_backend": "nftables", - "firewalld_table_owner": "yes", - "firewalld_source_zones": "absent", - "firewalld_default_runtime": "public", - "firewalld_default_permanent": "public", - "firewalld_runtime_nft": "ready", - "firewalld_interface_inventory_ready": True, - "firewalld_command_available": False, + "legacy_firewalld_table_present": False, + "legacy_manifest_state": "absent", + "fabric_transaction_state": "absent", + "emergency_owner_state": "absent", + "vpc_transaction_state": "absent", + "vpc_snapshot_state": "absent", + "fabric_desired_state": "absent", + "vpc_state": "absent", "nftables_active": True, "nftables_enabled": True, "nftables_load_state": "loaded", @@ -974,118 +836,47 @@ def test_capability_probe_rejects_firewalld_enabled_for_next_boot(): assert capabilities["nftables"]["ready"] is False assert capabilities["vpc_ready"] is False assert "firewall_manager_conflict" in capabilities["reasons"] + assert "firewalld_unsupported_requires_maintenance" in capabilities["reasons"] assert "firewall_owner_not_nftables" in capabilities["reasons"] def firewalld_capability_output( - *, nft_active="inactive", nft_enabled="masked", nft_load="masked", - backend="nftables", table_owner="yes", source_zones="absent", - firewalld_enabled="enabled", default_runtime="public", - default_permanent="public", runtime_nft="ready", + *, firewalld_active="active", firewalld_enabled="enabled", + firewalld_load="loaded", nft_active="inactive", + nft_enabled="masked", nft_load="masked", live_table=False, ): replacements = ( - ("__VCAP_TOOL__firewall_cmd=", - "__VCAP_TOOL__firewall_cmd=/usr/bin/firewall-cmd"), ("__VCAP_SERVICE__firewall=inactive", - "__VCAP_SERVICE__firewall=active"), + f"__VCAP_SERVICE__firewall={firewalld_active}"), ("__VCAP_SERVICE__nftables=active", f"__VCAP_SERVICE__nftables={nft_active}"), ("__VCAP_SERVICE_ENABLED__firewall=masked", f"__VCAP_SERVICE_ENABLED__firewall={firewalld_enabled}"), ("__VCAP_SERVICE_LOAD__firewall=masked", - "__VCAP_SERVICE_LOAD__firewall=loaded"), + f"__VCAP_SERVICE_LOAD__firewall={firewalld_load}"), ("__VCAP_SERVICE_ENABLED__nftables=enabled", f"__VCAP_SERVICE_ENABLED__nftables={nft_enabled}"), ("__VCAP_SERVICE_LOAD__nftables=loaded", f"__VCAP_SERVICE_LOAD__nftables={nft_load}"), - ("__VCAP_FIREWALL_CONF__backend=nftables", - f"__VCAP_FIREWALL_CONF__backend={backend}"), - ("__VCAP_FIREWALL_CONF__table_owner=yes", - f"__VCAP_FIREWALL_CONF__table_owner={table_owner}"), - ("__VCAP_FIREWALL_CONF__source_zones=absent", - f"__VCAP_FIREWALL_CONF__source_zones={source_zones}"), - ("__VCAP_FIREWALL_CONF__default_runtime=public", - f"__VCAP_FIREWALL_CONF__default_runtime={default_runtime}"), - ("__VCAP_FIREWALL_CONF__default_permanent=public", - f"__VCAP_FIREWALL_CONF__default_permanent={default_permanent}"), - ("__VCAP_FIREWALL_CONF__runtime_nft=ready", - f"__VCAP_FIREWALL_CONF__runtime_nft={runtime_nft}"), ) output = CAPABILITY_OUTPUT for old, new in replacements: output = output.replace(old, new) + if live_table: + output = output.replace( + '{"nftables":[{"table":{"family":"inet","name":"filter"}}]}', + '{"nftables":[{"table":{"family":"inet","name":"filter"}},' + '{"table":{"family":"inet","name":"firewalld"}}]}', + ) return output -@pytest.mark.parametrize(("nft_active", "nft_enabled", "nft_load"), [ - ("inactive", "disabled", "loaded"), - ("inactive", "masked", "masked"), - ("unknown", "unknown", "not-found"), -]) -def test_capability_probe_accepts_supported_firewalld_nft_owner( - nft_active, nft_enabled, nft_load, -): - capabilities = parse_host_capabilities(firewalld_capability_output( - nft_active=nft_active, nft_enabled=nft_enabled, nft_load=nft_load, - ), public_network="bridge:br0") - - assert capabilities["network"]["firewall_manager"] == "firewalld" - assert capabilities["network"]["firewall_mode"] == "firewalld_nft" - assert capabilities["nftables"]["firewall_mode"] == "firewalld_nft" - # Raw nft remains required for the panel-owned tables even though - # firewalld owns the host's primary firewall lifecycle. - assert capabilities["nftables"]["ready"] is True - assert capabilities["vpc_ready"] is True - assert capabilities["reasons"] == [] - - -@pytest.mark.parametrize(("backend", "table_owner", "reason"), [ - ("iptables", "yes", "firewalld_backend_not_nftables"), - ("unknown", "yes", "firewalld_backend_not_nftables"), - ("nftables", "no", "firewalld_table_owner_not_enabled"), - ("nftables", "unknown", "firewalld_table_owner_not_enabled"), -]) -def test_capability_probe_rejects_unsupported_firewalld_configuration( - backend, table_owner, reason, -): - capabilities = parse_host_capabilities(firewalld_capability_output( - backend=backend, table_owner=table_owner, - ), public_network="bridge:br0") - - assert capabilities["network"]["firewall_mode"] == "unsupported" - assert capabilities["nftables"]["ready"] is False - assert capabilities["vpc_ready"] is False - assert reason in capabilities["reasons"] -@pytest.mark.parametrize(("default_runtime", "default_permanent"), [ - ("trusted", "trusted"), - ("public", "drop"), - ("unknown", "public"), -]) -def test_capability_probe_rejects_unsafe_or_drifted_firewalld_default_zone( - default_runtime, default_permanent, -): - capabilities = parse_host_capabilities(firewalld_capability_output( - default_runtime=default_runtime, - default_permanent=default_permanent, - ), public_network="bridge:br0") - assert capabilities["network"]["firewall_mode"] == "unsupported" - assert capabilities["nftables"]["ready"] is False - assert capabilities["vpc_ready"] is False - assert "firewalld_default_zone_unsafe_or_drifted" in capabilities["reasons"] -def test_capability_probe_requires_live_firewalld_nft_owner_evidence(): - capabilities = parse_host_capabilities(firewalld_capability_output( - runtime_nft="unknown", - ), public_network="bridge:br0") - assert capabilities["network"]["firewall_mode"] == "unsupported" - assert capabilities["nftables"]["ready"] is False - assert capabilities["vpc_ready"] is False - assert "firewalld_runtime_nft_backend_unverified" in capabilities["reasons"] def test_capability_probe_rejects_runtime_only_firewalld_enablement(): @@ -1096,7 +887,7 @@ def test_capability_probe_rejects_runtime_only_firewalld_enablement(): assert capabilities["network"]["firewall_manager"] == "firewalld" assert capabilities["network"]["firewall_mode"] == "unsupported" assert capabilities["vpc_ready"] is False - assert "firewalld_service_not_enabled" in capabilities["reasons"] + assert "firewalld_unsupported_requires_maintenance" in capabilities["reasons"] def test_capability_probe_rejects_runtime_only_native_nft_enablement(): @@ -1121,17 +912,9 @@ def test_capability_probe_rejects_an_active_competing_nftables_service(): assert capabilities["network"]["firewall_mode"] == "unsupported" assert capabilities["nftables"]["ready"] is False assert "firewall_manager_conflict" in capabilities["reasons"] + assert "firewalld_unsupported_requires_maintenance" in capabilities["reasons"] -@pytest.mark.parametrize("source_zones", ["present", "unknown"]) -def test_capability_probe_rejects_firewalld_source_zone_ambiguity(source_zones): - capabilities = parse_host_capabilities(firewalld_capability_output( - source_zones=source_zones, - ), public_network="bridge:br0") - - assert capabilities["network"]["firewall_mode"] == "unsupported" - assert capabilities["nftables"]["ready"] is False - assert "firewalld_source_zones_present_or_unknown" in capabilities["reasons"] def test_capability_probe_accepts_native_owner_when_firewalld_is_not_found(): @@ -1163,7 +946,89 @@ def test_capability_probe_rejects_inactive_but_unmasked_firewalld_competitor(): native_capabilities = parse_host_capabilities( native, public_network="bridge:br0") assert native_capabilities["network"]["firewall_mode"] == "unsupported" - assert "firewall_competing_service_unknown" in native_capabilities["reasons"] + assert "firewalld_unsupported_requires_maintenance" in ( + native_capabilities["reasons"]) + + +@pytest.mark.parametrize(("family", "table"), ( + ("inet", "firewalld"), + ("inet", "virt_cloud_fabric_emergency"), + ("bridge", "virt_cloud_fabric_emergency"), +)) +def test_capability_probe_rejects_live_legacy_firewall_table(family, table): + output = CAPABILITY_OUTPUT.replace( + '{"nftables":[{"table":{"family":"inet","name":"filter"}}]}', + '{"nftables":[{"table":{"family":"inet","name":"filter"}},' + f'{{"table":{{"family":"{family}","name":"{table}"}}}}]}}', + ) + capabilities = parse_host_capabilities( + output, public_network="bridge:br0", + ) + + assert capabilities["network"]["firewall_manager"] == "nftables" + assert capabilities["network"]["firewall_mode"] == "unsupported" + assert capabilities["network"]["firewall_manager_evidence"][ + "legacy_firewalld_table_present"] is True + assert capabilities["nftables"]["ready"] is False + assert capabilities["vpc_ready"] is False + assert "legacy_firewalld_state_requires_cleanup" in capabilities["reasons"] + + +@pytest.mark.parametrize(("marker", "reason"), ( + ("manifest", "legacy_firewalld_state_requires_cleanup"), + ("vpc_snapshot", "legacy_firewalld_state_requires_cleanup"), + ("fabric_desired_state", "legacy_firewalld_state_requires_cleanup"), + ("vpc_state", "legacy_firewalld_state_requires_cleanup"), + ("emergency_owner", "legacy_firewalld_state_requires_cleanup"), + ("fabric_transaction", "firewall_transaction_requires_recovery_or_cleanup"), + ("vpc_transaction", "firewall_transaction_requires_recovery_or_cleanup"), +)) +def test_capability_probe_rejects_legacy_firewall_artifact(marker, reason): + output = CAPABILITY_OUTPUT.replace( + f"__VCAP_LEGACY_FIREWALL__{marker}=absent", + f"__VCAP_LEGACY_FIREWALL__{marker}=present", + ) + capabilities = parse_host_capabilities(output, public_network="bridge:br0") + + assert capabilities["probe_valid"] is True + assert capabilities["vpc_ready"] is False + assert capabilities["network"]["firewall_mode"] == "unsupported" + evidence_key = { + "manifest": "legacy_manifest_state", + "fabric_transaction": "fabric_transaction_state", + "emergency_owner": "emergency_owner_state", + "vpc_transaction": "vpc_transaction_state", + "vpc_snapshot": "vpc_snapshot_state", + }.get(marker, marker) + assert capabilities["network"]["firewall_manager_evidence"][ + evidence_key + ] == "present" + assert reason in capabilities["reasons"] + + +@pytest.mark.parametrize("failure", ("missing", "duplicate", "unknown")) +@pytest.mark.parametrize("marker", ( + "manifest", "fabric_transaction", "emergency_owner", "vpc_transaction", + "vpc_snapshot", "fabric_desired_state", "vpc_state", +)) +def test_capability_probe_fails_closed_on_invalid_legacy_firewall_evidence( + marker, failure, +): + line = f"__VCAP_LEGACY_FIREWALL__{marker}=absent" + if failure == "missing": + replacement = "" + elif failure == "duplicate": + replacement = line + "\n" + line + else: + replacement = f"__VCAP_LEGACY_FIREWALL__{marker}=unknown" + capabilities = parse_host_capabilities( + CAPABILITY_OUTPUT.replace(line, replacement), + public_network="bridge:br0", + ) + + assert capabilities["probe_valid"] is False + assert capabilities["vpc_ready"] is False + assert "legacy_firewall_evidence_invalid" in capabilities["reasons"] diff --git a/engine/tests/test_vpc_dataplane.py b/engine/tests/test_vpc_dataplane.py index b72d1337..1fec6e0f 100644 --- a/engine/tests/test_vpc_dataplane.py +++ b/engine/tests/test_vpc_dataplane.py @@ -320,33 +320,28 @@ def test_rendered_data_plane_has_exact_routes_proxy_arp_and_atomic_nft_batch(): @pytest.mark.parametrize("mode", ("networkmanager_nmcli", "ifupdown")) def test_rp_filter_is_set_before_first_up_and_reasserted_after_manager(mode): script = render_apply_script( - _plan(), deployment_mode=mode, firewall_mode="firewalld_nft") + _plan(), deployment_mode=mode, firewall_mode="native_nftables") guard = script.index( "# Install and exactly read back the raw tenant guard") calls = [match.start() for match in re.finditer( r"^[ ]*vc_vpc_rp_filter_set \|\| exit 72$", script, re.MULTILINE)] - assert len(calls) == 4 + assert len(calls) == 3 first_up = script.index('"$IP" link set dev "$BRIDGE" up', guard) assert guard < calls[0] < first_up if mode == "networkmanager_nmcli": activation = script.index( '"$NMCLI" connection up uuid "$NM_PROFILE_UUID"', first_up) - trusted = script.index("connection.zone trusted", activation) else: activation = script.index( 'VC_FABRIC_APPLYING=1 "$IFUP" --force "$BRIDGE"', first_up) - trusted = script.index( - '"$FIREWALL_CMD" --zone=trusted --add-interface="$BRIDGE"', - activation, - ) reassert = script.index("vc_vpc_rp_filter_set || exit 72", activation) - assert first_up < activation < reassert < trusted + assert first_up < activation < reassert def test_rp_filter_failure_and_rollback_remain_fail_closed(): script = render_apply_script( - _plan(), firewall_mode="firewalld_nft") + _plan(), firewall_mode="native_nftables") helper = _shell_block( script, "vc_vpc_rp_filter_set() {", "vc_vpc_native_ifupdown_rollback_reopen() {") assert helper.index("RP_FILTER_CHANGED=yes") < helper.index( @@ -365,9 +360,7 @@ def test_rp_filter_failure_and_rollback_remain_fail_closed(): 'cat "$TXN/rp_filter.before" > ' '"/proc/sys/net/ipv4/conf/$BRIDGE/rp_filter"') restore_nft = rollback.index('NFT_ROLLBACK="$TXN/rollback.nft"') - reopen_trusted = rollback.index( - 'connection.zone trusted', restore_nft) - assert quarantine < restore_raw < restore_nft < reopen_trusted + assert quarantine < restore_raw < restore_nft def test_sysctl_artifact_contract_binds_fixed_shape_and_identity(tmp_path): diff --git a/engine/tests/test_vpc_firewalld_dataplane.py b/engine/tests/test_vpc_nft_dataplane.py similarity index 55% rename from engine/tests/test_vpc_firewalld_dataplane.py rename to engine/tests/test_vpc_nft_dataplane.py index 45c93517..19ef3676 100644 --- a/engine/tests/test_vpc_firewalld_dataplane.py +++ b/engine/tests/test_vpc_nft_dataplane.py @@ -1,8 +1,12 @@ -"""Firewalld/nft ownership contracts for the VPC host transaction.""" +"""Native nftables ownership contracts for the VPC host transaction.""" from __future__ import annotations import os +from pathlib import Path +import shlex +import stat import subprocess +import sys from dataclasses import replace import pytest @@ -10,13 +14,10 @@ import pytest from virsh_panel.vpc import VPC_SCHEMA_VERSION from virsh_panel.fabric import _NMCLI_FILENAME_RESOLVER_SH, _nft_rules_text from virsh_panel.vpc_dataplane import ( - FIREWALL_MODE_FIREWALLD, FIREWALL_MODE_NATIVE, - FIREWALL_ZONE_OWNERSHIP_SH, - FIREWALL_ZONE_VALIDATION_SH, NETWORKMANAGER_VPC_PROFILE_SH, NFT_LIVE_SNAPSHOT_SH, - NFT_MANIFEST_VALIDATION_SH, + VPC_NATIVE_TRANSACTION_GUARD_SH, VpcDataplaneError, VpcPrivateHostAdapter, parse_vpc_dataplane_plan, @@ -45,13 +46,13 @@ def _vpc_state_loader_functions(script: str) -> str: def _plan(): return parse_vpc_dataplane_plan({ "schema_version": VPC_SCHEMA_VERSION, - "vpc_id": "vpc-firewalld", + "vpc_id": "vpc-native-nft", "region": "hk", "fabric_id": "fabric-hk", "fabric_revision": 1, "cidr": "10.44.0.0/24", "desired_revision": 1, - "operation_id": "vpc-firewalld-op-0001", + "operation_id": "vpc-native-nft-op-0001", "subnets": [{"subnet_id": "subnet-a", "cidr": "10.44.0.0/25"}], "hosts": [ {"host_id": "host-a", "members": [{ @@ -87,51 +88,14 @@ def _empty_local_plan(): ) -@pytest.mark.parametrize("firewall_mode", [ - FIREWALL_MODE_NATIVE, - FIREWALL_MODE_FIREWALLD, -]) -def test_apply_refuses_a_host_without_local_members(firewall_mode): +def test_apply_refuses_a_host_without_local_members(): with pytest.raises(VpcDataplaneError) as rejected: render_apply_script( - _empty_local_plan(), firewall_mode=firewall_mode) + _empty_local_plan(), firewall_mode=FIREWALL_MODE_NATIVE) assert rejected.value.status == 409 assert rejected.value.error_code == "vpc_apply_no_local_members" -@pytest.mark.parametrize("deployment_mode", ["networkmanager_nmcli", "ifupdown"]) -def test_firewalld_scripts_are_posix_and_never_reload_global_rules(deployment_mode): - plan = _plan() - scripts = ( - render_apply_script( - plan, deployment_mode=deployment_mode, - firewall_mode=FIREWALL_MODE_FIREWALLD), - render_observe_script(plan, firewall_mode=FIREWALL_MODE_FIREWALLD), - render_remove_script( - _empty_local_plan(), deployment_mode=deployment_mode, - firewall_mode=FIREWALL_MODE_FIREWALLD), - ) - for script in scripts: - result = subprocess.run( - ["/bin/sh", "-n"], input=script, text=True, - capture_output=True, check=False, - ) - assert result.returncode == 0, result.stderr - assert "FIREWALL_MODE=firewalld_nft" in script - assert "/etc/virt-cloud/nftables.conf" in script - assert "# virt-cloud-nft-manifest-v1" in script - assert "--reload" not in script - assert "--runtime-to-permanent" not in script - assert '--permanent --zone=trusted --add-interface=' not in script - assert '--permanent --zone=trusted --remove-interface=' not in script - - for script in (scripts[0], scripts[2]): - assert "vc_firewall_mode_ready || exit 71" in script - assert "systemctl" in script - assert "FirewallBackend" in script - assert "NftablesTableOwner" in script - assert "vc_firewall_mode_ready || firewall_mode_state=mismatch" in scripts[1] - assert "__VC_VPC_FIREWALL_STATE__=$firewall_mode_state" in scripts[1] @pytest.mark.parametrize("transaction", ["apply", "remove"]) @@ -141,11 +105,11 @@ def test_vpc_transactions_strictly_bind_fabric_digest_and_tenant_set( script = ( render_apply_script( plan, deployment_mode="networkmanager_nmcli", - firewall_mode=FIREWALL_MODE_FIREWALLD) + firewall_mode=FIREWALL_MODE_NATIVE) if transaction == "apply" else render_remove_script( _empty_local_plan(), deployment_mode="networkmanager_nmcli", - firewall_mode=FIREWALL_MODE_FIREWALLD) + firewall_mode=FIREWALL_MODE_NATIVE) ) guard = _fabric_guard_functions(script) loader = _vpc_state_loader_functions(script) @@ -270,19 +234,16 @@ NFT=vc_test_nft "vc_nft_table_matches_snapshot inet virt_cloud_wg_fabric", last_exact) assert last_strict < last_exact < last_snapshot < commit_at - if transaction == "apply": - restore_trusted = script.rindex("connection.zone trusted") - assert last_snapshot < restore_trusted < commit_at + assert "connection.zone trusted" not in script + assert "firewall-cmd" not in script def test_vpc_observe_strictly_binds_fabric_digest_semantics_and_tenant_set( tmp_path): script = render_observe_script( - _plan(), firewall_mode=FIREWALL_MODE_FIREWALLD) - snapshot_start = script.index("vc_nft_table_absent() {") - snapshot_end = script.index( - "vc_firewalld_interface_zone() {", snapshot_start) - snapshot_functions = script[snapshot_start:snapshot_end] + _plan(), firewall_mode=FIREWALL_MODE_NATIVE) + assert NFT_LIVE_SNAPSHOT_SH in script + snapshot_functions = NFT_LIVE_SNAPSHOT_SH guard_start = script.index("vc_fabric_nft_strict_shape() {") loader_start = script.index( 'VPC_STATE_DIR="/etc/virt-cloud/vpc-dataplane"', guard_start) @@ -398,137 +359,24 @@ NFT=vc_test_nft in observe_function) -def test_firewalld_apply_owns_only_vpc_bridge_zone_and_manifest_include(): - plan = _plan() - script = render_apply_script( - plan, deployment_mode="ifupdown", - firewall_mode=FIREWALL_MODE_FIREWALLD) - include = f'include "/etc/nftables.d/virt-cloud-vpc-{plan.resources.token}.nft"' - assert include in script - assert 'NFT_MAIN="$NFT_MANIFEST"' in script - assert "virt-cloud-wg-fabric[.]nft|virt-cloud-vpc-" in script - assert 'vc_firewalld_bridge_zone_expect active managed "$BRIDGE"' in script - assert '--zone=trusted --add-interface="$BRIDGE"' in script - assert '--zone=trusted --change-interface="$BRIDGE"' in script - assert '--zone=trusted --remove-interface="$BRIDGE"' in script - assert '--permanent --zone=trusted --add-interface="$BRIDGE"' not in script - assert '--permanent --zone=trusted --remove-interface="$BRIDGE"' not in script - assert "--zone=trusted --add-interface=eth" not in script - assert "--zone=trusted --add-interface=wg-fabric0" not in script - assert f"\nauto {plan.resources.bridge}\n" not in script - assert 'ifcfg-$BRIDGE' in script - assert "firewall_mode|firewalld_nft" in script - assert "ZONE_OWNERSHIP=absent" in script - assert 'ZONE_OWNERSHIP=managed' in script - assert '[ -f "$OWNER_FILE" ] && [ ! -L "$OWNER_FILE" ] &&' in script - assert 'grep -Fqx "$OWNER_MARKER" "$OWNER_FILE" || exit 71' in script - assert '[ ! -e "$IF_FRAGMENT" ] && [ ! -L "$IF_FRAGMENT" ] || exit 71' in script - assert '"$IP" link show dev "$BRIDGE" >/dev/null 2>&1 && exit 71 || true' in script - assert 'vc_firewalld_bridge_zone_expect preapply "$ZONE_OWNERSHIP" "$BRIDGE" || exit 71' in script - assert script.index('vc_firewalld_bridge_zone_expect preapply "$ZONE_OWNERSHIP"') < script.index( - 'mkdir -m 700 "$TXN"') - nft_ready = script.rindex("vc_vpc_desired_nft_ready || exit 72") - assert script.index('"$NFT" -f "$NFT_BATCH"') < nft_ready - assert nft_ready < script.rindex('VC_FABRIC_APPLYING=1 "$IFUP" --force "$BRIDGE"') - assert nft_ready < script.rindex('--zone=trusted --add-interface="$BRIDGE"') - trusted_add = script.rindex('--zone=trusted --add-interface="$BRIDGE"') - assert trusted_add < script.index( - '[ -e "/etc/sysconfig/network-scripts/ifcfg-$BRIDGE" ]', trusted_add) - - -def test_firewalld_networkmanager_profile_is_fail_closed_on_disk_and_temporary_in_memory(): - script = render_apply_script( - _plan(), deployment_mode="networkmanager_nmcli", - firewall_mode=FIREWALL_MODE_FIREWALLD) - assert "connection.autoconnect no connection.autoconnect-retries 0" in script - assert "connection.autoconnect yes" not in script - assert "GENERAL.FILENAME" not in script - assert '-f UUID,FILENAME' in script - assert script.count("vc_nm_filename_for_uuid() {") == 1 - assert 'zone_count != 0' in script - assert 'substr($0, 13) != expected_auto' in script - temporary = ('"$NMCLI" connection modify --temporary uuid ' - '"$NM_PROFILE_UUID" connection.zone trusted') - assert temporary in script - assert 'vc_nm_profile_disk_validate "$NM_PROFILE_FILE" "$NM_PROFILE_UUID" || exit 72' in script - fabric_ready = script.rindex("vc_vpc_fabric_nft_live_ready || exit 74") - tenant_ready = script.index( - "vpc_state_nft_set_matches || exit 74", fabric_ready) - fabric_snapshot = script.index( - "vc_nft_table_matches_snapshot inet virt_cloud_wg_fabric", - tenant_ready) - temporary_index = script.rindex("connection.zone trusted") - blocked_up = script.rindex('"$NMCLI" connection up uuid "$NM_PROFILE_UUID"') - assert fabric_ready < tenant_ready < fabric_snapshot < temporary_index - assert blocked_up < fabric_ready - assert temporary_index < script.rindex('"$NMCLI" device reapply "$BRIDGE"') - assert 'vc_nm_profile_expect active || exit 72' in script -def test_firewalld_remove_deletes_only_exact_owned_zone_and_manifest_entry(): - script = render_remove_script( - _empty_local_plan(), deployment_mode="ifupdown", - firewall_mode=FIREWALL_MODE_FIREWALLD) - assert 'state_firewall_record=$(awk' in script - assert '[ "$state_firewall_record" = "1|$FIREWALL_MODE" ] || exit 71' in script - assert '--zone=trusted --remove-interface="$BRIDGE"' in script - assert '--permanent --zone=trusted --remove-interface="$BRIDGE"' not in script - assert '--permanent --zone=trusted --add-interface="$BRIDGE"' not in script - assert 'vc_firewalld_bridge_zone_expect blocked managed "$BRIDGE"' in script - assert 'ifcfg-$BRIDGE' in script - assert "grep -Eq '^[[:space:]]*(auto|allow-" in script - assert "awk -v needle=\"$NFT_INCLUDE\" '$0 != needle'" in script - assert 'vc_nft_manifest_validate "$NFT_MAIN_TMP"' in script - assert 'grep -Fqx "$NFT_INCLUDE" "$NFT_MAIN" && exit 79' in script - nm_script = render_remove_script( - _empty_local_plan(), deployment_mode="networkmanager_nmcli", - firewall_mode=FIREWALL_MODE_FIREWALLD) - assert "GENERAL.FILENAME" not in nm_script - assert nm_script.count("vc_nm_filename_for_uuid() {") == 1 - assert ('connection modify --temporary uuid "$NM_PROFILE_UUID" ' - 'connection.zone ""') in nm_script - assert '"$NMCLI" device reapply "$BRIDGE"' in nm_script - assert 'vc_firewalld_bridge_zone_expect blocked absent "$BRIDGE" || exit 72' in nm_script - assert ( - '[ "$(vc_firewalld_interface_zone permanent "$BRIDGE")" = none ] ' - '|| exit 79' - ) in script - untrust = script.index('--zone=trusted --remove-interface="$BRIDGE" >/dev/null || exit 72') - first_live_compare = script.index( - "vc_vpc_persisted_snapshots_match || exit 71") - nft_delete = script.index('"$NFT" -f "$NFT_REMOVE_BATCH"') - assert untrust < first_live_compare < nft_delete - - -def test_firewalld_observe_proves_manifest_zone_state_and_raw_tables(): - script = render_observe_script( - _plan(), firewall_mode=FIREWALL_MODE_FIREWALLD) - assert 'vc_nft_manifest_validate "$NFT_MANIFEST"' in script - assert 'grep -Fqx "$NFT_INCLUDE" "$NFT_MAIN"' in script - assert '__VC_VPC_FIREWALL_MODE__=$FIREWALL_MODE' in script - assert '__VC_VPC_FIREWALL_ZONE__=$firewall_zone' in script - assert '__VC_VPC_FIREWALL_STATE__=$firewall_mode_state' in script - assert '[ "$firewall_mode_state" = owned ]' in script - assert '[ "$permanent_zone" = none ]' in script - assert "GENERAL.FILENAME" not in script - assert '-f UUID,FILENAME' in script - assert script.count("vc_nm_filename_for_uuid() {") == 1 - assert "zone_count != 0" in script - assert "vc_nm_profile_expect active" in script - assert 'ifcfg-$BRIDGE' in script - assert "__VC_VPC_STATE_FIREWALL_MODE__=" in script - assert "state_firewall_observed=invalid" in script - assert 'list table inet "$TABLE"' in script - assert 'list table bridge "$TABLE"' in script - assert 'list table arp "$TABLE"' in script + + + + def test_firewall_mode_is_validated_and_native_state_stays_explicit(): native = render_apply_script(_plan(), firewall_mode=FIREWALL_MODE_NATIVE) assert "FIREWALL_MODE=native_nftables" in native assert "firewall_mode|native_nftables" in native - assert '[ ! -e "$NFT_MANIFEST" ] && [ ! -L "$NFT_MANIFEST" ] || exit 71' in native + assert '[ ! -e "$LEGACY_MANIFEST" ] && [ ! -L "$LEGACY_MANIFEST" ] || exit 71' in native assert "'0|'|'1|native_nftables'" in native + assert '[ "$vpc_a" = native_nftables ] ||' in native + assert "nft_snapshot)" in native + assert "vpc_state_fail || return 1" in native + assert "firewall-cmd" not in native native_ifup = render_apply_script( _plan(), deployment_mode="ifupdown", firewall_mode=FIREWALL_MODE_NATIVE) @@ -537,268 +385,278 @@ def test_firewall_mode_is_validated_and_native_state_stays_explicit(): explicit_up = native_ifup.rindex( 'VC_FABRIC_APPLYING=1 "$IFUP" --force "$BRIDGE"') assert nft_ready < explicit_up - with pytest.raises(VpcDataplaneError) as invalid: - render_apply_script(_plan(), firewall_mode="firewalld") - assert invalid.value.error_code == "vpc_dataplane_firewall_mode_conflict" + for render, plan in ( + (render_apply_script, _plan()), + (render_observe_script, _plan()), + (render_remove_script, _empty_local_plan()), + ): + with pytest.raises(VpcDataplaneError) as invalid: + render(plan, firewall_mode="firewalld_nft") + assert invalid.value.error_code == "vpc_dataplane_firewall_mode_conflict" + + +def _transaction_stat_tool(tmp_path: Path) -> Path: + tool = tmp_path / "stat" + tool.write_text( + f"#!{sys.executable}\n" + "import os, stat, sys\n" + "value = os.lstat(sys.argv[3])\n" + "mode = format(stat.S_IMODE(value.st_mode), 'o')\n" + "fields = {'%u:%g:%a': '0:0:' + mode, '%s': str(value.st_size)}\n" + "print(fields[sys.argv[2]])\n", + encoding="utf-8", + ) + tool.chmod(0o700) + return tool -def test_manifest_validator_rejects_foreign_lines_duplicates_and_bad_mode(tmp_path): - manifest = tmp_path / "nftables.conf" - script = "\n".join(( - 'vc_test_stat() { printf \'0:%s\\n\' "$VC_STAT_MODE"; }', - "STAT=vc_test_stat", - NFT_MANIFEST_VALIDATION_SH, - 'vc_nft_manifest_validate "$MANIFEST"', - )) - env = dict(os.environ, MANIFEST=str(manifest), VC_STAT_MODE="600") - - manifest.write_text( - "# virt-cloud-nft-manifest-v1\n" - 'include "/etc/nftables.d/virt-cloud-wg-fabric.nft"\n' - 'include "/etc/nftables.d/virt-cloud-vpc-0123456789a.nft"\n' - ) - manifest.chmod(0o600) - valid = subprocess.run( - ["/bin/sh", "-eu", "-c", script], env=env, +def test_native_transaction_marker_is_single_line_and_private(tmp_path): + txn = tmp_path / "transaction" + txn.mkdir(mode=0o700) + stat_tool = _transaction_stat_tool(tmp_path) + result = subprocess.run( + ["/bin/sh", "-eu", "-c", "\n".join(( + f"TXN={txn}", + f"STAT={stat_tool}", + VPC_NATIVE_TRANSACTION_GUARD_SH, + "vc_vpc_write_native_transaction_mode", + "vc_vpc_native_transaction_owned", + ))], capture_output=True, text=True, check=False, ) - assert valid.returncode == 0, valid.stderr - invalid_payloads = ( - manifest.read_text() + 'include "/etc/nftables.d/operator.nft"\n', - manifest.read_text() + - 'include "/etc/nftables.d/virt-cloud-vpc-0123456789a.nft"\n', - "# foreign-owner\n" - 'include "/etc/nftables.d/virt-cloud-wg-fabric.nft"\n', - ) - for payload in invalid_payloads: - manifest.write_text(payload) - rejected = subprocess.run( - ["/bin/sh", "-eu", "-c", script], env=env, - capture_output=True, text=True, check=False, + assert result.returncode == 0, result.stderr + marker = txn / "firewall_mode" + assert marker.read_text(encoding="utf-8") == "native_nftables\n" + assert stat.S_IMODE(marker.stat().st_mode) == 0o600 + + +@pytest.mark.parametrize("legacy", ( + "markerless", + "markerless_committed", + "markerless_wg_merged", + "firewalld_marker", + "firewalld_committed", + "firewalld_files_removed", + "firewalld_state", + "snapshot_state", + "nft_persist", + "reassert", + "desired_verify", +)) +def test_native_transaction_guard_preserves_and_rejects_legacy_journal( + tmp_path, legacy, +): + txn = tmp_path / "transaction" + txn.mkdir(mode=0o700) + if not legacy.startswith("markerless"): + (txn / "firewall_mode").write_text( + "firewalld_nft\n" if legacy.startswith("firewalld_") + else "native_nftables\n", + encoding="utf-8", ) - assert rejected.returncode != 0 + (txn / "firewall_mode").chmod(0o600) + for phase in ("committed", "files_removed", "wg_merged"): + if legacy.endswith("_" + phase): + (txn / "state").write_text(phase + "\n", encoding="utf-8") + if legacy == "firewalld_state": + (txn / "state.before").write_text( + "firewall_mode|firewalld_nft\n", encoding="utf-8") + elif legacy == "snapshot_state": + (txn / "state.before").write_text( + "firewall_mode|native_nftables\nnft_snapshot|legacy\n", + encoding="utf-8", + ) + elif legacy == "nft_persist": + (txn / "nft.persist.before.inet").write_text( + "legacy\n", encoding="utf-8") + elif legacy == "reassert": + (txn / "reassert.nft").write_text("legacy\n", encoding="utf-8") + elif legacy == "desired_verify": + (txn / "nft.desired.verify.inet").write_text( + "legacy\n", encoding="utf-8") + for path in txn.iterdir(): + path.chmod(0o600) + before = { + path.name: (path.read_bytes(), stat.S_IMODE(path.stat().st_mode)) + for path in txn.iterdir() + } + mutation = tmp_path / "mutation" + stat_tool = _transaction_stat_tool(tmp_path) + result = subprocess.run( + ["/bin/sh", "-eu", "-c", "\n".join(( + f"TXN={txn}", + f"STAT={stat_tool}", + VPC_NATIVE_TRANSACTION_GUARD_SH, + f"MUTATION={mutation}", + "vc_vpc_native_transaction_owned || exit 79", + ': > "$MUTATION"', + ))], + capture_output=True, text=True, check=False, + ) - manifest.write_text( - "# virt-cloud-nft-manifest-v1\n" - 'include "/etc/nftables.d/virt-cloud-wg-fabric.nft"\n' + assert result.returncode == 79, result.stderr + assert not mutation.exists() + assert txn.is_dir() + after = { + path.name: (path.read_bytes(), stat.S_IMODE(path.stat().st_mode)) + for path in txn.iterdir() + } + assert after == before + + +@pytest.mark.parametrize("legacy", ( + "legacy_manifest", + "fabric_transaction", + "emergency_owner", + "other_transaction", + "inet_snapshot", + "bridge_snapshot", + "arp_snapshot", + "vpc_firewalld_state", + "vpc_nft_snapshot_state", + "fabric_firewalld_mode", + "fabric_underlay", + "fabric_firewall_rule", + "live_firewalld", + "live_inet_emergency", + "live_bridge_emergency", +)) +def test_firewall_boundary_blocks_global_legacy_state_before_mutation( + tmp_path, legacy, +): + etc = tmp_path / "etc" / "virt-cloud" + state_dir = etc / "vpc-dataplane" + run = tmp_path / "run" / "virt-cloud" + state_dir.mkdir(parents=True, mode=0o700) + run.mkdir(parents=True, mode=0o700) + txn = etc / ".vpc-dataplane-current.txn" + evidence: Path | None = None + nft_output = "table inet filter\n" + if legacy == "legacy_manifest": + evidence = etc / "nftables.conf" + evidence.write_text("legacy\n", encoding="utf-8") + elif legacy == "fabric_transaction": + evidence = etc / ".wg-fabric-txn" + evidence.mkdir(mode=0o700) + elif legacy == "emergency_owner": + evidence = run / "wg-fabric-emergency.owner" + evidence.write_text("legacy\n", encoding="utf-8") + elif legacy == "other_transaction": + evidence = etc / ".vpc-dataplane-foreign.txn" + evidence.mkdir(mode=0o700) + elif legacy.endswith("_snapshot"): + family = legacy.removesuffix("_snapshot") + evidence = state_dir / f"legacy.nft.{family}.live" + evidence.write_text("legacy\n", encoding="utf-8") + elif legacy in {"vpc_firewalld_state", "vpc_nft_snapshot_state"}: + record = ( + "firewall_mode|firewalld_nft\n" + if legacy == "vpc_firewalld_state" + else "firewall_mode|native_nftables\nnft_snapshot|legacy\n" + ) + evidence = state_dir / "legacy.state" + evidence.write_text(record, encoding="utf-8") + elif legacy.startswith("live_"): + nft_output = { + "live_firewalld": "table inet firewalld\n", + "live_inet_emergency": ( + "table inet virt_cloud_fabric_emergency\n"), + "live_bridge_emergency": ( + "table bridge virt_cloud_fabric_emergency\n"), + }[legacy] + else: + record = { + "fabric_firewalld_mode": "firewall|firewalld_nft\n", + "fabric_underlay": "underlay|203.0.113.1|51820\n", + "fabric_firewall_rule": "firewall_rule|legacy\n", + }[legacy] + evidence = etc / "wg-fabric0.conf" + evidence.write_text(record, encoding="utf-8") + if evidence is not None and evidence.is_file(): + evidence.chmod(0o600) + evidence_bytes = evidence.read_bytes() if evidence and evidence.is_file() else None + guard = VPC_NATIVE_TRANSACTION_GUARD_SH.replace( + "/etc/virt-cloud", str(etc), + ).replace( + "/run/virt-cloud", str(run), ) - env["VC_STAT_MODE"] = "644" - wrong_mode = subprocess.run( - ["/bin/sh", "-eu", "-c", script], env=env, + mutation = tmp_path / "mutation" + stat_tool = _transaction_stat_tool(tmp_path) + result = subprocess.run( + ["/bin/sh", "-eu", "-c", "\n".join(( + f"TXN={txn}", + f"STAT={stat_tool}", + f"MUTATION={mutation}", + f"NFT_OUTPUT={shlex.quote(nft_output)}", + "vc_test_nft() {", + " [ \"$*\" = \"list tables\" ] || return 2", + " printf '%s' \"$NFT_OUTPUT\"", + "}", + "NFT=vc_test_nft", + guard, + "vc_vpc_firewall_boundary_clean || exit 79", + ': > "$MUTATION"', + ))], capture_output=True, text=True, check=False, ) - assert wrong_mode.returncode != 0 - -def test_firewalld_zone_probe_normalizes_unbound_and_detects_foreign_zone(): - script = "\n".join(( - r''' -vc_test_firewall() { - permanent=no - [ "${1:-}" != --permanent ] || { permanent=yes; shift; } - case "$1" in - --get-zones) printf '%s\n' 'public trusted' ;; - --zone=public) - [ "$VC_ZONE" = public ] && return 0 || return 1 ;; - --zone=trusted) - [ "$VC_ZONE" = trusted ] && return 0 || return 1 ;; - *) return 2 ;; - esac -} -FIREWALL_CMD=vc_test_firewall -'''.strip(), - FIREWALL_ZONE_VALIDATION_SH, - 'vc_firewalld_interface_zone runtime vcv-test', - )) - for configured, expected in (("", "none\n"), ("trusted", "trusted\n"), - ("public", "public\n")): - result = subprocess.run( - ["/bin/sh", "-eu", "-c", script], - env=dict(os.environ, VC_ZONE=configured), - capture_output=True, text=True, check=False, - ) - assert result.returncode == 0, result.stderr - assert result.stdout == expected + assert result.returncode == 79, result.stderr + assert not mutation.exists() + if evidence is not None: + assert evidence.exists() + if evidence_bytes is not None: + assert evidence.read_bytes() == evidence_bytes -@pytest.mark.parametrize("zones", ( - "", - "public bad.zone trusted", - "x" * 65, - " ".join(f"zone{index}" for index in range(65)), -)) -def test_firewalld_zone_probe_rejects_invalid_or_unbounded_zone_lists(zones): - script = "\n".join(( - r''' -vc_test_firewall() { - permanent=no - [ "${1:-}" != --permanent ] || { permanent=yes; shift; } - case "$1" in - --get-zones) printf '%s\n' "$VC_ZONES" ;; - --zone=*) return 1 ;; - *) return 2 ;; - esac -} -FIREWALL_CMD=vc_test_firewall -'''.strip(), - FIREWALL_ZONE_VALIDATION_SH, - 'vc_firewalld_interface_zone runtime vcv-test', - )) - result = subprocess.run( - ["/bin/sh", "-eu", "-c", script], - env=dict(os.environ, VC_ZONES=zones), - capture_output=True, text=True, check=False, +def test_apply_and_remove_validate_native_transaction_before_host_mutation(): + scripts = ( + render_apply_script( + _plan(), deployment_mode="networkmanager_nmcli", + firewall_mode=FIREWALL_MODE_NATIVE, + ), + render_remove_script( + _empty_local_plan(), deployment_mode="networkmanager_nmcli", + firewall_mode=FIREWALL_MODE_NATIVE, + ), + ) + for script in scripts: + boundary = script.index("vc_vpc_firewall_boundary_clean || exit 79") + transaction = script.index( + 'if [ -e "$TXN" ] || [ -L "$TXN" ]; then', boundary) + ownership = script.index( + "vc_vpc_native_transaction_owned || exit 79", transaction) + quarantine = script.index("vc_nm_device_quarantine || exit 72", ownership) + assert boundary < transaction < ownership < quarantine + + create = script.index('mkdir -m 700 "$TXN"', ownership) + marker = script.index( + "vc_vpc_write_native_transaction_mode || exit 79", create) + inflight = script.index('> "$TXN/inflight"', marker) + assert create < marker < inflight + + +def test_observe_refuses_readiness_while_any_transaction_path_exists(): + script = render_observe_script( + _plan(), firewall_mode=FIREWALL_MODE_NATIVE) + readiness = script.index("firewall_mode_state=owned") + transaction = script.index( + '[ ! -e "$TXN" ] && [ ! -L "$TXN" ] || ' + "firewall_mode_state=mismatch", + readiness, ) - assert result.returncode != 0 + marker = script.index("__VC_VPC_FIREWALL_STATE__", transaction) + assert readiness < transaction < marker + + + + -def test_firewalld_zone_ownership_rejects_first_apply_prebound_trusted(): - script = "\n".join(( - r''' -vc_test_firewall() { - permanent=no - [ "$1" != --permanent ] || { permanent=yes; shift; } - case "$1" in - --get-zones) printf '%s\n' 'public trusted' ;; - --get-default-zone) printf '%s\n' "${VC_DEFAULT_ZONE:-public}" ;; - --zone=public|--zone=trusted) - wanted=$(printf '%s\n' "$1" | sed 's/^--zone=//') - if [ "$permanent" = yes ]; then configured=$VC_PERMANENT_ZONE - else configured=$VC_RUNTIME_ZONE - fi - [ "$configured" = "$wanted" ] && return 0 || return 1 - ;; - *) return 2 ;; - esac -} -FIREWALL_CMD=vc_test_firewall -'''.strip(), - FIREWALL_ZONE_VALIDATION_SH, - FIREWALL_ZONE_OWNERSHIP_SH, - 'vc_firewalld_bridge_zone_expect "$VC_EXPECTED" "$VC_OWNER" vcv-test', - )) - def run(expected, owner, runtime, permanent, default="public"): - return subprocess.run( - ["/bin/sh", "-eu", "-c", script], - env=dict( - os.environ, - VC_EXPECTED=expected, - VC_OWNER=owner, - VC_RUNTIME_ZONE=runtime, - VC_PERMANENT_ZONE=permanent, - VC_DEFAULT_ZONE=default, - ), - capture_output=True, text=True, check=False, - ) - assert run("blocked", "absent", "", "").returncode == 0 - assert run("blocked", "managed", "public", "").returncode != 0 - assert run("preapply", "absent", "public", "").returncode != 0 - assert run("nm-blocked", "managed-nm", "public", "").returncode == 0 - assert run("preapply", "managed", "public", "").returncode != 0 - assert run("preapply", "managed-nm", "public", "").returncode == 0 - assert run("preapply", "absent", "trusted", "").returncode != 0 - assert run("preapply", "managed", "trusted", "").returncode == 0 - assert run("active", "managed", "trusted", "").returncode == 0 - assert run("active", "managed", "trusted", "trusted").returncode != 0 - assert run("blocked", "managed", "", "", default="trusted").returncode != 0 -def test_networkmanager_default_zone_quarantine_requires_active_owned_profile(tmp_path): - profile = tmp_path / "vcv-test.nmconnection" - profile.write_text( - "[connection]\n" - "id=vcv-test\n" - "uuid=11111111-1111-4111-8111-111111111111\n" - "type=bridge\n" - "interface-name=vcv-test\n" - "stable-id=virt-cloud-vpc-dataplane-v1 vpc=vpc-test\n" - "autoconnect=false\n" - "autoconnect-retries=0\n" - "\n[ipv4]\nmethod=disabled\n", - ) - profile.chmod(0o600) - helper = NETWORKMANAGER_VPC_PROFILE_SH.replace( - "/etc/NetworkManager/system-connections/", f"{tmp_path}/") - script = "\n".join(( - r''' -vc_test_stat() { printf '%s\n' '0:600'; } -vc_test_nmcli() { - field=; fields=; active_query=no - while [ "$#" -gt 0 ]; do - case "$1" in - -g) shift; field=$1 ;; - -f) shift; fields=$1 ;; - --active) active_query=yes ;; - esac - shift - done - if [ "$fields:$active_query" = UUID:yes ]; then - [ "$VC_ACTIVE" = yes ] && printf '%s\n' "$NM_PROFILE_UUID" - return 0 - fi - case "$field" in - connection.type) printf '%s\n' bridge ;; - connection.interface-name) printf '%s\n' "$BRIDGE" ;; - connection.stable-id) printf '%s\n' "$NM_PROFILE_MARKER" ;; - connection.autoconnect) printf '%s\n' no ;; - connection.autoconnect-retries) printf '%s\n' 0 ;; - connection.zone) printf '%s\n' "$VC_MEMORY_ZONE" ;; - ipv4.method|ipv6.method) printf '%s\n' disabled ;; - bridge.stp) printf '%s\n' no ;; - *) printf '%s\n' -- ;; - esac -} -vc_test_firewall() { - permanent=no - [ "$1" != --permanent ] || { permanent=yes; shift; } - case "$1" in - --get-zones) printf '%s\n' 'public trusted' ;; - --get-default-zone) printf '%s\n' public ;; - --zone=public|--zone=trusted) - wanted=${1#--zone=} - if [ "$permanent" = yes ]; then configured= - else configured=public - fi - [ "$configured" = "$wanted" ] && return 0 || return 1 - ;; - *) return 2 ;; - esac -} -STAT=vc_test_stat -NMCLI=vc_test_nmcli -FIREWALL_CMD=vc_test_firewall -BRIDGE=vcv-test -NM_PROFILE_UUID=11111111-1111-4111-8111-111111111111 -NM_PROFILE_MARKER='virt-cloud-vpc-dataplane-v1 vpc=vpc-test' -'''.strip(), - FIREWALL_ZONE_VALIDATION_SH, - FIREWALL_ZONE_OWNERSHIP_SH, - helper, - 'NM_PROFILE_FILE=$PROFILE', - 'vc_nm_profile_expect blocked-active', - )) - base_env = dict(os.environ, PROFILE=str(profile), VC_MEMORY_ZONE="") - accepted = subprocess.run( - ["/bin/sh", "-eu", "-c", script], - env=dict(base_env, VC_ACTIVE="yes"), - capture_output=True, text=True, check=False, - ) - assert accepted.returncode == 0, accepted.stderr - inactive = subprocess.run( - ["/bin/sh", "-eu", "-c", script], - env=dict(base_env, VC_ACTIVE="no"), - capture_output=True, text=True, check=False, - ) - assert inactive.returncode != 0 - trusted_memory = subprocess.run( - ["/bin/sh", "-eu", "-c", script], - env=dict(base_env, VC_ACTIVE="yes", VC_MEMORY_ZONE="trusted"), - capture_output=True, text=True, check=False, - ) - assert trusted_memory.returncode != 0 def test_networkmanager_disk_profile_requires_manual_boot_and_no_zone(tmp_path): @@ -856,34 +714,6 @@ def test_networkmanager_disk_profile_requires_manual_boot_and_no_zone(tmp_path): ) -def test_firewalld_rollback_quarantines_before_one_atomic_nft_restore(): - script = render_apply_script( - _plan(), deployment_mode="networkmanager_nmcli", - firewall_mode=FIREWALL_MODE_FIREWALLD) - rollback = script[script.index("rollback() {"):script.index( - "trap rollback EXIT HUP INT TERM")] - quarantine = ('connection modify --temporary uuid "$NM_PROFILE_UUID" ' - 'connection.zone ""') - assert quarantine in rollback - assert rollback.index(quarantine) < rollback.index( - "while IFS='|' read -r destination device table via metric") - assert rollback.index(quarantine) < rollback.index( - 'NFT_ROLLBACK="$TXN/rollback.nft"') - assert ("printf 'delete table inet virt_cloud_wg_fabric\\n' >> " - '"$NFT_ROLLBACK"') in rollback - assert 'cat "$TXN/fabric.nft.before" >> "$NFT_ROLLBACK"' in rollback - assert '"$NFT" delete table inet virt_cloud_wg_fabric' not in rollback - assert rollback.count('"$NFT" -f "$NFT_ROLLBACK"') == 1 - restore_trust = ('connection modify --temporary uuid "$NM_PROFILE_UUID" ' - 'connection.zone trusted') - assert rollback.index('vc_vpc_fabric_nft_live_ready') < rollback.index( - restore_trust) - restore_index = rollback.index(restore_trust) - fabric_exact = rollback.rfind( - 'vc_nft_table_matches_snapshot inet virt_cloud_wg_fabric', - 0, restore_index) - assert fabric_exact >= 0 - assert '"$TXN/fabric.nft.before"' in rollback[fabric_exact:restore_index] def test_fabric_snapshot_verdict_drift_blocks_exact_restore_guard(tmp_path): @@ -1025,17 +855,16 @@ CMP=/usr/bin/cmp def test_apply_rollback_requires_exact_post_apply_tables_before_delete(): script = render_apply_script( _plan(), deployment_mode="networkmanager_nmcli", - firewall_mode=FIREWALL_MODE_FIREWALLD) + firewall_mode=FIREWALL_MODE_NATIVE) apply_call = script.index('"$NFT" -f "$NFT_BATCH" >/dev/null || exit 72') applied_flag = script.index("NFT_APPLIED=yes", apply_call) assert apply_call < applied_flag assert "NFT_APPLIED=no" in script assert "FABRIC_NFT_APPLIED=no" in script assert '"$NFT" -s list table "$vc_nft_snapshot_family"' in script - assert 'vc_vpc_nft_snapshots_equal "$TXN/nft.desired"' in script - assert 'nft_snapshot|inet|/etc/virt-cloud/vpc-dataplane/' in script - assert 'nft_snapshot|bridge|/etc/virt-cloud/vpc-dataplane/' in script - assert 'nft_snapshot|arp|/etc/virt-cloud/vpc-dataplane/' in script + assert 'vc_vpc_capture_nft_snapshot "$TXN/nft.desired"' in script + assert 'vc_vpc_nft_matches_snapshot "$TXN/nft.desired"' in script + assert "nft_snapshot|" not in script rollback = script[script.index("rollback() {"):script.index( "trap rollback EXIT HUP INT TERM")] @@ -1051,250 +880,36 @@ def test_apply_rollback_requires_exact_post_apply_tables_before_delete(): def test_networkmanager_rollback_restores_nontrusted_existing_profile_state(): - for firewall_mode in (FIREWALL_MODE_NATIVE, FIREWALL_MODE_FIREWALLD): - script = render_apply_script( - _plan(), deployment_mode="networkmanager_nmcli", - firewall_mode=firewall_mode) - rollback = script[script.index("rollback() {"):script.index( - "trap rollback EXIT HUP INT TERM")] - restore = rollback.index( - '[ "$NM_PROFILE_WAS_PRESENT" = yes ]') - restored_nft = rollback.index( - 'if [ "$FIREWALL_ZONE_BLOCKED" = yes ]; then') - release = rollback.index("vc_nm_device_release", restore) - activate = rollback.index( - 'connection up uuid "$NM_PROFILE_UUID"', release) - topology = rollback.index( - "vc_vpc_bridge_topology_matches", activate) - assert restored_nft < restore < release < activate < topology - assert '[ "$NM_WAS_ACTIVE" = yes ]' in rollback[restore:topology] - assert "vc_nm_device_expect_managed yes" in rollback[restore:] - - -def test_rollback_keeps_journal_when_old_trusted_acl_is_not_proven_safe(): script = render_apply_script( _plan(), deployment_mode="networkmanager_nmcli", - firewall_mode=FIREWALL_MODE_FIREWALLD) + firewall_mode=FIREWALL_MODE_NATIVE) rollback = script[script.index("rollback() {"):script.index( "trap rollback EXIT HUP INT TERM")] - unsafe = rollback.index('[ "$FIREWALL_PREVIOUS_NFT_SAFE" != yes ]') - mark_failed = rollback.index("ROLLBACK_FAILED=yes", unsafe) - journal = rollback.index( - "if [ \"$ROLLBACK_FAILED\" = yes ]; then printf '%s\\n' " - "rollback_failed", mark_failed) - assert unsafe < mark_failed < journal + nft_restore = rollback.index('"$NFT" -f "$NFT_ROLLBACK"') + restore = rollback.index('[ "$NM_PROFILE_WAS_PRESENT" = yes ]') + release = rollback.index("vc_nm_device_release", restore) + activate = rollback.index( + 'connection up uuid "$NM_PROFILE_UUID"', release) + topology = rollback.index( + "vc_vpc_bridge_topology_matches", activate) + assert nft_restore < restore < release < activate < topology + assert '[ "$NM_WAS_ACTIVE" = yes ]' in rollback[restore:topology] + assert "vc_nm_device_expect_managed yes" in rollback[restore:] + -def test_trusted_bridge_is_quarantined_before_old_acl_readback(tmp_path): - script = render_apply_script( - _plan(), deployment_mode="ifupdown", - firewall_mode=FIREWALL_MODE_FIREWALLD) - quarantine_comment = script.index( - "# Emergency quarantine deliberately runs before") - hard_down = script.index( - '"$IP" link set dev "$BRIDGE" down >/dev/null || exit 72', - quarantine_comment) - remove_trust = script.index( - '--zone=trusted --remove-interface="$BRIDGE"', quarantine_comment) - fabric_owner = script.index( - '[ -f /etc/virt-cloud/wg-fabric0.owner ]', remove_trust) - manifest_check = script.index( - 'vc_nft_manifest_validate "$NFT_MANIFEST" || exit 71', remove_trust) - rpdb_check = script.index("vc_ip_rule_scope_check", remove_trust) - fabric_readback = script.index( - 'if "$NFT" -s list table inet virt_cloud_wg_fabric', remove_trust) - old_acl_readback = script.index( - "vc_vpc_persisted_nft_ready || exit 71", remove_trust) - assert hard_down < remove_trust - assert remove_trust < fabric_owner < manifest_check < rpdb_check - assert remove_trust < fabric_readback - assert remove_trust < old_acl_readback - -def test_networkmanager_quarantine_preserves_device_until_acl_is_blocked(tmp_path): - script = render_apply_script( - _plan(), deployment_mode="networkmanager_nmcli", - firewall_mode=FIREWALL_MODE_FIREWALLD) - early = script.index("# Emergency quarantine deliberately runs before") - quarantine_call = script.index("vc_nm_device_quarantine || exit 72", early) - mode_guard = script.index("vc_firewall_mode_ready || exit 71", quarantine_call) - remove_trust = script.index( - '--zone=trusted --remove-interface="$BRIDGE"', mode_guard) - fabric_owner = script.index( - '[ -f /etc/virt-cloud/wg-fabric0.owner ]', remove_trust) - assert quarantine_call < mode_guard < remove_trust < fabric_owner - clear_zone = NETWORKMANAGER_VPC_PROFILE_SH.index( - 'connection modify --temporary uuid "$NM_PROFILE_UUID"') - unmanaged = NETWORKMANAGER_VPC_PROFILE_SH.index( - '"$NMCLI" device set "$BRIDGE" managed no', clear_zone) - hard_down = NETWORKMANAGER_VPC_PROFILE_SH.index( - '"$IP" link set dev "$BRIDGE" down', unmanaged) - assert clear_zone < unmanaged < hard_down - assert 'connection down uuid "$NM_PROFILE_UUID"' not in script - assert 'device disconnect "$BRIDGE"' not in script - - zone_state = tmp_path / "zone" - managed_state = tmp_path / "managed" - link_state = tmp_path / "link" - event_log = tmp_path / "events" - zone_state.write_text("trusted\n") - managed_state.write_text("yes\n") - link_state.write_text("up\n") - harness = "\n".join(( - "set -eu", - "BRIDGE=vcv-test", - "NM_PROFILE_UUID=11111111-1111-4111-8111-111111111111", - "FIREWALL_MODE=firewalld_nft", - f"ZONE_STATE={zone_state}", - f"MANAGED_STATE={managed_state}", - f"LINK_STATE={link_state}", - f"EVENT_LOG={event_log}", - r''' -vc_test_ip() { - if [ "$1:$2:$3" = "link:show:dev" ]; then return 0; fi - if [ "$1:$2:$3:$4:$5" = "link:set:dev:vcv-test:down" ]; then - printf '%s\n' down > "$LINK_STATE" - printf '%s\n' down >> "$EVENT_LOG" - return 0 - fi - if [ "$1:$2:$3" = "-o:link:show" ]; then - [ "$(cat "$LINK_STATE")" = up ] && - printf '%s\n' '9: vcv-test: ' || - printf '%s\n' '9: vcv-test: ' - return 0 - fi - return 2 -} -vc_test_nmcli() { - case "$*" in - "-t --escape no -g GENERAL.CON-UUID device show vcv-test") - printf '%s\n' "$NM_PROFILE_UUID" ;; - "-t --escape no -g connection.zone connection show uuid $NM_PROFILE_UUID") - cat "$ZONE_STATE" ;; - "-t --escape no -g GENERAL.NM-MANAGED device show vcv-test") - cat "$MANAGED_STATE" ;; - "connection modify --temporary uuid $NM_PROFILE_UUID connection.zone ") - : > "$ZONE_STATE" - printf '%s\n' clear-zone >> "$EVENT_LOG" ;; - "device set vcv-test managed no") - printf '%s\n' no > "$MANAGED_STATE" - printf '%s\n' unmanaged >> "$EVENT_LOG" ;; - *) return 2 ;; - esac -} -IP=vc_test_ip -NMCLI=vc_test_nmcli -'''.strip(), - NETWORKMANAGER_VPC_PROFILE_SH, - "vc_nm_device_quarantine", - "false # simulate a later ACL verification failure", - )) - failed = subprocess.run( - ["/bin/sh", "-eu", "-c", harness], capture_output=True, - text=True, check=False, - ) - assert failed.returncode != 0 - assert zone_state.read_text() == "" - assert managed_state.read_text() == "no\n" - assert link_state.read_text() == "down\n" - assert event_log.read_text().splitlines() == [ - "clear-zone", "unmanaged", "down", - ] -def test_networkmanager_trusted_isolated_retry_requires_exact_state(): - harness = "\n".join(( - "set -eu", - "BRIDGE=vcv-test", - "NM_PROFILE_UUID=11111111-1111-4111-8111-111111111111", - "FIREWALL_MODE=firewalld_nft", - 'BRIDGE_TOPOLOGY_CAPTURED="${VC_CAPTURED:-yes}"', - NETWORKMANAGER_VPC_PROFILE_SH, - r''' -vc_nm_profile_base_validate() { [ "$VC_BASE_VALID" = yes ]; } -vc_nm_profile_field() { - [ "$1" = connection.zone ] || return 1 - printf '%s\n' "$VC_MEMORY_ZONE" -} -vc_nm_device_expect_managed() { - [ "$1" = no ] && [ "$VC_MANAGED" = no ] -} -vc_vpc_bridge_topology_matches() { [ "$VC_TOPOLOGY" = same ]; } -vc_test_nmcli() { - case "$*" in - "-t --escape no -f UUID connection show --active") - [ "$VC_ACTIVE" = no ] || printf '%s\n' "$NM_PROFILE_UUID" - ;; - "-t --escape no -g GENERAL.CON-UUID device show vcv-test") - [ "${VC_DEVICE_QUERY_FAIL:-no}" = no ] || return 3 - printf '%s\n' "$VC_DEVICE_UUID" - ;; - *) return 2 ;; - esac -} -vc_test_ip() { - case "$*" in - "link show dev vcv-test") return 0 ;; - "-o link show dev vcv-test") - [ "${VC_LINK_QUERY_FAIL:-no}" = no ] || return 3 - case "$VC_LINK" in - up) printf '%s\n' '9: vcv-test: ' ;; - down) printf '%s\n' '9: vcv-test: ' ;; - empty) : ;; - malformed) printf '%s\n' malformed ;; - *) return 2 ;; - esac - ;; - *) return 2 ;; - esac -} -NMCLI=vc_test_nmcli -IP=vc_test_ip -vc_nm_trusted_isolated_retry_ready -'''.strip(), - )) - exact = { - "VC_BASE_VALID": "yes", - "VC_MEMORY_ZONE": "trusted", - "VC_ACTIVE": "no", - "VC_MANAGED": "no", - "VC_DEVICE_UUID": "11111111-1111-4111-8111-111111111111", - "VC_LINK": "down", - "VC_TOPOLOGY": "same", - } - def run(**changes): - values = dict(exact) - values.update(changes) - return subprocess.run( - ["/bin/sh", "-eu", "-c", harness], - env=dict(os.environ, **values), capture_output=True, text=True, - check=False, - ) - accepted = run() - assert accepted.returncode == 0, accepted.stderr - assert run(VC_BASE_VALID="no").returncode != 0 - assert run(VC_MEMORY_ZONE="").returncode != 0 - assert run(VC_ACTIVE="yes").returncode != 0 - assert run(VC_MANAGED="yes").returncode != 0 - assert run( - VC_DEVICE_UUID="22222222-2222-4222-8222-222222222222" - ).returncode != 0 - assert run(VC_LINK="up").returncode != 0 - assert run(VC_DEVICE_QUERY_FAIL="yes").returncode != 0 - assert run(VC_LINK_QUERY_FAIL="yes").returncode != 0 - assert run(VC_LINK="empty").returncode != 0 - assert run(VC_LINK="malformed").returncode != 0 - assert run(VC_CAPTURED="no").returncode != 0 - assert run(VC_TOPOLOGY="drifted").returncode != 0 def test_vpc_bridge_topology_parser_propagates_malformed_output(): script = render_apply_script( _plan(), deployment_mode="networkmanager_nmcli", - firewall_mode=FIREWALL_MODE_FIREWALLD) + firewall_mode=FIREWALL_MODE_NATIVE) start = script.index("vc_vpc_bridge_port_names() {") - end = script.index("# Emergency quarantine deliberately runs before", start) + end = script.index("# Never mutate even the bridge state", start) helpers = script[start:end] harness = "\n".join(( "set -eu", @@ -1327,7 +942,7 @@ BRIDGE_CMD=vc_test_bridge def test_vpc_bridge_link_state_requires_one_well_formed_record(): script = render_apply_script( _plan(), deployment_mode="ifupdown", - firewall_mode=FIREWALL_MODE_FIREWALLD) + firewall_mode=FIREWALL_MODE_NATIVE) start = script.index("vc_vpc_bridge_link_state() {") end = script.index( "vc_vpc_native_ifupdown_rollback_reopen() {", start) @@ -1363,7 +978,7 @@ IP=vc_test_ip assert run("9: foreign: \n").returncode != 0 assert run("9: vcv-test: \n", fail="yes").returncode != 0 - early = script.index("# Emergency quarantine deliberately runs before") + early = script.index("# Link quarantine deliberately runs before") hard_down = script.index( '"$IP" link set dev "$BRIDGE" down >/dev/null || exit 72', early) exact_down = script.index( @@ -1372,40 +987,12 @@ IP=vc_test_ip assert hard_down < exact_down -def test_networkmanager_trusted_isolated_retry_is_proven_before_zone_cleanup(): - script = render_apply_script( - _plan(), deployment_mode="networkmanager_nmcli", - firewall_mode=FIREWALL_MODE_FIREWALLD) - state = script.index("state_revision=$(awk") - owner = script.index( - 'grep -Fqx "$OWNER_MARKER" "$OWNER_FILE" || exit 71', state) - profile = script.index("vc_nm_profile_base_validate || exit 71", owner) - candidate = script.index("NM_TRUSTED_ISOLATED_CANDIDATE=yes", profile) - ports = script.index( - "BRIDGE_PORTS_BEFORE=$(vc_vpc_bridge_port_names) || exit 71", - candidate) - exact = script.index( - "vc_nm_trusted_isolated_retry_ready || exit 71", ports) - quarantine = script.index("vc_nm_device_quarantine || exit 72", exact) - mode_guard = script.index("vc_firewall_mode_ready || exit 71", quarantine) - clear_runtime = script.index( - 'if [ "$VC_VPC_ZONE_RUNTIME" = trusted ]; then', mode_guard) - assert state < owner < profile < candidate < ports < exact - assert exact < quarantine < mode_guard < clear_runtime - assert ('[ "$NM_TRUSTED_ISOLATED_CANDIDATE" = no ] ||\n' - ' [ "$NM_TRUSTED_ISOLATED_RETRY" = yes ] || exit 71') in script - assert ('[ "$VC_VPC_ZONE_RUNTIME" = trusted ] &&\n' - ' [ "$NM_TRUSTED_ISOLATED_RETRY" != yes ]') in script - - rollback = script[script.index("rollback() {"):script.index( - "trap rollback EXIT HUP INT TERM")] - assert '[ "$NM_TRUSTED_ISOLATED_RETRY" != yes ]' in rollback def test_ifupdown_apply_rollback_never_ifdowns_a_bridge_before_port_proof(): script = render_apply_script( _plan(), deployment_mode="ifupdown", - firewall_mode=FIREWALL_MODE_FIREWALLD) + firewall_mode=FIREWALL_MODE_NATIVE) rollback = script[script.index("rollback() {"):script.index( "trap rollback EXIT HUP INT TERM")] assert '"$IFDOWN" --force "$BRIDGE"' not in rollback @@ -1434,7 +1021,7 @@ def test_native_ifupdown_rollback_reopens_only_exact_sampled_up_existing_bridge( helper_start = script.index( "vc_vpc_native_ifupdown_rollback_reopen() {") helper_end = script.index( - "# Emergency quarantine deliberately runs before", helper_start) + "# Never mutate even the bridge state", helper_start) helper = script[helper_start:helper_end] exact = helper.index( 'vc_vpc_nft_matches_snapshot "$TXN/nft.precommit.before"') @@ -1451,7 +1038,7 @@ def test_native_ifupdown_rollback_reopens_only_exact_sampled_up_existing_bridge( "trap rollback EXIT HUP INT TERM")] restore = rollback.index( 'if [ "$MODE" = ifupdown ] &&\n' - ' [ "$FIREWALL_MODE" = native_nftables ]') + ' [ "$BRIDGE_CREATED" = no ]') restore_call = rollback.index( "vc_vpc_native_ifupdown_rollback_reopen", restore) assert '[ "$BRIDGE_CREATED" = no ]' in rollback[restore:restore_call] @@ -1464,7 +1051,7 @@ def test_native_ifupdown_reopen_failure_reisolates_bridge(tmp_path): firewall_mode=FIREWALL_MODE_NATIVE) start = script.index("vc_vpc_native_ifupdown_rollback_reopen() {") end = script.index( - "# Emergency quarantine deliberately runs before", start) + "# Never mutate even the bridge state", start) helper = script[start:end] link_state = tmp_path / "link" topology_calls = tmp_path / "topology-calls" @@ -1686,7 +1273,7 @@ vc_nm_created_profile_remove def test_destructive_nft_batches_recheck_full_state_after_dry_run(): apply = render_apply_script( _plan(), deployment_mode="networkmanager_nmcli", - firewall_mode=FIREWALL_MODE_FIREWALLD) + firewall_mode=FIREWALL_MODE_NATIVE) baseline = apply.index( 'vc_vpc_capture_nft_precommit "$TXN/nft.precommit.before"') apply_check = apply.index('"$NFT" -c -f "$NFT_BATCH"', baseline) @@ -1698,17 +1285,6 @@ def test_destructive_nft_batches_recheck_full_state_after_dry_run(): 'vc_vpc_capture_nft_snapshot "$TXN/nft.desired"', apply_commit) assert baseline < apply_check < apply_precommit < apply_commit < apply_post - reassert = apply.index('NFT_REASSERT="$TXN/reassert.nft"') - reassert_check = apply.index('"$NFT" -c -f "$NFT_REASSERT"', reassert) - reassert_precommit = apply.index( - 'vc_vpc_nft_matches_snapshot "$TXN/nft.desired"', reassert_check) - reassert_commit = apply.index( - '"$NFT" -f "$NFT_REASSERT"', reassert_precommit) - reassert_post = apply.index( - 'vc_vpc_capture_nft_snapshot "$TXN/nft.desired.verify"', - reassert_commit) - assert reassert_check < reassert_precommit < reassert_commit < reassert_post - rollback = apply[apply.index("rollback() {"):apply.index( "trap rollback EXIT HUP INT TERM")] rollback_check = rollback.index('"$NFT" -c -f "$NFT_ROLLBACK"') @@ -1730,7 +1306,7 @@ def test_destructive_nft_batches_recheck_full_state_after_dry_run(): remove = render_remove_script( _empty_local_plan(), deployment_mode="ifupdown", - firewall_mode=FIREWALL_MODE_FIREWALLD) + firewall_mode=FIREWALL_MODE_NATIVE) remove_capture = remove.index( 'vc_remove_capture_nft_precommit "$TXN/nft.remove.precommit"') remove_check = remove.index( @@ -1748,14 +1324,14 @@ def test_destructive_nft_batches_recheck_full_state_after_dry_run(): def test_fake_nft_dry_run_replacement_never_reaches_destructive_commit(tmp_path): apply = render_apply_script( _plan(), deployment_mode="networkmanager_nmcli", - firewall_mode=FIREWALL_MODE_FIREWALLD) + firewall_mode=FIREWALL_MODE_NATIVE) apply_start = apply.index("vc_vpc_capture_nft_precommit() {") - apply_end = apply.index("vc_vpc_nft_snapshots_equal() {", apply_start) + apply_end = apply.index("vc_firewall_mode_ready() {", apply_start) apply_helpers = apply[apply_start:apply_end] remove = render_remove_script( _empty_local_plan(), deployment_mode="ifupdown", - firewall_mode=FIREWALL_MODE_FIREWALLD) + firewall_mode=FIREWALL_MODE_NATIVE) remove_start = remove.index("vc_remove_capture_nft_precommit() {") remove_end = remove.index('case "$PHASE" in', remove_start) remove_helpers = remove[remove_start:remove_end] @@ -1900,33 +1476,8 @@ RACE_TABLE=$TABLE assert not applied.exists() -def test_firewalld_remove_compares_all_canonical_tables_before_atomic_delete(): - script = render_remove_script( - _empty_local_plan(), deployment_mode="ifupdown", - firewall_mode=FIREWALL_MODE_FIREWALLD) - first_compare = script.index( - 'vc_nft_table_matches_snapshot "$family" "$name"') - fresh_compare = script.index("vc_vpc_persisted_snapshots_match", first_compare) - apply_delete = script.index('"$NFT" -f "$NFT_REMOVE_BATCH"', fresh_compare) - assert first_compare < fresh_compare < apply_delete - assert '[ "$nft_table_count" = 3 ] || exit 71' in script - assert '"$NFT" delete table "$family" "$name"' not in script - - -def test_remove_acl_mismatch_fails_after_quarantining_trusted_bridge(tmp_path): - script = render_remove_script( - _empty_local_plan(), deployment_mode="ifupdown", - firewall_mode=FIREWALL_MODE_FIREWALLD) - early = script.index("# Minimum state/manager proof") - hard_down = script.index( - '"$IP" link set dev "$BRIDGE" down >/dev/null || exit 72', early) - remove_trust = script.index( - '--zone=trusted --remove-interface="$BRIDGE"', hard_down) - fabric_owner = script.index( - '[ -f /etc/virt-cloud/wg-fabric0.owner ]', remove_trust) - first_live_compare = script.index( - "vc_vpc_persisted_snapshots_match || exit 71", remove_trust) - assert hard_down < remove_trust < fabric_owner < first_live_compare + + def test_adapter_normalizes_legacy_test_contract_but_rejects_unknown_mode(): adapter = VpcPrivateHostAdapter(object(), {}) diff --git a/engine/virsh_panel/fabric.py b/engine/virsh_panel/fabric.py index 3dd56068..c9381632 100644 --- a/engine/virsh_panel/fabric.py +++ b/engine/virsh_panel/fabric.py @@ -46,8 +46,7 @@ FABRIC_LIBVIRT_GATE_UNITS = ( ) FABRIC_NFT_TABLE = "virt_cloud_wg_fabric" FABRIC_NFT_RULES = "/etc/nftables.d/virt-cloud-wg-fabric.nft" -FABRIC_NFT_MANIFEST = "/etc/virt-cloud/nftables.conf" -FABRIC_NFT_MANIFEST_OWNER = "# virt-cloud-nft-manifest-v1" +LEGACY_FIREWALLD_NFT_MANIFEST = "/etc/virt-cloud/nftables.conf" FABRIC_NFT_MAIN_CANDIDATES = ( "/etc/nftables.conf", "/etc/sysconfig/nftables.conf", @@ -60,10 +59,7 @@ FABRIC_LOCK = "/run/virt-cloud/wg-fabric0.lock" FABRIC_TXN = "/etc/virt-cloud/.wg-fabric-txn" FABRIC_DEPLOYMENT_MODES = frozenset(("networkmanager_nmcli", "ifupdown")) FIREWALL_MODE_NATIVE = "native_nftables" -FIREWALL_MODE_FIREWALLD = "firewalld_nft" -FABRIC_FIREWALL_MODES = frozenset(( - FIREWALL_MODE_NATIVE, FIREWALL_MODE_FIREWALLD, -)) +FABRIC_FIREWALL_MODES = frozenset((FIREWALL_MODE_NATIVE,)) FABRIC_FAILURE_PHASES = frozenset(("identity", "observe", "apply")) FABRIC_REMOTE_READ_TIMEOUT_SECONDS = 35 FABRIC_REMOTE_APPLY_TIMEOUT_SECONDS = 300 @@ -125,7 +121,6 @@ _KEY = re.compile(r"[A-Za-z0-9+/]{42}==\Z|[A-Za-z0-9+/]{43}=\Z") _FABRIC_KEY = re.compile(r"[A-Za-z0-9][A-Za-z0-9_.-]{0,63}\Z") _READINESS_REASON = re.compile(r"[a-z][a-z0-9_]{0,63}\Z") _SAFE_INTERFACE = re.compile(r"[A-Za-z0-9_.:-]{1,15}\Z") -_SAFE_FIREWALL_ZONE = re.compile(r"[A-Za-z0-9_-]{1,64}\Z") _OPERATION_ID = re.compile(r"[0-9a-f]{32}\Z") _RFC1918_NETWORKS = ( ipaddress.IPv4Network("10.0.0.0/8"), @@ -433,7 +428,8 @@ vpc_state_load() { vpc_state_fail || return 1 vpc_digest_count=$((vpc_digest_count + 1)) ;; firewall_mode) - case "$vpc_a" in native_nftables|firewalld_nft) : ;; *) vpc_state_fail || return 1 ;; esac + [ "$vpc_a" = native_nftables ] || + vpc_state_fail || return 1 [ -z "$vpc_b$vpc_c$vpc_d" ] || vpc_state_fail || return 1 vpc_state_firewall_mode=$vpc_a vpc_firewall_count=$((vpc_firewall_count + 1)) ;; @@ -524,9 +520,9 @@ vpc_state_load() { [ "$vpc_table_count" = 1 ] || vpc_state_fail || return 1 # Native state written before the firewall-owner contract omitted - # this line. It can be upgraded only while the outer runtime guard - # still proves nftables.service is the sole owner. Firewalld state - # must always be explicit, and duplicate markers remain invalid. + # this line. It can be accepted only while the outer runtime guard + # proves nftables.service is the sole owner. Any explicit legacy + # firewalld marker was rejected above and requires maintenance cleanup. if [ "$vpc_firewall_count" = 0 ]; then [ "$FIREWALL_MODE" = native_nftables ] || vpc_state_fail || return 1 vpc_state_firewall_mode=native_nftables @@ -534,17 +530,7 @@ vpc_state_load() { [ "$vpc_firewall_count" = 1 ] || vpc_state_fail || return 1 fi [ "$vpc_state_firewall_mode" = "$FIREWALL_MODE" ] || vpc_state_fail || return 1 - if [ "$FIREWALL_MODE" = firewalld_nft ]; then - [ "$vpc_snapshot_count" = 3 ] || vpc_state_fail || return 1 - for vpc_snapshot_family in inet bridge arp; do - case " $vpc_snapshot_families " in - *" $vpc_snapshot_family "*) : ;; - *) vpc_state_fail || return 1 ;; - esac - done - else - [ "$vpc_snapshot_count" = 0 ] || vpc_state_fail || return 1 - fi + [ "$vpc_snapshot_count" = 0 ] || vpc_state_fail || return 1 # Validate membership after the complete record has been parsed so a # malicious route cannot precede the cidr line and bypass the check. while IFS='|' read -r vpc_route_prefix vpc_route_dev vpc_route_tbl vpc_route_via vpc_route_metric vpc_route_extra; do @@ -849,119 +835,6 @@ def _is_usable_endpoint_ipv4(address: ipaddress.IPv4Address) -> bool: ) -def _firewalld_endpoint_zone_evidence( - capabilities: Mapping[str, Any], - endpoint_ip: str, -) -> tuple[dict[str, str] | None, str | None]: - """Resolve one local endpoint's effective firewalld zone fail closed. - - Runtime firewalld bindings take precedence over the default zone. On - NetworkManager hosts the permanent value comes from the unique active - connection's root-owned keyfile evidence; firewalld permanent XML may be - absent or agree, but may never conflict. ifupdown has no such external - owner and therefore uses the permanent binding/default directly. - """ - network = capabilities.get("network") - if not isinstance(network, Mapping): - return None, "firewalld_endpoint_zone_unverified" - - def zone_map(value: Any) -> dict[str, str] | None: - if not isinstance(value, Mapping) or len(value) > 4096: - return None - result: dict[str, str] = {} - for raw_interface, raw_zone in value.items(): - if (not isinstance(raw_interface, str) - or _SAFE_INTERFACE.fullmatch(raw_interface) is None - or not isinstance(raw_zone, str) - or _SAFE_FIREWALL_ZONE.fullmatch(raw_zone) is None - or raw_interface in result): - return None - result[raw_interface] = raw_zone - return result - - address_map = network.get("local_ipv4_interfaces") - interfaces = ( - address_map.get(endpoint_ip) - if isinstance(address_map, Mapping) else None - ) - if (not isinstance(interfaces, list) or len(interfaces) != 1 - or not isinstance(interfaces[0], str) - or _SAFE_INTERFACE.fullmatch(interfaces[0]) is None): - return None, "firewalld_endpoint_zone_unverified" - interface = interfaces[0] - if (interface in {"lo", FABRIC_INTERFACE} - or interface.startswith(( - "vcv-", "vnet", "tap", "macvtap", "fwpr", "fwln"))): - return None, "firewalld_endpoint_zone_unsafe_or_drifted" - - evidence = network.get("firewall_manager_evidence") - if not isinstance(evidence, Mapping): - return None, "firewalld_endpoint_zone_unverified" - default_runtime = evidence.get("firewalld_default_runtime") - default_permanent = evidence.get("firewalld_default_permanent") - if (not isinstance(default_runtime, str) - or _SAFE_FIREWALL_ZONE.fullmatch(default_runtime) is None - or not isinstance(default_permanent, str) - or _SAFE_FIREWALL_ZONE.fullmatch(default_permanent) is None): - return None, "firewalld_endpoint_zone_unverified" - runtime_bindings = zone_map( - network.get("firewalld_runtime_interface_zones")) - permanent_bindings = zone_map( - network.get("firewalld_permanent_interface_zones")) - if runtime_bindings is None or permanent_bindings is None: - return None, "firewalld_endpoint_zone_unverified" - runtime_zone = runtime_bindings.get(interface, default_runtime) - - manager = network.get("network_manager") - if manager == "networkmanager": - nm_bindings = network.get( - "networkmanager_persistent_interface_zones") - if not isinstance(nm_bindings, Mapping) or len(nm_bindings) > 4096: - return None, "firewalld_endpoint_zone_unverified" - normalized_nm: dict[str, str] = {} - for raw_interface, raw_zone in nm_bindings.items(): - if (not isinstance(raw_interface, str) - or _SAFE_INTERFACE.fullmatch(raw_interface) is None - or not isinstance(raw_zone, str) - or (raw_zone not in {"", "?"} and - _SAFE_FIREWALL_ZONE.fullmatch(raw_zone) is None) - or raw_interface in normalized_nm): - return None, "firewalld_endpoint_zone_unverified" - normalized_nm[raw_interface] = raw_zone - if interface not in normalized_nm: - return None, "firewalld_endpoint_zone_unverified" - if normalized_nm[interface] == "?": - return None, "firewalld_endpoint_zone_unverified" - permanent_zone = normalized_nm[interface] or default_permanent - permanent_binding = permanent_bindings.get(interface) - if (permanent_binding is not None - and permanent_binding != permanent_zone): - return None, "firewalld_endpoint_zone_unsafe_or_drifted" - elif manager == "ifupdown": - ifcfg_interfaces = network.get("ifcfg_zone_interfaces") - if (not isinstance(ifcfg_interfaces, list) - or len(ifcfg_interfaces) > 256 - or any(not isinstance(item, str) - or _SAFE_INTERFACE.fullmatch(item) is None - for item in ifcfg_interfaces)): - return None, "firewalld_endpoint_zone_unverified" - if interface in ifcfg_interfaces: - return None, "firewalld_endpoint_zone_unsafe_or_drifted" - permanent_zone = permanent_bindings.get( - interface, default_permanent) - else: - return None, "firewalld_endpoint_zone_unverified" - - if (runtime_zone == "trusted" or permanent_zone == "trusted" - or runtime_zone != permanent_zone): - return None, "firewalld_endpoint_zone_unsafe_or_drifted" - return ({ - "fabric_endpoint_interface": interface, - "fabric_endpoint_runtime_zone": runtime_zone, - "fabric_endpoint_permanent_zone": permanent_zone, - }, None) - - def _fabric_endpoint_rp_filter_evidence( capabilities: Mapping[str, Any], endpoint_ip: str, @@ -1208,15 +1081,11 @@ def _profile_firewall_mode(profile: Mapping[str, Any] | None) -> str: nftables = capabilities.get("nftables") if isinstance( capabilities, Mapping) else None mode = network.get("firewall_mode") if isinstance(network, Mapping) else None - expected_owner = { - FIREWALL_MODE_NATIVE: "nftables", - FIREWALL_MODE_FIREWALLD: "firewalld", - }.get(mode) - if (mode not in FABRIC_FIREWALL_MODES - or network.get("firewall_manager") != expected_owner + if (mode != FIREWALL_MODE_NATIVE + or network.get("firewall_manager") != "nftables" or not isinstance(nftables, Mapping) or nftables.get("ready") is not True - or nftables.get("firewall_mode") != mode): + or nftables.get("firewall_mode") != FIREWALL_MODE_NATIVE): raise FabricError( "宿主机防火墙管理器与 Fabric 部署模式不匹配", 409, "fabric_host_firewall_mode_conflict") @@ -2294,34 +2163,6 @@ vc_fabric_nft_empty_tenant_set() { '''.strip() -_NFT_MANIFEST_VALIDATION_SH = r''' -vc_nft_manifest_validate() { - manifest=$1 - [ -f "$manifest" ] && [ ! -L "$manifest" ] || return 1 - [ "$($STAT -c '%u:%a' "$manifest" 2>/dev/null)" = 0:600 ] || return 1 - awk ' - NR == 1 { - if ($0 != "# virt-cloud-nft-manifest-v1") exit 1 - next - } - $0 == "include \"/etc/nftables.d/virt-cloud-wg-fabric.nft\"" { - if (++fabric != 1 || seen[$0]++) exit 1 - next - } - $0 ~ /^include "\/etc\/nftables[.]d\/virt-cloud-vpc-[0-9a-f]+[.]nft"$/ { - token=$0 - sub(/^include "\/etc\/nftables[.]d\/virt-cloud-vpc-/, "", token) - sub(/[.]nft"$/, "", token) - if (length(token) != 11 || token !~ /^[0-9a-f]+$/ || seen[$0]++) exit 1 - next - } - { exit 1 } - END { if (NR < 2 || fabric != 1) exit 1 } - ' "$manifest" -} -'''.strip() - - _NMCLI_FILENAME_RESOLVER_SH = r''' vc_nm_filename_for_uuid() { vc_nm_filename_wanted=$1 @@ -2373,50 +2214,6 @@ vc_nm_filename_for_uuid() { _FIREWALL_MODE_GUARD_SH = r''' -vc_firewall_effective_config() { - vc_fw_backend=nftables - vc_fw_table_owner=yes - if [ -e /etc/firewalld/firewalld.conf ]; then - [ -f /etc/firewalld/firewalld.conf ] && - [ ! -L /etc/firewalld/firewalld.conf ] || return 1 - vc_fw_values=$(awk -F= ' - /^[[:space:]]*#/ {next} - /^[[:space:]]*FirewallBackend[[:space:]]*=/ { - value=$2; gsub(/^[[:space:]]+|[[:space:]]+$/, "", value) - backend=value; backend_count++ - } - /^[[:space:]]*NftablesTableOwner[[:space:]]*=/ { - value=$2; gsub(/^[[:space:]]+|[[:space:]]+$/, "", value) - owner=value; owner_count++ - } - END { - if (backend_count > 1 || owner_count > 1) exit 1 - printf "%s|%s", backend_count ? backend : "nftables", owner_count ? owner : "yes" - } - ' /etc/firewalld/firewalld.conf 2>/dev/null) || return 1 - vc_fw_backend=${vc_fw_values%%|*} - vc_fw_table_owner=${vc_fw_values#*|} - fi - [ "$vc_fw_backend" = nftables ] && [ "$vc_fw_table_owner" = yes ] -} -vc_firewalld_live_nft_ready() { - [ -n "${NFT:-}" ] || return 1 - vc_nft_dd=$(find_tool /usr/bin/dd /bin/dd /usr/sbin/dd /sbin/dd) || return 1 - # Keep runtime evidence bounded and private. Only the JSON table object is - # inspected; the ruleset is never emitted through readiness or logs. - vc_firewalld_json=$($NFT -j list table inet firewalld 2>/dev/null | "$vc_nft_dd" bs=262145 count=1 2>/dev/null) || return 1 - vc_firewalld_json_size=$(printf '%s' "$vc_firewalld_json" | wc -c | tr -d ' ') - [ "$vc_firewalld_json_size" -gt 0 ] 2>/dev/null && - [ "$vc_firewalld_json_size" -le 262144 ] 2>/dev/null || return 1 - vc_firewalld_compact=$(printf '%s' "$vc_firewalld_json" | tr -d '[:space:]') || return 1 - vc_firewalld_table=$(printf '%s\n' "$vc_firewalld_compact" | - grep -Eo '"table":\{[^{{}}]*\}' | - grep -E '"family":"inet"' | - grep -E '"name":"firewalld"') || return 1 - [ "$(printf '%s\n' "$vc_firewalld_table" | awk 'NF {count++} END {print count + 0}')" = 1 ] || return 1 - printf '%s\n' "$vc_firewalld_table" | grep -Eq '"flags":\[[^]]*"owner"' || return 1 - printf '%s\n' "$vc_firewalld_table" | grep -Eq '"flags":\[[^]]*"persist"' -} vc_underlay_route_fields() { vc_route_expected_from=${1:-} awk -v expected_from="$vc_route_expected_from" ' @@ -2480,679 +2277,157 @@ vc_underlay_route_path() { printf '%s' "$vc_route_plain_fields" } vc_firewall_mode_ready() { - vc_nft_active=$($SYSTEMCTL is-active nftables 2>/dev/null || true) - vc_nft_enabled=$($SYSTEMCTL is-enabled nftables 2>/dev/null || true) - vc_nft_load=$($SYSTEMCTL show -p LoadState --value nftables 2>/dev/null || true) - vc_firewalld_active=$($SYSTEMCTL is-active firewalld 2>/dev/null || true) - vc_firewalld_enabled=$($SYSTEMCTL is-enabled firewalld 2>/dev/null || true) + [ "$FIREWALL_MODE" = native_nftables ] || return 1 + [ "$($SYSTEMCTL show -p LoadState --value nftables 2>/dev/null || true)" = loaded ] || return 1 + [ "$($SYSTEMCTL is-active nftables 2>/dev/null || true)" = active ] || return 1 + [ "$($SYSTEMCTL is-enabled nftables 2>/dev/null || true)" = enabled ] || return 1 vc_firewalld_load=$($SYSTEMCTL show -p LoadState --value firewalld 2>/dev/null || true) - vc_nft_absent_or_disabled=no - if [ "$vc_nft_load" = not-found ]; then - vc_nft_absent_or_disabled=yes - elif [ "$vc_nft_active" = inactive ]; then - case "$vc_nft_enabled" in disabled|masked) vc_nft_absent_or_disabled=yes ;; esac - fi - vc_firewalld_absent_or_masked=no if [ "$vc_firewalld_load" = not-found ]; then - vc_firewalld_absent_or_masked=yes - elif [ "$vc_firewalld_active" = inactive ] && [ "$vc_firewalld_enabled" = masked ]; then - vc_firewalld_absent_or_masked=yes - fi - case "$FIREWALL_MODE" in - native_nftables) - [ "$vc_nft_load" = loaded ] || return 1 - [ "$vc_nft_active" = active ] || return 1 - [ "$vc_nft_enabled" = enabled ] || return 1 - [ "$vc_firewalld_absent_or_masked" = yes ] || return 1 - ;; - firewalld_nft) - [ "$vc_firewalld_load" = loaded ] || return 1 - [ "$vc_firewalld_active" = active ] || return 1 - [ "$vc_firewalld_enabled" = enabled ] || return 1 - [ "$vc_nft_absent_or_disabled" = yes ] || return 1 - [ -n "${FIREWALL_CMD:-}" ] || return 1 - [ "$($FIREWALL_CMD --state 2>/dev/null || true)" = running ] || return 1 - vc_firewall_effective_config || return 1 - vc_firewalld_live_nft_ready || return 1 - ;; - *) return 1 ;; - esac - return 0 -} -vc_firewall_zone() { - vc_zone_scope=$1 - vc_zone_iface=${2:-$IFACE} - case "$vc_zone_scope" in runtime|permanent) : ;; *) return 1 ;; esac - if [ "$vc_zone_scope" = permanent ]; then - vc_zone_list=$($FIREWALL_CMD --permanent --get-zones 2>/dev/null) || return 1 - else - vc_zone_list=$($FIREWALL_CMD --get-zones 2>/dev/null) || return 1 - fi - vc_zone_count=0 - vc_zone_match=none - for vc_zone in $vc_zone_list; do - case "$vc_zone" in ''|*[!A-Za-z0-9_-]*) return 1 ;; esac - [ "$(printf '%s' "$vc_zone" | awk 'length($0)<=64')" = "$vc_zone" ] || return 1 - vc_zone_count=$((vc_zone_count + 1)) - [ "$vc_zone_count" -le 64 ] || return 1 - if [ "$vc_zone_scope" = permanent ]; then - if $FIREWALL_CMD --permanent --zone="$vc_zone" --query-interface="$vc_zone_iface" >/dev/null 2>&1; then - [ "$vc_zone_match" = none ] || return 1 - vc_zone_match=$vc_zone - else - vc_zone_status=$? - [ "$vc_zone_status" = 1 ] || return 1 - fi - elif $FIREWALL_CMD --zone="$vc_zone" --query-interface="$vc_zone_iface" >/dev/null 2>&1; then - [ "$vc_zone_match" = none ] || return 1 - vc_zone_match=$vc_zone - else - vc_zone_status=$? - [ "$vc_zone_status" = 1 ] || return 1 - fi - done - [ "$vc_zone_count" -gt 0 ] || return 1 - printf '%s' "$vc_zone_match" -} -vc_firewall_zone_exists() { - vc_zone_scope=$1 - vc_zone_wanted=$2 - case "$vc_zone_wanted" in ''|*[!A-Za-z0-9_-]*) return 1 ;; esac - if [ "$vc_zone_scope" = permanent ]; then - vc_zone_values=$($FIREWALL_CMD --permanent --get-zones 2>/dev/null) || return 1 - elif [ "$vc_zone_scope" = runtime ]; then - vc_zone_values=$($FIREWALL_CMD --get-zones 2>/dev/null) || return 1 - else - return 1 + return 0 fi - vc_zone_value_count=0 - vc_zone_value_found=no - for vc_zone_value in $vc_zone_values; do - case "$vc_zone_value" in ''|*[!A-Za-z0-9_-]*) return 1 ;; esac - [ "$(printf '%s' "$vc_zone_value" | awk 'length($0)<=64')" = "$vc_zone_value" ] || return 1 - vc_zone_value_count=$((vc_zone_value_count + 1)) - [ "$vc_zone_value_count" -le 64 ] || return 1 - [ "$vc_zone_value" != "$vc_zone_wanted" ] || vc_zone_value_found=yes - done - [ "$vc_zone_value_count" -gt 0 ] && [ "$vc_zone_value_found" = yes ] + [ "$vc_firewalld_load" = masked ] || return 1 + [ "$($SYSTEMCTL is-active firewalld 2>/dev/null || true)" = inactive ] || return 1 + [ "$($SYSTEMCTL is-enabled firewalld 2>/dev/null || true)" = masked ] } -vc_firewall_quarantined() { - vc_quarantine_iface=${1:-$IFACE} - vc_quarantine_allow_default=${2:-no} - vc_quarantine_permanent=$(vc_firewall_zone permanent "$vc_quarantine_iface") || return 1 - [ "$vc_quarantine_permanent" = none ] || return 1 - vc_quarantine_runtime=$(vc_firewall_zone runtime "$vc_quarantine_iface") || return 1 - [ "$vc_quarantine_runtime" != trusted ] || return 1 - if [ "$vc_quarantine_runtime" = none ]; then +vc_native_transaction_journal_valid_or_absent() { + vc_journal=/etc/virt-cloud/.wg-fabric-txn + if [ ! -e "$vc_journal" ] && [ ! -L "$vc_journal" ]; then return 0 fi - [ "$vc_quarantine_allow_default" = yes ] || return 1 - vc_quarantine_default=$($FIREWALL_CMD --get-default-zone 2>/dev/null) || return 1 - vc_firewall_zone_exists runtime "$vc_quarantine_default" || return 1 - [ "$vc_quarantine_default" != trusted ] || return 1 - [ "$vc_quarantine_runtime" = "$vc_quarantine_default" ] -} -vc_fabric_emergency_owner_read() { - VC_FABRIC_EMERGENCY_OWNER_TOKEN= - vc_emergency_owner_path=${EMERGENCY_OWNER:-/run/virt-cloud/wg-fabric-emergency.owner} - [ -f "$vc_emergency_owner_path" ] && [ ! -L "$vc_emergency_owner_path" ] || return 1 - [ "$($STAT -c '%u:%a' "$vc_emergency_owner_path" 2>/dev/null)" = 0:600 ] || return 1 - vc_emergency_owner_line=$(cat "$vc_emergency_owner_path" 2>/dev/null) || return 1 - VC_FABRIC_EMERGENCY_OWNER_TOKEN=$(printf '%s\n' "$vc_emergency_owner_line" | awk -F: ' - NF == 2 && $1 == "virt-cloud-fabric-emergency-v2" && - length($2) == 24 && $2 ~ /^[A-Za-z0-9]+$/ {print $2; found=1} - END {exit found ? 0 : 1} - ') || return 1 - [ -n "$VC_FABRIC_EMERGENCY_OWNER_TOKEN" ] -} -vc_fabric_emergency_owner_clear() { - vc_emergency_owner_expected=$1 - vc_fabric_emergency_owner_read || return 1 - [ "$VC_FABRIC_EMERGENCY_OWNER_TOKEN" = "$vc_emergency_owner_expected" ] || return 1 - vc_emergency_owner_path=${EMERGENCY_OWNER:-/run/virt-cloud/wg-fabric-emergency.owner} - rm -f "$vc_emergency_owner_path" || return 1 - [ ! -e "$vc_emergency_owner_path" ] && [ ! -L "$vc_emergency_owner_path" ] -} -vc_fabric_emergency_shape() { - # This table is deliberately independent of both NetworkManager and - # firewalld. It has a later base-chain priority than the persistent Fabric - # ACL, so its drop remains effective even while the interface is still in - # firewalld's trusted zone. Its random token is published in a root-only - # owner intent before the nft transaction; fixed-marker lookalikes are - # external even when their shape is otherwise identical. - vc_emergency_shape_token=$2 - case "$vc_emergency_shape_token" in ''|*[!A-Za-z0-9]*) return 1 ;; esac - [ "$(printf '%s' "$vc_emergency_shape_token" | wc -c | tr -d ' ')" = 24 ] || return 1 - printf '%s\n' "$1" | awk -v token="$vc_emergency_shape_token" ' - function trim(value) { - gsub(/^[[:space:]]+|[[:space:]]+$/, "", value) - return value - } + [ -n "${STAT:-}" ] && [ -n "${AWK:-}" ] || return 1 + [ -d "$vc_journal" ] && [ ! -L "$vc_journal" ] || return 1 + [ "$($STAT -c '%u:%g:%a' "$vc_journal" 2>/dev/null)" = 0:0:700 ] || return 1 + for vc_journal_file in state meta; do + [ -f "$vc_journal/$vc_journal_file" ] && + [ ! -L "$vc_journal/$vc_journal_file" ] || return 1 + [ "$($STAT -c '%u:%g:%a' "$vc_journal/$vc_journal_file" 2>/dev/null)" = 0:0:600 ] || return 1 + done + vc_journal_state_size=$($STAT -c '%s' "$vc_journal/state" 2>/dev/null) || return 1 + vc_journal_meta_size=$($STAT -c '%s' "$vc_journal/meta" 2>/dev/null) || return 1 + case "$vc_journal_state_size:$vc_journal_meta_size" in + *[!0-9:]*|:*) return 1 ;; + esac + [ "$vc_journal_state_size" -le 32 ] && + [ "$vc_journal_meta_size" -le 65536 ] || return 1 + vc_journal_state=$($AWK ' + NR == 1 && NF == 1 && + $0 ~ /^(preparing|ready|committed|cleanup_helper)$/ {value=$0; ok=1} + {rows++} + END {if (ok && rows == 1) print value; else exit 1} + ' "$vc_journal/state") || return 1 + $AWK -F= -v state="$vc_journal_state" ' + function val() {return substr($0, index($0, "=") + 1)} { - line=trim($0) - if (line == "") next - if (line == "table inet virt_cloud_fabric_emergency {") {table++; next} - if (line == "chain input {") {input_chain++; current="input"; next} - if (line == "chain output {") {output_chain++; current="output"; next} - if (line == "chain forward {") {forward_chain++; current="forward"; next} - if (line == "}") {closes++; current=""; next} - plain="type filter hook " current " priority 300; policy accept;" - symbolic="type filter hook " current " priority filter + 300; policy accept;" - if (current != "" && (line == plain || line == symbolic)) { - meta[current]++; next - } - ingress="iifname \"wg-fabric0\" counter drop comment \"virt-cloud-fabric-emergency-v2:" token "\"" - egress="oifname \"wg-fabric0\" counter drop comment \"virt-cloud-fabric-emergency-v2:" token "\"" - vpc_ingress="iifname \"vcv-*\" counter drop comment \"virt-cloud-fabric-emergency-v2:" token "\"" - vpc_egress="oifname \"vcv-*\" counter drop comment \"virt-cloud-fabric-emergency-v2:" token "\"" - if ((current == "input" || current == "forward") && line == ingress) { - ingress_rules[current]++; next - } - if ((current == "output" || current == "forward") && line == egress) { - egress_rules[current]++; next - } - if ((current == "input" || current == "forward") && line == vpc_ingress) { - vpc_ingress_rules[current]++; next - } - if ((current == "output" || current == "forward") && line == vpc_egress) { - vpc_egress_rules[current]++; next + rows++ + if (length($0) > 1024 || index($0, "=") < 2) {bad=1; next} + value=val() + if ($1=="schema") {schema++; if (value!="1") bad=1; next} + if ($1=="phase") {phase++; if (value!="preparing") bad=1; next} + if ($1=="firewall_mode") {firewall++; if (value!="native_nftables") bad=1; next} + if ($1=="plan_digest") {digest++; if (length(value)!=64 || value !~ /^[0-9a-f]+$/) bad=1; next} + if ($1=="deployment_mode") {mode++; if (value!="networkmanager_nmcli" && value!="ifupdown") bad=1; next} + if ($1=="connection_uuid") { + uuid++ + candidate=value; gsub(/-/, "", candidate) + if (length(value)!=36 || length(candidate)!=32 || + candidate !~ /^[0-9A-Fa-f]+$/ || + substr(value,9,1)!="-" || substr(value,14,1)!="-" || + substr(value,19,1)!="-" || substr(value,24,1)!="-") bad=1 + next } + if ($1=="nft_main") {nft_main++; if (value!="/etc/nftables.conf" && value!="/etc/sysconfig/nftables.conf") bad=1; next} + if ($1=="iface_before") {iface++; if (value!="yes" && value!="no") bad=1; iface_value=value; next} + if ($1=="iface_up_before") {iface_up++; if (value!="yes" && value!="no") bad=1; iface_up_value=value; next} + if ($1=="rp_filter_before") {rpf++; if (value!="missing" && value!="0" && value!="1" && value!="2") bad=1; rpf_value=value; next} + if ($1=="nft_before") {nft++; if (value!="yes" && value!="no") bad=1; nft_value=value; next} + if ($1=="service_enabled_before") {service_enabled++; if (value=="" || value ~ /[^A-Za-z0-9_-]/) bad=1; next} + if ($1=="service_active_before") {service_active++; if (value!="active" && value!="inactive" && value!="activating" && value!="deactivating" && value!="unknown") bad=1; next} + if ($1=="timer_enabled_before") {timer_enabled++; if (value=="" || value ~ /[^A-Za-z0-9_-]/) bad=1; next} + if ($1=="timer_active_before") {timer_active++; if (value!="active" && value!="inactive" && value!="activating" && value!="deactivating" && value!="unknown") bad=1; next} bad=1 } END { - exit (!bad && table==1 && input_chain==1 && output_chain==1 && - forward_chain==1 && closes==4 && meta["input"]==1 && - meta["output"]==1 && meta["forward"]==1 && - ingress_rules["input"]==1 && egress_rules["output"]==1 && - ingress_rules["forward"]==1 && - egress_rules["forward"]==1 && - vpc_ingress_rules["input"]==1 && - vpc_egress_rules["output"]==1 && - vpc_ingress_rules["forward"]==1 && - vpc_egress_rules["forward"]==1) ? 0 : 1 - } - ' -} -vc_fabric_emergency_bridge_shape() { - vc_emergency_shape_token=$2 - case "$vc_emergency_shape_token" in ''|*[!A-Za-z0-9]*) return 1 ;; esac - [ "$(printf '%s' "$vc_emergency_shape_token" | wc -c | tr -d ' ')" = 24 ] || return 1 - printf '%s\n' "$1" | awk -v token="$vc_emergency_shape_token" ' - function trim(value) {gsub(/^[[:space:]]+|[[:space:]]+$/, "", value); return value} - { - line=trim($0) - if (line == "") next - if (line == "table bridge virt_cloud_fabric_emergency {") {table++; next} - if (line == "chain input {") {input_chain++; current="input"; next} - if (line == "chain output {") {output_chain++; current="output"; next} - if (line == "chain forward {") {forward_chain++; current="forward"; next} - if (line == "}") {closes++; current=""; next} - plain="type filter hook " current " priority 300; policy accept;" - symbolic="type filter hook " current " priority filter + 300; policy accept;" - if (current != "" && (line == plain || line == symbolic)) {meta[current]++; next} - ingress="ibrname \"vcv-*\" counter drop comment \"virt-cloud-fabric-emergency-v2:" token "\"" - egress="obrname \"vcv-*\" counter drop comment \"virt-cloud-fabric-emergency-v2:" token "\"" - canonical_ingress="meta " ingress - canonical_egress="meta " egress - if ((current == "input" || current == "forward") && - (line == ingress || line == canonical_ingress)) { - ingress_rules[current]++; next - } - if ((current == "output" || current == "forward") && - (line == egress || line == canonical_egress)) { - egress_rules[current]++; next + if (bad || schema!=1 || firewall!=1 || digest!=1) exit 1 + if (state=="preparing" && phase==1) { + exit (rows==4 && mode==0 && uuid==0 && nft_main==0 && + iface==0 && iface_up==0 && rpf==0 && nft==0 && + service_enabled==0 && service_active==0 && + timer_enabled==0 && timer_active==0) ? 0 : 1 } - bad=1 - } - END { - exit (!bad && table==1 && input_chain==1 && output_chain==1 && - forward_chain==1 && closes==4 && meta["input"]==1 && - meta["output"]==1 && meta["forward"]==1 && - ingress_rules["input"]==1 && egress_rules["output"]==1 && - ingress_rules["forward"]==1 && egress_rules["forward"]==1) ? 0 : 1 + if (phase!=0 || rows!=14 || mode!=1 || uuid!=1 || nft_main!=1 || + iface!=1 || iface_up!=1 || rpf!=1 || nft!=1 || + service_enabled!=1 || service_active!=1 || + timer_enabled!=1 || timer_active!=1) exit 1 + if ((iface_value=="no" && (iface_up_value!="no" || rpf_value!="missing")) || + (iface_value=="yes" && rpf_value=="missing") || + (iface_value=="no" && nft_value!="no")) exit 1 } - ' -} -vc_fabric_emergency_install() { - [ "$FIREWALL_MODE" = firewalld_nft ] || return 0 - [ -n "${NFT:-}" ] && [ -n "${CMP:-}" ] || return 1 - VC_FABRIC_EMERGENCY_SNAPSHOT= - VC_FABRIC_EMERGENCY_BRIDGE_SNAPSHOT= - VC_FABRIC_EMERGENCY_TOKEN= - vc_emergency_owner_path=${EMERGENCY_OWNER:-/run/virt-cloud/wg-fabric-emergency.owner} - vc_emergency_owner_present=no - if [ -e "$vc_emergency_owner_path" ] || [ -L "$vc_emergency_owner_path" ]; then - vc_fabric_emergency_owner_read || return 1 - vc_emergency_owner_present=yes - vc_emergency_token=$VC_FABRIC_EMERGENCY_OWNER_TOKEN - fi - vc_emergency_tables=$($NFT list tables 2>/dev/null) || return 1 - vc_emergency_inet_present=no - vc_emergency_bridge_present=no - if printf '%s\n' "$vc_emergency_tables" | - grep -Eq 'table inet virt_cloud_fabric_emergency([[:space:]]|$)'; then - vc_emergency_inet_present=yes - fi - if printf '%s\n' "$vc_emergency_tables" | - grep -Eq 'table bridge virt_cloud_fabric_emergency([[:space:]]|$)'; then - vc_emergency_bridge_present=yes - fi - if [ "$vc_emergency_owner_present" = yes ] && - [ "$vc_emergency_inet_present" = yes ] && - [ "$vc_emergency_bridge_present" = yes ]; then - vc_emergency_live=$($NFT -s list table inet virt_cloud_fabric_emergency 2>/dev/null) || return 1 - vc_emergency_bridge_live=$($NFT -s list table bridge virt_cloud_fabric_emergency 2>/dev/null) || return 1 - vc_fabric_emergency_shape "$vc_emergency_live" "$vc_emergency_token" || return 1 - vc_fabric_emergency_bridge_shape "$vc_emergency_bridge_live" "$vc_emergency_token" || return 1 - elif [ "$vc_emergency_owner_present" = yes ] && - [ "$vc_emergency_inet_present" = no ] && - [ "$vc_emergency_bridge_present" = no ]; then - # A crash either preceded nft commit or followed a verified delete. - # Only the exact root-owned intent may be cleared before retrying. - vc_fabric_emergency_owner_clear "$vc_emergency_token" || return 1 - vc_emergency_owner_present=no - elif [ "$vc_emergency_inet_present" = yes ] || - [ "$vc_emergency_bridge_present" = yes ]; then - # Never adopt a same-name table without the independent random intent, - # even if an external table deliberately copies the complete shape. - return 1 - fi - if [ "$vc_emergency_owner_present" = no ]; then - vc_emergency_owner_tmp=$(mktemp /run/virt-cloud/.fabric-emergency-owner.XXXXXXXXXXXXXXXXXXXXXXXX) || return 1 - vc_emergency_token=${vc_emergency_owner_tmp##*.} - case "$vc_emergency_token" in ''|*[!A-Za-z0-9]*) - rm -f "$vc_emergency_owner_tmp" - return 1 - ;; - esac - if [ "$(printf '%s' "$vc_emergency_token" | wc -c | tr -d ' ')" != 24 ]; then - rm -f "$vc_emergency_owner_tmp" - return 1 - fi - printf '%s:%s\n' virt-cloud-fabric-emergency-v2 "$vc_emergency_token" > "$vc_emergency_owner_tmp" || { - rm -f "$vc_emergency_owner_tmp"; return 1; - } - chmod 600 "$vc_emergency_owner_tmp" || { - rm -f "$vc_emergency_owner_tmp"; return 1; - } - [ ! -e "$vc_emergency_owner_path" ] && [ ! -L "$vc_emergency_owner_path" ] || { - rm -f "$vc_emergency_owner_tmp"; return 1; - } - # Publishing by hard link cannot overwrite a foreign file that races - # the existence check, unlike a plain mv to the fixed owner path. - ln "$vc_emergency_owner_tmp" "$vc_emergency_owner_path" 2>/dev/null || { - rm -f "$vc_emergency_owner_tmp"; return 1; - } - rm -f "$vc_emergency_owner_tmp" || return 1 - vc_fabric_emergency_owner_read || return 1 - [ "$VC_FABRIC_EMERGENCY_OWNER_TOKEN" = "$vc_emergency_token" ] || return 1 - vc_emergency_tmp=$(mktemp /run/virt-cloud/.fabric-emergency.XXXXXX) || return 1 - cat > "$vc_emergency_tmp" <<__VC_FABRIC_EMERGENCY__ -table inet virt_cloud_fabric_emergency { - chain input { - type filter hook input priority 300; policy accept; - iifname "wg-fabric0" counter drop comment "virt-cloud-fabric-emergency-v2:$vc_emergency_token" - iifname "vcv-*" counter drop comment "virt-cloud-fabric-emergency-v2:$vc_emergency_token" - } - chain output { - type filter hook output priority 300; policy accept; - oifname "wg-fabric0" counter drop comment "virt-cloud-fabric-emergency-v2:$vc_emergency_token" - oifname "vcv-*" counter drop comment "virt-cloud-fabric-emergency-v2:$vc_emergency_token" - } - chain forward { - type filter hook forward priority 300; policy accept; - iifname "wg-fabric0" counter drop comment "virt-cloud-fabric-emergency-v2:$vc_emergency_token" - oifname "wg-fabric0" counter drop comment "virt-cloud-fabric-emergency-v2:$vc_emergency_token" - iifname "vcv-*" counter drop comment "virt-cloud-fabric-emergency-v2:$vc_emergency_token" - oifname "vcv-*" counter drop comment "virt-cloud-fabric-emergency-v2:$vc_emergency_token" - } -} -table bridge virt_cloud_fabric_emergency { - chain input { - type filter hook input priority 300; policy accept; - ibrname "vcv-*" counter drop comment "virt-cloud-fabric-emergency-v2:$vc_emergency_token" - } - chain output { - type filter hook output priority 300; policy accept; - obrname "vcv-*" counter drop comment "virt-cloud-fabric-emergency-v2:$vc_emergency_token" - } - chain forward { - type filter hook forward priority 300; policy accept; - ibrname "vcv-*" counter drop comment "virt-cloud-fabric-emergency-v2:$vc_emergency_token" - obrname "vcv-*" counter drop comment "virt-cloud-fabric-emergency-v2:$vc_emergency_token" - } + ' "$vc_journal/meta" } -__VC_FABRIC_EMERGENCY__ - chmod 600 "$vc_emergency_tmp" || { rm -f "$vc_emergency_tmp"; return 1; } - $NFT -c -f "$vc_emergency_tmp" >/dev/null 2>&1 || { - rm -f "$vc_emergency_tmp" - vc_fabric_emergency_owner_clear "$vc_emergency_token" || true - return 1 - } - vc_emergency_precommit=$($NFT list tables 2>/dev/null) || { - rm -f "$vc_emergency_tmp"; return 1; - } - if printf '%s\n' "$vc_emergency_precommit" | - grep -Eq 'table (inet|bridge) virt_cloud_fabric_emergency([[:space:]]|$)'; then - rm -f "$vc_emergency_tmp" - vc_fabric_emergency_owner_clear "$vc_emergency_token" || true - return 1 - fi - $NFT -f "$vc_emergency_tmp" >/dev/null 2>&1 || { - # Local nft batches are atomic, but preserve the intent on an - # ambiguous command outcome so the next locked run can inspect it. - rm -f "$vc_emergency_tmp"; return 1; - } - rm -f "$vc_emergency_tmp" || return 1 - vc_emergency_live=$($NFT -s list table inet virt_cloud_fabric_emergency 2>/dev/null) || return 1 - vc_emergency_bridge_live=$($NFT -s list table bridge virt_cloud_fabric_emergency 2>/dev/null) || return 1 - vc_fabric_emergency_shape "$vc_emergency_live" "$vc_emergency_token" || return 1 - vc_fabric_emergency_bridge_shape "$vc_emergency_bridge_live" "$vc_emergency_token" || return 1 - fi - [ "$(printf '%s' "$vc_emergency_live" | wc -c | tr -d ' ')" -le 8192 ] 2>/dev/null || return 1 - [ "$(printf '%s' "$vc_emergency_bridge_live" | wc -c | tr -d ' ')" -le 8192 ] 2>/dev/null || return 1 - vc_fabric_emergency_owner_read || return 1 - [ "$VC_FABRIC_EMERGENCY_OWNER_TOKEN" = "$vc_emergency_token" ] || return 1 - VC_FABRIC_EMERGENCY_SNAPSHOT=$vc_emergency_live - VC_FABRIC_EMERGENCY_BRIDGE_SNAPSHOT=$vc_emergency_bridge_live - VC_FABRIC_EMERGENCY_TOKEN=$vc_emergency_token - [ -n "$VC_FABRIC_EMERGENCY_SNAPSHOT" ] && - [ -n "$VC_FABRIC_EMERGENCY_BRIDGE_SNAPSHOT" ] -} -vc_fabric_emergency_adopt() { - # Native nftables normally needs no independent emergency table. It must, - # however, recover a guard left by a failed firewalld-mode transaction - # before the configured backend was corrected. Adoption is read-only: - # never create or modify a table here, and only remember a table backed by - # the exact root-only random owner intent and complete expected shape. - [ "$FIREWALL_MODE" = native_nftables ] || return 0 - [ -n "${NFT:-}" ] && [ -n "${CMP:-}" ] || return 1 - VC_FABRIC_EMERGENCY_SNAPSHOT= - VC_FABRIC_EMERGENCY_BRIDGE_SNAPSHOT= - VC_FABRIC_EMERGENCY_TOKEN= - vc_emergency_owner_path=${EMERGENCY_OWNER:-/run/virt-cloud/wg-fabric-emergency.owner} - vc_emergency_owner_present=no - if [ -e "$vc_emergency_owner_path" ] || [ -L "$vc_emergency_owner_path" ]; then - vc_fabric_emergency_owner_read || return 1 - vc_emergency_owner_present=yes - vc_emergency_token=$VC_FABRIC_EMERGENCY_OWNER_TOKEN - fi - vc_emergency_tables=$($NFT list tables 2>/dev/null) || return 1 - vc_emergency_inet_present=no - vc_emergency_bridge_present=no - if printf '%s\n' "$vc_emergency_tables" | - grep -Eq 'table inet virt_cloud_fabric_emergency([[:space:]]|$)'; then - vc_emergency_inet_present=yes - fi - if printf '%s\n' "$vc_emergency_tables" | - grep -Eq 'table bridge virt_cloud_fabric_emergency([[:space:]]|$)'; then - vc_emergency_bridge_present=yes - fi - if [ "$vc_emergency_owner_present" = no ]; then - # A same-name table without the independent intent is external even if - # it copies every rule and comment. Native mode must not touch it. - [ "$vc_emergency_inet_present" = no ] && - [ "$vc_emergency_bridge_present" = no ] || return 1 - return 0 - fi - if [ "$vc_emergency_inet_present" = no ] && - [ "$vc_emergency_bridge_present" = no ]; then - # Crash-before-nft-commit (or crash-after-verified-delete). Re-prove - # both the exact intent and absence immediately before clearing it. - vc_fabric_emergency_owner_read || return 1 - [ "$VC_FABRIC_EMERGENCY_OWNER_TOKEN" = "$vc_emergency_token" ] || return 1 - vc_emergency_tables=$($NFT list tables 2>/dev/null) || return 1 - if printf '%s\n' "$vc_emergency_tables" | - grep -Eq 'table (inet|bridge) virt_cloud_fabric_emergency([[:space:]]|$)'; then - return 1 - fi - vc_fabric_emergency_owner_clear "$vc_emergency_token" || return 1 - vc_emergency_tables=$($NFT list tables 2>/dev/null) || return 1 - ! printf '%s\n' "$vc_emergency_tables" | - grep -Eq 'table (inet|bridge) virt_cloud_fabric_emergency([[:space:]]|$)' - return - fi - [ "$vc_emergency_inet_present" = yes ] && - [ "$vc_emergency_bridge_present" = yes ] || return 1 - vc_emergency_live=$($NFT -s list table inet virt_cloud_fabric_emergency 2>/dev/null) || return 1 - vc_emergency_bridge_live=$($NFT -s list table bridge virt_cloud_fabric_emergency 2>/dev/null) || return 1 - vc_fabric_emergency_shape "$vc_emergency_live" "$vc_emergency_token" || return 1 - vc_fabric_emergency_bridge_shape "$vc_emergency_bridge_live" "$vc_emergency_token" || return 1 - [ "$(printf '%s' "$vc_emergency_live" | wc -c | tr -d ' ')" -le 8192 ] 2>/dev/null || return 1 - [ "$(printf '%s' "$vc_emergency_bridge_live" | wc -c | tr -d ' ')" -le 8192 ] 2>/dev/null || return 1 - vc_fabric_emergency_owner_read || return 1 - [ "$VC_FABRIC_EMERGENCY_OWNER_TOKEN" = "$vc_emergency_token" ] || return 1 - vc_emergency_precommit=$($NFT -s list table inet virt_cloud_fabric_emergency 2>/dev/null) || return 1 - vc_emergency_bridge_precommit=$($NFT -s list table bridge virt_cloud_fabric_emergency 2>/dev/null) || return 1 - vc_fabric_emergency_shape "$vc_emergency_precommit" "$vc_emergency_token" || return 1 - vc_fabric_emergency_bridge_shape "$vc_emergency_bridge_precommit" "$vc_emergency_token" || return 1 - [ "$vc_emergency_precommit" = "$vc_emergency_live" ] || return 1 - [ "$vc_emergency_bridge_precommit" = "$vc_emergency_bridge_live" ] || return 1 - VC_FABRIC_EMERGENCY_SNAPSHOT=$vc_emergency_precommit - VC_FABRIC_EMERGENCY_BRIDGE_SNAPSHOT=$vc_emergency_bridge_precommit - VC_FABRIC_EMERGENCY_TOKEN=$vc_emergency_token - [ -n "$VC_FABRIC_EMERGENCY_SNAPSHOT" ] && - [ -n "$VC_FABRIC_EMERGENCY_BRIDGE_SNAPSHOT" ] -} -vc_fabric_native_adopt_vpc_state_empty() { - # Native may clear a firewalld transaction guard only when there is no - # durable VPC dataplane to migrate. Moving an existing VPC between the - # two persistence owners is a maintenance-window transaction; merely - # accepting its old state here would mix manifest/include ownership and - # could expose a bridge with stale ACL semantics. - [ "$FIREWALL_MODE" = native_nftables ] || return 0 - if [ -z "${VC_FABRIC_EMERGENCY_SNAPSHOT:-}" ] && - [ -z "${VC_FABRIC_EMERGENCY_BRIDGE_SNAPSHOT:-}" ] && - [ -z "${VC_FABRIC_EMERGENCY_TOKEN:-}" ]; then - return 0 - fi - [ -n "${VC_FABRIC_EMERGENCY_SNAPSHOT:-}" ] && - [ -n "${VC_FABRIC_EMERGENCY_BRIDGE_SNAPSHOT:-}" ] && - [ -n "${VC_FABRIC_EMERGENCY_TOKEN:-}" ] || return 1 - vc_native_vpc_state_dir=${VPC_STATE_DIR:-/etc/virt-cloud/vpc-dataplane} - [ ! -L "$vc_native_vpc_state_dir" ] || return 1 - if [ ! -e "$vc_native_vpc_state_dir" ]; then - return 0 - fi - [ -d "$vc_native_vpc_state_dir" ] && - [ "$($STAT -c '%u:%a' "$vc_native_vpc_state_dir" 2>/dev/null)" = 0:700 ] || +vc_legacy_firewalld_state_absent() { + [ -n "$NFT" ] || return 1 + vc_fabric_config=/etc/virt-cloud/wg-fabric0.conf + if [ -e "$vc_fabric_config" ] || [ -L "$vc_fabric_config" ]; then + [ -f "$vc_fabric_config" ] && [ ! -L "$vc_fabric_config" ] || return 1 + [ "$($STAT -c '%u:%g:%a' "$vc_fabric_config" 2>/dev/null)" = 0:0:600 ] || return 1 + vc_fabric_config_size=$($STAT -c '%s' "$vc_fabric_config" 2>/dev/null) || return 1 + case "$vc_fabric_config_size" in ''|*[!0-9]*) return 1 ;; esac + [ "$vc_fabric_config_size" -le 65536 ] || return 1 + $AWK -F'|' ' + $1=="firewall" {firewall++; if (NF!=2 || $2!="native_nftables") bad=1; next} + $1=="underlay" || $1=="firewall_rule" {bad=1; next} + END {exit (bad || firewall>1) ? 1 : 0} + ' "$vc_fabric_config" || return 1 + fi + [ ! -e /run/virt-cloud/wg-fabric-emergency.owner ] && + [ ! -L /run/virt-cloud/wg-fabric-emergency.owner ] || return 1 + [ ! -e /etc/virt-cloud/nftables.conf ] && + [ ! -L /etc/virt-cloud/nftables.conf ] || return 1 + vc_native_transaction_journal_valid_or_absent || return 1 + vc_vpc_transaction_count=0 + for vc_vpc_transaction in /etc/virt-cloud/.vpc-dataplane-*.txn; do + [ -e "$vc_vpc_transaction" ] || [ -L "$vc_vpc_transaction" ] || continue + vc_vpc_transaction_count=$((vc_vpc_transaction_count + 1)) + [ "$vc_vpc_transaction_count" -le 256 ] || return 1 return 1 - # A missing state file does not make orphan owner/proxy/snapshot data safe - # to reinterpret. Cover ordinary names and both POSIX hidden-name forms; - # unmatched globs are ignored only after an existence/symlink check. - for vc_native_vpc_entry in "$vc_native_vpc_state_dir"/* \ - "$vc_native_vpc_state_dir"/.[!.]* \ - "$vc_native_vpc_state_dir"/..?*; do - [ -e "$vc_native_vpc_entry" ] || [ -L "$vc_native_vpc_entry" ] || - continue + done + vc_vpc_legacy_snapshot_count=0 + for vc_vpc_legacy_snapshot in \ + /etc/virt-cloud/vpc-dataplane/*.nft.inet.live \ + /etc/virt-cloud/vpc-dataplane/*.nft.bridge.live \ + /etc/virt-cloud/vpc-dataplane/*.nft.arp.live; do + [ -e "$vc_vpc_legacy_snapshot" ] || + [ -L "$vc_vpc_legacy_snapshot" ] || continue + vc_vpc_legacy_snapshot_count=$((vc_vpc_legacy_snapshot_count + 1)) + [ "$vc_vpc_legacy_snapshot_count" -le 256 ] || return 1 return 1 done - return 0 -} -vc_fabric_emergency_remove() { - case "$FIREWALL_MODE" in - firewalld_nft) : ;; - native_nftables) - if [ -z "${VC_FABRIC_EMERGENCY_SNAPSHOT:-}" ] && - [ -z "${VC_FABRIC_EMERGENCY_BRIDGE_SNAPSHOT:-}" ] && - [ -z "${VC_FABRIC_EMERGENCY_TOKEN:-}" ]; then - return 0 - fi - ;; - *) return 1 ;; - esac - [ -n "${VC_FABRIC_EMERGENCY_SNAPSHOT:-}" ] || return 1 - [ -n "${VC_FABRIC_EMERGENCY_BRIDGE_SNAPSHOT:-}" ] || return 1 - [ -n "${VC_FABRIC_EMERGENCY_TOKEN:-}" ] || return 1 - vc_fabric_emergency_owner_read || return 1 - [ "$VC_FABRIC_EMERGENCY_OWNER_TOKEN" = "$VC_FABRIC_EMERGENCY_TOKEN" ] || return 1 - vc_emergency_live=$($NFT -s list table inet virt_cloud_fabric_emergency 2>/dev/null) || return 1 - vc_emergency_bridge_live=$($NFT -s list table bridge virt_cloud_fabric_emergency 2>/dev/null) || return 1 - vc_fabric_emergency_shape "$vc_emergency_live" "$VC_FABRIC_EMERGENCY_TOKEN" || return 1 - vc_fabric_emergency_bridge_shape "$vc_emergency_bridge_live" "$VC_FABRIC_EMERGENCY_TOKEN" || return 1 - [ "$vc_emergency_live" = "$VC_FABRIC_EMERGENCY_SNAPSHOT" ] || return 1 - [ "$vc_emergency_bridge_live" = "$VC_FABRIC_EMERGENCY_BRIDGE_SNAPSHOT" ] || return 1 - vc_emergency_tmp=$(mktemp /run/virt-cloud/.fabric-emergency-delete.XXXXXX) || return 1 - printf '%s\n' 'delete table inet virt_cloud_fabric_emergency' \ - 'delete table bridge virt_cloud_fabric_emergency' > "$vc_emergency_tmp" || { - rm -f "$vc_emergency_tmp"; return 1; - } - chmod 600 "$vc_emergency_tmp" || { rm -f "$vc_emergency_tmp"; return 1; } - $NFT -c -f "$vc_emergency_tmp" >/dev/null 2>&1 || { - rm -f "$vc_emergency_tmp"; return 1; - } - vc_emergency_precommit=$($NFT -s list table inet virt_cloud_fabric_emergency 2>/dev/null) || { - rm -f "$vc_emergency_tmp"; return 1; - } - vc_emergency_bridge_precommit=$($NFT -s list table bridge virt_cloud_fabric_emergency 2>/dev/null) || { - rm -f "$vc_emergency_tmp"; return 1; - } - vc_fabric_emergency_owner_read || { - rm -f "$vc_emergency_tmp"; return 1; - } - [ "$VC_FABRIC_EMERGENCY_OWNER_TOKEN" = "$VC_FABRIC_EMERGENCY_TOKEN" ] || { - rm -f "$vc_emergency_tmp"; return 1; - } - vc_fabric_emergency_shape "$vc_emergency_precommit" "$VC_FABRIC_EMERGENCY_TOKEN" || { - rm -f "$vc_emergency_tmp"; return 1; - } - vc_fabric_emergency_bridge_shape "$vc_emergency_bridge_precommit" "$VC_FABRIC_EMERGENCY_TOKEN" || { - rm -f "$vc_emergency_tmp"; return 1; - } - [ "$vc_emergency_precommit" = "$VC_FABRIC_EMERGENCY_SNAPSHOT" ] || { - rm -f "$vc_emergency_tmp"; return 1; - } - [ "$vc_emergency_bridge_precommit" = "$VC_FABRIC_EMERGENCY_BRIDGE_SNAPSHOT" ] || { - rm -f "$vc_emergency_tmp"; return 1; - } - $NFT -f "$vc_emergency_tmp" >/dev/null 2>&1 || { - rm -f "$vc_emergency_tmp"; return 1; - } - rm -f "$vc_emergency_tmp" || return 1 - vc_emergency_tables=$($NFT list tables 2>/dev/null) || return 1 - printf '%s\n' "$vc_emergency_tables" | - grep -Eq 'table (inet|bridge) virt_cloud_fabric_emergency([[:space:]]|$)' && return 1 - vc_fabric_emergency_owner_clear "$VC_FABRIC_EMERGENCY_TOKEN" || return 1 - VC_FABRIC_EMERGENCY_SNAPSHOT= - VC_FABRIC_EMERGENCY_BRIDGE_SNAPSHOT= - VC_FABRIC_EMERGENCY_TOKEN= - return 0 -} -vc_fabric_nm_profile_secure_hash() { - vc_nm_hash_path=$1 - [ -f "$vc_nm_hash_path" ] && [ ! -L "$vc_nm_hash_path" ] || return 1 - [ "$($STAT -c '%u:%g:%a' "$vc_nm_hash_path" 2>/dev/null)" = 0:0:600 ] || return 1 - vc_nm_hash_size=$($STAT -c '%s' "$vc_nm_hash_path" 2>/dev/null) || return 1 - [ "$vc_nm_hash_size" -gt 0 ] 2>/dev/null && - [ "$vc_nm_hash_size" -le 65536 ] 2>/dev/null || return 1 - "$VC_NM_SHA256" "$vc_nm_hash_path" 2>/dev/null | - awk 'NF==2 && length($1)==64 && $1 ~ /^[0-9a-f]+$/ {value=$1; count++} - END {if (count==1) print value; else exit 1}' -} -vc_fabric_nm_profile_semantics_match() { - vc_nm_compare_runtime=$1 - vc_nm_compare_shadow=$2 - # Closed grammar for profiles emitted by this platform. NM serializes an - # in-memory modification into its fixed /run shadow, reorders keys and - # drops comments; only the runtime zone and exact shadow metadata differ. - # Values (including the private key) stay entirely inside awk. Unknown or - # duplicate settings fail closed; no parsed key/value is ever emitted. - awk -v shadow="$vc_nm_compare_shadow" -v persistent="$NM_PROFILE" \ - -v firewall_mode="$FIREWALL_MODE" -v wanted_id="$CONNECTION" \ - -v wanted_uuid="$CONNECTION_UUID" ' - function host32(value, parts, octet, count, pos) { - if (split(value, parts, "/") != 2 || parts[2] != "32" || - split(parts[1], octet, ".") != 4) return 0 - for (pos=1; pos<=4; pos++) - if (octet[pos] !~ /^(0|[1-9][0-9]*)$/ || octet[pos] > 255) return 0 - return 1 - } - FNR==1 {file++; section=""} - { - if (file > 2 || FNR > 256 || length($0) > 4096) {bad=1; next} - line=$0 - if (line == "" || line ~ /^#/) next - if (line ~ /^\[[A-Za-z0-9_.-]+\]$/) { - section=substr(line,2,length(line)-2) - if (sections[file,section]++ || - (section!="connection" && section!="wireguard" && - section!="ipv4" && section!="ipv6" && section!="proxy" && - !(file==2 && shadow=="yes" && section==".nmmeta"))) bad=1 - next - } - separator=index(line,"=") - if (section=="" || !separator) {bad=1; next} - key=substr(line,1,separator-1) - value=substr(line,separator+1) - if (key !~ /^[A-Za-z0-9_.-]+$/ || seen[file,section,key]++) {bad=1; next} - permitted=0 - if (section=="connection") { - permitted=(key=="id" || key=="uuid" || key=="type" || - key=="interface-name" || key=="autoconnect" || key=="autoconnect-retries") - if (key=="zone" && file==2 && shadow=="yes") { - permitted=1 - if (value!="" && (value!="trusted" || firewall_mode!="firewalld_nft")) bad=1 - next - } - } else if (section=="wireguard") { - permitted=(key=="private-key" || key=="listen-port") - } else if (section=="ipv4") { - permitted=(key=="method" || key=="address1" || - key=="never-default" || key=="route-metric") - if (key ~ /^route[1-9][0-9]*$/) { - number=substr(key,6)+0 - if (number > 63 || !host32(value)) bad=1 - routes[file]++; permitted=1 - } - } else if (section=="ipv6") { - permitted=(key=="method") - } else if (section==".nmmeta" && file==2 && shadow=="yes") { - permitted=(key=="shadowed-storage" || key=="shadowed-owned") - if (!permitted || (key=="shadowed-storage" && value!=persistent) || - (key=="shadowed-owned" && value!="true")) bad=1 - next - } - if (!permitted) {bad=1; next} - values[file,section,key]=value - settings[section,key]=1 - } - END { - if (file!=2 || bad) exit 1 - for (file=1; file<=2; file++) { - if (sections[file,"connection"]!=1 || sections[file,"wireguard"]!=1 || - sections[file,"ipv4"]!=1 || sections[file,"ipv6"]!=1 || routes[file]<1) exit 1 - if (values[file,"connection","id"]!=wanted_id || - values[file,"connection","uuid"]!=wanted_uuid || - values[file,"connection","type"]!="wireguard" || - values[file,"connection","interface-name"]!="wg-fabric0" || - values[file,"connection","autoconnect"]!="false" || - values[file,"connection","autoconnect-retries"]!="0" || - values[file,"ipv4","method"]!="manual" || - !host32(values[file,"ipv4","address1"]) || - values[file,"ipv4","never-default"]!="true" || - values[file,"ipv4","route-metric"]!="50" || - values[file,"ipv6","method"]!="disabled") exit 1 - private_key=values[file,"wireguard","private-key"] - if (length(private_key)!=44 || private_key !~ /^[A-Za-z0-9+\/]+=$/) exit 1 - port=values[file,"wireguard","listen-port"] - if (port !~ /^[1-9][0-9]*$/ || port<1 || port>65535) exit 1 - for (number=1; number<=routes[file]; number++) - if (seen[file,"ipv4","route" number]!=1) exit 1 - } - if (shadow=="yes" && (sections[2,".nmmeta"]!=1 || - seen[2,".nmmeta","shadowed-storage"]!=1 || - seen[2,".nmmeta","shadowed-owned"]!=1)) exit 1 - for (setting in settings) - if (values[1,setting]!=values[2,setting]) exit 1 - } - ' "$NM_PROFILE" "$vc_nm_compare_runtime" >/dev/null + vc_vpc_state_count=0 + for vc_vpc_state_file in /etc/virt-cloud/vpc-dataplane/*.state; do + [ -e "$vc_vpc_state_file" ] || [ -L "$vc_vpc_state_file" ] || continue + vc_vpc_state_count=$((vc_vpc_state_count + 1)) + [ "$vc_vpc_state_count" -le 256 ] || return 1 + [ -f "$vc_vpc_state_file" ] && [ ! -L "$vc_vpc_state_file" ] || return 1 + [ "$($STAT -c '%u:%g:%a' "$vc_vpc_state_file" 2>/dev/null)" = 0:0:600 ] || return 1 + vc_vpc_state_size=$($STAT -c '%s' "$vc_vpc_state_file" 2>/dev/null) || return 1 + case "$vc_vpc_state_size" in ''|*[!0-9]*) return 1 ;; esac + [ "$vc_vpc_state_size" -le 65536 ] || return 1 + $AWK -F'|' ' + $1=="firewall_mode" {firewall++; if (NF!=2 || $2!="native_nftables") bad=1; next} + $1=="nft_snapshot" {bad=1; next} + END {exit (bad || firewall>1) ? 1 : 0} + ' "$vc_vpc_state_file" || return 1 + done + vc_legacy_tables=$($NFT list tables 2>/dev/null) || return 1 + ! printf '%s\n' "$vc_legacy_tables" | + grep -Eq 'table (inet firewalld|(inet|bridge) virt_cloud_fabric_emergency)([[:space:]]|$)' } vc_fabric_nm_profile_owned() { vc_nm_digest_mode=${1:-expected} [ -n "${NMCLI:-}" ] || return 1 [ -f "$NM_PROFILE" ] && [ ! -L "$NM_PROFILE" ] || return 1 - VC_NM_SHA256=$(find_tool /usr/bin/sha256sum /bin/sha256sum /usr/sbin/sha256sum /sbin/sha256sum) || return 1 - vc_nm_persistent_hash=$(vc_fabric_nm_profile_secure_hash "$NM_PROFILE") || return 1 + [ "$($STAT -c '%u:%g:%a' "$NM_PROFILE" 2>/dev/null)" = 0:0:600 ] || return 1 + [ "$($STAT -c '%s' "$NM_PROFILE" 2>/dev/null)" -le 65536 ] 2>/dev/null || return 1 vc_nm_first=; vc_nm_second= { IFS= read -r vc_nm_first || true; IFS= read -r vc_nm_second || true; } < "$NM_PROFILE" [ "$vc_nm_first" = '# virt-cloud-wg-fabric-v1' ] || return 1 @@ -3181,22 +2456,11 @@ vc_fabric_nm_profile_owned() { ') || return 1 [ "$vc_nm_profile_match" = yes ] || return 1 vc_nm_filename=$(vc_nm_filename_for_uuid "$CONNECTION_UUID") || return 1 - case "$vc_nm_filename" in - "$NM_PROFILE") - vc_fabric_nm_profile_semantics_match "$NM_PROFILE" no || return 1 ;; - /run/NetworkManager/system-connections/virt-cloud-wg-fabric0.nmconnection) - for vc_nm_runtime_dir in /run /run/NetworkManager /run/NetworkManager/system-connections; do - [ -d "$vc_nm_runtime_dir" ] && [ ! -L "$vc_nm_runtime_dir" ] || return 1 - vc_nm_runtime_owner=$($STAT -c '%u:%g:%a' "$vc_nm_runtime_dir" 2>/dev/null) || return 1 - case "$vc_nm_runtime_owner" in 0:0:700|0:0:750|0:0:755) : ;; *) return 1 ;; esac - done - vc_nm_runtime_hash=$(vc_fabric_nm_profile_secure_hash "$vc_nm_filename") || return 1 - vc_fabric_nm_profile_semantics_match "$vc_nm_filename" yes || return 1 - [ "$(vc_fabric_nm_profile_secure_hash "$vc_nm_filename")" = "$vc_nm_runtime_hash" ] || return 1 ;; - *) return 1 ;; - esac - [ "$(vc_fabric_nm_profile_secure_hash "$NM_PROFILE")" = "$vc_nm_persistent_hash" ] || return 1 - [ "$(vc_nm_filename_for_uuid "$CONNECTION_UUID")" = "$vc_nm_filename" ] + [ "$vc_nm_filename" = "$NM_PROFILE" ] || return 1 + vc_nm_zone=$($NMCLI -t --escape no -g connection.zone \ + connection show uuid "$CONNECTION_UUID" 2>/dev/null | sed -n '1p') || return 1 + [ -z "$vc_nm_zone" ] || [ "$vc_nm_zone" = -- ] || return 1 + [ "$(vc_nm_filename_for_uuid "$CONNECTION_UUID")" = "$NM_PROFILE" ] } vc_fabric_nm_active() { "$IP" link show dev "$IFACE" >/dev/null 2>&1 || return 1 @@ -3256,255 +2520,10 @@ vc_fabric_ifup_fragment_owned() { ' "$IFUPDOWN_FRAGMENT" } vc_fabric_quarantine() { - [ "$FIREWALL_MODE" = firewalld_nft ] || return 0 - vc_fabric_quarantine_mode=${1:-disconnect} - case "$vc_fabric_quarantine_mode" in down|disconnect) : ;; *) return 1 ;; esac - VC_FABRIC_QUARANTINE_RUNTIME_BEFORE= - VC_FABRIC_QUARANTINE_PERMANENT_BEFORE= - vc_fabric_link_present=no - vc_fabric_link_up_before=no - vc_fabric_nm_managed_before= - vc_fabric_memory_before= - if "$IP" link show dev "$IFACE" >/dev/null 2>&1; then - vc_fabric_link_present=yes - [ -f "$OWNER" ] && [ ! -L "$OWNER" ] || return 1 - [ "$($STAT -c '%u:%a' "$OWNER" 2>/dev/null)" = 0:600 ] || return 1 - [ "$(cat "$OWNER" 2>/dev/null)" = virt-cloud-wg-fabric-v1 ] || return 1 - "$WG" show "$IFACE" >/dev/null 2>&1 || return 1 - if "$IP" -o link show dev "$IFACE" 2>/dev/null | - grep -Eq '<[^>]*UP([,>])'; then - vc_fabric_link_up_before=yes - fi - fi - # The independent nft guard is the first mutable action after minimum - # owner/type proof. Profile, nmcli, and zone validation may now fail while - # the still-UP trusted interface remains unable to carry any packet. - vc_fabric_emergency_install || return 1 - if [ "$DEPLOYMENT_MODE" = networkmanager_nmcli ]; then - if [ -f "$NM_PROFILE" ] && [ ! -L "$NM_PROFILE" ]; then - vc_fabric_nm_profile_owned any || return 1 - if [ "$vc_fabric_link_present" = yes ]; then - vc_fabric_nm_managed_before=$(vc_fabric_nm_managed_state) || return 1 - fi - vc_fabric_memory_before=$( - "$NMCLI" -t --escape no -g connection.zone \ - connection show uuid "$CONNECTION_UUID" 2>/dev/null | - sed -n '1p' - ) || return 1 - case "$vc_fabric_memory_before" in - ''|--|trusted) : ;; - *) return 1 ;; - esac - else - [ "$vc_fabric_link_present" = no ] || return 1 - vc_fabric_memory_before= - fi - # A live NetworkManager-owned Fabric link must be made unmanaged and - # administratively down before *any* firewalld zone query or mutation. - # This preserves the WireGuard topology while ensuring that a failed - # zone query cannot leave a trusted overlay carrying traffic. - if [ "$vc_fabric_link_present" = yes ]; then - vc_fabric_nm_quarantine || return 1 - fi - vc_fabric_runtime=$(vc_firewall_zone runtime) || return 1 - vc_fabric_permanent=$(vc_firewall_zone permanent) || return 1 - [ "$vc_fabric_permanent" = none ] || return 1 - VC_FABRIC_QUARANTINE_PERMANENT_BEFORE=$vc_fabric_permanent - if [ "$vc_fabric_memory_before" = trusted ]; then - if [ "$vc_fabric_link_present" = yes ]; then - [ "$vc_fabric_nm_managed_before" = yes ] || return 1 - fi - VC_FABRIC_QUARANTINE_RUNTIME_BEFORE=trusted - elif [ "$vc_fabric_link_present" = yes ] && - [ "$vc_fabric_nm_managed_before" = no ] && - [ "$vc_fabric_link_up_before" = no ]; then - # This is the only retryable pre-journal state: a prior panel run - # already proved ownership, made the device unmanaged and lowered - # the link before losing its caller. Preserve that evidence in the - # transaction instead of pretending the default zone was original. - VC_FABRIC_QUARANTINE_RUNTIME_BEFORE=quarantined - else - [ "$vc_fabric_runtime" = none ] || return 1 - VC_FABRIC_QUARANTINE_RUNTIME_BEFORE=none - fi - if [ "$vc_fabric_memory_before" = trusted ]; then - # The hard isolation above has already completed. It is now safe - # to clear the temporary profile zone and remove any surviving - # runtime binding without NetworkManager reactivating the link. - "$NMCLI" connection modify --temporary uuid "$CONNECTION_UUID" connection.zone "" >/dev/null 2>&1 || return 1 - fi - if [ "$vc_fabric_runtime" = trusted ]; then - [ "$VC_FABRIC_QUARANTINE_RUNTIME_BEFORE" = trusted ] || return 1 - if "$FIREWALL_CMD" --zone=trusted --query-interface="$IFACE" >/dev/null 2>&1; then - "$FIREWALL_CMD" --zone=trusted --remove-interface="$IFACE" >/dev/null 2>&1 || return 1 - else - vc_fabric_zone_query_status=$? - [ "$vc_fabric_zone_query_status" = 1 ] || return 1 - fi - fi - if [ -f "$NM_PROFILE" ]; then - vc_fabric_memory_zone=$($NMCLI -t --escape no -g connection.zone connection show uuid "$CONNECTION_UUID" 2>/dev/null | sed -n '1p') || return 1 - [ -z "$vc_fabric_memory_zone" ] || [ "$vc_fabric_memory_zone" = -- ] || return 1 - vc_fabric_quarantined_runtime=$(vc_firewall_zone runtime) || return 1 - [ "$vc_fabric_quarantined_runtime" = none ] || - [ "$vc_fabric_quarantined_runtime" = "$($FIREWALL_CMD --get-default-zone 2>/dev/null)" ] || return 1 - if [ "$vc_fabric_link_present" = yes ]; then - vc_fabric_nm_expect_managed no || return 1 - fi - vc_firewall_quarantined "$IFACE" yes - else - vc_firewall_quarantined "$IFACE" no - fi - elif [ "$DEPLOYMENT_MODE" = ifupdown ]; then - if [ "$vc_fabric_link_present" = yes ]; then - "$IP" link set dev "$IFACE" down >/dev/null 2>&1 || return 1 - fi - vc_fabric_runtime=$(vc_firewall_zone runtime) || return 1 - vc_fabric_permanent=$(vc_firewall_zone permanent) || return 1 - VC_FABRIC_QUARANTINE_RUNTIME_BEFORE=$vc_fabric_runtime - VC_FABRIC_QUARANTINE_PERMANENT_BEFORE=$vc_fabric_permanent - [ "$vc_fabric_permanent" = none ] || return 1 - case "$vc_fabric_runtime" in - trusted) - vc_fabric_ifup_fragment_owned any || return 1 - [ ! -e "/etc/sysconfig/network-scripts/ifcfg-$IFACE" ] && - [ ! -L "/etc/sysconfig/network-scripts/ifcfg-$IFACE" ] || return 1 - "$FIREWALL_CMD" --zone=trusted --remove-interface="$IFACE" >/dev/null 2>&1 || return 1 - ;; - none) : ;; - *) return 1 ;; - esac - vc_firewall_quarantined "$IFACE" no - else - return 1 - fi + return 0 } vc_fabric_retry_quarantine_ready() { - # This state is produced only by a previous panel apply that failed after - # the early quarantine but before its durable transaction journal. It is - # a safe retry input only when every manager, profile, zone and link fact - # still proves the same panel-owned isolation boundary. - [ "$FIREWALL_MODE" = firewalld_nft ] || return 1 - [ -f "$CONF" ] && [ ! -L "$CONF" ] || return 1 - [ -f "$OWNER" ] && [ ! -L "$OWNER" ] || return 1 - [ "$($STAT -c '%u:%a' "$CONF" 2>/dev/null)" = 0:600 ] || return 1 - [ "$($STAT -c '%u:%a' "$OWNER" 2>/dev/null)" = 0:600 ] || return 1 - [ "$(cat "$OWNER" 2>/dev/null)" = virt-cloud-wg-fabric-v1 ] || return 1 - "$IP" link show dev "$IFACE" >/dev/null 2>&1 || return 1 - "$WG" show "$IFACE" >/dev/null 2>&1 || return 1 - "$IP" -o link show dev "$IFACE" 2>/dev/null | - grep -Eq '<[^>]*UP([,>])' && return 1 || true - if [ "$DEPLOYMENT_MODE" = networkmanager_nmcli ]; then - vc_fabric_nm_profile_owned any || return 1 - vc_fabric_nm_expect_managed no || return 1 - vc_retry_nm_uuid=$( - "$NMCLI" -t --escape no -g GENERAL.CON-UUID \ - device show "$IFACE" 2>/dev/null | sed -n '1p' - ) || return 1 - [ -z "$vc_retry_nm_uuid" ] || - [ "$vc_retry_nm_uuid" = "$CONNECTION_UUID" ] || return 1 - vc_retry_nm_zone=$( - "$NMCLI" -t --escape no -g connection.zone \ - connection show uuid "$CONNECTION_UUID" 2>/dev/null | - sed -n '1p' - ) || return 1 - [ -z "$vc_retry_nm_zone" ] || [ "$vc_retry_nm_zone" = -- ] || return 1 - vc_firewall_quarantined "$IFACE" yes - elif [ "$DEPLOYMENT_MODE" = ifupdown ]; then - vc_fabric_ifup_fragment_owned any || return 1 - [ ! -e "$NM_PROFILE" ] && [ ! -L "$NM_PROFILE" ] || return 1 - [ ! -e "/etc/sysconfig/network-scripts/ifcfg-$IFACE" ] && - [ ! -L "/etc/sysconfig/network-scripts/ifcfg-$IFACE" ] || return 1 - vc_firewall_quarantined "$IFACE" no - else - return 1 - fi -} -vc_firewall_effective_zone() { - vc_effective_scope=$1 - vc_effective_iface=$2 - vc_effective_zone=$(vc_firewall_zone "$vc_effective_scope" "$vc_effective_iface") || return 1 - if [ "$vc_effective_zone" = none ]; then - # firewalld exposes one default zone; --get-default-zone has no - # permanent variant and applies to both configuration views. - vc_effective_zone=$($FIREWALL_CMD --get-default-zone 2>/dev/null) || return 1 - vc_firewall_zone_exists "$vc_effective_scope" "$vc_effective_zone" || return 1 - fi - # A public/physical underlay that is already trusted would bypass every - # port policy in this integration. Never accept or create that topology. - [ "$vc_effective_zone" != trusted ] || return 1 - printf '%s' "$vc_effective_zone" -} -vc_firewall_underlay_zone() { - vc_underlay_zone_iface=$1 - vc_underlay_runtime_zone=$(vc_firewall_effective_zone runtime "$vc_underlay_zone_iface") || return 1 - if [ "${DEPLOYMENT_MODE:-}" = networkmanager_nmcli ]; then - [ -n "${NMCLI:-}" ] || return 1 - vc_nm_active=$($NMCLI -t --escape no -f UUID,DEVICE connection show --active 2>/dev/null) || return 1 - vc_nm_uuid=$(printf '%s\n' "$vc_nm_active" | awk -F: -v wanted="$vc_underlay_zone_iface" ' - $2 == wanted {print $1; count++} - END {if (count != 1) exit 1} - ') || return 1 - printf '%s\n' "$vc_nm_uuid" | awk ' - length($0)==36 && $0 ~ /^[0-9A-Fa-f-]+$/ && - substr($0,9,1)=="-" && substr($0,14,1)=="-" && - substr($0,19,1)=="-" && substr($0,24,1)=="-" {ok=1} - END {exit ok ? 0 : 1}' || return 1 - vc_nm_file=$(vc_nm_filename_for_uuid "$vc_nm_uuid") || return 1 - case "$vc_nm_file" in /etc/NetworkManager/system-connections/*.nmconnection) : ;; *) return 1 ;; esac - [ -f "$vc_nm_file" ] && [ ! -L "$vc_nm_file" ] || return 1 - [ "$($STAT -c '%u:%a' "$vc_nm_file" 2>/dev/null)" = 0:600 ] || return 1 - vc_underlay_permanent_zone=$(awk -F= ' - /^\[/ {section=$0; next} - section=="[connection]" && $1=="zone" { - if (++count > 1) exit 1 - value=substr($0, index($0, "=")+1) - } - END {if (count <= 1) printf "%s", value} - ' "$vc_nm_file") || return 1 - if [ -z "$vc_underlay_permanent_zone" ]; then - vc_underlay_permanent_zone=$($FIREWALL_CMD --get-default-zone 2>/dev/null) || return 1 - fi - vc_firewall_zone_exists permanent "$vc_underlay_permanent_zone" || return 1 - vc_permanent_binding=$(vc_firewall_zone permanent "$vc_underlay_zone_iface") || return 1 - [ "$vc_permanent_binding" = none ] || [ "$vc_permanent_binding" = "$vc_underlay_permanent_zone" ] || return 1 - elif [ "${DEPLOYMENT_MODE:-}" = ifupdown ]; then - # RHEL-style ifcfg files can cause firewall-cmd to persist a ZONE when - # changing even a runtime binding. They are outside the Debian - # ifupdown contract and make ownership ambiguous. - [ ! -e "/etc/sysconfig/network-scripts/ifcfg-$vc_underlay_zone_iface" ] && - [ ! -L "/etc/sysconfig/network-scripts/ifcfg-$vc_underlay_zone_iface" ] || return 1 - vc_underlay_permanent_zone=$(vc_firewall_effective_zone permanent "$vc_underlay_zone_iface") || return 1 - else - return 1 - fi - [ "$vc_underlay_runtime_zone" = "$vc_underlay_permanent_zone" ] || return 1 - [ "$vc_underlay_runtime_zone" != trusted ] || return 1 - printf '%s' "$vc_underlay_runtime_zone" -} -vc_firewall_no_source_zones() { - vc_source_scope=$1 - if [ "$vc_source_scope" = permanent ]; then - vc_source_zones=$($FIREWALL_CMD --permanent --get-zones 2>/dev/null) || return 1 - elif [ "$vc_source_scope" = runtime ]; then - vc_source_zones=$($FIREWALL_CMD --get-zones 2>/dev/null) || return 1 - else - return 1 - fi - vc_source_zone_count=0 - for vc_source_zone in $vc_source_zones; do - case "$vc_source_zone" in ''|*[!A-Za-z0-9_-]*) return 1 ;; esac - [ "$(printf '%s' "$vc_source_zone" | awk 'length($0)<=64')" = "$vc_source_zone" ] || return 1 - vc_source_zone_count=$((vc_source_zone_count + 1)) - [ "$vc_source_zone_count" -le 64 ] || return 1 - if [ "$vc_source_scope" = permanent ]; then - vc_sources=$($FIREWALL_CMD --permanent --zone="$vc_source_zone" --list-sources 2>/dev/null) || return 1 - else - vc_sources=$($FIREWALL_CMD --zone="$vc_source_zone" --list-sources 2>/dev/null) || return 1 - fi - [ -z "$vc_sources" ] || return 1 - done - [ "$vc_source_zone_count" -gt 0 ] || return 1 + return 1 } vc_underlay_local_interface() { vc_underlay_endpoint=$1 @@ -3512,10 +2531,8 @@ vc_underlay_local_interface() { for (i=1;i<=4;i++) if ($i !~ /^[0-9]+$/ || $i>255) exit 1 ok=1 } END {exit ok ? 0 : 1}' || return 1 - # Both firewall modes require the advertised endpoint to be a fresh, - # unique, local global IPv4 address. This also keeps firewalld rich-rule - # matching correct after any upstream DNAT; a NAT-only advertised address - # cannot be proven safe and is intentionally unsupported by the platform. + # The advertised endpoint must be a fresh, unique, local global IPv4 + # address. A NAT-only advertised address cannot be proven safe. vc_underlay_addresses=$($IP -o -4 address show up scope global 2>/dev/null) || return 1 vc_underlay_iface=$(printf '%s\n' "$vc_underlay_addresses" | awk -v wanted="$vc_underlay_endpoint" ' $3 == "inet" {split($4, address, "/"); if (address[1] == wanted) {print $2; count++}} @@ -3547,43 +2564,7 @@ vc_underlay_discover() { [ "$vc_underlay_probe_count" -le 64 ] || return 1 done [ "$vc_underlay_probe_count" -gt 0 ] || return 1 - vc_underlay_runtime_zone=$(vc_firewall_underlay_zone "$vc_underlay_iface") || return 1 - vc_firewall_no_source_zones runtime || return 1 - vc_firewall_no_source_zones permanent || return 1 - printf '%s|%s' "$vc_underlay_iface" "$vc_underlay_runtime_zone" -} -vc_firewall_rich_rule() { - vc_rule_endpoint=$1 - vc_rule_port=$2 - printf '%s\n' "$vc_rule_endpoint" | awk -F. 'NF==4 { - for (i=1;i<=4;i++) if ($i !~ /^[0-9]+$/ || $i>255) exit 1 - ok=1 - } END {exit ok ? 0 : 1}' || return 1 - case "$vc_rule_port" in ''|*[!0-9]*) return 1 ;; esac - [ "$vc_rule_port" -ge 1 ] 2>/dev/null && [ "$vc_rule_port" -le 65535 ] 2>/dev/null || return 1 - printf 'rule family="ipv4" destination address="%s/32" port port="%s" protocol="udp" accept' \ - "$vc_rule_endpoint" "$vc_rule_port" -} -vc_firewall_rule_state() { - vc_rule_scope=$1 - vc_rule_zone=$2 - vc_rule_text=$3 - vc_firewall_zone_exists "$vc_rule_scope" "$vc_rule_zone" || return 1 - if [ "$vc_rule_scope" = permanent ]; then - if $FIREWALL_CMD --permanent --zone="$vc_rule_zone" --query-rich-rule="$vc_rule_text" >/dev/null 2>&1; then - printf '%s' yes - else - vc_rule_status=$? - [ "$vc_rule_status" = 1 ] || return 1 - printf '%s' no - fi - elif $FIREWALL_CMD --zone="$vc_rule_zone" --query-rich-rule="$vc_rule_text" >/dev/null 2>&1; then - printf '%s' yes - else - vc_rule_status=$? - [ "$vc_rule_status" = 1 ] || return 1 - printf '%s' no - fi + printf '%s' "$vc_underlay_iface" } '''.strip() @@ -3683,7 +2664,6 @@ CONF="$BASE/wg-fabric0.conf" OWNER="$BASE/wg-fabric0.owner" KEY="$BASE/wg-fabric0.key" NFT_RULES=/etc/nftables.d/virt-cloud-wg-fabric.nft -NFT_MANIFEST=/etc/virt-cloud/nftables.conf JOURNAL="$BASE/.wg-fabric-txn" NM_PROFILE=/etc/NetworkManager/system-connections/virt-cloud-wg-fabric0.nmconnection IFUPDOWN_FRAGMENT=/etc/network/interfaces.d/virt-cloud-wg-fabric0 @@ -3739,7 +2719,6 @@ STAT=$(find_tool /usr/bin/stat /bin/stat /usr/sbin/stat /sbin/stat) || exit 70 SHA256=$(find_tool /usr/bin/sha256sum /bin/sha256sum /usr/sbin/sha256sum /sbin/sha256sum) || exit 70 CMP=$(find_tool /usr/bin/cmp /bin/cmp /usr/sbin/cmp /sbin/cmp) || exit 70 SYSTEMCTL=$(find_tool /usr/bin/systemctl /bin/systemctl /usr/sbin/systemctl /sbin/systemctl) || exit 70 -FIREWALL_CMD=$(find_tool /usr/bin/firewall-cmd /usr/sbin/firewall-cmd /bin/firewall-cmd /sbin/firewall-cmd || true) AWK=$(find_tool /usr/bin/awk /bin/awk /usr/sbin/awk /sbin/awk) || exit 70 BRIDGE_CMD=$(find_tool /usr/sbin/bridge /usr/bin/bridge /sbin/bridge /bin/bridge) || exit 70 vc_fabric_recovery_checkpoint() { @@ -3767,22 +2746,17 @@ vc_fabric_recovery_checkpoint() { } __VC_FABRIC_SYSCTL_VALIDATION__ __VC_VPC_SYSCTL_VALIDATION__ -vc_fabric_emergency_reconcile_needed() { - # This pre-lock condition is deliberately read-only and conservative. The - # complete helper later proves the random owner token and both table - # shapes while holding the Fabric flock. Any owner/table evidence (or an - # unreadable nft inventory) must schedule that authoritative recovery. - vc_repair_emergency_owner=$EMERGENCY_OWNER - if [ -e "$vc_repair_emergency_owner" ] || - [ -L "$vc_repair_emergency_owner" ]; then +vc_legacy_firewalld_state_detected() { + # Every transaction journal needs the locked helper: valid native state + # must be recovered, while legacy or malformed state must fail closed and + # remain untouched. Other legacy artifacts and unreadable nft inventory + # take the same conservative path. + if [ -e /etc/virt-cloud/.wg-fabric-txn ] || + [ -L /etc/virt-cloud/.wg-fabric-txn ]; then return 0 fi - vc_repair_emergency_tables=$($NFT list tables 2>/dev/null) || return 0 - if printf '%s\n' "$vc_repair_emergency_tables" | - grep -Eq 'table (inet|bridge) virt_cloud_fabric_emergency([[:space:]]|$)'; then - return 0 - fi - return 1 + if vc_legacy_firewalld_state_absent; then return 1; fi + return 0 } vc_primary_inactive_retry_ready() { [ "$($SYSTEMCTL is-active virt-cloud-wg-fabric.service 2>/dev/null || true)" = inactive ] || return 1 @@ -3832,68 +2806,21 @@ vc_fabric_rpf_repair_needed() { [ "$(cat "$vc_repair_rpf_path" 2>/dev/null)" = 2 ] || return 0 return 1 } -vc_firewalld_runtime_repair_needed() { - # Return success only when a configured firewalld-backed dataplane has - # lost one of its deliberately runtime-only trusted bindings. firewalld - # restores permanent rich rules across restart, but these overlay/tenant - # bindings must be rebuilt by the panel after raw nft ACL readback. - [ "$FIREWALL_MODE" = firewalld_nft ] || return 1 - [ -n "$FIREWALL_CMD" ] || return 1 - [ "$($SYSTEMCTL is-active virt-cloud-wg-fabric.service 2>/dev/null || true)" = active ] || return 1 - [ "$($FIREWALL_CMD --state 2>/dev/null || true)" = running ] || return 1 - if [ ! -f "$CONF" ] || [ -L "$CONF" ] || - [ ! -f "$OWNER" ] || [ -L "$OWNER" ] || - [ "$($STAT -c '%u:%a' "$CONF" 2>/dev/null)" != 0:600 ] || - [ "$($STAT -c '%u:%a' "$OWNER" 2>/dev/null)" != 0:600 ] || - [ "$(cat "$OWNER" 2>/dev/null)" != virt-cloud-wg-fabric-v1 ]; then - return 0 - fi - "$IP" link show dev "$IFACE" >/dev/null 2>&1 || return 0 - "$FIREWALL_CMD" --zone=trusted --query-interface="$IFACE" >/dev/null 2>&1 || return 0 - vc_repair_vpc_dir="$BASE/vpc-dataplane" - if [ -e "$vc_repair_vpc_dir" ] || [ -L "$vc_repair_vpc_dir" ]; then - [ -d "$vc_repair_vpc_dir" ] && [ ! -L "$vc_repair_vpc_dir" ] || return 0 - [ "$($STAT -c '%u:%a' "$vc_repair_vpc_dir" 2>/dev/null)" = 0:700 ] || return 0 - else - return 1 - fi - vc_repair_vpc_count=0 - for vc_repair_owner in "$vc_repair_vpc_dir"/*.owner; do - [ -e "$vc_repair_owner" ] || [ -L "$vc_repair_owner" ] || continue - vc_repair_vpc_count=$((vc_repair_vpc_count + 1)) - [ "$vc_repair_vpc_count" -le __VC_MAX_VPC_STATE_FILES__ ] || return 0 - vc_repair_token=${vc_repair_owner##*/} - vc_repair_token=${vc_repair_token%.owner} - if ! printf '%s\n' "$vc_repair_token" | grep -Eq '^[0-9a-f]{11}$' || - [ ! -f "$vc_repair_owner" ] || [ -L "$vc_repair_owner" ] || - [ "$($STAT -c '%u:%a' "$vc_repair_owner" 2>/dev/null)" != 0:600 ] || - ! grep -Fqx 'virt-cloud-vpc-dataplane-v1' "$vc_repair_owner"; then - return 0 - fi - vc_repair_bridge="vcv-$vc_repair_token" - "$IP" link show dev "$vc_repair_bridge" >/dev/null 2>&1 || return 0 - "$FIREWALL_CMD" --zone=trusted --query-interface="$vc_repair_bridge" >/dev/null 2>&1 || return 0 - done - return 1 -} if [ "${1:-}" = --needs-repair ]; then - # Retry a failed boot gate, or repair runtime-only trusted bindings lost by - # a firewalld service restart. Healthy dataplanes remain untouched. + # Retry a failed boot gate or repair runtime drift. Healthy dataplanes + # remain untouched. if "$SYSTEMCTL" is-failed --quiet virt-cloud-wg-fabric.service; then exit 0 fi if vc_primary_inactive_retry_ready; then exit 0 fi - if vc_fabric_emergency_reconcile_needed; then + if vc_legacy_firewalld_state_detected; then exit 0 fi if vc_fabric_rpf_repair_needed; then exit 0 fi - if vc_firewalld_runtime_repair_needed; then - exit 0 - fi exit 1 fi if [ "${1:-}" = --retry-gated-units ]; then @@ -4058,7 +2985,7 @@ if [ "${1:-}" = --retry-gated-units ]; then } retry_primary_failed=no retry_primary_inactive=no - retry_emergency_guard=no + retry_legacy_firewalld=no retry_runtime_drift=no retry_primary_state=$($SYSTEMCTL is-active virt-cloud-wg-fabric.service 2>/dev/null || true) if "$SYSTEMCTL" is-failed --quiet virt-cloud-wg-fabric.service; then @@ -4067,19 +2994,16 @@ if [ "${1:-}" = --retry-gated-units ]; then if vc_primary_inactive_retry_ready; then retry_primary_inactive=yes fi - if vc_fabric_emergency_reconcile_needed; then - retry_emergency_guard=yes + if vc_legacy_firewalld_state_detected; then + retry_legacy_firewalld=yes fi if vc_fabric_rpf_repair_needed; then retry_runtime_drift=yes fi - if vc_firewalld_runtime_repair_needed; then - retry_runtime_drift=yes - fi [ "$retry_primary_failed" = yes ] || [ "$retry_primary_inactive" = yes ] || - [ "$retry_emergency_guard" = yes ] || + [ "$retry_legacy_firewalld" = yes ] || [ "$retry_runtime_drift" = yes ] || exit 0 - if { [ "$retry_emergency_guard" = yes ] || + if { [ "$retry_legacy_firewalld" = yes ] || [ "$retry_runtime_drift" = yes ]; } && [ "$retry_primary_failed" = no ] && [ "$retry_primary_inactive" = no ]; then @@ -4088,9 +3012,8 @@ if [ "${1:-}" = --retry-gated-units ]; then # Never restart an active primary unit: libvirt units Require # it and systemd would propagate that restart. Execute the # same complete, flocked, ACL-first helper inside this - # independent retry service instead. It is the only path that - # may remove a committed emergency guard after full VPC - # recovery and readback. + # independent retry service instead. Unsupported legacy state + # remains untouched and makes that helper fail closed. if [ "$DEPLOYMENT_MODE" = ifupdown ]; then exec /bin/sh "$HELPER" --ifupdown-service fi @@ -4132,7 +3055,6 @@ if [ "${1:-}" = --retry-gated-units ]; then fi __VC_NMCLI_FILENAME_RESOLVER__ __VC_FIREWALL_MODE_GUARD__ -__VC_NFT_MANIFEST_VALIDATION__ __VC_FABRIC_NFT_VALIDATION__ __VC_FABRIC_WG_READBACK__ __VC_VPC_STATE_LOADER__ @@ -4143,7 +3065,7 @@ if [ "$DEPLOYMENT_MODE" = networkmanager_nmcli ]; then NMCLI=$(find_tool /usr/bin/nmcli /usr/sbin/nmcli /bin/nmcli /sbin/nmcli || true) fi [ "$DEPLOYMENT_MODE" = networkmanager_nmcli ] || [ "$DEPLOYMENT_MODE" = ifupdown ] || exit 71 -[ "$FIREWALL_MODE" = native_nftables ] || [ "$FIREWALL_MODE" = firewalld_nft ] || exit 71 +[ "$FIREWALL_MODE" = native_nftables ] || exit 71 if [ -L /run/virt-cloud ] || { [ -e /run/virt-cloud ] && [ ! -d /run/virt-cloud ]; }; then exit 71; fi if [ -e /run/virt-cloud ]; then [ -d /run/virt-cloud ] && [ ! -L /run/virt-cloud ] || exit 71 @@ -4174,20 +3096,12 @@ if "$IP" link show dev "$IFACE" >/dev/null 2>&1; then [ "$(cat "$OWNER" 2>/dev/null)" = virt-cloud-wg-fabric-v1 ] || exit 71 "$WG" show "$IFACE" >/dev/null 2>&1 || exit 71 fi -if [ "$FIREWALL_MODE" = firewalld_nft ]; then - # Install the manager-independent packet stop before any profile, nmcli, or - # zone lookup. VPC bridges are hard-lowered next; only then may the normal - # Fabric quarantine inspect mutable manager state. - vc_fabric_emergency_install || exit 71 -else - vc_fabric_emergency_adopt || exit 71 - vc_fabric_native_adopt_vpc_state_empty || exit 71 - if "$IP" link show dev "$IFACE" >/dev/null 2>&1; then - if [ "$DEPLOYMENT_MODE" = networkmanager_nmcli ]; then - vc_fabric_nm_quarantine || exit 71 - else - "$IP" link set dev "$IFACE" down >/dev/null 2>&1 || exit 71 - fi +vc_legacy_firewalld_state_absent || exit 79 +if "$IP" link show dev "$IFACE" >/dev/null 2>&1; then + if [ "$DEPLOYMENT_MODE" = networkmanager_nmcli ]; then + vc_fabric_nm_quarantine || exit 71 + else + "$IP" link set dev "$IFACE" down >/dev/null 2>&1 || exit 71 fi fi vc_bridge_port_names() { @@ -4345,9 +3259,6 @@ vc_vpc_emergency_down() { [ "$vc_vpc_emergency_failed" = no ] } vc_vpc_emergency_down || exit 71 -if [ "$FIREWALL_MODE" = firewalld_nft ]; then - vc_fabric_quarantine down || exit 71 -fi vc_fabric_sysctl_converge() { [ -d /etc/sysctl.d ] && [ ! -L /etc/sysctl.d ] || return 1 [ "$($STAT -c '%u:%a' /etc/sysctl.d 2>/dev/null)" = 0:755 ] || return 1 @@ -4759,17 +3670,13 @@ nft_plan_owned() { recover_journal() { [ -e "$JOURNAL" ] || return 0 journal_iface_recreated=no - if [ "$FIREWALL_MODE" = firewalld_nft ] && - "$IP" link show dev "$IFACE" >/dev/null 2>&1; then - if [ "$DEPLOYMENT_MODE" = networkmanager_nmcli ]; then - NMCLI=$(find_tool /usr/bin/nmcli /usr/sbin/nmcli /bin/nmcli /sbin/nmcli) || exit 79 - fi - # A corrupt/incomplete journal is itself a validation failure. Close - # the owned overlay before trusting any journal metadata so that such - # a failure cannot preserve a trusted interface without its guard. - vc_fabric_quarantine || exit 79 - fi journal_secure || exit 79 + # Repeat the complete native journal schema check inside recovery itself. + # No alternate entry point may reap a merely marked, malformed, or retired + # backend journal. + vc_native_transaction_journal_valid_or_absent || exit 79 + journal_early_firewall_mode=$(journal_value firewall_mode 2>/dev/null || true) + [ "$journal_early_firewall_mode" = native_nftables ] || exit 79 journal_state=$(cat "$JOURNAL/state" 2>/dev/null || true) case "$journal_state" in cleanup_helper) @@ -4803,14 +3710,9 @@ recover_journal() { exit 0 ;; guarded) - # A synchronous apply rollback restored its pre-transaction - # artifacts but deliberately retained the independent emergency - # stop because VPC readiness was not re-proved in the trap. The - # current complete helper is already running under that guard; - # discard the hand-off journal and continue into normal full - # Fabric/VPC recovery, whose final gate alone may remove it. - rm -rf "$JOURNAL" || exit 79 - return 0 + # This phase was produced only by the retired firewalld recovery + # protocol. Ordinary reconciliation must never consume it. + exit 79 ;; committed) rm -rf "$JOURNAL" || exit 79 @@ -4841,34 +3743,10 @@ recover_journal() { journal_service_active=$(journal_value service_active_before 2>/dev/null || true) journal_timer_enabled=$(journal_value timer_enabled_before 2>/dev/null || true) journal_timer_active=$(journal_value timer_active_before 2>/dev/null || true) - journal_firewalld_runtime_zone=$(journal_value firewalld_runtime_zone_before 2>/dev/null || true) - journal_firewalld_permanent_zone=$(journal_value firewalld_permanent_zone_before 2>/dev/null || true) - journal_underlay_endpoint=$(journal_value underlay_endpoint 2>/dev/null || true) - journal_underlay_port=$(journal_value underlay_port 2>/dev/null || true) - journal_underlay_zone=$(journal_value underlay_zone 2>/dev/null || true) - journal_underlay_rule_owner=$(journal_value underlay_rule_owner 2>/dev/null || true) - journal_underlay_rule_runtime=$(journal_value underlay_rule_runtime_before 2>/dev/null || true) - journal_underlay_rule_permanent=$(journal_value underlay_rule_permanent_before 2>/dev/null || true) - journal_old_underlay_remove=$(journal_value old_underlay_rule_remove 2>/dev/null || printf no) - journal_old_underlay_endpoint=$(journal_value old_underlay_endpoint 2>/dev/null || true) - journal_old_underlay_port=$(journal_value old_underlay_port 2>/dev/null || true) - journal_old_underlay_zone=$(journal_value old_underlay_zone 2>/dev/null || true) if [ -z "$journal_firewall_mode" ]; then case "$journal_nft_main" in /etc/nftables.conf|/etc/sysconfig/nftables.conf) journal_firewall_mode=native_nftables - journal_firewalld_runtime_zone=na - journal_firewalld_permanent_zone=na - journal_underlay_endpoint= - journal_underlay_port= - journal_underlay_zone= - journal_underlay_rule_owner=none - journal_underlay_rule_runtime=na - journal_underlay_rule_permanent=na - journal_old_underlay_remove=no - journal_old_underlay_endpoint= - journal_old_underlay_port= - journal_old_underlay_zone= ;; *) exit 79 ;; esac @@ -4910,57 +3788,9 @@ recover_journal() { networkmanager_nmcli|ifupdown) : ;; *) exit 79 ;; esac - case "$journal_firewall_mode" in - native_nftables) - case "$journal_nft_main" in /etc/nftables.conf|/etc/sysconfig/nftables.conf) : ;; *) exit 79 ;; esac - [ "$journal_firewalld_runtime_zone" = na ] && - [ "$journal_firewalld_permanent_zone" = na ] || exit 79 - [ -z "$journal_underlay_endpoint$journal_underlay_port$journal_underlay_zone" ] && - [ "$journal_underlay_rule_owner:$journal_underlay_rule_runtime:$journal_underlay_rule_permanent" = none:na:na ] || exit 79 - [ "$journal_old_underlay_remove" = no ] && - [ -z "$journal_old_underlay_endpoint$journal_old_underlay_port$journal_old_underlay_zone" ] || exit 79 - ;; - firewalld_nft) - [ "$journal_nft_main" = /etc/virt-cloud/nftables.conf ] || exit 79 - case "$journal_firewalld_runtime_zone:$journal_firewalld_permanent_zone" in - none:none|trusted:none|quarantined:none) : ;; - *) exit 79 ;; - esac - if [ "${RECOVER_ONLY:-no}" != yes ] || - [ "${RECOVER_CURRENT:-no}" = yes ]; then - [ "$journal_underlay_endpoint" = "$UNDERLAY_ENDPOINT" ] || exit 79 - [ "$journal_underlay_port" = "$UNDERLAY_PORT" ] || exit 79 - else - printf '%s\n' "$journal_underlay_endpoint" | awk -F. 'NF==4 { - for (i=1;i<=4;i++) if ($i !~ /^[0-9]+$/ || $i>255) exit 1 - ok=1 - } END {exit ok ? 0 : 1}' || exit 79 - case "$journal_underlay_port" in ''|*[!0-9]*) exit 79 ;; esac - [ "$journal_underlay_port" -ge 1 ] 2>/dev/null && - [ "$journal_underlay_port" -le 65535 ] 2>/dev/null || exit 79 - fi - case "$journal_underlay_zone" in ''|*[!A-Za-z0-9_-]*) exit 79 ;; esac - case "$journal_underlay_rule_owner" in panel|external) : ;; *) exit 79 ;; esac - case "$journal_underlay_rule_runtime:$journal_underlay_rule_permanent" in - yes:yes|yes:no|no:yes|no:no) : ;; - *) exit 79 ;; - esac - case "$journal_old_underlay_remove" in - no) - [ -z "$journal_old_underlay_endpoint$journal_old_underlay_port$journal_old_underlay_zone" ] || exit 79 ;; - yes) - printf '%s\n' "$journal_old_underlay_endpoint" | awk -F. 'NF==4 { - for (i=1;i<=4;i++) if ($i !~ /^[0-9]+$/ || $i>255) exit 1 - ok=1 - } END {exit ok ? 0 : 1}' || exit 79 - case "$journal_old_underlay_port" in ''|*[!0-9]*) exit 79 ;; esac - [ "$journal_old_underlay_port" -ge 1 ] 2>/dev/null && - [ "$journal_old_underlay_port" -le 65535 ] 2>/dev/null || exit 79 - case "$journal_old_underlay_zone" in ''|*[!A-Za-z0-9_-]*) exit 79 ;; esac - [ "$(printf '%s' "$journal_old_underlay_zone" | awk 'length($0)<=64')" = "$journal_old_underlay_zone" ] || exit 79 ;; - *) exit 79 ;; - esac - ;; + [ "$journal_firewall_mode" = native_nftables ] || exit 79 + case "$journal_nft_main" in + /etc/nftables.conf|/etc/sysconfig/nftables.conf) : ;; *) exit 79 ;; esac case "$journal_iface_before:$journal_iface_up_before:$journal_nft_before" in @@ -5078,21 +3908,7 @@ recover_journal() { journal_first_deploy_artifact_safe "$OWNER" owner || exit 79 journal_first_deploy_artifact_safe "$SYSCTL_FILE" sysctl || exit 79 journal_first_deploy_artifact_safe "$NFT_RULES" nft_rules || exit 79 - case "$journal_firewall_mode" in - firewalld_nft) - journal_marker_secure nft_main.absent || exit 79 - [ ! -e "$JOURNAL/nft_main.present" ] && - [ ! -L "$JOURNAL/nft_main.present" ] && - [ ! -e "$JOURNAL/nft_main" ] && - [ ! -L "$JOURNAL/nft_main" ] || exit 79 - journal_first_deploy_artifact_safe \ - "$journal_nft_main" nft_main || exit 79 - ;; - native_nftables) - journal_native_nft_main_safe || exit 79 - ;; - *) exit 79 ;; - esac + journal_native_nft_main_safe || exit 79 journal_recover_owner_desired=no if [ -e "$OWNER" ] || [ -L "$OWNER" ]; then journal_recover_owner_desired=yes @@ -5175,28 +3991,8 @@ recover_journal() { journal_nmcli=$(find_tool /usr/bin/nmcli /usr/sbin/nmcli /bin/nmcli /sbin/nmcli) || exit 79 NMCLI=$journal_nmcli fi - if [ "$journal_firewall_mode" = firewalld_nft ]; then - # Journal metadata is authoritative during --recover-only. Hard-close - # the owned link before the first zone query or mutation in recovery. - vc_fabric_quarantine down || exit 79 - if [ "$journal_mode" = networkmanager_nmcli ]; then - if [ -f "$NM_PROFILE" ] && [ ! -L "$NM_PROFILE" ]; then - NMCLI=$journal_nmcli - vc_fabric_nm_profile_owned any || exit 79 - journal_memory_zone=$("$journal_nmcli" -t --escape no -g connection.zone connection show uuid "$journal_connection_uuid" 2>/dev/null | sed -n '1p') - [ -z "$journal_memory_zone" ] || [ "$journal_memory_zone" = -- ] || exit 79 - if [ "$journal_current_iface_state" = present ]; then - vc_fabric_nm_expect_managed no || exit 79 - fi - vc_firewall_quarantined "$IFACE" yes || exit 79 - else - vc_firewall_quarantined "$IFACE" no || exit 79 - fi - else - vc_firewall_quarantined "$IFACE" no || exit 79 - fi - elif [ "$journal_current_iface_state" = present ]; then - # Native nftables has no firewalld emergency zone. Prevent + if [ "$journal_current_iface_state" = present ]; then + # Prevent # NetworkManager from racing the raw-sysctl restore by making the # validated panel profile unmanaged before link-down. if [ "$journal_mode" = networkmanager_nmcli ]; then @@ -5428,20 +4224,13 @@ recover_journal() { "$journal_nmcli" connection load "$NM_PROFILE" >/dev/null 2>&1 || exit 79 if [ "$journal_iface_before" = yes ]; then NMCLI=$journal_nmcli - if [ "$journal_firewall_mode" = firewalld_nft ]; then - # Every firewalld recovery path stays unmanaged/down until - # the final ACL and rule readbacks below. This also - # preserves a deliberate pre-journal quarantine. - vc_fabric_nm_quarantine || exit 79 + vc_fabric_nm_release || exit 79 + if [ "$journal_iface_up_before" = yes ]; then + "$journal_nmcli" connection up uuid "$journal_old_uuid" >/dev/null 2>&1 || exit 79 else - vc_fabric_nm_release || exit 79 - if [ "$journal_iface_up_before" = yes ]; then - "$journal_nmcli" connection up uuid "$journal_old_uuid" >/dev/null 2>&1 || exit 79 - else - # Preserve a previously administratively-down link. A - # recovery must not turn it on as a side effect. - "$IP" link set dev "$IFACE" down >/dev/null 2>&1 || exit 79 - fi + # Preserve a previously administratively-down link. A + # recovery must not turn it on as a side effect. + "$IP" link set dev "$IFACE" down >/dev/null 2>&1 || exit 79 fi fi else @@ -5465,112 +4254,10 @@ recover_journal() { IFUPDOWN_VC_FABRIC_OPERATION_ID="${VC_FABRIC_OPERATION_ID:-}" \ "$IFUP_PATH" --force "$IFACE" >/dev/null 2>&1 || exit 79 fi - if [ "$journal_firewall_mode" = firewalld_nft ]; then - journal_runtime_zone=$(vc_firewall_zone runtime) || exit 79 - journal_permanent_zone=$(vc_firewall_zone permanent) || exit 79 - if [ "$journal_mode" = networkmanager_nmcli ]; then - if [ -f "$NM_PROFILE" ] && [ ! -L "$NM_PROFILE" ]; then - NMCLI=$journal_nmcli - vc_fabric_nm_profile_owned any || exit 79 - journal_memory_zone=$("$journal_nmcli" -t --escape no -g connection.zone connection show uuid "$journal_old_uuid" 2>/dev/null | sed -n '1p') - [ -z "$journal_memory_zone" ] || [ "$journal_memory_zone" = -- ] || exit 79 - if [ "$(vc_firewall_zone runtime)" != none ]; then - vc_fabric_nm_active || exit 79 - fi - vc_firewall_quarantined "$IFACE" yes || exit 79 - else - vc_firewall_quarantined "$IFACE" no || exit 79 - fi - else - vc_firewall_quarantined "$IFACE" no || exit 79 - fi - [ "$journal_firewalld_permanent_zone" = none ] || exit 79 - case "$journal_firewalld_runtime_zone" in - trusted) - if [ "$journal_mode" = networkmanager_nmcli ]; then - [ -n "$journal_nmcli" ] || exit 79 - "$journal_nmcli" connection modify --temporary uuid "$journal_old_uuid" connection.zone trusted >/dev/null 2>&1 || exit 79 - else - [ ! -e "/etc/sysconfig/network-scripts/ifcfg-$IFACE" ] && - [ ! -L "/etc/sysconfig/network-scripts/ifcfg-$IFACE" ] || exit 79 - "$FIREWALL_CMD" --zone=trusted --add-interface="$IFACE" >/dev/null 2>&1 || exit 79 - fi - ;; - none) : ;; - quarantined) - vc_fabric_quarantine down || exit 79 - ;; - *) exit 79 ;; - esac - [ "$(vc_firewall_zone permanent)" = none ] || exit 79 - if [ "$journal_firewalld_runtime_zone" = quarantined ]; then - if [ "$journal_mode" = networkmanager_nmcli ]; then - vc_fabric_nm_expect_managed no || exit 79 - vc_firewall_quarantined "$IFACE" yes || exit 79 - else - vc_firewall_quarantined "$IFACE" no || exit 79 - fi - "$IP" -o link show dev "$IFACE" 2>/dev/null | - grep -Eq '<[^>]*UP([,>])' && exit 79 || true - else - [ "$(vc_firewall_zone runtime)" = "$journal_firewalld_runtime_zone" ] || exit 79 - fi - if [ "$journal_old_underlay_remove" = yes ]; then - journal_old_rule=$(vc_firewall_rich_rule "$journal_old_underlay_endpoint" "$journal_old_underlay_port") || exit 79 - journal_old_runtime=$(vc_firewall_rule_state runtime "$journal_old_underlay_zone" "$journal_old_rule") || exit 79 - journal_old_permanent=$(vc_firewall_rule_state permanent "$journal_old_underlay_zone" "$journal_old_rule") || exit 79 - case "$journal_old_runtime" in - yes) : ;; - no) "$FIREWALL_CMD" --zone="$journal_old_underlay_zone" --add-rich-rule="$journal_old_rule" >/dev/null 2>&1 || exit 79 ;; - *) exit 79 ;; - esac - case "$journal_old_permanent" in - yes) : ;; - no) "$FIREWALL_CMD" --permanent --zone="$journal_old_underlay_zone" --add-rich-rule="$journal_old_rule" >/dev/null 2>&1 || exit 79 ;; - *) exit 79 ;; - esac - [ "$(vc_firewall_rule_state runtime "$journal_old_underlay_zone" "$journal_old_rule")" = yes ] || exit 79 - [ "$(vc_firewall_rule_state permanent "$journal_old_underlay_zone" "$journal_old_rule")" = yes ] || exit 79 - fi - journal_rule=$(vc_firewall_rich_rule "$journal_underlay_endpoint" "$journal_underlay_port") || exit 79 - journal_rule_runtime=$(vc_firewall_rule_state runtime "$journal_underlay_zone" "$journal_rule") || exit 79 - journal_rule_permanent=$(vc_firewall_rule_state permanent "$journal_underlay_zone" "$journal_rule") || exit 79 - if [ "$journal_underlay_rule_owner" = panel ]; then - if [ "$journal_underlay_rule_runtime" = no ] && [ "$journal_rule_runtime" = yes ]; then - "$FIREWALL_CMD" --zone="$journal_underlay_zone" --remove-rich-rule="$journal_rule" >/dev/null 2>&1 || exit 79 - else - [ "$journal_underlay_rule_runtime" = "$journal_rule_runtime" ] || exit 79 - fi - if [ "$journal_underlay_rule_permanent" = no ] && [ "$journal_rule_permanent" = yes ]; then - "$FIREWALL_CMD" --permanent --zone="$journal_underlay_zone" --remove-rich-rule="$journal_rule" >/dev/null 2>&1 || exit 79 - else - [ "$journal_underlay_rule_permanent" = "$journal_rule_permanent" ] || exit 79 - fi - [ "$(vc_firewall_rule_state runtime "$journal_underlay_zone" "$journal_rule")" = "$journal_underlay_rule_runtime" ] || exit 79 - [ "$(vc_firewall_rule_state permanent "$journal_underlay_zone" "$journal_rule")" = "$journal_underlay_rule_permanent" ] || exit 79 - else - [ "$journal_underlay_rule_owner" = external ] && - [ "$journal_rule_runtime:$journal_rule_permanent" = yes:yes ] || exit 79 - fi - if [ "$journal_mode" = networkmanager_nmcli ] && - [ "$journal_firewalld_runtime_zone" = trusted ] && - [ "$journal_iface_before" = yes ]; then - # Only exact post-restore nft/zone/rich-rule evidence permits the - # recovered profile to manage and activate wg-fabric0 again. - vc_fabric_nm_release || exit 79 - if [ "$journal_iface_up_before" = yes ]; then - "$journal_nmcli" connection up uuid "$journal_old_uuid" >/dev/null 2>&1 || exit 79 - else - "$IP" link set dev "$IFACE" down >/dev/null 2>&1 || exit 79 - fi - fi - fi # Manager activation must preserve the value restored while down. Never # correct a post-activation mismatch on a live link; retain the journal. if ! journal_verify_rp_filter; then - if [ "$journal_firewall_mode" = firewalld_nft ]; then - vc_fabric_quarantine down >/dev/null 2>&1 || true - elif [ "$journal_mode" = networkmanager_nmcli ]; then + if [ "$journal_mode" = networkmanager_nmcli ]; then vc_fabric_nm_quarantine >/dev/null 2>&1 || true else "$IP" link set dev "$IFACE" down >/dev/null 2>&1 || true @@ -5639,8 +4326,8 @@ if [ "$RECOVER_ONLY" = yes ]; then exit 0 fi # A first deployment can be interrupted before its generator is published. -# Recovery first proves the journal and complete plan while the emergency -# packet stop remains installed; only normal configuration needs the live +# Recovery first proves the journal and complete plan while the Fabric and VPC +# interfaces remain quarantined; only normal configuration needs the live gate # generator. An old helper can likewise restore a journaled newer generator # before this current-plan assertion is meaningful. VC_FABRIC_HELPER_PHASE=gate @@ -5694,7 +4381,7 @@ if [ "${1:-}" = --ifdown ]; then "$IP" -d link show dev "$IFACE" 2>/dev/null | grep -Eq '(^|[[:space:]])wireguard([[:space:]]|$)' || exit 71 "$IP" link del dev "$IFACE" || exit 72 fi - vc_fabric_emergency_remove || exit 79 + vc_legacy_firewalld_state_absent || exit 79 exit 0 fi NMCLI= @@ -5777,13 +4464,6 @@ if [ "$DEPLOYMENT_MODE" = ifupdown ] && exit 0 fi fi -if [ "$FIREWALL_MODE" = firewalld_nft ] && - [ "${VC_FABRIC_APPLYING:-0}" != 1 ]; then - # Quarantine from immutable helper/profile ownership before parsing the - # mutable config or underlay. A later validation failure therefore - # cannot leave an already-trusted overlay live without its raw nft guard. - vc_fabric_quarantine || exit 71 -fi vc_firewall_mode_ready || exit 71 VC_FABRIC_HELPER_PHASE=config [ -f "$CONF" ] || exit 72 @@ -5868,13 +4548,6 @@ fabric_seen=0 revision_seen=0 digest_seen=0 firewall_seen=0 -underlay_seen=0 -firewall_rule_seen=0 -underlay_scope= -underlay_endpoint= -underlay_iface= -underlay_zone= -underlay_rule_owner= underlay_probes= listen_seen=0 network_seen=0 @@ -5909,27 +4582,10 @@ while IFS='|' read -r kind a b c d; do [ "$a" = "$FIREWALL_MODE" ] || exit 72 [ "$firewall_seen" = 0 ] || exit 72 firewall_seen=1 ;; - underlay) - [ "$FIREWALL_MODE" = firewalld_nft ] && [ "$a" = local ] || exit 72 - is_ipv4 "$b" && [ "$b" = "$UNDERLAY_ENDPOINT" ] || exit 72 - case "$c" in ''|*[!A-Za-z0-9_.:-]*) exit 72 ;; esac - [ "$(printf '%s' "$c" | awk 'length($0)<=15')" = "$c" ] || exit 72 - case "$d" in ''|*[!A-Za-z0-9_-]*) exit 72 ;; esac - [ "$(printf '%s' "$d" | awk 'length($0)<=64')" = "$d" ] || exit 72 - [ "$underlay_seen" = 0 ] || exit 72 - underlay_scope=$a - underlay_endpoint=$b - underlay_iface=$c - underlay_zone=$d - underlay_seen=1 ;; - firewall_rule) - [ "$FIREWALL_MODE" = firewalld_nft ] || exit 72 - is_port "$a" && [ "$a" = "$UNDERLAY_PORT" ] || exit 72 - case "$b" in panel|external) : ;; *) exit 72 ;; esac - [ -z "$c$d" ] || exit 72 - [ "$firewall_rule_seen" = 0 ] || exit 72 - underlay_rule_owner=$b - firewall_rule_seen=1 ;; + underlay|firewall_rule) + # These records belonged to the retired firewalld integration. + # Refuse old plans instead of silently adopting their ownership. + exit 72 ;; network) is_overlay_network "$a" || exit 72 [ "$a" = "$VPC_FABRIC_NETWORK" ] || exit 72 @@ -5976,19 +4632,6 @@ done < "$CONF" [ "$FIREWALL_MODE" = native_nftables ] || exit 72 firewall_seen=1 } -[ "$FIREWALL_MODE" = firewalld_nft ] && { - [ "$underlay_seen" = 1 ] && [ "$firewall_rule_seen" = 1 ] || exit 72 - underlay_live=$(vc_underlay_discover "$UNDERLAY_ENDPOINT" "$underlay_probes") || exit 71 - [ "$underlay_live" = "$underlay_iface|$underlay_zone" ] || exit 71 - underlay_rule=$(vc_firewall_rich_rule "$UNDERLAY_ENDPOINT" "$UNDERLAY_PORT") || exit 71 - underlay_runtime_state=$(vc_firewall_rule_state runtime "$underlay_zone" "$underlay_rule") || exit 71 - underlay_permanent_state=$(vc_firewall_rule_state permanent "$underlay_zone" "$underlay_rule") || exit 71 - if [ "$underlay_rule_owner" = external ]; then - [ "$underlay_runtime_state" = yes ] && [ "$underlay_permanent_state" = yes ] || exit 71 - fi -} || { - [ "$underlay_seen" = 0 ] && [ "$firewall_rule_seen" = 0 ] || exit 72 -} [ "$schema_seen" = 1 ] && [ "$fabric_seen" = 1 ] && [ "$revision_seen" = 1 ] && [ "$digest_seen" = 1 ] && [ "$firewall_seen" = 1 ] && [ "$listen_seen" = 1 ] && [ "$network_seen" = 1 ] && [ "$overlay_seen" = 1 ] || exit 72 is_hex64 "$digest_value" || exit 72 ACTIVE_DIGEST="$digest_value" @@ -6032,9 +4675,8 @@ fi [ -f "$OWNER" ] && [ "$($STAT -c '%u' "$OWNER" 2>/dev/null)" = 0 ] || exit 71 [ "$($STAT -c '%a' "$OWNER" 2>/dev/null)" = 600 ] || exit 71 if [ -f "$OWNER" ] && [ "$(cat "$OWNER")" != "virt-cloud-wg-fabric-v1" ]; then exit 71; fi -# A periodic repair can run while the overlay is already trusted. Quarantine -# it before even inspecting/replacing the guard so a missing table never -# creates a trusted-without-ACL repair window. +# A periodic repair can run while the overlay is already live. Close the +# panel-owned link before inspecting or replacing its nft guard. vc_fabric_quarantine || exit 71 VC_FABRIC_HELPER_PHASE=nft NFT_RULES=/etc/nftables.d/virt-cloud-wg-fabric.nft @@ -6047,27 +4689,22 @@ nft_first=; nft_second= [ "$nft_first" = '# virt-cloud-wg-fabric-v1' ] || exit 71 [ "$nft_second" = "# fabric-plan-digest=$EXPECTED_DIGEST" ] || exit 71 nft_persisted=no -if [ "$FIREWALL_MODE" = native_nftables ]; then - nft_main_count=0 - for nft_main in /etc/nftables.conf /etc/sysconfig/nftables.conf; do - [ -f "$nft_main" ] && [ ! -L "$nft_main" ] || continue - [ "$($STAT -c '%u' "$nft_main" 2>/dev/null)" = 0 ] || exit 71 - nft_main_mode=$($STAT -c '%a' "$nft_main" 2>/dev/null || true) - case "$nft_main_mode" in [0-7][0145][0145]) : ;; *) exit 71 ;; esac - nft_main_count=$((nft_main_count + 1)) - if grep -Fqx 'include "/etc/nftables.d/virt-cloud-wg-fabric.nft"' "$nft_main" || - grep -Fqx 'include "/etc/nftables.d/*.nft"' "$nft_main" || - grep -Fqx 'include "/etc/nftables.d/*"' "$nft_main"; then - nft_persisted=yes - fi - done - [ "$nft_main_count" = 1 ] && [ "$nft_persisted" = yes ] || exit 71 - [ ! -e "$NFT_MANIFEST" ] && [ ! -L "$NFT_MANIFEST" ] || exit 71 -else - [ "$FIREWALL_MODE" = firewalld_nft ] || exit 71 - vc_nft_manifest_validate "$NFT_MANIFEST" || exit 71 - nft_persisted=yes -fi +nft_main_count=0 +for nft_main in /etc/nftables.conf /etc/sysconfig/nftables.conf; do + [ -f "$nft_main" ] && [ ! -L "$nft_main" ] || continue + [ "$($STAT -c '%u' "$nft_main" 2>/dev/null)" = 0 ] || exit 71 + nft_main_mode=$($STAT -c '%a' "$nft_main" 2>/dev/null || true) + case "$nft_main_mode" in [0-7][0145][0145]) : ;; *) exit 71 ;; esac + nft_main_count=$((nft_main_count + 1)) + if grep -Fqx 'include "/etc/nftables.d/virt-cloud-wg-fabric.nft"' "$nft_main" || + grep -Fqx 'include "/etc/nftables.d/*.nft"' "$nft_main" || + grep -Fqx 'include "/etc/nftables.d/*"' "$nft_main"; then + nft_persisted=yes + fi +done +[ "$nft_main_count" = 1 ] && [ "$nft_persisted" = yes ] || exit 71 +[ ! -e /etc/virt-cloud/nftables.conf ] && +[ ! -L /etc/virt-cloud/nftables.conf ] || exit 71 validate_nft_table() { vc_fabric_nft_strict_shape "$1" || return 1 # The strict grammar above already proves all eight complete expressions. @@ -6099,31 +4736,14 @@ if nft_table=$("$NFT" -s list table inet virt_cloud_wg_fabric 2>/dev/null); then else "$NFT" -f "$NFT_RULES" || exit 72 nft_table=$("$NFT" -s list table inet virt_cloud_wg_fabric 2>/dev/null) || exit 72 - validate_nft_table "$nft_table" || exit 71 -fi -vpc_state_refresh_nft || exit 71 -fabric_acl_ready || exit 71 -if [ "$FIREWALL_MODE" = firewalld_nft ]; then - underlay_runtime_state=$(vc_firewall_rule_state runtime "$underlay_zone" "$underlay_rule") || exit 71 - underlay_permanent_state=$(vc_firewall_rule_state permanent "$underlay_zone" "$underlay_rule") || exit 71 - if [ "$underlay_rule_owner" = panel ]; then - if [ "$underlay_permanent_state" = no ]; then - "$FIREWALL_CMD" --permanent --zone="$underlay_zone" --add-rich-rule="$underlay_rule" >/dev/null 2>&1 || exit 72 - fi - if [ "$underlay_runtime_state" = no ]; then - "$FIREWALL_CMD" --zone="$underlay_zone" --add-rich-rule="$underlay_rule" >/dev/null 2>&1 || exit 72 - fi - else - [ "$underlay_rule_owner" = external ] && - [ "$underlay_runtime_state:$underlay_permanent_state" = yes:yes ] || exit 71 - fi - [ "$(vc_firewall_rule_state runtime "$underlay_zone" "$underlay_rule")" = yes ] || exit 71 - [ "$(vc_firewall_rule_state permanent "$underlay_zone" "$underlay_rule")" = yes ] || exit 71 + validate_nft_table "$nft_table" || exit 71 fi +vpc_state_refresh_nft || exit 71 +fabric_acl_ready || exit 71 # NetworkManager can create WireGuard while activating a profile. Create the # already-reserved, panel-owned link administratively down first so its runtime # rp_filter is converged and read back before manager activation can carry any -# packet. The emergency guard remains installed throughout this transition. +# packet. Legacy emergency artifacts were rejected before this transition. VC_FABRIC_HELPER_PHASE=activation if ! "$IP" link show dev "$IFACE" >/dev/null 2>&1; then "$IP" link add dev "$IFACE" type wireguard || exit 72 @@ -6134,40 +4754,20 @@ manager_owner=missing if [ "$DEPLOYMENT_MODE" = networkmanager_nmcli ]; then vc_fabric_nm_profile_owned expected || exit 71 "$NMCLI" connection load "$NM_PROFILE" >/dev/null 2>&1 || exit 72 - if [ "$FIREWALL_MODE" = firewalld_nft ]; then - [ "$(vc_firewall_zone permanent)" = none ] || exit 71 - fabric_acl_ready || exit 71 - "$NMCLI" connection modify --temporary uuid "$CONNECTION_UUID" connection.zone trusted >/dev/null 2>&1 || exit 72 - [ "$("$NMCLI" -t --escape no -g connection.zone connection show uuid "$CONNECTION_UUID" 2>/dev/null)" = trusted ] || exit 71 - grep -Eq '^zone=' "$NM_PROFILE" && exit 71 || true - fi + nm_memory_zone=$("$NMCLI" -t --escape no -g connection.zone connection show uuid "$CONNECTION_UUID" 2>/dev/null | sed -n '1p') + [ -z "$nm_memory_zone" ] || [ "$nm_memory_zone" = -- ] || exit 71 + grep -Eq '^zone=' "$NM_PROFILE" && exit 71 || true vc_fabric_nm_release || exit 72 "$NMCLI" connection up uuid "$CONNECTION_UUID" >/dev/null 2>&1 || exit 72 active_connection=$("$NMCLI" -g GENERAL.CONNECTION device show "$IFACE" 2>/dev/null || true) [ "$active_connection" = "$CONNECTION" ] || exit 72 vc_fabric_nm_active || exit 72 - if [ "$FIREWALL_MODE" = firewalld_nft ]; then - [ "$(vc_firewall_zone runtime)" = trusted ] || exit 71 - [ "$(vc_firewall_zone permanent)" = none ] || exit 71 - fi manager_owner=owned else validate_ifupdown_fragment || exit 71 [ ! -e "/etc/sysconfig/network-scripts/ifcfg-$IFACE" ] && [ ! -L "/etc/sysconfig/network-scripts/ifcfg-$IFACE" ] || exit 71 "$IFQUERY" "$IFACE" >/dev/null 2>&1 || exit 72 - if [ "$FIREWALL_MODE" = firewalld_nft ]; then - runtime_zone=$(vc_firewall_zone runtime) || exit 71 - permanent_zone=$(vc_firewall_zone permanent) || exit 71 - case "$runtime_zone:$permanent_zone" in - none:none|trusted:none) : ;; - *) exit 71 ;; - esac - fabric_acl_ready || exit 71 - "$FIREWALL_CMD" --zone=trusted --add-interface="$IFACE" >/dev/null 2>&1 || exit 72 - [ "$(vc_firewall_zone permanent)" = none ] || exit 71 - [ "$(vc_firewall_zone runtime)" = trusted ] || exit 71 - fi manager_owner=owned fi if "$IP" link show dev "$IFACE" >/dev/null 2>&1; then @@ -6183,108 +4783,6 @@ recover_vpc_dataplanes() { vpc_boot_bridge_port_names() { vc_bridge_port_names "$1" } - vpc_boot_nft_snapshots_match() { - for vpc_boot_family in inet bridge arp; do - vpc_boot_snapshot="$VPC_STATE_DIR/$vpc_token.nft.$vpc_boot_family.live" - [ -f "$vpc_boot_snapshot" ] && [ ! -L "$vpc_boot_snapshot" ] || - return 1 - [ "$($STAT -c '%u:%a' "$vpc_boot_snapshot" 2>/dev/null)" = 0:600 ] || - return 1 - vpc_boot_live=$(mktemp /run/virt-cloud/.vpc-nft-live.XXXXXX) || - return 1 - _tmp_vpc_nft_batch=$vpc_boot_live - if ! "$NFT" -s list table "$vpc_boot_family" "$vpc_nft_name" \ - > "$vpc_boot_live" 2>/dev/null || - ! "$CMP" -s "$vpc_boot_live" "$vpc_boot_snapshot"; then - rm -f "$vpc_boot_live" - _tmp_vpc_nft_batch= - return 1 - fi - rm -f "$vpc_boot_live" - _tmp_vpc_nft_batch= - done - } - if [ "$FIREWALL_MODE" = firewalld_nft ] && [ -d "$VPC_STATE_DIR" ]; then - # First pass is intentionally independent of state contents. A bad - # later state record must not leave an earlier panel-owned bridge live - # and trusted. The reserved token + root-owned marker is the minimum - # evidence needed to disconnect/down the bridge; full profile/state - # ownership is validated only after every bridge has been closed. - vpc_bulk_quarantine_failed=no - for vpc_quarantine_state in "$VPC_STATE_DIR"/*.state; do - [ -e "$vpc_quarantine_state" ] || [ -L "$vpc_quarantine_state" ] || continue - vpc_quarantine_token=${vpc_quarantine_state##*/} - vpc_quarantine_token=${vpc_quarantine_token%.state} - if ! printf '%s\n' "$vpc_quarantine_token" | - grep -Eq '^[0-9a-f]{11}$'; then - vpc_bulk_quarantine_failed=yes - continue - fi - vpc_quarantine_bridge="vcv-$vpc_quarantine_token" - vpc_quarantine_owner="$VPC_STATE_DIR/$vpc_quarantine_token.owner" - if [ ! -f "$vpc_quarantine_owner" ] || [ -L "$vpc_quarantine_owner" ] || - [ "$($STAT -c '%u:%a' "$vpc_quarantine_owner" 2>/dev/null)" != 0:600 ] || - ! grep -Fqx 'virt-cloud-vpc-dataplane-v1' "$vpc_quarantine_owner"; then - vpc_bulk_quarantine_failed=yes - continue - fi - if "$IP" link show dev "$vpc_quarantine_bridge" >/dev/null 2>&1; then - if ! "$IP" -d link show dev "$vpc_quarantine_bridge" 2>/dev/null | - grep -Eq '(^|[[:space:]])bridge[[:space:]]+(forward_delay|stp_state|hello_time|ageing_time)([[:space:]]|$)'; then - vpc_bulk_quarantine_failed=yes - continue - fi - if ! vpc_quarantine_ports_before=$(vc_bridge_port_names \ - "$vpc_quarantine_bridge"); then - vpc_bulk_quarantine_failed=yes - continue - fi - if [ "$DEPLOYMENT_MODE" = networkmanager_nmcli ]; then - "$NMCLI" device set "$vpc_quarantine_bridge" managed no >/dev/null 2>&1 || - vpc_bulk_quarantine_failed=yes - vpc_quarantine_managed=$( - "$NMCLI" -t --escape no -g GENERAL.NM-MANAGED \ - device show "$vpc_quarantine_bridge" 2>/dev/null | sed -n '1p' - ) || vpc_bulk_quarantine_failed=yes - [ "$vpc_quarantine_managed" = no ] || vpc_bulk_quarantine_failed=yes - fi - if ! vc_vpc_bridge_force_down "$vpc_quarantine_bridge"; then - vpc_bulk_quarantine_failed=yes - continue - fi - if ! vpc_quarantine_ports_after=$(vc_bridge_port_names \ - "$vpc_quarantine_bridge"); then - vpc_bulk_quarantine_failed=yes - continue - fi - [ "$vpc_quarantine_ports_after" = "$vpc_quarantine_ports_before" ] || - vpc_bulk_quarantine_failed=yes - fi - vpc_quarantine_permanent=$(vc_firewall_zone permanent "$vpc_quarantine_bridge" 2>/dev/null || printf foreign) - [ "$vpc_quarantine_permanent" = none ] || vpc_bulk_quarantine_failed=yes - vpc_quarantine_runtime=$(vc_firewall_zone runtime "$vpc_quarantine_bridge" 2>/dev/null || printf foreign) - if [ "$vpc_quarantine_runtime" = trusted ]; then - # Unmanaging or lowering the link does not promise that - # firewalld drops an existing binding. Once panel ownership, - # bridge type, port stability and link-down are proven, remove - # trusted explicitly before parsing any later state record. - "$FIREWALL_CMD" --zone=trusted --remove-interface="$vpc_quarantine_bridge" >/dev/null 2>&1 || - vpc_bulk_quarantine_failed=yes - vpc_quarantine_runtime=$(vc_firewall_zone runtime "$vpc_quarantine_bridge" 2>/dev/null || printf foreign) - fi - if [ "$DEPLOYMENT_MODE" = networkmanager_nmcli ]; then - # NetworkManager/firewalld may expose an unbound device through - # the non-trusted default zone. That is a safe quarantine; - # foreign or trusted bindings still fail closed. - vc_firewall_quarantined "$vpc_quarantine_bridge" yes || - vpc_bulk_quarantine_failed=yes - else - vc_firewall_quarantined "$vpc_quarantine_bridge" no || - vpc_bulk_quarantine_failed=yes - fi - done - [ "$vpc_bulk_quarantine_failed" = no ] || exit 71 - fi # A VPC transaction journal means its apply/remove outcome is not yet # reconciled. Never let Fabric repair replay state across that boundary. for vpc_txn in "$BASE"/.vpc-dataplane-*.txn; do @@ -6292,6 +4790,12 @@ recover_vpc_dataplanes() { exit 79 done [ -d "$VPC_STATE_DIR" ] || return 0 + # Snapshot artifacts were written only by the retired firewalld mode. + # Their presence requires an explicit maintenance cleanup/migration. + for vpc_legacy_snapshot in "$VPC_STATE_DIR"/*.nft.*.live; do + [ -e "$vpc_legacy_snapshot" ] || [ -L "$vpc_legacy_snapshot" ] || continue + exit 79 + done # Validate every VPC policy-rule slot before repairing any VPC bridge, # route, sysctl or proxy entry. A collision in a later state file must not # leave earlier VPCs partially repaired on each timer run. @@ -6363,73 +4867,9 @@ recover_vpc_dataplanes() { [ -f "$vpc_nft_file" ] && [ ! -L "$vpc_nft_file" ] || exit 71 [ "$($STAT -c '%u:%a' "$vpc_nft_file" 2>/dev/null)" = 0:600 ] || exit 71 grep -Fqx '# virt-cloud-vpc-dataplane-v1' "$vpc_nft_file" || exit 71 - if [ "$FIREWALL_MODE" = firewalld_nft ]; then - grep -Fqx "include \"$vpc_nft_file\"" "$NFT_MANIFEST" || exit 71 - fi vpc_state_firewall_count=$(awk -F'|' '$1=="firewall_mode" {count++; value=$2} END {print (count + 0) "|" value}' "$vpc_state_file") - if [ "$FIREWALL_MODE" = native_nftables ]; then - case "$vpc_state_firewall_count" in 0\||1\|native_nftables) : ;; *) exit 71 ;; esac - else - [ "$FIREWALL_MODE" = firewalld_nft ] && - [ "$vpc_state_firewall_count" = '1|firewalld_nft' ] || exit 71 - fi - # Timer repair must close an already-trusted bridge before inspecting - # or replacing its nft tables. Validate the manager artifact first so - # no foreign profile/fragment is ever modified. - if [ "$FIREWALL_MODE" = firewalld_nft ]; then - vpc_quarantine_allow_default=no - if [ "$DEPLOYMENT_MODE" = networkmanager_nmcli ]; then - vpc_quarantine_allow_default=yes - vpc_nm_profile_count=$("$NMCLI" -t --escape no -f NAME connection show 2>/dev/null | awk -v wanted="$vpc_bridge" '$0==wanted {count++} END {print count + 0}') || exit 71 - [ "$vpc_nm_profile_count" = 1 ] || exit 71 - vpc_nm_uuid=$("$NMCLI" -t --escape no -g connection.uuid connection show "$vpc_bridge" 2>/dev/null | sed -n '1p') || exit 71 - printf '%s\n' "$vpc_nm_uuid" | grep -Eq '^[0-9A-Fa-f]{8}-[0-9A-Fa-f]{4}-[0-9A-Fa-f]{4}-[0-9A-Fa-f]{4}-[0-9A-Fa-f]{12}$' || exit 71 - vpc_nm_file=$(vc_nm_filename_for_uuid "$vpc_nm_uuid") || exit 71 - case "$vpc_nm_file" in /etc/NetworkManager/system-connections/*.nmconnection) : ;; *) exit 71 ;; esac - [ -f "$vpc_nm_file" ] && [ ! -L "$vpc_nm_file" ] || exit 71 - [ "$($STAT -c '%u:%a' "$vpc_nm_file" 2>/dev/null)" = 0:600 ] || exit 71 - awk -F= -v wanted_id="$vpc_bridge" -v wanted_uuid="$vpc_nm_uuid" \ - -v wanted_marker="virt-cloud-vpc-dataplane-v1 vpc=$vpc_id" ' - /^\[/ {in_connection=($0=="[connection]"); next} - in_connection && $1=="id" {id++; if (substr($0,4)!=wanted_id) bad=1; next} - in_connection && $1=="uuid" {uuid++; if (substr($0,6)!=wanted_uuid) bad=1; next} - in_connection && $1=="type" {type++; if (substr($0,6)!="bridge") bad=1; next} - in_connection && $1=="interface-name" {iface++; if (substr($0,16)!=wanted_id) bad=1; next} - in_connection && $1=="stable-id" {stable++; if (substr($0,11)!=wanted_marker) bad=1; next} - in_connection && $1=="autoconnect" {auto++; if (substr($0,13)!="false") bad=1; next} - in_connection && $1=="autoconnect-retries" {retry++; if (substr($0,21)!="0") bad=1; next} - in_connection && $1=="zone" {zone++} - END {exit (bad || id!=1 || uuid!=1 || type!=1 || iface!=1 || stable!=1 || auto!=1 || retry!=1 || zone!=0) ? 1 : 0} - ' "$vpc_nm_file" || exit 71 - [ "$(vc_firewall_zone permanent "$vpc_bridge")" = none ] || exit 71 - vc_firewall_quarantined "$vpc_bridge" yes || exit 71 - [ "$("$NMCLI" -t --escape no -g GENERAL.NM-MANAGED device show "$vpc_bridge" 2>/dev/null | sed -n '1p')" = no ] || exit 71 - "$NMCLI" connection modify --temporary uuid "$vpc_nm_uuid" connection.zone "" >/dev/null 2>&1 || exit 72 - vpc_nm_memory_zone=$("$NMCLI" -t --escape no -g connection.zone connection show uuid "$vpc_nm_uuid" 2>/dev/null | sed -n '1p') - [ -z "$vpc_nm_memory_zone" ] || [ "$vpc_nm_memory_zone" = -- ] || exit 71 - vc_firewall_quarantined "$vpc_bridge" yes || exit 71 - else - vpc_if_fragment="/etc/network/interfaces.d/virt-cloud-vpc-$vpc_token" - [ -f "$vpc_if_fragment" ] && [ ! -L "$vpc_if_fragment" ] || exit 71 - [ "$($STAT -c '%u:%a' "$vpc_if_fragment" 2>/dev/null)" = 0:600 ] || exit 71 - awk -v owner="# virt-cloud-vpc-dataplane-v1 vpc=$vpc_id revision=$vpc_revision digest=$vpc_digest" -v bridge="$vpc_bridge" ' - NR==1 && $0==owner {next} - NR==2 && $0=="iface " bridge " inet manual" {next} - NR==3 && $0==" bridge_ports none" {next} - NR==4 && $0==" bridge-stp off" {next} - {bad=1} - END {exit (NR==4 && !bad) ? 0 : 1} - ' "$vpc_if_fragment" || exit 71 - [ ! -e "/etc/sysconfig/network-scripts/ifcfg-$vpc_bridge" ] && - [ ! -L "/etc/sysconfig/network-scripts/ifcfg-$vpc_bridge" ] || exit 71 - [ "$(vc_firewall_zone permanent "$vpc_bridge")" = none ] || exit 71 - vpc_runtime_zone=$(vc_firewall_zone runtime "$vpc_bridge") || exit 71 - if [ "$vpc_runtime_zone" = trusted ]; then - "$FIREWALL_CMD" --zone=trusted --remove-interface="$vpc_bridge" >/dev/null 2>&1 || exit 72 - fi - fi - vc_firewall_quarantined "$vpc_bridge" "$vpc_quarantine_allow_default" || exit 71 - fi + case "$vpc_state_firewall_count" in 0\||1\|native_nftables) : ;; *) exit 79 ;; esac + [ "$(awk -F'|' '$1=="nft_snapshot" {count++} END {print count + 0}' "$vpc_state_file")" = 0 ] || exit 79 [ "$(wc -c < "$vpc_nft_file")" -le 262144 ] 2>/dev/null || exit 71 vpc_nft_name="virt_cloud_vpc_$(printf '%s' "$vpc_hash" | awk '{print substr($0,1,12)}')" vpc_nft_tables=$(awk '$1=="table" {print $2 "|" $3}' "$vpc_nft_file") @@ -6438,85 +4878,7 @@ recover_vpc_dataplanes() { for vpc_required_family in inet bridge arp; do [ "$(printf '%s\n' "$vpc_nft_tables" | grep -Fxc "$vpc_required_family|$vpc_nft_name")" = 1 ] || exit 71 done - if [ "$FIREWALL_MODE" = firewalld_nft ]; then - # A comment is not a semantic snapshot: an out-of-band rule can - # retain every owner marker while changing a drop verdict to - # accept. Trust is restored only after all three stateless nft - # dumps match the root-owned snapshots written by VPC apply. - [ "$(awk -F'|' '$1=="nft_snapshot" {count++} END {print count + 0}' "$vpc_state_file")" = 3 ] || exit 71 - vpc_snapshot_marker= - vpc_nft_reload=no - vpc_existing_families= - for vpc_family in inet bridge arp; do - vpc_snapshot_file="$VPC_STATE_DIR/$vpc_token.nft.$vpc_family.live" - vpc_snapshot_record=$(awk -F'|' -v family="$vpc_family" -v path="$vpc_snapshot_file" ' - $1=="nft_snapshot" && $2==family && $3==path && NF==4 {count++; marker=$4} - END {print (count + 0) "|" marker} - ' "$vpc_state_file") || exit 71 - [ "${vpc_snapshot_record%%|*}" = 1 ] || exit 71 - vpc_family_marker=${vpc_snapshot_record#*|} - printf '%s\n' "$vpc_family_marker" | - grep -Eq "^virt-cloud-vpc-dataplane-v1:$vpc_token:[0-9a-f]{16}$" || exit 71 - if [ -n "$vpc_snapshot_marker" ]; then - [ "$vpc_family_marker" = "$vpc_snapshot_marker" ] || exit 71 - else - vpc_snapshot_marker=$vpc_family_marker - fi - [ -f "$vpc_snapshot_file" ] && [ ! -L "$vpc_snapshot_file" ] || exit 71 - [ "$($STAT -c '%u:%a' "$vpc_snapshot_file" 2>/dev/null)" = 0:600 ] || exit 71 - grep -Fq "comment \"$vpc_snapshot_marker:" "$vpc_snapshot_file" || exit 71 - vpc_live_snapshot=$(mktemp /run/virt-cloud/.vpc-nft-live.XXXXXX) || exit 72 - _tmp_vpc_nft_batch=$vpc_live_snapshot - if "$NFT" -s list table "$vpc_family" "$vpc_nft_name" > "$vpc_live_snapshot" 2>/dev/null; then - # Existing-but-different tables are never deleted by the - # timer. They stay quarantined for explicit reconciliation. - "$CMP" -s "$vpc_live_snapshot" "$vpc_snapshot_file" || exit 71 - vpc_existing_families="$vpc_existing_families $vpc_family" - else - vpc_nft_reload=yes - fi - rm -f "$vpc_live_snapshot" - _tmp_vpc_nft_batch= - done - if [ "$vpc_nft_reload" = yes ]; then - vpc_nft_batch=$(mktemp /run/virt-cloud/.vpc-nft-batch.XXXXXX) || exit 72 - _tmp_vpc_nft_batch=$vpc_nft_batch - : > "$vpc_nft_batch" - for vpc_family in $vpc_existing_families; do - vpc_snapshot_file="$VPC_STATE_DIR/$vpc_token.nft.$vpc_family.live" - vpc_live_snapshot=$(mktemp /run/virt-cloud/.vpc-nft-live.XXXXXX) || exit 72 - "$NFT" -s list table "$vpc_family" "$vpc_nft_name" > "$vpc_live_snapshot" 2>/dev/null || exit 71 - "$CMP" -s "$vpc_live_snapshot" "$vpc_snapshot_file" || exit 71 - rm -f "$vpc_live_snapshot" - printf 'delete table %s %s\n' "$vpc_family" "$vpc_nft_name" >> "$vpc_nft_batch" - done - cat "$vpc_nft_file" >> "$vpc_nft_batch" - "$NFT" -c -f "$vpc_nft_batch" >/dev/null 2>&1 || exit 72 - for vpc_family in $vpc_existing_families; do - vpc_snapshot_file="$VPC_STATE_DIR/$vpc_token.nft.$vpc_family.live" - vpc_live_snapshot=$(mktemp /run/virt-cloud/.vpc-nft-live.XXXXXX) || exit 72 - _tmp_vpc_nft_batch=$vpc_live_snapshot - "$NFT" -s list table "$vpc_family" "$vpc_nft_name" > "$vpc_live_snapshot" 2>/dev/null || exit 71 - "$CMP" -s "$vpc_live_snapshot" "$vpc_snapshot_file" || exit 71 - rm -f "$vpc_live_snapshot" - _tmp_vpc_nft_batch=$vpc_nft_batch - done - "$NFT" -f "$vpc_nft_batch" >/dev/null 2>&1 || exit 72 - rm -f "$vpc_nft_batch" - _tmp_vpc_nft_batch= - fi - for vpc_family in inet bridge arp; do - vpc_snapshot_file="$VPC_STATE_DIR/$vpc_token.nft.$vpc_family.live" - vpc_live_snapshot=$(mktemp /run/virt-cloud/.vpc-nft-live.XXXXXX) || exit 72 - _tmp_vpc_nft_batch=$vpc_live_snapshot - "$NFT" -s list table "$vpc_family" "$vpc_nft_name" > "$vpc_live_snapshot" 2>/dev/null || exit 71 - "$CMP" -s "$vpc_live_snapshot" "$vpc_snapshot_file" || exit 71 - rm -f "$vpc_live_snapshot" - _tmp_vpc_nft_batch= - done - else - # Native nftables retains the legacy ownership-shape readback; - # firewalld state is the only mode that may become trusted here. + # Native nftables ownership-shape readback. vpc_expected_marker_count=$(grep -Eoc 'comment "virt-cloud-vpc-dataplane-v1:[^"]+"' "$vpc_nft_file" || true) vpc_expected_marker_unique=$(grep -Eo 'comment "virt-cloud-vpc-dataplane-v1:[^"]+"' "$vpc_nft_file" | LC_ALL=C sort -u | "$SHA256" | awk '{print $1}') [ "$vpc_expected_marker_count" -gt 0 ] 2>/dev/null || exit 71 @@ -6574,7 +4936,6 @@ recover_vpc_dataplanes() { [ "$(LC_ALL=C sort -u "$vpc_live_marker_file" | "$SHA256" | awk '{print $1}')" = "$vpc_expected_marker_unique" ] || exit 71 rm -f "$vpc_live_marker_file" _tmp_vpc_nft_batch= - fi vpc_sysctl_file="/etc/sysctl.d/99-virt-cloud-vpc-$vpc_token.conf" vc_vpc_sysctl_artifact_owned "$vpc_sysctl_file" "$vpc_id" \ "$vpc_revision" "$vpc_digest" "$vpc_bridge" || exit 71 @@ -6641,17 +5002,8 @@ recover_vpc_dataplanes() { [ "$(nm_vpc_field connection.autoconnect uuid "$vpc_nm_uuid")" = no ] || exit 71 [ "$(nm_vpc_field connection.autoconnect-retries uuid "$vpc_nm_uuid")" = 0 ] || exit 71 [ "$(nm_vpc_field connection.stable-id uuid "$vpc_nm_uuid")" = "virt-cloud-vpc-dataplane-v1 vpc=$vpc_id" ] || exit 71 - if [ "$FIREWALL_MODE" = firewalld_nft ]; then - [ "$(vc_firewall_zone permanent "$vpc_bridge")" = none ] || exit 71 - "$NMCLI" connection modify --temporary uuid "$vpc_nm_uuid" connection.zone "" >/dev/null 2>&1 || exit 72 - [ "$("$NMCLI" -t --escape no -g GENERAL.NM-MANAGED device show "$vpc_bridge" 2>/dev/null | sed -n '1p')" = no ] || exit 71 - nm_vpc_zone=$(nm_vpc_field connection.zone uuid "$vpc_nm_uuid") - [ -z "$nm_vpc_zone" ] || [ "$nm_vpc_zone" = -- ] || exit 71 - vpc_runtime_zone=$(vc_firewall_zone runtime "$vpc_bridge") || exit 71 - vpc_default_zone=$($FIREWALL_CMD --get-default-zone 2>/dev/null) || exit 71 - [ "$vpc_default_zone" != trusted ] || exit 71 - [ "$vpc_runtime_zone" = none ] || [ "$vpc_runtime_zone" = "$vpc_default_zone" ] || exit 71 - fi + nm_vpc_zone=$(nm_vpc_field connection.zone uuid "$vpc_nm_uuid") + [ -z "$nm_vpc_zone" ] || [ "$nm_vpc_zone" = -- ] || exit 71 if "$IP" link show dev "$vpc_bridge" >/dev/null 2>&1; then "$NMCLI" device set "$vpc_bridge" managed yes >/dev/null 2>&1 || exit 72 [ "$("$NMCLI" -t --escape no -g GENERAL.NM-MANAGED device show "$vpc_bridge" 2>/dev/null | sed -n '1p')" = yes ] || exit 72 @@ -6664,24 +5016,16 @@ recover_vpc_dataplanes() { [ -f "$vpc_if_fragment" ] && [ ! -L "$vpc_if_fragment" ] || exit 71 [ "$($STAT -c '%u:%a' "$vpc_if_fragment" 2>/dev/null)" = 0:600 ] || exit 71 grep -Fqx "# virt-cloud-vpc-dataplane-v1 vpc=$vpc_id revision=$vpc_revision digest=$vpc_digest" "$vpc_if_fragment" || exit 71 - if [ "$FIREWALL_MODE" = firewalld_nft ]; then - awk -v bridge="$vpc_bridge" ' - NR==1 && index($0,"# virt-cloud-vpc-dataplane-v1 vpc=")==1 {next} - NR==2 && $0=="iface " bridge " inet manual" {next} - NR==3 && $0==" bridge_ports none" {next} - NR==4 && $0==" bridge-stp off" {next} - {bad=1} - END {exit (NR==4 && !bad) ? 0 : 1} - ' "$vpc_if_fragment" || exit 71 - [ ! -e "/etc/sysconfig/network-scripts/ifcfg-$vpc_bridge" ] && - [ ! -L "/etc/sysconfig/network-scripts/ifcfg-$vpc_bridge" ] || exit 71 - [ "$(vc_firewall_zone permanent "$vpc_bridge")" = none ] || exit 71 - vpc_runtime_zone=$(vc_firewall_zone runtime "$vpc_bridge") || exit 71 - if [ "$vpc_runtime_zone" = trusted ]; then - "$FIREWALL_CMD" --zone=trusted --remove-interface="$vpc_bridge" >/dev/null 2>&1 || exit 72 - fi - vc_firewall_quarantined "$vpc_bridge" || exit 71 - fi + awk -v bridge="$vpc_bridge" ' + NR==1 && index($0,"# virt-cloud-vpc-dataplane-v1 vpc=")==1 {next} + NR==2 && $0=="iface " bridge " inet manual" {next} + NR==3 && $0==" bridge_ports none" {next} + NR==4 && $0==" bridge-stp off" {next} + {bad=1} + END {exit (NR==4 && !bad) ? 0 : 1} + ' "$vpc_if_fragment" || exit 71 + [ ! -e "/etc/sysconfig/network-scripts/ifcfg-$vpc_bridge" ] && + [ ! -L "/etc/sysconfig/network-scripts/ifcfg-$vpc_bridge" ] || exit 71 "$IFQUERY" "$vpc_bridge" >/dev/null 2>&1 || exit 72 VC_FABRIC_APPLYING=1 "$IFUP" --force "$vpc_bridge" >/dev/null 2>&1 || exit 72 fi @@ -6747,40 +5091,6 @@ EOF vc_ip_rule_preflight "$((vpc_priority + 1))" "$IFACE" "$vpc_cidr" "$vpc_table" 71 72 - || exit $? vc_ip_rule_ensure "$vpc_priority" - "$vpc_cidr" "$vpc_table" 71 72 - - || exit $? vc_ip_rule_ensure "$((vpc_priority + 1))" "$IFACE" "$vpc_cidr" "$vpc_table" 71 72 - - || exit $? - # Trust is deliberately last. The bridge exists and routes are - # converged only after all three panel nft tables were atomically - # restored and read back above. - if [ "$FIREWALL_MODE" = firewalld_nft ]; then - # Routes/profile work above is intentionally performed while the - # bridge is quarantined. Re-read all three canonical tables at - # the last possible point before restoring trusted. - vpc_boot_nft_snapshots_match || exit 71 - if [ "$DEPLOYMENT_MODE" = networkmanager_nmcli ]; then - "$NMCLI" connection modify --temporary uuid "$vpc_nm_uuid" connection.zone trusted >/dev/null 2>&1 || exit 72 - "$NMCLI" device reapply "$vpc_bridge" >/dev/null 2>&1 || exit 72 - [ "$(nm_vpc_field connection.zone uuid "$vpc_nm_uuid")" = trusted ] || exit 71 - [ -f "$vpc_nm_file" ] && [ ! -L "$vpc_nm_file" ] && - [ "$($STAT -c '%u:%a' "$vpc_nm_file" 2>/dev/null)" = 0:600 ] || exit 71 - awk -F= -v wanted_id="$vpc_bridge" -v wanted_uuid="$vpc_nm_uuid" \ - -v wanted_marker="virt-cloud-vpc-dataplane-v1 vpc=$vpc_id" ' - /^\[/ {in_connection=($0=="[connection]"); next} - in_connection && $1=="id" {id++; if (substr($0,4)!=wanted_id) bad=1; next} - in_connection && $1=="uuid" {uuid++; if (substr($0,6)!=wanted_uuid) bad=1; next} - in_connection && $1=="type" {type++; if (substr($0,6)!="bridge") bad=1; next} - in_connection && $1=="interface-name" {iface++; if (substr($0,16)!=wanted_id) bad=1; next} - in_connection && $1=="stable-id" {stable++; if (substr($0,11)!=wanted_marker) bad=1; next} - in_connection && $1=="autoconnect" {auto++; if (substr($0,13)!="false") bad=1} - in_connection && $1=="autoconnect-retries" {retry++; if (substr($0,21)!="0") bad=1} - in_connection && $1=="zone" {zone++} - END {exit (bad || id!=1 || uuid!=1 || type!=1 || iface!=1 || stable!=1 || auto!=1 || retry!=1 || zone!=0) ? 1 : 0} - ' "$vpc_nm_file" || exit 71 - else - vc_firewall_quarantined "$vpc_bridge" || exit 71 - "$FIREWALL_CMD" --zone=trusted --add-interface="$vpc_bridge" >/dev/null 2>&1 || exit 72 - fi - [ "$(vc_firewall_zone permanent "$vpc_bridge")" = none ] || exit 71 - [ "$(vc_firewall_zone runtime "$vpc_bridge")" = trusted ] || exit 71 - fi done } "$WG" set "$IFACE" private-key "$KEY" @@ -6852,18 +5162,6 @@ for route in $allowed_routes; do done vc_fabric_sysctl_file_owned "$SYSCTL_FILE" "$EXPECTED_DIGEST" || exit 71 [ "$(read_sysctl "/proc/sys/net/ipv4/conf/$IFACE/rp_filter")" = 2 ] || exit 71 -if [ "$FIREWALL_MODE" = firewalld_nft ] || - [ -n "${VC_FABRIC_EMERGENCY_SNAPSHOT:-}" ]; then - # All Fabric and VPC topology/readback gates have completed. Re-prove the - # final ACL and trusted binding immediately before removing the independent - # emergency stop. - fabric_acl_ready || exit 71 - if [ "$FIREWALL_MODE" = firewalld_nft ]; then - [ "$(vc_firewall_zone permanent)" = none ] || exit 71 - [ "$(vc_firewall_zone runtime)" = trusted ] || exit 71 - fi - vc_fabric_emergency_remove || exit 79 -fi VC_FABRIC_HELPER_PHASE=complete exit 0 ''' @@ -6926,7 +5224,7 @@ _UNIT = '''[Unit] # virt-cloud-wg-fabric-v1 # fabric-plan-digest=__VC_FABRIC_DIGEST__ Description=Virt Cloud host WireGuard fabric -After=network-online.target NetworkManager.service networking.service nftables.service firewalld.service +After=network-online.target NetworkManager.service networking.service nftables.service Wants=network-online.target Before=libvirtd.service virtqemud.service virtproxyd.service libvirt-guests.service @@ -6943,8 +5241,8 @@ WantedBy=multi-user.target _REPAIR_UNIT = '''[Unit] # virt-cloud-wg-fabric-v1 # fabric-plan-digest=__VC_FABRIC_DIGEST__ -Description=Virt Cloud host WireGuard fabric and firewalld runtime repair -After=network-online.target NetworkManager.service networking.service nftables.service firewalld.service +Description=Virt Cloud host WireGuard fabric runtime repair +After=network-online.target NetworkManager.service networking.service nftables.service Wants=network-online.target [Service] @@ -6957,7 +5255,7 @@ TimeoutStartSec=5min _TIMER_UNIT = '''[Unit] # virt-cloud-wg-fabric-v1 # fabric-plan-digest=__VC_FABRIC_DIGEST__ -Description=Retry failed Fabric boot gates and firewalld runtime bindings +Description=Retry failed Fabric boot gates [Timer] OnBootSec=30s @@ -6979,26 +5277,22 @@ nft_first=; nft_second= [ "$nft_first" = '# virt-cloud-wg-fabric-v1' ] || exit 71 [ "$nft_second" = "# fabric-plan-digest=$EXPECTED_DIGEST" ] || exit 71 nft_persisted=no -if [ "$FIREWALL_MODE" = native_nftables ]; then - nft_main_count=0 - for nft_main in /etc/nftables.conf /etc/sysconfig/nftables.conf; do - [ -f "$nft_main" ] && [ ! -L "$nft_main" ] || continue - [ "$($STAT -c '%u' "$nft_main" 2>/dev/null)" = 0 ] || exit 71 - nft_main_mode=$($STAT -c '%a' "$nft_main" 2>/dev/null || true) - case "$nft_main_mode" in [0-7][0145][0145]) : ;; *) exit 71 ;; esac - nft_main_count=$((nft_main_count + 1)) - if grep -Fqx 'include "/etc/nftables.d/virt-cloud-wg-fabric.nft"' "$nft_main" || - grep -Fqx 'include "/etc/nftables.d/*.nft"' "$nft_main" || - grep -Fqx 'include "/etc/nftables.d/*"' "$nft_main"; then - nft_persisted=yes - fi - done - [ "$nft_main_count" = 1 ] && [ "$nft_persisted" = yes ] || exit 71 -else - [ "$FIREWALL_MODE" = firewalld_nft ] || exit 71 - vc_nft_manifest_validate "$NFT_MANIFEST" || exit 71 - nft_persisted=yes -fi +nft_main_count=0 +for nft_main in /etc/nftables.conf /etc/sysconfig/nftables.conf; do + [ -f "$nft_main" ] && [ ! -L "$nft_main" ] || continue + [ "$($STAT -c '%u' "$nft_main" 2>/dev/null)" = 0 ] || exit 71 + nft_main_mode=$($STAT -c '%a' "$nft_main" 2>/dev/null || true) + case "$nft_main_mode" in [0-7][0145][0145]) : ;; *) exit 71 ;; esac + nft_main_count=$((nft_main_count + 1)) + if grep -Fqx 'include "/etc/nftables.d/virt-cloud-wg-fabric.nft"' "$nft_main" || + grep -Fqx 'include "/etc/nftables.d/*.nft"' "$nft_main" || + grep -Fqx 'include "/etc/nftables.d/*"' "$nft_main"; then + nft_persisted=yes + fi +done +[ "$nft_main_count" = 1 ] && [ "$nft_persisted" = yes ] || exit 71 +[ ! -e /etc/virt-cloud/nftables.conf ] && +[ ! -L /etc/virt-cloud/nftables.conf ] || exit 71 validate_nft_table() { vc_fabric_nft_strict_shape "$1" || return 1 nft_owner_count=$(printf '%s\n' "$1" | grep -Ec 'comment "virt-cloud-wg-fabric-v1:[0-9a-f]{64}"' || true) @@ -7439,9 +5733,7 @@ verify_rollback_rp_filter() { ROLLBACK_FAILED=yes fi if [ "$ROLLBACK_FAILED" = yes ] && [ "$IFACE_BEFORE" = yes ]; then - if [ "$FIREWALL_MODE" = firewalld_nft ]; then - vc_fabric_quarantine down >/dev/null 2>&1 || true - elif [ "$DEPLOYMENT_MODE" = networkmanager_nmcli ]; then + if [ "$DEPLOYMENT_MODE" = networkmanager_nmcli ]; then vc_fabric_nm_quarantine >/dev/null 2>&1 || true else "$IP" link set dev "$IFACE" down >/dev/null 2>&1 || true @@ -7527,45 +5819,27 @@ transaction_recovery_same_current_plan() { $1=="connection_uuid" {uuid++; if (substr($0,index($0,"=")+1)!=wanted_uuid) bad=1; next} $1=="deployment_mode" {mode++; if (substr($0,index($0,"=")+1)!=wanted_mode) bad=1; next} $1=="firewall_mode" {firewall++; if (substr($0,index($0,"=")+1)!=wanted_firewall) bad=1; next} - $1=="underlay_endpoint" {endpoint++; next} - $1=="underlay_port" {port++; next} + $1 ~ /^(firewalld_|underlay_)/ {bad=1; next} END { exit (bad || schema!=1 || digest!=1 || uuid!=1 || mode!=1 || - firewall!=1 || endpoint!=1 || port!=1) ? 1 : 0 + firewall!=1) ? 1 : 0 } ' "$transaction_current_meta" >/dev/null } -transaction_recovery_current_underlay_matches() { - # These public constants are already rendered into the guarded current - # helper. The outer apply has not yet reached underlay preflight, so do - # not read its uninitialized mutable UNDERLAY_* variables or eval files. - if [ "$FIREWALL_MODE" = native_nftables ]; then - transaction_current_endpoint= - transaction_current_port= - else - transaction_current_endpoint=$(awk -F'"' ' - /^UNDERLAY_ENDPOINT="[0-9.]+"$/ {value=$2; count++} - END {if (count==1) print value; else exit 1} - ' "$_tmp_helper") || return 1 - transaction_current_port=$(awk -F'"' ' - /^UNDERLAY_PORT="[0-9]+"$/ {value=$2; count++} - END {if (count==1) print value; else exit 1} - ' "$_tmp_helper") || return 1 - fi - awk -F= -v wanted_endpoint="$transaction_current_endpoint" \ - -v wanted_port="$transaction_current_port" ' - $1=="underlay_endpoint" {endpoint++; if (substr($0,index($0,"=")+1)!=wanted_endpoint) bad=1} - $1=="underlay_port" {port++; if (substr($0,index($0,"=")+1)!=wanted_port) bad=1} - END {exit (bad || endpoint!=1 || port!=1) ? 1 : 0} - ' "$transaction_current_meta" >/dev/null -} if [ -L "$BASE" ] || { [ -e "$BASE" ] && [ ! -d "$BASE" ]; }; then exit 71; fi TRANSACTION_RECOVERED=no if [ -e "$BASE/.wg-fabric-txn" ]; then + vc_native_transaction_journal_valid_or_absent || exit 79 [ -d "$BASE/.wg-fabric-txn" ] && [ ! -L "$BASE/.wg-fabric-txn" ] || exit 79 [ "$(stat -c '%u:%a' "$BASE/.wg-fabric-txn" 2>/dev/null)" = 0:700 ] || exit 79 [ -f "$BASE/.wg-fabric-txn/state" ] && [ ! -L "$BASE/.wg-fabric-txn/state" ] || exit 79 [ "$($STAT -c '%u:%a' "$BASE/.wg-fabric-txn/state" 2>/dev/null)" = 0:600 ] || exit 79 + if [ -f "$BASE/.wg-fabric-txn/meta" ] && + grep -Eq '^(firewalld_|underlay_(zone|rule))' "$BASE/.wg-fabric-txn/meta"; then + exit 79 + fi + journal_existing_firewall=$(awk -F= '$1=="firewall_mode" {value=$2; count++} END {if (count==1) print value}' "$BASE/.wg-fabric-txn/meta" 2>/dev/null || true) + [ "$journal_existing_firewall" = native_nftables ] || exit 79 journal_existing_state=$(cat "$BASE/.wg-fabric-txn/state" 2>/dev/null || true) case "$journal_existing_state" in committed|preparing) rm -rf "$BASE/.wg-fabric-txn" || exit 79 ;; @@ -7604,12 +5878,8 @@ if [ -e "$BASE/.wg-fabric-txn" ]; then TRANSACTION_RECOVERED=yes ;; guarded) - # The previous trap fully restored its snapshots and retained the - # already-adopted emergency guard. Continue this new transaction - # from that safe state; its post-commit reconcile will run the - # complete VPC recovery before releasing the guard. - rm -rf "$BASE/.wg-fabric-txn" || exit 79 - TRANSACTION_RECOVERED=yes + # Legacy guarded journals require explicit maintenance cleanup. + exit 79 ;; ready) # Complete an interrupted revision before creating a new journal. @@ -7685,7 +5955,6 @@ if [ -e "$BASE/.wg-fabric-txn" ]; then [ "$(sed -n '3p' "$_tmp_helper")" = "# fabric-plan-digest=$EXPECTED_DIGEST" ] && [ "$(sed -n '4p' "$_tmp_helper")" = "PLAN_DIGEST=\"$EXPECTED_DIGEST\"" ] || exit 79 grep -Fq -- '--recover-current' "$_tmp_helper" || exit 79 - transaction_recovery_current_underlay_matches || exit 79 vc_fabric_operation_checkpoint journal_dispatch VC_FABRIC_LOCK_HELD=1 /bin/sh "$_tmp_helper" --recover-current || exit 79 else @@ -7716,22 +5985,12 @@ if [ "$TRANSACTION_RECOVERED" = yes ]; then PREFLIGHT_IFACE_UP_BEFORE=yes fi fi - if [ "$FIREWALL_MODE" = firewalld_nft ]; then - vc_fabric_quarantine down || exit 79 - PREFLIGHT_FIREWALL_RUNTIME_ZONE_BEFORE=$VC_FABRIC_QUARANTINE_RUNTIME_BEFORE - PREFLIGHT_FIREWALL_PERMANENT_ZONE_BEFORE=$VC_FABRIC_QUARANTINE_PERMANENT_BEFORE - if [ -f "$CONF" ] && - [ "$PREFLIGHT_FIREWALL_RUNTIME_ZONE_BEFORE" = quarantined ]; then - vc_fabric_retry_quarantine_ready || exit 79 - fi - else - vc_fabric_emergency_adopt || exit 79 - if [ "$PREFLIGHT_IFACE_BEFORE" = yes ]; then - if [ "$DEPLOYMENT_MODE" = networkmanager_nmcli ]; then - vc_fabric_nm_quarantine || exit 79 - else - "$IP" link set dev "$IFACE" down >/dev/null 2>&1 || exit 79 - fi + vc_legacy_firewalld_state_absent || exit 79 + if [ "$PREFLIGHT_IFACE_BEFORE" = yes ]; then + if [ "$DEPLOYMENT_MODE" = networkmanager_nmcli ]; then + vc_fabric_nm_quarantine || exit 79 + else + "$IP" link set dev "$IFACE" down >/dev/null 2>&1 || exit 79 fi fi fi @@ -7750,6 +6009,7 @@ chmod 600 "$_tmp_txn_dir/state" cat > "$_tmp_txn_dir/meta" </dev/null)" = preparing ] || exit 71 [ "$(sed -n 's/^schema=//p' "$_tmp_txn_dir/meta")" = 1 ] || exit 71 [ "$(sed -n 's/^phase=//p' "$_tmp_txn_dir/meta")" = preparing ] || exit 71 +[ "$(sed -n 's/^firewall_mode=//p' "$_tmp_txn_dir/meta")" = native_nftables ] || exit 71 [ "$(sed -n 's/^plan_digest=//p' "$_tmp_txn_dir/meta")" = "$EXPECTED_DIGEST" ] || exit 71 [ ! -e "$TXN" ] && [ ! -L "$TXN" ] || exit 79 sync @@ -7798,29 +6059,6 @@ SERVICE_ENABLED_BEFORE=$($SYSTEMCTL is-enabled virt-cloud-wg-fabric.service 2>/d SERVICE_ACTIVE_BEFORE=$($SYSTEMCTL is-active virt-cloud-wg-fabric.service 2>/dev/null || true) TIMER_ENABLED_BEFORE=$($SYSTEMCTL is-enabled virt-cloud-wg-fabric-reconcile.timer 2>/dev/null || true) TIMER_ACTIVE_BEFORE=$($SYSTEMCTL is-active virt-cloud-wg-fabric-reconcile.timer 2>/dev/null || true) -FIREWALL_RUNTIME_ZONE_BEFORE=na -FIREWALL_PERMANENT_ZONE_BEFORE=na -if [ "$FIREWALL_MODE" = firewalld_nft ]; then - if [ -n "${PREFLIGHT_FIREWALL_RUNTIME_ZONE_BEFORE:-}" ]; then - FIREWALL_RUNTIME_ZONE_BEFORE=$PREFLIGHT_FIREWALL_RUNTIME_ZONE_BEFORE - FIREWALL_PERMANENT_ZONE_BEFORE=$PREFLIGHT_FIREWALL_PERMANENT_ZONE_BEFORE - vc_fabric_quarantine down || exit 71 - else - vc_fabric_quarantine down || exit 71 - FIREWALL_RUNTIME_ZONE_BEFORE=$VC_FABRIC_QUARANTINE_RUNTIME_BEFORE - FIREWALL_PERMANENT_ZONE_BEFORE=$VC_FABRIC_QUARANTINE_PERMANENT_BEFORE - fi - [ "$FIREWALL_PERMANENT_ZONE_BEFORE" = none ] || exit 71 - if [ -f "$CONF" ]; then - case "$FIREWALL_RUNTIME_ZONE_BEFORE" in - trusted) : ;; - quarantined) vc_fabric_retry_quarantine_ready || exit 71 ;; - *) exit 71 ;; - esac - else - [ "$FIREWALL_RUNTIME_ZONE_BEFORE" = none ] || exit 71 - fi -fi IFACE_BEFORE=${PREFLIGHT_IFACE_BEFORE:-no} IFACE_UP_BEFORE=${PREFLIGHT_IFACE_UP_BEFORE:-no} RP_FILTER_BEFORE=missing @@ -7853,18 +6091,6 @@ service_enabled_before=$SERVICE_ENABLED_BEFORE service_active_before=$SERVICE_ACTIVE_BEFORE timer_enabled_before=$TIMER_ENABLED_BEFORE timer_active_before=$TIMER_ACTIVE_BEFORE -firewalld_runtime_zone_before=$FIREWALL_RUNTIME_ZONE_BEFORE -firewalld_permanent_zone_before=$FIREWALL_PERMANENT_ZONE_BEFORE -underlay_endpoint=$UNDERLAY_ENDPOINT -underlay_port=$UNDERLAY_PORT -underlay_zone=$UNDERLAY_ZONE -underlay_rule_owner=$UNDERLAY_RULE_OWNER -underlay_rule_runtime_before=$UNDERLAY_RULE_RUNTIME_BEFORE -underlay_rule_permanent_before=$UNDERLAY_RULE_PERMANENT_BEFORE -old_underlay_rule_remove=$OLD_UNDERLAY_RULE_REMOVE -old_underlay_endpoint=$OLD_UNDERLAY_ENDPOINT -old_underlay_port=$OLD_UNDERLAY_PORT -old_underlay_zone=$OLD_UNDERLAY_ZONE EOF chmod 600 "$TXN/meta.tmp" mv -f "$TXN/meta.tmp" "$TXN/meta" @@ -7893,7 +6119,6 @@ rollback() { trap - EXIT HUP INT TERM set +e set +u - ROLLBACK_GUARDED=no if [ "$COMMITTED" = yes ]; then # Durable commit is followed by an unlocked, synchronous reconcile. # Once the flock has been released, TXN may already belong to a newer @@ -7919,9 +6144,7 @@ rollback() { # Quarantine the Fabric interface before replacing or removing its nft # guard. NetworkManager is made unmanaged before link-down so it cannot # reactivate the interface while rollback restores the prior ACL. - if [ "$FIREWALL_MODE" = firewalld_nft ]; then - vc_fabric_quarantine || ROLLBACK_FAILED=yes - elif [ "$DEPLOYMENT_MODE" = networkmanager_nmcli ] && + if [ "$DEPLOYMENT_MODE" = networkmanager_nmcli ] && "$IP" link show dev "$IFACE" >/dev/null 2>&1; then vc_fabric_nm_quarantine || ROLLBACK_FAILED=yes elif "$IP" link show dev "$IFACE" >/dev/null 2>&1; then @@ -7962,7 +6185,7 @@ rollback() { restore_file "$NFT_RULES" nft_rules restore_file "$NFT_MAIN" nft_main # Restore the old nft table in one checked netlink batch while the - # firewalld interface remains quarantined. Never expose a delete/load gap. + # interface remains down. Never expose a delete/load gap. if [ "$ROLLBACK_FAILED" = no ]; then rollback_nft_phase= rollback_nft_expected_file= @@ -8138,11 +6361,7 @@ rollback() { "$IP" address restore < "$TXN/address.save" || ROLLBACK_FAILED=yes "$IP" route flush table all dev "$IFACE" || ROLLBACK_FAILED=yes "$IP" route restore < "$TXN/route.save" || ROLLBACK_FAILED=yes - if [ "$FIREWALL_MODE" = firewalld_nft ]; then - # Keep the restored topology hard-down until nft and firewalld have - # both been restored and read back below. - "$IP" link set dev "$IFACE" down || ROLLBACK_FAILED=yes - elif [ "$IFACE_UP_BEFORE" = yes ]; then + if [ "$IFACE_UP_BEFORE" = yes ]; then "$IP" link set dev "$IFACE" up || ROLLBACK_FAILED=yes else "$IP" link set dev "$IFACE" down || ROLLBACK_FAILED=yes @@ -8152,7 +6371,7 @@ rollback() { if [ -f "$TXN/nm_profile.present" ]; then rollback_old_uuid=$(awk -F= '$1=="uuid" {print $2; count++} END {if (count!=1) exit 1}' "$NM_PROFILE" 2>/dev/null) || ROLLBACK_FAILED=yes [ "$ROLLBACK_FAILED" = yes ] || "$NMCLI" connection load "$NM_PROFILE" >/dev/null 2>&1 || ROLLBACK_FAILED=yes - if [ "$ROLLBACK_FAILED" = no ] && [ "$FIREWALL_MODE" = native_nftables ] && [ "$IFACE_BEFORE" = yes ]; then + if [ "$ROLLBACK_FAILED" = no ] && [ "$IFACE_BEFORE" = yes ]; then vc_fabric_nm_release || ROLLBACK_FAILED=yes if [ "$IFACE_UP_BEFORE" = yes ]; then "$NMCLI" connection up uuid "$rollback_old_uuid" >/dev/null 2>&1 || ROLLBACK_FAILED=yes @@ -8170,121 +6389,6 @@ rollback() { ROLLBACK_FAILED=yes fi fi - if [ "$FIREWALL_MODE" = firewalld_nft ]; then - rollback_runtime_zone=$(vc_firewall_zone runtime 2>/dev/null || printf foreign) - rollback_permanent_zone=$(vc_firewall_zone permanent 2>/dev/null || printf foreign) - if [ "$DEPLOYMENT_MODE" = networkmanager_nmcli ]; then - if [ -f "$NM_PROFILE" ] && [ ! -L "$NM_PROFILE" ]; then - vc_fabric_nm_profile_owned any || ROLLBACK_FAILED=yes - rollback_memory_zone=$("$NMCLI" -t --escape no -g connection.zone connection show uuid "$CONNECTION_UUID" 2>/dev/null | sed -n '1p') - [ -z "$rollback_memory_zone" ] || [ "$rollback_memory_zone" = -- ] || ROLLBACK_FAILED=yes - if [ "$ROLLBACK_FAILED" = no ] && - [ "$FIREWALL_RUNTIME_ZONE_BEFORE" != quarantined ] && - [ "$(vc_firewall_zone runtime 2>/dev/null || printf foreign)" != none ]; then - vc_fabric_nm_active || ROLLBACK_FAILED=yes - fi - [ "$ROLLBACK_FAILED" = yes ] || vc_firewall_quarantined "$IFACE" yes || ROLLBACK_FAILED=yes - else - vc_firewall_quarantined "$IFACE" no || ROLLBACK_FAILED=yes - fi - else - vc_firewall_quarantined "$IFACE" no || ROLLBACK_FAILED=yes - fi - if [ "$ROLLBACK_FAILED" = no ]; then - [ "$FIREWALL_PERMANENT_ZONE_BEFORE" = none ] || ROLLBACK_FAILED=yes - case "$FIREWALL_RUNTIME_ZONE_BEFORE" in - trusted) - if [ "$DEPLOYMENT_MODE" = networkmanager_nmcli ]; then - rollback_old_uuid= - if [ -f "$NM_PROFILE" ] && [ ! -L "$NM_PROFILE" ]; then - rollback_old_uuid=$(awk -F= '$1=="uuid" {print $2; count++} END {if (count!=1) exit 1}' "$NM_PROFILE" 2>/dev/null) || ROLLBACK_FAILED=yes - [ "$ROLLBACK_FAILED" = yes ] || "$NMCLI" connection load "$NM_PROFILE" >/dev/null 2>&1 || ROLLBACK_FAILED=yes - [ "$ROLLBACK_FAILED" = yes ] || "$NMCLI" connection modify --temporary uuid "$rollback_old_uuid" connection.zone trusted >/dev/null 2>&1 || ROLLBACK_FAILED=yes - else - ROLLBACK_FAILED=yes - fi - else - [ ! -e "/etc/sysconfig/network-scripts/ifcfg-$IFACE" ] && - [ ! -L "/etc/sysconfig/network-scripts/ifcfg-$IFACE" ] || ROLLBACK_FAILED=yes - [ "$ROLLBACK_FAILED" = yes ] || - "$FIREWALL_CMD" --zone=trusted --add-interface="$IFACE" >/dev/null 2>&1 || ROLLBACK_FAILED=yes - fi - ;; - none) : ;; - quarantined) - vc_fabric_quarantine down || ROLLBACK_FAILED=yes - ;; - *) ROLLBACK_FAILED=yes ;; - esac - [ "$(vc_firewall_zone permanent 2>/dev/null || printf foreign)" = none ] || ROLLBACK_FAILED=yes - if [ "$ROLLBACK_FAILED" = no ] && - [ "$FIREWALL_RUNTIME_ZONE_BEFORE" = quarantined ]; then - if [ "$DEPLOYMENT_MODE" = networkmanager_nmcli ]; then - vc_fabric_nm_expect_managed no || ROLLBACK_FAILED=yes - vc_firewall_quarantined "$IFACE" yes || ROLLBACK_FAILED=yes - else - vc_firewall_quarantined "$IFACE" no || ROLLBACK_FAILED=yes - fi - "$IP" -o link show dev "$IFACE" 2>/dev/null | - grep -Eq '<[^>]*UP([,>])' && ROLLBACK_FAILED=yes || true - elif [ "$ROLLBACK_FAILED" = no ]; then - [ "$(vc_firewall_zone runtime 2>/dev/null || printf foreign)" = "$FIREWALL_RUNTIME_ZONE_BEFORE" ] || ROLLBACK_FAILED=yes - fi - fi - if [ "$ROLLBACK_FAILED" = no ] && [ "$OLD_UNDERLAY_RULE_REMOVE" = yes ]; then - rollback_old_runtime=$(vc_firewall_rule_state runtime "$OLD_UNDERLAY_ZONE" "$OLD_UNDERLAY_RULE" 2>/dev/null || printf unknown) - rollback_old_permanent=$(vc_firewall_rule_state permanent "$OLD_UNDERLAY_ZONE" "$OLD_UNDERLAY_RULE" 2>/dev/null || printf unknown) - case "$rollback_old_runtime" in - yes) : ;; - no) "$FIREWALL_CMD" --zone="$OLD_UNDERLAY_ZONE" --add-rich-rule="$OLD_UNDERLAY_RULE" >/dev/null 2>&1 || ROLLBACK_FAILED=yes ;; - *) ROLLBACK_FAILED=yes ;; - esac - case "$rollback_old_permanent" in - yes) : ;; - no) "$FIREWALL_CMD" --permanent --zone="$OLD_UNDERLAY_ZONE" --add-rich-rule="$OLD_UNDERLAY_RULE" >/dev/null 2>&1 || ROLLBACK_FAILED=yes ;; - *) ROLLBACK_FAILED=yes ;; - esac - [ "$ROLLBACK_FAILED" = yes ] || - [ "$(vc_firewall_rule_state runtime "$OLD_UNDERLAY_ZONE" "$OLD_UNDERLAY_RULE" 2>/dev/null || printf unknown)" = yes ] || ROLLBACK_FAILED=yes - [ "$ROLLBACK_FAILED" = yes ] || - [ "$(vc_firewall_rule_state permanent "$OLD_UNDERLAY_ZONE" "$OLD_UNDERLAY_RULE" 2>/dev/null || printf unknown)" = yes ] || ROLLBACK_FAILED=yes - fi - rollback_rule=$(vc_firewall_rich_rule "$UNDERLAY_ENDPOINT" "$UNDERLAY_PORT" 2>/dev/null || true) - [ -n "$rollback_rule" ] || ROLLBACK_FAILED=yes - if [ "$ROLLBACK_FAILED" = no ] && [ "$UNDERLAY_RULE_OWNER" = panel ]; then - rollback_rule_runtime=$(vc_firewall_rule_state runtime "$UNDERLAY_ZONE" "$rollback_rule" 2>/dev/null || printf unknown) - rollback_rule_permanent=$(vc_firewall_rule_state permanent "$UNDERLAY_ZONE" "$rollback_rule" 2>/dev/null || printf unknown) - if [ "$UNDERLAY_RULE_RUNTIME_BEFORE" = no ] && [ "$rollback_rule_runtime" = yes ]; then - "$FIREWALL_CMD" --zone="$UNDERLAY_ZONE" --remove-rich-rule="$rollback_rule" >/dev/null 2>&1 || ROLLBACK_FAILED=yes - elif [ "$UNDERLAY_RULE_RUNTIME_BEFORE" != "$rollback_rule_runtime" ]; then - ROLLBACK_FAILED=yes - fi - if [ "$UNDERLAY_RULE_PERMANENT_BEFORE" = no ] && [ "$rollback_rule_permanent" = yes ]; then - "$FIREWALL_CMD" --permanent --zone="$UNDERLAY_ZONE" --remove-rich-rule="$rollback_rule" >/dev/null 2>&1 || ROLLBACK_FAILED=yes - elif [ "$UNDERLAY_RULE_PERMANENT_BEFORE" != "$rollback_rule_permanent" ]; then - ROLLBACK_FAILED=yes - fi - [ "$(vc_firewall_rule_state runtime "$UNDERLAY_ZONE" "$rollback_rule" 2>/dev/null || printf unknown)" = "$UNDERLAY_RULE_RUNTIME_BEFORE" ] || ROLLBACK_FAILED=yes - [ "$(vc_firewall_rule_state permanent "$UNDERLAY_ZONE" "$rollback_rule" 2>/dev/null || printf unknown)" = "$UNDERLAY_RULE_PERMANENT_BEFORE" ] || ROLLBACK_FAILED=yes - elif [ "$ROLLBACK_FAILED" = no ]; then - [ "$UNDERLAY_RULE_OWNER" = external ] && - [ "$(vc_firewall_rule_state runtime "$UNDERLAY_ZONE" "$rollback_rule" 2>/dev/null || printf unknown)" = yes ] && - [ "$(vc_firewall_rule_state permanent "$UNDERLAY_ZONE" "$rollback_rule" 2>/dev/null || printf unknown)" = yes ] || ROLLBACK_FAILED=yes - fi - if [ "$ROLLBACK_FAILED" = no ] && - [ "$DEPLOYMENT_MODE" = networkmanager_nmcli ] && - [ "$FIREWALL_RUNTIME_ZONE_BEFORE" = trusted ] && - [ "$IFACE_BEFORE" = yes ]; then - # Activation is the final step. A failed zone/rich-rule readback - # therefore leaves the panel-owned link unmanaged and down. - vc_fabric_nm_release || ROLLBACK_FAILED=yes - if [ "$ROLLBACK_FAILED" = no ] && [ "$IFACE_UP_BEFORE" = yes ]; then - "$NMCLI" connection up uuid "$rollback_old_uuid" >/dev/null 2>&1 || ROLLBACK_FAILED=yes - elif [ "$ROLLBACK_FAILED" = no ]; then - "$IP" link set dev "$IFACE" down >/dev/null 2>&1 || ROLLBACK_FAILED=yes - fi - fi - fi # Manager activation must not silently reapply the transaction value. # This final check is read-only; a mismatch fails closed with the journal. verify_rollback_rp_filter @@ -8293,45 +6397,6 @@ rollback() { case "$SERVICE_ENABLED_BEFORE" in enabled|enabled-runtime) $SYSTEMCTL enable virt-cloud-wg-fabric.service >/dev/null 2>&1 || ROLLBACK_FAILED=yes ;; *) $SYSTEMCTL disable virt-cloud-wg-fabric.service >/dev/null 2>&1 || true ;; esac case "$TIMER_ENABLED_BEFORE" in enabled|enabled-runtime) $SYSTEMCTL enable virt-cloud-wg-fabric-reconcile.timer >/dev/null 2>&1 || ROLLBACK_FAILED=yes ;; *) $SYSTEMCTL disable virt-cloud-wg-fabric-reconcile.timer >/dev/null 2>&1 || true ;; esac fi - if [ "$ROLLBACK_FAILED" = no ] && - { [ -n "${VC_FABRIC_EMERGENCY_SNAPSHOT:-}" ] || - [ -n "${VC_FABRIC_EMERGENCY_BRIDGE_SNAPSHOT:-}" ] || - [ -n "${VC_FABRIC_EMERGENCY_TOKEN:-}" ]; }; then - [ -n "${VC_FABRIC_EMERGENCY_SNAPSHOT:-}" ] && - [ -n "${VC_FABRIC_EMERGENCY_BRIDGE_SNAPSHOT:-}" ] && - [ -n "${VC_FABRIC_EMERGENCY_TOKEN:-}" ] || ROLLBACK_FAILED=yes - if [ "$IFACE_BEFORE" = yes ]; then - "$WG" showconf "$IFACE" > "$TXN/wg.verify" 2>/dev/null || ROLLBACK_FAILED=yes - [ "$ROLLBACK_FAILED" = yes ] || "$CMP" -s "$TXN/wg.verify" "$TXN/wg.conf" || ROLLBACK_FAILED=yes - "$IP" address save dev "$IFACE" > "$TXN/address.verify" 2>/dev/null || ROLLBACK_FAILED=yes - [ "$ROLLBACK_FAILED" = yes ] || "$CMP" -s "$TXN/address.verify" "$TXN/address.save" || ROLLBACK_FAILED=yes - "$IP" route save table all dev "$IFACE" > "$TXN/route.verify" 2>/dev/null || ROLLBACK_FAILED=yes - [ "$ROLLBACK_FAILED" = yes ] || "$CMP" -s "$TXN/route.verify" "$TXN/route.save" || ROLLBACK_FAILED=yes - elif "$IP" link show dev "$IFACE" >/dev/null 2>&1; then - ROLLBACK_FAILED=yes - fi - rm -f "$TXN/wg.verify" "$TXN/address.verify" "$TXN/route.verify" || ROLLBACK_FAILED=yes - if [ "$ROLLBACK_FAILED" = no ]; then - # The transaction trap cannot duplicate the authoritative VPC - # three-family ACL/bridge/route/rule/sysctl/zone verifier. Hand - # the completely restored old state to the normal helper while - # retaining both emergency tables. This is deliberately reported - # as an incomplete rollback until that helper proves every VPC. - rollback_state_tmp=$(mktemp "$TXN/.state.XXXXXX") || ROLLBACK_FAILED=yes - if [ "$ROLLBACK_FAILED" = no ]; then - printf '%s\n' guarded > "$rollback_state_tmp" || ROLLBACK_FAILED=yes - [ "$ROLLBACK_FAILED" = yes ] || chmod 600 "$rollback_state_tmp" || ROLLBACK_FAILED=yes - [ "$ROLLBACK_FAILED" = yes ] || mv -f "$rollback_state_tmp" "$TXN/state" || ROLLBACK_FAILED=yes - rollback_state_tmp= - [ "$ROLLBACK_FAILED" = yes ] || sync || ROLLBACK_FAILED=yes - fi - [ -z "${rollback_state_tmp:-}" ] || rm -f "$rollback_state_tmp" - if [ "$ROLLBACK_FAILED" = no ]; then - ROLLBACK_GUARDED=yes - ROLLBACK_FAILED=yes - fi - fi - fi # Keep the transaction's complete recovery helper installed until every # other rollback/readback has succeeded. A first-deploy rollback uses the # same durable cleanup_helper terminal phase as boot recovery so any crash @@ -8344,11 +6409,7 @@ rollback() { restore_file "$HELPER" helper fi if [ "$ROLLBACK_FAILED" = yes ]; then - if [ "$ROLLBACK_GUARDED" = yes ]; then - printf '%s\n' '__VC_FABRIC_ROLLBACK__=guarded' >&2 - else - printf '%s\n' '__VC_FABRIC_ROLLBACK__=failed' >&2 - fi + printf '%s\n' '__VC_FABRIC_ROLLBACK__=failed' >&2 # Keep the durable snapshot so a boot or a later maintenance run can # retry recovery. Deleting it here would turn a partial rollback into # an unrecoverable mixed host state. @@ -8373,12 +6434,6 @@ def _config_text(plan: FabricPlan, host: FabricNode, lines = [f"schema|{FABRIC_SCHEMA_VERSION}", f"fabric|{plan.fabric_id}", f"revision|{plan.revision}", f"digest|{plan.digest}", f"firewall|{firewall_mode}"] - if firewall_mode == FIREWALL_MODE_FIREWALLD: - lines.extend(( - "underlay|local|%s|__VC_UNDERLAY_INTERFACE__|__VC_UNDERLAY_ZONE__" - % host.endpoint.rsplit(":", 1)[0], - "firewall_rule|%s|__VC_UNDERLAY_RULE_OWNER__" % host.listen_port, - )) lines.extend(( f"network|{plan.overlay_cidr}", f"overlay|{host.overlay_ip}", f"listen|{host.listen_port}", @@ -8433,10 +6488,8 @@ def _nm_profile_template(plan: FabricPlan, host: FabricNode, "autoconnect=false", "autoconnect-retries=0", ] - # Firewalld trust is deliberately runtime-only. The helper first - # restores and verifies the panel-owned nft guard, then binds this - # dedicated overlay interface to trusted. A persistent keyfile zone - # could expose the interface during boot before those ACLs exist. + # The dedicated overlay profile never carries a firewall zone. Native + # nftables owns the complete boundary before the profile is activated. lines.extend([ "", "[wireguard]", @@ -8616,7 +6669,6 @@ def _apply_script(plan: FabricPlan, host: FabricNode, "__VC_NMCLI_FILENAME_RESOLVER__", _NMCLI_FILENAME_RESOLVER_SH).replace( "__VC_FIREWALL_MODE_GUARD__", _FIREWALL_MODE_GUARD_SH).replace( - "__VC_NFT_MANIFEST_VALIDATION__", _NFT_MANIFEST_VALIDATION_SH).replace( "__VC_FABRIC_NFT_VALIDATION__", _FABRIC_NFT_VALIDATION_SH).replace( "__VC_FABRIC_WG_READBACK__", _FABRIC_WG_READBACK_SH).replace( "__VC_FABRIC_SYSCTL_VALIDATION__", _FABRIC_SYSCTL_VALIDATION_SH) @@ -8677,9 +6729,6 @@ def _apply_script(plan: FabricPlan, host: FabricNode, preflight = f'''OLD_MANAGED_PEERS= OLD_MANAGED_ROUTES= OLD_FIREWALL_MODE= -OLD_UNDERLAY= -OLD_FIREWALL_RULE_PORT= -OLD_FIREWALL_RULE_OWNER= if [ -f "$CONF" ]; then while IFS='|' read -r kind a b c d; do case "$kind" in @@ -8688,19 +6737,13 @@ while IFS='|' read -r kind a b c d; do firewall) [ -z "$OLD_FIREWALL_MODE" ] || exit 71 OLD_FIREWALL_MODE=$a ;; - underlay) - [ -z "$OLD_UNDERLAY" ] || exit 71 - OLD_UNDERLAY="$a|$b|$c|$d" ;; - firewall_rule) - [ -z "$OLD_FIREWALL_RULE_OWNER" ] || exit 71 - [ -z "$c$d" ] || exit 71 - OLD_FIREWALL_RULE_PORT=$a - OLD_FIREWALL_RULE_OWNER=$b ;; + underlay|firewall_rule) + # Explicit maintenance is required for a legacy firewalld plan. + exit 79 ;; esac done < "$CONF" -# Legacy panel configs predate the explicit backend record and are native -# nftables only. Never infer a firewalld migration from the current service -# state; that requires a clean first deployment. +# Legacy panel configs that predate the explicit backend record are native +# nftables only. Never infer a backend migration from host service state. [ -n "$OLD_FIREWALL_MODE" ] || OLD_FIREWALL_MODE=native_nftables [ "$OLD_FIREWALL_MODE" = "$FIREWALL_MODE" ] || exit 71 fi @@ -8708,73 +6751,6 @@ vc_fabric_operation_checkpoint preflight_underlay LOCAL_ENDPOINT={_q(host.endpoint.rsplit(':', 1)[0])} UNDERLAY_PROBES={_q(underlay_probe_ips)} LOCAL_UNDERLAY_INTERFACE=$(vc_underlay_local_interface "$LOCAL_ENDPOINT") || exit 71 -UNDERLAY_ENDPOINT= -UNDERLAY_PORT= -UNDERLAY_INTERFACE= -UNDERLAY_ZONE= -UNDERLAY_RULE= -UNDERLAY_RULE_OWNER=none -UNDERLAY_RULE_RUNTIME_BEFORE=na -UNDERLAY_RULE_PERMANENT_BEFORE=na -OLD_UNDERLAY_RULE_REMOVE=no -OLD_UNDERLAY_ENDPOINT= -OLD_UNDERLAY_PORT= -OLD_UNDERLAY_ZONE= -OLD_UNDERLAY_RULE= -if [ "$FIREWALL_MODE" = firewalld_nft ]; then - UNDERLAY_ENDPOINT=$LOCAL_ENDPOINT - UNDERLAY_PORT={_q(str(host.listen_port))} - underlay_live=$(vc_underlay_discover "$UNDERLAY_ENDPOINT" "$UNDERLAY_PROBES") || exit 71 - UNDERLAY_INTERFACE=${{underlay_live%%|*}} - UNDERLAY_ZONE=${{underlay_live#*|}} - [ "$UNDERLAY_INTERFACE" = "$LOCAL_UNDERLAY_INTERFACE" ] && - [ -n "$UNDERLAY_ZONE" ] || exit 71 - UNDERLAY_RULE=$(vc_firewall_rich_rule "$UNDERLAY_ENDPOINT" "$UNDERLAY_PORT") || exit 71 - UNDERLAY_RULE_RUNTIME_BEFORE=$(vc_firewall_rule_state runtime "$UNDERLAY_ZONE" "$UNDERLAY_RULE") || exit 71 - UNDERLAY_RULE_PERMANENT_BEFORE=$(vc_firewall_rule_state permanent "$UNDERLAY_ZONE" "$UNDERLAY_RULE") || exit 71 - if [ -f "$CONF" ]; then - old_underlay_scope=${{OLD_UNDERLAY%%|*}} - old_underlay_rest=${{OLD_UNDERLAY#*|}} - OLD_UNDERLAY_ENDPOINT=${{old_underlay_rest%%|*}} - old_underlay_rest=${{old_underlay_rest#*|}} - old_underlay_iface=${{old_underlay_rest%%|*}} - OLD_UNDERLAY_ZONE=${{old_underlay_rest#*|}} - [ "$old_underlay_scope" = local ] || exit 71 - case "$old_underlay_iface" in ''|*[!A-Za-z0-9_.:-]*) exit 71 ;; esac - [ "$(printf '%s' "$old_underlay_iface" | awk 'length($0)<=15')" = "$old_underlay_iface" ] || exit 71 - case "$OLD_UNDERLAY_ZONE" in ''|*[!A-Za-z0-9_-]*) exit 71 ;; esac - [ "$(printf '%s' "$OLD_UNDERLAY_ZONE" | awk 'length($0)<=64')" = "$OLD_UNDERLAY_ZONE" ] || exit 71 - OLD_UNDERLAY_PORT=$OLD_FIREWALL_RULE_PORT - OLD_UNDERLAY_RULE=$(vc_firewall_rich_rule "$OLD_UNDERLAY_ENDPOINT" "$OLD_UNDERLAY_PORT") || exit 71 - case "$OLD_FIREWALL_RULE_OWNER" in panel|external) : ;; *) exit 71 ;; esac - old_rule_runtime=$(vc_firewall_rule_state runtime "$OLD_UNDERLAY_ZONE" "$OLD_UNDERLAY_RULE") || exit 71 - old_rule_permanent=$(vc_firewall_rule_state permanent "$OLD_UNDERLAY_ZONE" "$OLD_UNDERLAY_RULE") || exit 71 - [ "$old_rule_runtime:$old_rule_permanent" = yes:yes ] || exit 71 - if [ "$OLD_UNDERLAY_ENDPOINT|$OLD_UNDERLAY_PORT|$OLD_UNDERLAY_ZONE" = "$UNDERLAY_ENDPOINT|$UNDERLAY_PORT|$UNDERLAY_ZONE" ]; then - UNDERLAY_RULE_OWNER=$OLD_FIREWALL_RULE_OWNER - else - case "$UNDERLAY_RULE_RUNTIME_BEFORE:$UNDERLAY_RULE_PERMANENT_BEFORE" in - no:no) UNDERLAY_RULE_OWNER=panel ;; - yes:yes) UNDERLAY_RULE_OWNER=external ;; - *) exit 71 ;; - esac - if [ "$OLD_FIREWALL_RULE_OWNER" = panel ]; then - OLD_UNDERLAY_RULE_REMOVE=yes - fi - fi - if [ "$UNDERLAY_RULE_OWNER" = external ]; then - [ "$UNDERLAY_RULE_RUNTIME_BEFORE:$UNDERLAY_RULE_PERMANENT_BEFORE" = yes:yes ] || exit 71 - fi - else - case "$UNDERLAY_RULE_RUNTIME_BEFORE:$UNDERLAY_RULE_PERMANENT_BEFORE" in - no:no) UNDERLAY_RULE_OWNER=panel ;; - yes:yes) UNDERLAY_RULE_OWNER=external ;; - *) exit 71 ;; - esac - fi -else - [ -z "$OLD_UNDERLAY$OLD_FIREWALL_RULE_OWNER" ] || exit 71 -fi if [ "$DEPLOYMENT_MODE" = networkmanager_nmcli ]; then existing_name_uuid=$("$NMCLI" -t --escape no -g connection.uuid connection show "$CONNECTION" 2>/dev/null || true) existing_name_count=$(printf '%s\n' "$existing_name_uuid" | awk 'NF {{count++}} END {{print count+0}}') @@ -8949,7 +6925,7 @@ vc_fabric_apply_failure_log() {{ [ "$vc_apply_log_status" -le 255 ] 2>/dev/null || vc_apply_log_status=1 [ "$vc_apply_log_status" != 0 ] || return 0 case "${{VC_FABRIC_APPLY_DETAIL:-}}" in - entry|final_acl|final_tenants|final_identity|final_link|final_addresses|final_peers|final_endpoints|final_allowed_ips|final_routes|final_overlay_paths|final_zone|commit|convergence|complete) + entry|final_acl|final_tenants|final_identity|final_link|final_addresses|final_peers|final_endpoints|final_allowed_ips|final_routes|final_overlay_paths|commit|convergence|complete) vc_apply_log_detail=$VC_FABRIC_APPLY_DETAIL ;; *) vc_apply_log_detail=unknown ;; esac @@ -8978,7 +6954,6 @@ vc_fabric_operation_checkpoint() {{ _tmp_key= _tmp_owner= _tmp_conf= -_tmp_conf_template= _tmp_helper= _tmp_unit= _tmp_repair= @@ -8995,7 +6970,7 @@ _tmp_txn_dir= _tmp_restore= _tmp_cleanup_state= cleanup() {{ - for _fabric_tmp in "$_tmp_key" "$_tmp_owner" "$_tmp_conf" "$_tmp_conf_template" "$_tmp_helper" "$_tmp_unit" "$_tmp_repair" "$_tmp_timer" "$_tmp_gate" "$_tmp_nm" "$_tmp_ifupdown" "$_tmp_sysctl" "$_tmp_sysctl_desired" "$_tmp_nft" "$_tmp_nft_main" "$_tmp_nft_rules" "$_tmp_restore" "$_tmp_cleanup_state"; do + for _fabric_tmp in "$_tmp_key" "$_tmp_owner" "$_tmp_conf" "$_tmp_helper" "$_tmp_unit" "$_tmp_repair" "$_tmp_timer" "$_tmp_gate" "$_tmp_nm" "$_tmp_ifupdown" "$_tmp_sysctl" "$_tmp_sysctl_desired" "$_tmp_nft" "$_tmp_nft_main" "$_tmp_nft_rules" "$_tmp_restore" "$_tmp_cleanup_state"; do [ -z "$_fabric_tmp" ] || rm -f "$_fabric_tmp" || true done [ -z "$_tmp_txn_dir" ] || rm -rf -- "$_tmp_txn_dir" || true @@ -9016,7 +6991,7 @@ TIMER_UNIT=/etc/systemd/system/{FABRIC_TIMER} GATE_GENERATOR={_q(FABRIC_LIBVIRT_GATE_GENERATOR)} LOCK={_q(FABRIC_LOCK)} NFT_RULES={_q(FABRIC_NFT_RULES)} -NFT_MANIFEST={_q(FABRIC_NFT_MANIFEST)} +LEGACY_NFT_MANIFEST={_q(LEGACY_FIREWALLD_NFT_MANIFEST)} NM_PROFILE={_q(FABRIC_NM_PROFILE)} IFUPDOWN_FRAGMENT={_q(FABRIC_IFUPDOWN_FRAGMENT)} SYSCTL_FILE={_q(FABRIC_SYSCTL)} @@ -9035,19 +7010,16 @@ IP=$(find_tool /usr/sbin/ip /usr/bin/ip /sbin/ip /bin/ip) || exit 70 NFT=$(find_tool /usr/sbin/nft /usr/bin/nft /sbin/nft /bin/nft) || exit 70 CMP=$(find_tool /usr/bin/cmp /bin/cmp /usr/sbin/cmp /sbin/cmp) || exit 70 SYSTEMCTL=$(find_tool /usr/bin/systemctl /bin/systemctl /usr/sbin/systemctl /sbin/systemctl) || exit 70 -FIREWALL_CMD=$(find_tool /usr/bin/firewall-cmd /usr/sbin/firewall-cmd /bin/firewall-cmd /sbin/firewall-cmd || true) STAT=$(find_tool /usr/bin/stat /bin/stat /usr/sbin/stat /sbin/stat) || exit 70 FLOCK=$(find_tool /usr/bin/flock /usr/sbin/flock /bin/flock /sbin/flock) || exit 70 +AWK=$(find_tool /usr/bin/awk /bin/awk /usr/sbin/awk /sbin/awk) || exit 70 {_NMCLI_FILENAME_RESOLVER_SH} {_FIREWALL_MODE_GUARD_SH} -{_NFT_MANIFEST_VALIDATION_SH} {_FABRIC_NFT_VALIDATION_SH} {_FABRIC_WG_READBACK_SH} {_FABRIC_SYSCTL_VALIDATION_SH} PREFLIGHT_IFACE_BEFORE=no PREFLIGHT_IFACE_UP_BEFORE=no -PREFLIGHT_FIREWALL_RUNTIME_ZONE_BEFORE= -PREFLIGHT_FIREWALL_PERMANENT_ZONE_BEFORE= NMCLI= IFQUERY= IFUP= @@ -9079,9 +7051,9 @@ vc_fabric_operation_checkpoint identity [ -f "$KEY" ] && [ ! -L "$KEY" ] || exit 71 [ "$($STAT -c '%u:%a' "$KEY" 2>/dev/null)" = 0:600 ] || exit 71 [ "$($WG pubkey < "$KEY")" = "$EXPECTED_PUBLIC_KEY" ] || exit 71 -# Close a proven panel-owned overlay before mutable firewall, sysctl, config, -# artifact, or nft preflight. NetworkManager devices are made unmanaged first -# so the daemon cannot race link-down with reactivation. +# Close a proven panel-owned overlay before mutable sysctl, config, artifact, +# or nft preflight. NetworkManager devices are made unmanaged first so the +# daemon cannot race link-down with reactivation. if "$IP" link show dev "$IFACE" >/dev/null 2>&1; then [ -f "$OWNER" ] && [ ! -L "$OWNER" ] || exit 71 [ "$($STAT -c '%u:%a' "$OWNER" 2>/dev/null)" = 0:600 ] || exit 71 @@ -9096,25 +7068,12 @@ if "$IP" link show dev "$IFACE" >/dev/null 2>&1; then PREFLIGHT_IFACE_UP_BEFORE=yes fi fi -if [ "$FIREWALL_MODE" = firewalld_nft ]; then - vc_fabric_quarantine down || exit 71 - PREFLIGHT_FIREWALL_RUNTIME_ZONE_BEFORE=$VC_FABRIC_QUARANTINE_RUNTIME_BEFORE - PREFLIGHT_FIREWALL_PERMANENT_ZONE_BEFORE=$VC_FABRIC_QUARANTINE_PERMANENT_BEFORE - if [ -f "$CONF" ] && - [ "$PREFLIGHT_FIREWALL_RUNTIME_ZONE_BEFORE" = quarantined ]; then - # A prior pre-journal failure intentionally leaves an owned interface - # isolated. Accept it only after the complete quarantine contract. - vc_fabric_retry_quarantine_ready || exit 71 - fi -else - vc_fabric_emergency_adopt || exit 71 - vc_fabric_native_adopt_vpc_state_empty || exit 71 - if [ "$PREFLIGHT_IFACE_BEFORE" = yes ]; then - if [ "$DEPLOYMENT_MODE" = networkmanager_nmcli ]; then - vc_fabric_nm_quarantine || exit 71 - else - "$IP" link set dev "$IFACE" down >/dev/null 2>&1 || exit 71 - fi +vc_legacy_firewalld_state_absent || exit 79 +if [ "$PREFLIGHT_IFACE_BEFORE" = yes ]; then + if [ "$DEPLOYMENT_MODE" = networkmanager_nmcli ]; then + vc_fabric_nm_quarantine || exit 71 + else + "$IP" link set dev "$IFACE" down >/dev/null 2>&1 || exit 71 fi fi vc_firewall_mode_ready || exit 71 @@ -9209,19 +7168,7 @@ if "$IP" link show dev "$IFACE" >/dev/null 2>&1; then "$WG" show "$IFACE" >/dev/null 2>&1 || exit 71 fi vc_fabric_operation_checkpoint preflight_firewall -if [ "$FIREWALL_MODE" = firewalld_nft ]; then - [ "$PREFLIGHT_FIREWALL_PERMANENT_ZONE_BEFORE" = none ] || exit 71 - if [ -f "$CONF" ]; then - case "$PREFLIGHT_FIREWALL_RUNTIME_ZONE_BEFORE" in - trusted) : ;; - quarantined) vc_fabric_retry_quarantine_ready || exit 71 ;; - *) exit 71 ;; - esac - else - [ "$PREFLIGHT_FIREWALL_RUNTIME_ZONE_BEFORE" = none ] || exit 71 - fi - vc_fabric_quarantine down || exit 71 -fi +vc_firewall_mode_ready || exit 71 MANAGED_PEERS={_q(peer_keys)} MANAGED_ROUTES={_q(route_keys)} vc_fabric_operation_checkpoint preflight_vpc @@ -9229,37 +7176,21 @@ vc_fabric_operation_checkpoint preflight_vpc vpc_state_load || exit 71 {preflight} vc_fabric_operation_checkpoint preflight_persistence -# Select exactly one persistence owner. Native nftables retains the distro -# main-file include. firewalld uses only the panel manifest loaded by the -# Fabric service/timer after firewalld; its own configuration is untouched. -if [ "$FIREWALL_MODE" = native_nftables ]; then - NFT_MAIN_COUNT=0 - for nft_candidate in /etc/nftables.conf /etc/sysconfig/nftables.conf; do - if [ -f "$nft_candidate" ] && [ ! -L "$nft_candidate" ] && - [ "$("$STAT" -c '%u' "$nft_candidate" 2>/dev/null)" = 0 ]; then - nft_main_mode=$("$STAT" -c '%a' "$nft_candidate" 2>/dev/null || true) - case "$nft_main_mode" in [0-7][0145][0145]) : ;; *) continue ;; esac - [ -n "$NFT_MAIN" ] || NFT_MAIN="$nft_candidate" - NFT_MAIN_COUNT=$((NFT_MAIN_COUNT + 1)) - fi - done - [ "$NFT_MAIN_COUNT" = 1 ] || exit 71 - [ ! -e "$NFT_MANIFEST" ] && [ ! -L "$NFT_MANIFEST" ] || exit 71 -else - [ "$FIREWALL_MODE" = firewalld_nft ] || exit 71 - NFT_MAIN="$NFT_MANIFEST" - if [ -e "$NFT_MANIFEST" ] || [ -L "$NFT_MANIFEST" ]; then - vc_nft_manifest_validate "$NFT_MANIFEST" || exit 71 - fi - for nft_candidate in /etc/nftables.conf /etc/sysconfig/nftables.conf; do - [ -f "$nft_candidate" ] && [ ! -L "$nft_candidate" ] || continue - if grep -Fqx 'include "/etc/nftables.d/virt-cloud-wg-fabric.nft"' "$nft_candidate" || - grep -Fqx 'include "/etc/nftables.d/*.nft"' "$nft_candidate" || - grep -Fqx 'include "/etc/nftables.d/*"' "$nft_candidate"; then - exit 71 - fi - done -fi +# Native nftables owns exactly one distro main-file include. A legacy +# firewalld manifest is a maintenance boundary, never an adoption source. +NFT_MAIN_COUNT=0 +for nft_candidate in /etc/nftables.conf /etc/sysconfig/nftables.conf; do + if [ -f "$nft_candidate" ] && [ ! -L "$nft_candidate" ] && + [ "$("$STAT" -c '%u' "$nft_candidate" 2>/dev/null)" = 0 ]; then + nft_main_mode=$("$STAT" -c '%a' "$nft_candidate" 2>/dev/null || true) + case "$nft_main_mode" in [0-7][0145][0145]) : ;; *) continue ;; esac + [ -n "$NFT_MAIN" ] || NFT_MAIN="$nft_candidate" + NFT_MAIN_COUNT=$((NFT_MAIN_COUNT + 1)) + fi +done +[ "$NFT_MAIN_COUNT" = 1 ] || exit 71 +[ ! -e "$LEGACY_NFT_MANIFEST" ] && +[ ! -L "$LEGACY_NFT_MANIFEST" ] || exit 79 if [ -L /etc/nftables.d ] || {{ [ -e /etc/nftables.d ] && [ ! -d /etc/nftables.d ]; }}; then exit 71; fi if [ -e /etc/nftables.d ]; then [ "$("$STAT" -c '%u:%a' /etc/nftables.d 2>/dev/null)" = 0:755 ] || exit 71 @@ -9315,28 +7246,16 @@ _tmp_nft=$(mktemp /etc/nftables.d/.virt-cloud-wg-fabric.XXXXXX) chmod 600 "$_tmp_nft" publish_desired_file "$_tmp_nft" "$NFT_RULES" nft_rules || exit 76 _tmp_nft= -if [ "$FIREWALL_MODE" = native_nftables ]; then - if ! grep -Fqx 'include "/etc/nftables.d/virt-cloud-wg-fabric.nft"' "$NFT_MAIN" && - ! grep -Fqx 'include "/etc/nftables.d/*.nft"' "$NFT_MAIN" && - ! grep -Fqx 'include "/etc/nftables.d/*"' "$NFT_MAIN"; then - _tmp_nft_main=$(mktemp "${{NFT_MAIN}}.virt-cloud.XXXXXX") - cat "$NFT_MAIN" > "$_tmp_nft_main" - printf '%s\\n' 'include "/etc/nftables.d/virt-cloud-wg-fabric.nft"' >> "$_tmp_nft_main" - "$NFT" -c -f "$_tmp_nft_main" >/dev/null 2>&1 || exit 72 - chmod "$($STAT -c '%a' "$NFT_MAIN")" "$_tmp_nft_main" - publish_desired_file "$_tmp_nft_main" "$NFT_MAIN" nft_main || exit 76 - _tmp_nft_main= - fi -else - if [ ! -f "$NFT_MANIFEST" ]; then - _tmp_nft_main=$(mktemp "$BASE/.nftables.conf.XXXXXX") - printf '%s\\n' '# virt-cloud-nft-manifest-v1' \ - 'include "/etc/nftables.d/virt-cloud-wg-fabric.nft"' > "$_tmp_nft_main" - chmod 600 "$_tmp_nft_main" - publish_desired_file "$_tmp_nft_main" "$NFT_MANIFEST" nft_main || exit 76 - _tmp_nft_main= - fi - vc_nft_manifest_validate "$NFT_MANIFEST" || exit 71 +if ! grep -Fqx 'include "/etc/nftables.d/virt-cloud-wg-fabric.nft"' "$NFT_MAIN" && + ! grep -Fqx 'include "/etc/nftables.d/*.nft"' "$NFT_MAIN" && + ! grep -Fqx 'include "/etc/nftables.d/*"' "$NFT_MAIN"; then + _tmp_nft_main=$(mktemp "${{NFT_MAIN}}.virt-cloud.XXXXXX") + cat "$NFT_MAIN" > "$_tmp_nft_main" + printf '%s\\n' 'include "/etc/nftables.d/virt-cloud-wg-fabric.nft"' >> "$_tmp_nft_main" + "$NFT" -c -f "$_tmp_nft_main" >/dev/null 2>&1 || exit 72 + chmod "$($STAT -c '%a' "$NFT_MAIN")" "$_tmp_nft_main" + publish_desired_file "$_tmp_nft_main" "$NFT_MAIN" nft_main || exit 76 + _tmp_nft_main= fi # Only after every read-only ownership, endpoint, route, address, and # persistence preflight has passed may this helper create or validate the @@ -9359,31 +7278,6 @@ printf '%s\n' "$fabric_acl_applied" > "$TXN/nft.applied.tmp" || exit 76 chmod 600 "$TXN/nft.applied.tmp" || exit 76 mv -f "$TXN/nft.applied.tmp" "$TXN/nft.applied" || exit 76 write_nft_phase refreshed || exit 76 -if [ "$FIREWALL_MODE" = firewalld_nft ]; then - underlay_runtime_state=$(vc_firewall_rule_state runtime "$UNDERLAY_ZONE" "$UNDERLAY_RULE") || exit 71 - underlay_permanent_state=$(vc_firewall_rule_state permanent "$UNDERLAY_ZONE" "$UNDERLAY_RULE") || exit 71 - if [ "$UNDERLAY_RULE_OWNER" = panel ]; then - if [ "$underlay_permanent_state" = no ]; then - "$FIREWALL_CMD" --permanent --zone="$UNDERLAY_ZONE" --add-rich-rule="$UNDERLAY_RULE" >/dev/null 2>&1 || exit 72 - fi - if [ "$underlay_runtime_state" = no ]; then - "$FIREWALL_CMD" --zone="$UNDERLAY_ZONE" --add-rich-rule="$UNDERLAY_RULE" >/dev/null 2>&1 || exit 72 - fi - else - [ "$UNDERLAY_RULE_OWNER" = external ] && - [ "$underlay_runtime_state:$underlay_permanent_state" = yes:yes ] || exit 71 - fi - [ "$(vc_firewall_rule_state runtime "$UNDERLAY_ZONE" "$UNDERLAY_RULE")" = yes ] || exit 71 - [ "$(vc_firewall_rule_state permanent "$UNDERLAY_ZONE" "$UNDERLAY_RULE")" = yes ] || exit 71 - if [ "$OLD_UNDERLAY_RULE_REMOVE" = yes ]; then - [ "$(vc_firewall_rule_state runtime "$OLD_UNDERLAY_ZONE" "$OLD_UNDERLAY_RULE")" = yes ] || exit 71 - [ "$(vc_firewall_rule_state permanent "$OLD_UNDERLAY_ZONE" "$OLD_UNDERLAY_RULE")" = yes ] || exit 71 - "$FIREWALL_CMD" --zone="$OLD_UNDERLAY_ZONE" --remove-rich-rule="$OLD_UNDERLAY_RULE" >/dev/null 2>&1 || exit 72 - "$FIREWALL_CMD" --permanent --zone="$OLD_UNDERLAY_ZONE" --remove-rich-rule="$OLD_UNDERLAY_RULE" >/dev/null 2>&1 || exit 72 - [ "$(vc_firewall_rule_state runtime "$OLD_UNDERLAY_ZONE" "$OLD_UNDERLAY_RULE")" = no ] || exit 71 - [ "$(vc_firewall_rule_state permanent "$OLD_UNDERLAY_ZONE" "$OLD_UNDERLAY_RULE")" = no ] || exit 71 - fi -fi [ -f "$KEY" ] && [ ! -L "$KEY" ] || exit 71 [ "$($STAT -c '%u:%a' "$KEY" 2>/dev/null)" = 0:600 ] || exit 71 [ "$($WG pubkey < "$KEY")" = "$EXPECTED_PUBLIC_KEY" ] || exit 71 @@ -9398,14 +7292,14 @@ if [ "$DEPLOYMENT_MODE" = networkmanager_nmcli ]; then {{ IFS= read -r nm_first || true; IFS= read -r nm_second || true; }} < "$NM_PROFILE" [ "$nm_first" = '# virt-cloud-wg-fabric-v1' ] || exit 71 case "$nm_second" in '# fabric-plan-digest='*) : ;; *) exit 71 ;; esac - awk -F= -v id="$CONNECTION" -v uuid="$CONNECTION_UUID" -v firewall_mode="$FIREWALL_MODE" ' + awk -F= -v id="$CONNECTION" -v uuid="$CONNECTION_UUID" ' $0 == "type=wireguard" {{type=1}} $0 == "interface-name=wg-fabric0" {{iface=1}} $0 == "id=" id {{name=1}} $0 == "uuid=" uuid {{identity=1}} $0 == "autoconnect=false" {{manual=1}} - $0 == "zone=trusted" {{trusted++}} - END {{exit (type && iface && name && identity && manual && trusted == 0) ? 0 : 1}} + /^zone=/ {{zone++}} + END {{exit (type && iface && name && identity && manual && zone == 0) ? 0 : 1}} ' "$NM_PROFILE" || exit 71 fi else @@ -9449,23 +7343,10 @@ vc_fabric_rp_filter_set || exit 71 if [ "$DEPLOYMENT_MODE" = networkmanager_nmcli ]; then "$NMCLI" connection load "$NM_PROFILE" >/dev/null 2>&1 || exit 72 vc_fabric_nm_profile_owned expected || exit 71 - if [ "$FIREWALL_MODE" = firewalld_nft ]; then - [ "$(vc_firewall_zone permanent)" = none ] || exit 71 - fabric_acl_pretrust=$($NFT -s list table inet virt_cloud_wg_fabric 2>/dev/null) || exit 72 - nft_plan_owned "$fabric_acl_pretrust" "$EXPECTED_DIGEST" || exit 71 - vpc_state_nft_set_matches || exit 71 - "$NMCLI" connection modify --temporary uuid {_q(connection_uuid)} connection.zone trusted >/dev/null 2>&1 || exit 72 - [ "$("$NMCLI" -t --escape no -g connection.zone connection show uuid {_q(connection_uuid)} 2>/dev/null)" = trusted ] || exit 71 - grep -Eq '^zone=' "$NM_PROFILE" && exit 71 || true - fi vc_fabric_nm_release || exit 72 "$NMCLI" connection up uuid {_q(connection_uuid)} >/dev/null 2>&1 || exit 72 "$IP" link show dev "$IFACE" >/dev/null 2>&1 || exit 72 vc_fabric_nm_active || exit 72 - if [ "$FIREWALL_MODE" = firewalld_nft ]; then - [ "$(vc_firewall_zone runtime)" = trusted ] || exit 71 - [ "$(vc_firewall_zone permanent)" = none ] || exit 71 - fi else [ ! -e "/etc/sysconfig/network-scripts/ifcfg-$IFACE" ] && [ ! -L "/etc/sysconfig/network-scripts/ifcfg-$IFACE" ] || exit 71 @@ -9478,20 +7359,6 @@ else IFUPDOWN_VC_FABRIC_OPERATION_ID="${{VC_FABRIC_OPERATION_ID:-}}" \\ "$IFUP" --force "$IFACE" >/dev/null 2>&1 || exit 72 "$IP" link show dev "$IFACE" >/dev/null 2>&1 || exit 72 - if [ "$FIREWALL_MODE" = firewalld_nft ]; then - runtime_zone=$(vc_firewall_zone runtime) || exit 71 - permanent_zone=$(vc_firewall_zone permanent) || exit 71 - case "$runtime_zone:$permanent_zone" in - none:none|trusted:none) : ;; - *) exit 71 ;; - esac - fabric_acl_pretrust=$($NFT -s list table inet virt_cloud_wg_fabric 2>/dev/null) || exit 72 - nft_plan_owned "$fabric_acl_pretrust" "$EXPECTED_DIGEST" || exit 71 - vpc_state_nft_set_matches || exit 71 - "$FIREWALL_CMD" --zone=trusted --add-interface="$IFACE" >/dev/null 2>&1 || exit 72 - [ "$(vc_firewall_zone permanent)" = none ] || exit 71 - [ "$(vc_firewall_zone runtime)" = trusted ] || exit 71 - fi fi "$WG" show "$IFACE" >/dev/null 2>&1 || exit 71 vc_fabric_sysctl_file_owned "$SYSCTL_FILE" "$EXPECTED_DIGEST" || exit 71 @@ -9515,17 +7382,8 @@ vc_fabric_rp_filter_set || exit 71 {route_commands} {path_checks} vc_fabric_operation_checkpoint persistence -_tmp_conf_template=$(mktemp "$BASE/.wg-fabric0.conf-template.XXXXXX") -{_here_doc('$_tmp_conf_template', config).rstrip()} _tmp_conf=$(mktemp "$BASE/.wg-fabric0.conf.XXXXXX") -awk -F'|' -v iface="$UNDERLAY_INTERFACE" -v zone="$UNDERLAY_ZONE" -v rule_owner="$UNDERLAY_RULE_OWNER" ' - BEGIN {{OFS="|"}} - $1=="underlay" {{$4=iface; $5=zone}} - $1=="firewall_rule" {{$3=rule_owner}} - {{print}} -' "$_tmp_conf_template" > "$_tmp_conf" -rm -f "$_tmp_conf_template" -_tmp_conf_template= +{_here_doc('$_tmp_conf', config).rstrip()} chmod 600 "$_tmp_conf" publish_desired_file "$_tmp_conf" "$CONF" conf || exit 76 _tmp_conf= @@ -9560,8 +7418,6 @@ done "$SYSTEMCTL" enable {FABRIC_SERVICE} >/dev/null "$SYSTEMCTL" enable {FABRIC_TIMER} >/dev/null vc_fabric_operation_checkpoint final_readback -if [ "$FIREWALL_MODE" = firewalld_nft ] || - [ -n "${{VC_FABRIC_EMERGENCY_SNAPSHOT:-}}" ]; then VC_FABRIC_APPLY_DETAIL=final_acl final_fabric_acl=$($NFT -s list table inet virt_cloud_wg_fabric 2>/dev/null) || exit 72 nft_plan_owned "$final_fabric_acl" "$EXPECTED_DIGEST" || exit 71 @@ -9656,16 +7512,7 @@ if [ "$FIREWALL_MODE" = firewalld_nft ] || done VC_FABRIC_APPLY_DETAIL=final_overlay_paths {path_checks} - if [ "$FIREWALL_MODE" = firewalld_nft ]; then - VC_FABRIC_APPLY_DETAIL=final_zone - [ "$(vc_firewall_zone permanent)" = none ] || exit 71 - [ "$(vc_firewall_zone runtime)" = trusted ] || exit 71 - fi - # Keep the two-table emergency guard through durable commit. The queued - # reconcile service (and its timer retry) runs the authoritative helper, - # which restores and proves every VPC before it alone removes the guard. : -fi VC_FABRIC_APPLY_DETAIL=commit _txn_state_tmp=$(mktemp "$TXN/.state.XXXXXX") printf '%s\n' committed > "$_txn_state_tmp" @@ -9682,8 +7529,8 @@ VC_FABRIC_APPLY_DETAIL=convergence # Release the transaction flock only after durable commit, then synchronously # run the independent dispatcher. It either waits for an inactive primary or, # when primary is already active, executes the complete helper to restore and -# read back every VPC before removing the emergency guard. The active timer -# remains the retry path if this convergence attempt fails. +# read back every VPC. The active timer remains the retry path if this +# convergence attempt fails. exec 9>&- "$SYSTEMCTL" start {FABRIC_REPAIR_SERVICE} >/dev/null 2>&1 || exit 79 [ ! -L "$LOCK" ] && [ -f "$LOCK" ] && @@ -9715,10 +7562,6 @@ printf '%s\n' "$final_post_reconcile_link" | awk -F'[<>]' 'NF >= 3 {{ count=split($2, flags, ",") for (i=1; i<=count; i++) if (flags[i] == "UP") found=1 }} END {{exit found ? 0 : 1}}' || exit 79 -if [ "$FIREWALL_MODE" = firewalld_nft ]; then - [ "$(vc_firewall_zone permanent)" = none ] || exit 79 - [ "$(vc_firewall_zone runtime)" = trusted ] || exit 79 -fi final_emergency_owner=/run/virt-cloud/wg-fabric-emergency.owner [ ! -e "$final_emergency_owner" ] && [ ! -L "$final_emergency_owner" ] || exit 79 final_emergency_tables=$($NFT list tables 2>/dev/null) || exit 79 @@ -9745,6 +7588,10 @@ def _observe_script( firewall_mode: str = FIREWALL_MODE_NATIVE, underlay_endpoint: str | None = None, ) -> str: + if firewall_mode not in FABRIC_FIREWALL_MODES: + raise FabricError( + "宿主机防火墙部署模式不受支持", 409, + "fabric_host_firewall_mode_conflict") target_lines = "\n".join( f'''if $PING -c 1 -W 1 {_q(target)} >/dev/null 2>&1; then printf '%s\\n' "__VC_FABRIC_PING__={target}|ok" @@ -9789,7 +7636,7 @@ IFUPDOWN_FRAGMENT={_q(FABRIC_IFUPDOWN_FRAGMENT)} SYSCTL_FILE={_q(FABRIC_SYSCTL)} LOCK={_q(FABRIC_LOCK)} NFT_RULES={_q(FABRIC_NFT_RULES)} -NFT_MANIFEST={_q(FABRIC_NFT_MANIFEST)} +LEGACY_NFT_MANIFEST={_q(LEGACY_FIREWALLD_NFT_MANIFEST)} HELPER=/usr/local/libexec/virt-cloud-wg-fabric UNIT=/etc/systemd/system/{FABRIC_SERVICE} REPAIR_UNIT=/etc/systemd/system/{FABRIC_REPAIR_SERVICE} @@ -9800,7 +7647,6 @@ find_tool() {{ for p in "$@"; do [ -x "$p" ] && {{ printf '%s' "$p"; return 0; } WG=$(find_tool /usr/bin/wg /usr/sbin/wg /bin/wg /sbin/wg) || exit 70 IP=$(find_tool /usr/sbin/ip /usr/bin/ip /sbin/ip /bin/ip) || exit 70 SYSTEMCTL=$(find_tool /usr/bin/systemctl /bin/systemctl /usr/sbin/systemctl /sbin/systemctl) || exit 70 -FIREWALL_CMD=$(find_tool /usr/bin/firewall-cmd /usr/sbin/firewall-cmd /bin/firewall-cmd /sbin/firewall-cmd || true) AWK=$(find_tool /usr/bin/awk /bin/awk /usr/sbin/awk /sbin/awk) || exit 70 PING=$(find_tool /usr/bin/ping /bin/ping /usr/sbin/ping /sbin/ping || true) NFT=$(find_tool /usr/sbin/nft /usr/bin/nft /sbin/nft /bin/nft || true) @@ -9808,11 +7654,8 @@ STAT=$(find_tool /usr/bin/stat /bin/stat /usr/sbin/stat /sbin/stat || true) FLOCK=$(find_tool /usr/bin/flock /usr/sbin/flock /bin/flock /sbin/flock) || exit 70 {_NMCLI_FILENAME_RESOLVER_SH} {_FIREWALL_MODE_GUARD_SH} -{_NFT_MANIFEST_VALIDATION_SH} {_FABRIC_NFT_VALIDATION_SH} {_FABRIC_SYSCTL_VALIDATION_SH} -firewall_mode_state=owned -vc_firewall_mode_ready || firewall_mode_state=mismatch read_observed_sysctl() {{ vc_read_proc_sysctl "$1" }} @@ -9869,6 +7712,10 @@ if [ ! -e "$LOCK" ]; then : > "$LOCK"; chmod 600 "$LOCK"; fi [ -f "$LOCK" ] && [ "$($STAT -c '%u:%a' "$LOCK" 2>/dev/null)" = 0:600 ] || exit 71 exec 9>>"$LOCK" "$FLOCK" -x 9 || exit 79 +firewall_mode_state=owned +vc_firewall_mode_ready && vc_legacy_firewalld_state_absent && +[ ! -e /etc/virt-cloud/.wg-fabric-txn ] && +[ ! -L /etc/virt-cloud/.wg-fabric-txn ] || firewall_mode_state=mismatch {_VPC_STATE_LOADER.replace('__VC_FABRIC_HOST_ID__', host_id or '').replace('__VC_FABRIC_ID__', fabric_id or '*').replace('__VC_FABRIC_NETWORK__', overlay_cidr or '').replace('__VC_VPC_STATE_OBSERVE__', 'yes')} vpc_observe_cleanup() {{ [ -z "${{VPC_PEER_FILE:-}}" ] || rm -f "$VPC_PEER_FILE" @@ -9917,17 +7764,6 @@ if [ "$DEPLOYMENT_MODE" = networkmanager_nmcli ] && [ -n "$NMCLI" ]; then vc_fabric_nm_active; then manager_active=active fi - if [ "$FIREWALL_MODE" = firewalld_nft ]; then - observed_zone=$(vc_firewall_zone runtime 2>/dev/null || printf foreign) - observed_permanent_zone=$(vc_firewall_zone permanent 2>/dev/null || printf foreign) - observed_memory_zone=$($NMCLI -t --escape no -g connection.zone connection show uuid "$CONNECTION_UUID" 2>/dev/null | sed -n '1p') - if [ "$observed_zone" != trusted ] || - [ "$observed_permanent_zone" != none ] || - [ "$observed_memory_zone" != trusted ] || - grep -Eq '^zone=' "$NM_PROFILE"; then - manager_active=inactive - fi - fi elif [ "$DEPLOYMENT_MODE" = ifupdown ] && [ -n "$IFQUERY" ]; then if [ ! -L /etc/network ] && [ ! -L /etc/network/interfaces.d ] && [ -f "$IFUPDOWN_FRAGMENT" ] && [ ! -L "$IFUPDOWN_FRAGMENT" ] && [ -n "$STAT" ] && @@ -9953,14 +7789,9 @@ elif [ "$DEPLOYMENT_MODE" = ifupdown ] && [ -n "$IFQUERY" ]; then "$IP" link show dev "$IFACE" >/dev/null 2>&1; then manager_active=active fi - if [ "$FIREWALL_MODE" = firewalld_nft ]; then - observed_zone=$(vc_firewall_zone runtime 2>/dev/null || printf foreign) - observed_permanent_zone=$(vc_firewall_zone permanent 2>/dev/null || printf foreign) - if [ "$observed_zone" != trusted ] || [ "$observed_permanent_zone" != none ] || - [ -e "/etc/sysconfig/network-scripts/ifcfg-$IFACE" ] || - [ -L "/etc/sysconfig/network-scripts/ifcfg-$IFACE" ]; then - manager_active=inactive - fi + if [ -e "/etc/sysconfig/network-scripts/ifcfg-$IFACE" ] || + [ -L "/etc/sysconfig/network-scripts/ifcfg-$IFACE" ]; then + manager_active=inactive fi fi if [ "$DEPLOYMENT_MODE" = networkmanager_nmcli ] && @@ -10018,12 +7849,6 @@ config_firewall= config_network= config_overlay= config_listen= -config_underlay_endpoint= -config_underlay_iface= -config_underlay_zone= -config_firewall_rule_owner= -config_firewall_rule_runtime=na -config_firewall_rule_permanent=na config_size_ok=no config_ok=no if [ -n "$STAT" ] && [ -f "$CONF" ] && [ ! -L "$CONF" ] && @@ -10093,8 +7918,6 @@ EOF config_network_count=0 config_overlay_count=0 config_listen_count=0 - config_underlay_count=0 - config_firewall_rule_count=0 config_peer_count=0 config_route_count=0 config_allowed= @@ -10126,7 +7949,7 @@ EOF config_digest=$config_a config_digest_count=$((config_digest_count + 1)) ;; firewall) - case "$config_a" in native_nftables|firewalld_nft) : ;; *) config_ok=no ;; esac + [ "$config_a" = native_nftables ] || config_ok=no [ -z "$config_b$config_c$config_d" ] || config_ok=no config_firewall=$config_a config_firewall_count=$((config_firewall_count + 1)) ;; @@ -10145,24 +7968,8 @@ EOF [ -z "$config_b$config_c$config_d" ] || config_ok=no config_listen=$config_a config_listen_count=$((config_listen_count + 1)) ;; - underlay) - [ "$config_a" = local ] || config_ok=no - config_is_ipv4 "$config_b" || config_ok=no - case "$config_c" in ''|*[!A-Za-z0-9_.-]*) config_ok=no ;; esac - [ "$(printf '%s' "$config_c" | awk 'length($0)<=15')" = "$config_c" ] || config_ok=no - case "$config_d" in ''|*[!A-Za-z0-9_-]*) config_ok=no ;; esac - [ "$(printf '%s' "$config_d" | awk 'length($0)<=64')" = "$config_d" ] || config_ok=no - config_underlay_endpoint=$config_b - config_underlay_iface=$config_c - config_underlay_zone=$config_d - config_underlay_count=$((config_underlay_count + 1)) ;; - firewall_rule) - config_is_port "$config_a" || config_ok=no - case "$config_b" in panel|external) : ;; *) config_ok=no ;; esac - [ -z "$config_c$config_d" ] || config_ok=no - config_firewall_rule_port=$config_a - config_firewall_rule_owner=$config_b - config_firewall_rule_count=$((config_firewall_rule_count + 1)) ;; + underlay|firewall_rule) + config_ok=no ;; peer) config_is_host_id "$config_a" || config_ok=no config_is_key "$config_b" || config_ok=no @@ -10196,23 +8003,7 @@ EOF [ "$config_listen_count" = 1 ] && [ "$config_peer_count" -gt 0 ] && [ "$config_peer_count" -le {MAX_FABRIC_NODES} ] && [ "$config_peer_count" = "$config_route_count" ] || config_ok=no - if [ "$config_firewall" = firewalld_nft ]; then - [ "$config_underlay_count" = 1 ] && - [ "$config_firewall_rule_count" = 1 ] && - [ "$config_firewall_rule_port" = "$config_listen" ] || config_ok=no - config_underlay_actual=$(vc_underlay_discover "$config_underlay_endpoint" "$config_peer_endpoint_ips" 2>/dev/null) || config_ok=no - [ "$config_underlay_actual" = "$config_underlay_iface|$config_underlay_zone" ] || config_ok=no - config_firewall_rule=$(vc_firewall_rich_rule "$config_underlay_endpoint" "$config_firewall_rule_port" 2>/dev/null) || config_ok=no - if [ -n "$config_firewall_rule" ]; then - config_firewall_rule_runtime=$(vc_firewall_rule_state runtime "$config_underlay_zone" "$config_firewall_rule" 2>/dev/null || printf unknown) - config_firewall_rule_permanent=$(vc_firewall_rule_state permanent "$config_underlay_zone" "$config_firewall_rule" 2>/dev/null || printf unknown) - fi - [ "$config_firewall_rule_runtime:$config_firewall_rule_permanent" = yes:yes ] || config_ok=no - else - [ "$config_firewall" = native_nftables ] || config_ok=no - [ "$config_underlay_count" = 0 ] && - [ "$config_firewall_rule_count" = 0 ] || config_ok=no - fi + [ "$config_firewall" = native_nftables ] || config_ok=no fi printf '%s\\n' "__VC_FABRIC_CONFIG_FABRIC__=$config_fabric" printf '%s\\n' "__VC_FABRIC_CONFIG_REVISION__=$config_revision" @@ -10220,10 +8011,6 @@ printf '%s\\n' "__VC_FABRIC_CONFIG_DIGEST__=$config_digest" printf '%s\\n' "__VC_FABRIC_CONFIG_FIREWALL__=$config_firewall" printf '%s\\n' "__VC_FABRIC_CONFIG_NETWORK__=$config_network" printf '%s\\n' "__VC_FABRIC_CONFIG_OVERLAY__=$config_overlay" -if [ "$config_firewall" = firewalld_nft ]; then - printf '%s\\n' "__VC_FABRIC_CONFIG_UNDERLAY__=$config_underlay_endpoint|$config_underlay_iface|$config_underlay_zone" - printf '%s\\n' "__VC_FABRIC_FIREWALL_RULE__=$config_firewall_rule_owner|$config_firewall_rule_runtime|$config_firewall_rule_permanent" -fi printf '%s\\n' "__VC_FABRIC_CONFIG_OK__=$config_ok" sysctl_persistence=missing if [ -e "$SYSCTL_FILE" ] || [ -L "$SYSCTL_FILE" ]; then @@ -10356,25 +8143,22 @@ if [ -n "$NFT" ] && [ -n "$STAT" ] && [ -f "$NFT_RULES" ] && [ ! -L "$NFT_RULES" {{ IFS= read -r nft_rule_first || true; IFS= read -r nft_rule_second || true; }} < "$NFT_RULES" if [ "$nft_rule_first" = '# virt-cloud-wg-fabric-v1' ] && [ "$nft_rule_second" = "# fabric-plan-digest=$(awk -F'|' '/^digest\\|/ {{print $2; exit}}' "$CONF" 2>/dev/null)" ]; then - if [ "$FIREWALL_MODE" = native_nftables ]; then - nft_main_count=0 - for nft_main in /etc/nftables.conf /etc/sysconfig/nftables.conf; do - [ -f "$nft_main" ] && [ ! -L "$nft_main" ] || continue - [ "$($STAT -c '%u' "$nft_main" 2>/dev/null)" = 0 ] || continue - nft_main_mode=$($STAT -c '%a' "$nft_main" 2>/dev/null || true) - case "$nft_main_mode" in [0-7][0145][0145]) : ;; *) continue ;; esac - nft_main_count=$((nft_main_count + 1)) - if grep -Fqx 'include "/etc/nftables.d/virt-cloud-wg-fabric.nft"' "$nft_main" || - grep -Fqx 'include "/etc/nftables.d/*.nft"' "$nft_main" || - grep -Fqx 'include "/etc/nftables.d/*"' "$nft_main"; then - nft_persistence=owned - fi - done - [ "$nft_main_count" = 1 ] || nft_persistence=missing - [ ! -e "$NFT_MANIFEST" ] && [ ! -L "$NFT_MANIFEST" ] || nft_persistence=missing - elif vc_nft_manifest_validate "$NFT_MANIFEST"; then - nft_persistence=owned - fi + nft_main_count=0 + for nft_main in /etc/nftables.conf /etc/sysconfig/nftables.conf; do + [ -f "$nft_main" ] && [ ! -L "$nft_main" ] || continue + [ "$($STAT -c '%u' "$nft_main" 2>/dev/null)" = 0 ] || continue + nft_main_mode=$($STAT -c '%a' "$nft_main" 2>/dev/null || true) + case "$nft_main_mode" in [0-7][0145][0145]) : ;; *) continue ;; esac + nft_main_count=$((nft_main_count + 1)) + if grep -Fqx 'include "/etc/nftables.d/virt-cloud-wg-fabric.nft"' "$nft_main" || + grep -Fqx 'include "/etc/nftables.d/*.nft"' "$nft_main" || + grep -Fqx 'include "/etc/nftables.d/*"' "$nft_main"; then + nft_persistence=owned + fi + done + [ "$nft_main_count" = 1 ] || nft_persistence=missing + [ ! -e "$LEGACY_NFT_MANIFEST" ] && + [ ! -L "$LEGACY_NFT_MANIFEST" ] || nft_persistence=missing fi fi if [ -n "$NFT" ]; then @@ -10478,16 +8262,6 @@ def _parse_markers(output: str) -> dict[str, Any]: result["config_network"] = line.split("=", 1)[1] elif line.startswith("__VC_FABRIC_CONFIG_OVERLAY__="): result["config_overlay"] = line.split("=", 1)[1] - elif line.startswith("__VC_FABRIC_CONFIG_UNDERLAY__="): - fields = line.split("=", 1)[1].split("|") - if len(fields) == 3 and all(fields): - result["config_underlay"] = tuple(fields) - elif line.startswith("__VC_FABRIC_FIREWALL_RULE__="): - fields = line.split("=", 1)[1].split("|") - if (len(fields) == 3 and fields[0] in {"panel", "external"} - and fields[1] in {"yes", "no", "unknown"} - and fields[2] in {"yes", "no", "unknown"}): - result["firewall_rule"] = tuple(fields) elif line.startswith("__VC_FABRIC_CONFIG_OK__="): result["config_ok"] = line.split("=", 1)[1] elif line.startswith("__VC_FABRIC_CONFIG_PEER__="): @@ -11034,20 +8808,6 @@ class FabricHostAdapter: } observed_sysctls = parsed.get("sysctls", {}) observed_rp_filters = parsed.get("rp_filters", {}) - firewall_rule_ok = firewall_mode == FIREWALL_MODE_NATIVE - if firewall_mode == FIREWALL_MODE_FIREWALLD: - raw_underlay = parsed.get("config_underlay") - raw_rule = parsed.get("firewall_rule") - firewall_rule_ok = ( - isinstance(raw_underlay, (list, tuple)) - and len(raw_underlay) == 3 - and raw_underlay[0] == host.endpoint.rsplit(":", 1)[0] - and all(isinstance(item, str) and item for item in raw_underlay) - and isinstance(raw_rule, (list, tuple)) - and len(raw_rule) == 3 - and raw_rule[0] in {"panel", "external"} - and tuple(raw_rule[1:]) == ("yes", "yes") - ) sysctl_ok = ( isinstance(observed_sysctls, Mapping) and observed_sysctls.get("ip_forward") == "1" @@ -11105,7 +8865,6 @@ class FabricHostAdapter: and parsed.get("manager_active") == "active" and parsed.get("firewall_mode") == firewall_mode and parsed.get("firewall_state") == "owned" - and firewall_rule_ok # A successful apply is not enough when an operator later brings # the interface down or changes forwarding/rp_filter. Require a # fresh kernel readback before exposing the Fabric as active. @@ -11139,8 +8898,6 @@ class FabricHostAdapter: "config_revision": parsed.get("config_revision"), "config_digest": parsed.get("config_digest"), "config_firewall_mode": parsed.get("config_firewall_mode"), - "config_underlay": parsed.get("config_underlay"), - "firewall_rule": parsed.get("firewall_rule"), "config_network": parsed.get("config_network"), "config_overlay": parsed.get("config_overlay"), "config_ok": parsed.get("config_ok"), @@ -11490,12 +9247,6 @@ class FabricCoordinator: "underlay_interface": safe_text( network.get("fabric_endpoint_interface"), 15) if isinstance(network, Mapping) else None, - "underlay_runtime_zone": safe_text( - network.get("fabric_endpoint_runtime_zone"), 64) - if isinstance(network, Mapping) else None, - "underlay_permanent_zone": safe_text( - network.get("fabric_endpoint_permanent_zone"), 64) - if isinstance(network, Mapping) else None, "ip_forward": safe_sysctl(network.get("ip_forward")) if isinstance(network, Mapping) else None, "rp_filter": safe_sysctl(network.get("rp_filter")) @@ -11700,23 +9451,6 @@ class FabricCoordinator: capabilities_with_endpoint["network"] = network_with_endpoint capabilities = capabilities_with_endpoint network = capabilities.get("network") - if firewall_mode == FIREWALL_MODE_FIREWALLD: - endpoint_evidence, endpoint_error = ( - _firewalld_endpoint_zone_evidence( - capabilities, endpoint_ip)) - if endpoint_error is not None or endpoint_evidence is None: - return self._readiness_result( - host_id, profile=live_profile, configured=True, - reachable=True, - error_code=(endpoint_error or - "firewalld_endpoint_zone_unverified")) - # Preserve raw capability evidence internally, but project - # only this endpoint's validated interface and effective - # zones through the administrator readiness response. - network_with_endpoint = dict(network) - network_with_endpoint.update(endpoint_evidence) - capabilities_with_endpoint["network"] = network_with_endpoint - capabilities = capabilities_with_endpoint return self._readiness_result( host_id, profile=live_profile, configured=True, reachable=True, capabilities=capabilities) @@ -11874,7 +9608,8 @@ class FabricCoordinator: return "network_manager_failed" if code in { "fabric_host_firewall_mode_conflict", - "firewalld_endpoint_zone_unverified"}: + "legacy_firewalld_state_requires_cleanup", + "firewalld_unsupported_requires_maintenance"}: return "firewall_conflict" if code in { "fabric_endpoint_mismatch", "fabric_endpoint_not_local", @@ -11944,7 +9679,7 @@ class FabricCoordinator: def _public(value: Mapping[str, Any]) -> dict[str, Any]: """Return a bounded status projection, never a raw durable record. - The durable row is local trusted state today, but keeping this + The durable row is local observed state today, but keeping this boundary recursive prevents a future diagnostic field (a command, private key, or raw remote output) from reaching an HTTP caller just because it was nested below ``nodes``. @@ -12250,20 +9985,6 @@ class FabricCoordinator: capabilities_with_endpoint["network"] = network_with_endpoint capabilities = capabilities_with_endpoint network = capabilities.get("network") - if firewall_mode == FIREWALL_MODE_FIREWALLD: - endpoint_evidence, endpoint_error = ( - _firewalld_endpoint_zone_evidence( - capabilities, endpoint_ip)) - if endpoint_error is not None or endpoint_evidence is None: - raise FabricError( - "Fabric 传输接口的 firewalld zone 无法安全确认", - 409, endpoint_error or - "firewalld_endpoint_zone_unverified") - network_with_endpoint = dict(network) - network_with_endpoint.update(endpoint_evidence) - capabilities_with_endpoint["network"] = network_with_endpoint - capabilities = capabilities_with_endpoint - network = network_with_endpoint if node.endpoint.rsplit(":", 1)[0] != endpoint_ip: raise FabricError( "Fabric 计划中的传输地址与宿主机配置不一致", @@ -13566,7 +11287,7 @@ class FabricCoordinator: # observer fail before it can emit markers. The narrow # identity probe is the recovery precondition: if that # succeeds for every repair target, the apply script can - # safely validate and converge its owned emergency state. + # safely validate and converge its native nft state. repair_targets.append((host, node, adapter)) # Verify every repair candidate without creating a replacement diff --git a/engine/virsh_panel/remote_host.py b/engine/virsh_panel/remote_host.py index 401eb4d0..58e8f9d5 100644 --- a/engine/virsh_panel/remote_host.py +++ b/engine/virsh_panel/remote_host.py @@ -44,7 +44,6 @@ _GEN_NAME = re.compile(r"gen-[0-9a-f]{32}\Z") _POOL_PATH = re.compile(r"/[A-Za-z0-9_.-]+(?:/[A-Za-z0-9_.-]+)*\Z") _BRIDGE_NAME = re.compile(r"[A-Za-z0-9][A-Za-z0-9_.-]{0,14}\Z") _IFACE_NAME = re.compile(r"[A-Za-z0-9_.:-]{1,15}\Z") -_FIREWALL_ZONE = re.compile(r"[A-Za-z0-9_-]{1,64}\Z") PUBLIC_POOL_KEY = re.compile(r"[0-9a-f]{32}\Z") PUBLIC_POOL_STATES = frozenset({"active", "draining", "disabled"}) MAX_PUBLIC_NETWORK_POOLS = 32 @@ -95,7 +94,6 @@ probe_binary wg /usr/bin/wg /usr/sbin/wg /bin/wg /sbin/wg probe_binary wg_quick /usr/bin/wg-quick /usr/sbin/wg-quick /bin/wg-quick /sbin/wg-quick probe_binary ip /usr/sbin/ip /usr/bin/ip /sbin/ip /bin/ip probe_binary nft /usr/sbin/nft /usr/bin/nft /sbin/nft /bin/nft -probe_binary firewall_cmd /usr/bin/firewall-cmd /usr/sbin/firewall-cmd /bin/firewall-cmd /sbin/firewall-cmd probe_binary nmcli /usr/bin/nmcli /usr/sbin/nmcli /bin/nmcli /sbin/nmcli probe_binary networkctl /usr/bin/networkctl /usr/sbin/networkctl /bin/networkctl /sbin/networkctl probe_binary ifup /usr/sbin/ifup /usr/bin/ifup /sbin/ifup /bin/ifup @@ -160,7 +158,6 @@ for rp_file in /proc/sys/net/ipv4/conf/*/rp_filter; do printf "__VCAP_RPF__%s=%s\n" "$rp_iface" "$rp_value" done -firewall_service_state=unknown probe_service() { name="$1" service="$2" @@ -174,7 +171,6 @@ probe_service() { active|inactive|failed|activating|deactivating) ;; *) status="unknown" ;; esac - if test "$name" = firewall; then firewall_service_state="$status"; fi printf "__VCAP_SERVICE__%s=%s\n" "$name" "$status" } probe_service network_manager NetworkManager @@ -220,341 +216,112 @@ probe_service_load() { probe_service_load firewall firewalld probe_service_load nftables nftables -# firewalld documents nftables/yes as the defaults when these keys (or the -# whole system configuration file) are absent. Project only those two -# normalized effective values; malformed, duplicate, or unsupported values -# remain unknown and fail closed in the parser. -firewall_backend=nftables -firewall_table_owner=yes -if test -e /etc/firewalld/firewalld.conf; then - if test -f /etc/firewalld/firewalld.conf && test ! -L /etc/firewalld/firewalld.conf; then - if test -x /usr/bin/grep; then firewall_grep=/usr/bin/grep; else firewall_grep=/bin/grep; fi - if test -x /usr/bin/tr; then firewall_tr=/usr/bin/tr; else firewall_tr=/bin/tr; fi - firewall_backend_lines=$($firewall_grep -E "^[[:space:]]*FirewallBackend[[:space:]]*=" /etc/firewalld/firewalld.conf 2>/dev/null || true) - firewall_owner_lines=$($firewall_grep -E "^[[:space:]]*NftablesTableOwner[[:space:]]*=" /etc/firewalld/firewalld.conf 2>/dev/null || true) - firewall_backend_count=$(printf "%s\n" "$firewall_backend_lines" | $firewall_grep -c "=" 2>/dev/null || true) - firewall_owner_count=$(printf "%s\n" "$firewall_owner_lines" | $firewall_grep -c "=" 2>/dev/null || true) - if test "$firewall_backend_count" = 1; then - firewall_backend=${firewall_backend_lines#*=} - firewall_backend=$(printf "%s" "$firewall_backend" | $firewall_tr -d "[:space:]") - elif test "$firewall_backend_count" != 0; then - firewall_backend=unknown - fi - if test "$firewall_owner_count" = 1; then - firewall_table_owner=${firewall_owner_lines#*=} - firewall_table_owner=$(printf "%s" "$firewall_table_owner" | $firewall_tr -d "[:space:]") - elif test "$firewall_owner_count" != 0; then - firewall_table_owner=unknown - fi - else - firewall_backend=unknown - firewall_table_owner=unknown +# Native nftables is the only supported firewall owner. firewalld service +# state remains in the probe solely as fail-closed migration evidence. +probe_legacy_firewall_path() { + name="$1" + path="$2" + state=absent + if test -e "$path" || test -L "$path"; then state=present; fi + printf "__VCAP_LEGACY_FIREWALL__%s=%s\n" "$name" "$state" +} +probe_legacy_firewall_path manifest /etc/virt-cloud/nftables.conf +probe_legacy_firewall_path fabric_transaction /etc/virt-cloud/.wg-fabric-txn +probe_legacy_firewall_path emergency_owner /run/virt-cloud/wg-fabric-emergency.owner +vpc_transaction_state=absent +for vpc_transaction in /etc/virt-cloud/.vpc-dataplane-*.txn; do + if test -e "$vpc_transaction" || test -L "$vpc_transaction"; then + vpc_transaction_state=present + break fi -fi -case "$firewall_backend" in nftables|iptables) : ;; *) firewall_backend=unknown ;; esac -case "$firewall_table_owner" in yes|no) : ;; *) firewall_table_owner=unknown ;; esac -printf "__VCAP_FIREWALL_CONF__backend=%s\n" "$firewall_backend" -printf "__VCAP_FIREWALL_CONF__table_owner=%s\n" "$firewall_table_owner" -firewall_source_zones=unknown -firewall_cmd= -for firewall_candidate in /usr/bin/firewall-cmd /usr/sbin/firewall-cmd /bin/firewall-cmd /sbin/firewall-cmd; do - if test -x "$firewall_candidate"; then firewall_cmd="$firewall_candidate"; break; fi done -if test "$firewall_service_state" = active && test -n "$firewall_cmd" && - test "$($firewall_cmd --state 2>/dev/null)" = running; then - firewall_source_zones=absent - for firewall_scope in runtime permanent; do - if test "$firewall_scope" = permanent; then - firewall_zones=$($firewall_cmd --permanent --get-zones 2>/dev/null) - else - firewall_zones=$($firewall_cmd --get-zones 2>/dev/null) - fi - if test $? != 0; then firewall_source_zones=unknown; break; fi - firewall_zone_count=0 - for firewall_zone in $firewall_zones; do - case "$firewall_zone" in ""|*[!A-Za-z0-9_-]*) firewall_source_zones=unknown; break 2 ;; esac - if test "${#firewall_zone}" -gt 64; then firewall_source_zones=unknown; break 2; fi - firewall_zone_count=$((firewall_zone_count + 1)) - if test "$firewall_zone_count" -gt 64; then firewall_source_zones=unknown; break 2; fi - if test "$firewall_scope" = permanent; then - firewall_sources=$($firewall_cmd --permanent --zone="$firewall_zone" --list-sources 2>/dev/null) - else - firewall_sources=$($firewall_cmd --zone="$firewall_zone" --list-sources 2>/dev/null) - fi - if test $? != 0; then firewall_source_zones=unknown; break 2; fi - if test -n "$firewall_sources"; then firewall_source_zones=present; break 2; fi - done - if test "$firewall_zone_count" -eq 0; then firewall_source_zones=unknown; break; fi - done -fi -printf "__VCAP_FIREWALL_CONF__source_zones=%s\n" "$firewall_source_zones" -firewall_default_runtime=unknown -firewall_default_permanent=unknown -if test "$firewall_service_state" = active && test -n "$firewall_cmd" && - test "$($firewall_cmd --state 2>/dev/null)" = running; then - firewall_default_runtime=$($firewall_cmd --get-default-zone 2>/dev/null) - if test $? != 0; then firewall_default_runtime=unknown; fi - # firewalld has one default zone for both views; --get-default-zone does not - # accept --permanent. Project the single authoritative value into both - # fields so the parser can retain its drift/fail-closed contract without - # invoking an unsupported CLI combination. - firewall_default_permanent=$firewall_default_runtime - case "$firewall_default_runtime" in ""|*[!A-Za-z0-9_-]*) firewall_default_runtime=unknown ;; esac - case "$firewall_default_permanent" in ""|*[!A-Za-z0-9_-]*) firewall_default_permanent=unknown ;; esac - if test "${#firewall_default_runtime}" -gt 64; then firewall_default_runtime=unknown; fi - if test "${#firewall_default_permanent}" -gt 64; then firewall_default_permanent=unknown; fi -fi -printf "__VCAP_FIREWALL_CONF__default_runtime=%s\n" "$firewall_default_runtime" -printf "__VCAP_FIREWALL_CONF__default_permanent=%s\n" "$firewall_default_permanent" - -# Keep the interface-to-zone inventory bounded and syntactically closed. It -# is consumed only as structured evidence for the selected local Fabric -# endpoint; arbitrary firewalld output is never returned by the API. -probe_firewall_interfaces() { - firewall_iface_scope="$1" - printf "__VCAP_FIREWALL_IFACE_BEGIN__%s\n" "$firewall_iface_scope" - firewall_iface_status=1 - if test "$firewall_service_state" = active && test -n "$firewall_cmd" && - test "$($firewall_cmd --state 2>/dev/null)" = running; then - if test "$firewall_iface_scope" = permanent; then - firewall_iface_zones=$($firewall_cmd --permanent --get-zones 2>/dev/null) - else - firewall_iface_zones=$($firewall_cmd --get-zones 2>/dev/null) - fi - if test $? = 0; then - firewall_iface_status=0 - firewall_iface_zone_count=0 - firewall_iface_binding_count=0 - for firewall_iface_zone in $firewall_iface_zones; do - case "$firewall_iface_zone" in ""|*[!A-Za-z0-9_-]*) firewall_iface_status=1; break ;; esac - if test "${#firewall_iface_zone}" -gt 64; then firewall_iface_status=1; break; fi - firewall_iface_zone_count=$((firewall_iface_zone_count + 1)) - if test "$firewall_iface_zone_count" -gt 64; then firewall_iface_status=1; break; fi - if test "$firewall_iface_scope" = permanent; then - firewall_ifaces=$($firewall_cmd --permanent --zone="$firewall_iface_zone" --list-interfaces 2>/dev/null) - else - firewall_ifaces=$($firewall_cmd --zone="$firewall_iface_zone" --list-interfaces 2>/dev/null) - fi - if test $? != 0; then firewall_iface_status=1; break; fi - for firewall_iface in $firewall_ifaces; do - case "$firewall_iface" in ""|*[!A-Za-z0-9_.:-]*) firewall_iface_status=1; break 2 ;; esac - if test "${#firewall_iface}" -gt 15; then firewall_iface_status=1; break 2; fi - firewall_iface_binding_count=$((firewall_iface_binding_count + 1)) - if test "$firewall_iface_binding_count" -gt 4096; then firewall_iface_status=1; break 2; fi - printf "%s|%s\n" "$firewall_iface" "$firewall_iface_zone" - done - done - if test "$firewall_iface_zone_count" -eq 0; then firewall_iface_status=1; fi - fi +printf "__VCAP_LEGACY_FIREWALL__vpc_transaction=%s\n" \ + "$vpc_transaction_state" +vpc_snapshot_state=absent +for vpc_snapshot in \ + /etc/virt-cloud/vpc-dataplane/*.nft.inet.live \ + /etc/virt-cloud/vpc-dataplane/*.nft.bridge.live \ + /etc/virt-cloud/vpc-dataplane/*.nft.arp.live; do + if test -e "$vpc_snapshot" || test -L "$vpc_snapshot"; then + vpc_snapshot_state=present + break fi - printf "__VCAP_FIREWALL_IFACE_END__%s=%s\n" "$firewall_iface_scope" "$firewall_iface_status" -} -probe_firewall_interfaces runtime -probe_firewall_interfaces permanent - -# For NetworkManager, permanent interface ownership lives in the active -# connection keyfile rather than in firewalld zone XML. Resolve every active -# interface through one UUID and parse only the exact [connection] zone key -# from a root-owned 0600 system profile. UUIDs, filenames and profile data -# are deliberately not emitted. -printf "%s\n" "__VCAP_NM_ZONE_BEGIN__" -firewall_nm_status=1 -firewall_nmcli= -firewall_stat= -firewall_systemctl= -for firewall_candidate in /usr/bin/nmcli /usr/sbin/nmcli /bin/nmcli /sbin/nmcli; do - if test -x "$firewall_candidate"; then firewall_nmcli="$firewall_candidate"; break; fi done -for firewall_candidate in /usr/bin/stat /bin/stat /usr/sbin/stat /sbin/stat; do - if test -x "$firewall_candidate"; then firewall_stat="$firewall_candidate"; break; fi +printf "__VCAP_LEGACY_FIREWALL__vpc_snapshot=%s\n" \ + "$vpc_snapshot_state" +legacy_stat= +for legacy_tool in /usr/bin/stat /bin/stat /usr/sbin/stat /sbin/stat; do + if test -x "$legacy_tool"; then legacy_stat="$legacy_tool"; break; fi done -for firewall_candidate in /usr/bin/systemctl /bin/systemctl /usr/sbin/systemctl /sbin/systemctl; do - if test -x "$firewall_candidate"; then firewall_systemctl="$firewall_candidate"; break; fi +legacy_awk= +for legacy_tool in /usr/bin/awk /bin/awk /usr/sbin/awk /sbin/awk; do + if test -x "$legacy_tool"; then legacy_awk="$legacy_tool"; break; fi done -if test -n "$firewall_nmcli" && test -n "$firewall_stat" && - test -n "$firewall_systemctl" && - test "$($firewall_systemctl is-active NetworkManager 2>/dev/null)" = active; then - firewall_nm_active=$($firewall_nmcli -t --escape no -f UUID,DEVICE connection show --active 2>/dev/null) - firewall_nm_active_status=$? - firewall_nm_profiles= - firewall_nm_profiles_status=1 - if test "$firewall_nm_active_status" = 0; then - # FILENAME is a list-mode field. It is not available as - # GENERAL.FILENAME in `connection show UUID` on current AlmaLinux 10. - # Capture one bounded inventory and resolve each active UUID exactly once; - # malformed rows, duplicate UUIDs and non-system keyfiles fail closed. - firewall_nm_profiles=$($firewall_nmcli -t --escape no -f UUID,FILENAME connection show 2>/dev/null) - firewall_nm_profiles_status=$? - fi - if test "$firewall_nm_active_status" = 0 && - test "$firewall_nm_profiles_status" = 0 && - test "${#firewall_nm_active}" -le 262144 && - test "${#firewall_nm_profiles}" -le 262144; then - firewall_nm_status=0 - firewall_nm_count=0 - while IFS=: read -r firewall_nm_uuid firewall_nm_iface firewall_nm_extra; do - test -n "$firewall_nm_uuid$firewall_nm_iface$firewall_nm_extra" || continue - # Active profiles without a kernel device do not own an ingress zone. - test -n "$firewall_nm_iface" || continue - if test -n "$firewall_nm_extra"; then firewall_nm_status=1; break; fi - case "$firewall_nm_iface" in ""|*[!A-Za-z0-9_.-]*) firewall_nm_status=1; break ;; esac - if test "${#firewall_nm_iface}" -gt 15; then firewall_nm_status=1; break; fi - firewall_nm_count=$((firewall_nm_count + 1)) - if test "$firewall_nm_count" -gt 4096; then firewall_nm_status=1; break; fi - firewall_nm_profile_valid=yes - case "$firewall_nm_uuid" in ????????-????-????-????-????????????) : ;; *) firewall_nm_profile_valid=no ;; esac - case "$firewall_nm_uuid" in *[!0-9A-Fa-f-]*) firewall_nm_profile_valid=no ;; esac - if test "$firewall_nm_profile_valid" != yes; then - printf "%s|?\n" "$firewall_nm_iface" - continue - fi - firewall_nm_file= - firewall_nm_file_count=0 - firewall_nm_profile_count=0 - firewall_nm_inventory_valid=yes - while IFS= read -r firewall_nm_profile_line || test -n "$firewall_nm_profile_line"; do - test -n "$firewall_nm_profile_line" || continue - firewall_nm_profile_count=$((firewall_nm_profile_count + 1)) - if test "$firewall_nm_profile_count" -gt 4096; then - firewall_nm_inventory_valid=no - break - fi - case "$firewall_nm_profile_line" in - *:*) : ;; - *) firewall_nm_inventory_valid=no; break ;; - esac - firewall_nm_list_uuid=${firewall_nm_profile_line%%:*} - firewall_nm_list_file=${firewall_nm_profile_line#*:} - case "$firewall_nm_list_uuid" in ????????-????-????-????-????????????) : ;; *) firewall_nm_inventory_valid=no; break ;; esac - case "$firewall_nm_list_uuid" in *[!0-9A-Fa-f-]*) firewall_nm_inventory_valid=no; break ;; esac - if test "${#firewall_nm_list_file}" -gt 4096; then - firewall_nm_inventory_valid=no - break - fi - if test "$firewall_nm_list_uuid" = "$firewall_nm_uuid"; then - firewall_nm_file_count=$((firewall_nm_file_count + 1)) - firewall_nm_file=$firewall_nm_list_file - fi - done <<__VCAP_NM_PROFILES__ -$firewall_nm_profiles -__VCAP_NM_PROFILES__ - if test "$firewall_nm_inventory_valid" != yes || - test "$firewall_nm_file_count" != 1 || test -z "$firewall_nm_file"; then - printf "%s|?\n" "$firewall_nm_iface" - continue - fi - case "$firewall_nm_file" in - /etc/NetworkManager/system-connections/*.nmconnection) : ;; - *) printf "%s|?\n" "$firewall_nm_iface"; continue ;; - esac - if test ! -f "$firewall_nm_file" || test -L "$firewall_nm_file" || - test "$($firewall_stat -c %u:%a "$firewall_nm_file" 2>/dev/null)" != 0:600; then - printf "%s|?\n" "$firewall_nm_iface" - continue - fi - firewall_nm_size=$($firewall_stat -c %s "$firewall_nm_file" 2>/dev/null) - case "$firewall_nm_size" in - ""|*[!0-9]*) printf "%s|?\n" "$firewall_nm_iface"; continue ;; - esac - if test "$firewall_nm_size" -gt 65536; then - printf "%s|?\n" "$firewall_nm_iface" - continue - fi - firewall_nm_section= - firewall_nm_zone= - firewall_nm_zone_count=0 - firewall_nm_line_count=0 - firewall_nm_file_valid=yes - while IFS= read -r firewall_nm_line || test -n "$firewall_nm_line"; do - firewall_nm_line_count=$((firewall_nm_line_count + 1)) - if test "$firewall_nm_line_count" -gt 4096; then firewall_nm_file_valid=no; break; fi - case "$firewall_nm_line" in - "[connection]") firewall_nm_section=connection ;; - \[*\]) firewall_nm_section=other ;; - zone=*) - if test "$firewall_nm_section" = connection; then - firewall_nm_zone_count=$((firewall_nm_zone_count + 1)) - firewall_nm_zone=${firewall_nm_line#zone=} - fi - ;; - esac - done < "$firewall_nm_file" - if test "$firewall_nm_file_valid" != yes || test "$firewall_nm_zone_count" -gt 1; then - printf "%s|?\n" "$firewall_nm_iface" - continue - fi - if test -n "$firewall_nm_zone"; then - case "$firewall_nm_zone" in - *[!A-Za-z0-9_-]*) printf "%s|?\n" "$firewall_nm_iface"; continue ;; - esac - if test "${#firewall_nm_zone}" -gt 64; then - printf "%s|?\n" "$firewall_nm_iface" - continue - fi - fi - printf "%s|%s\n" "$firewall_nm_iface" "$firewall_nm_zone" - done <<__VCAP_NM_ACTIVE__ -$firewall_nm_active -__VCAP_NM_ACTIVE__ +fabric_desired_state_legacy=absent +fabric_desired_state=/etc/virt-cloud/wg-fabric0.conf +if test -e "$fabric_desired_state" || test -L "$fabric_desired_state"; then + fabric_desired_state_legacy=unknown + if test -n "$legacy_stat" && test -n "$legacy_awk" && + test -f "$fabric_desired_state" && ! test -L "$fabric_desired_state" && + test "$("$legacy_stat" -c "%u:%g:%a" "$fabric_desired_state" 2>/dev/null)" = 0:0:600; then + fabric_desired_size=$("$legacy_stat" -c "%s" "$fabric_desired_state" 2>/dev/null) + case "$fabric_desired_size" in + ""|*[!0-9]*) ;; + *) + if test "$fabric_desired_size" -le 65536 2>/dev/null; then + fabric_desired_state_legacy=$("$legacy_awk" -F"|" " + length(\$0)>4096 {unsafe=1} + \$1==\"firewall\" {firewall++; if (NF!=2 || \$2!=\"native_nftables\") legacy=1; next} + \$1==\"underlay\" || \$1==\"firewall_rule\" {legacy=1; next} + END { + if (legacy) print \"present\" + else if (unsafe || firewall>1) print \"unknown\" + else print \"absent\" + } + " "$fabric_desired_state" 2>/dev/null) || fabric_desired_state_legacy=unknown + fi ;; + esac fi fi -printf "__VCAP_NM_ZONE_END__=%s\n" "$firewall_nm_status" - -# ifupdown hosts must not have a competing RHEL ifcfg zone owner for the -# endpoint interface. Only validated interface names are projected. -printf "%s\n" "__VCAP_IFCFG_ZONE_BEGIN__" -firewall_ifcfg_status=0 -firewall_ifcfg_count=0 -for firewall_ifcfg in /etc/sysconfig/network-scripts/ifcfg-*; do - if test ! -e "$firewall_ifcfg" && test ! -L "$firewall_ifcfg"; then continue; fi - firewall_ifcfg_iface=${firewall_ifcfg##*/ifcfg-} - case "$firewall_ifcfg_iface" in ""|*[!A-Za-z0-9_.:-]*) firewall_ifcfg_status=1; break ;; esac - if test "${#firewall_ifcfg_iface}" -gt 15; then firewall_ifcfg_status=1; break; fi - firewall_ifcfg_count=$((firewall_ifcfg_count + 1)) - if test "$firewall_ifcfg_count" -gt 256; then firewall_ifcfg_status=1; break; fi - printf "%s\n" "$firewall_ifcfg_iface" -done -printf "__VCAP_IFCFG_ZONE_END__=%s\n" "$firewall_ifcfg_status" -firewall_runtime_nft=unknown -firewall_nft= -firewall_dd= -firewall_wc= -firewall_tr_runtime= -firewall_grep_runtime= -for firewall_candidate in /usr/sbin/nft /usr/bin/nft /sbin/nft /bin/nft; do - if test -x "$firewall_candidate"; then firewall_nft="$firewall_candidate"; break; fi -done -for firewall_candidate in /usr/bin/dd /bin/dd /usr/sbin/dd /sbin/dd; do - if test -x "$firewall_candidate"; then firewall_dd="$firewall_candidate"; break; fi -done -for firewall_candidate in /usr/bin/wc /bin/wc /usr/sbin/wc /sbin/wc; do - if test -x "$firewall_candidate"; then firewall_wc="$firewall_candidate"; break; fi -done -for firewall_candidate in /usr/bin/tr /bin/tr /usr/sbin/tr /sbin/tr; do - if test -x "$firewall_candidate"; then firewall_tr_runtime="$firewall_candidate"; break; fi -done -for firewall_candidate in /usr/bin/grep /bin/grep /usr/sbin/grep /sbin/grep; do - if test -x "$firewall_candidate"; then firewall_grep_runtime="$firewall_candidate"; break; fi -done -if test "$firewall_service_state" = active && - test -n "$firewall_cmd" && test -n "$firewall_nft" && test -n "$firewall_dd" && - test -n "$firewall_wc" && test -n "$firewall_tr_runtime" && test -n "$firewall_grep_runtime" && - test "$($firewall_cmd --state 2>/dev/null)" = running; then - firewall_json=$($firewall_nft -j list table inet firewalld 2>/dev/null | $firewall_dd bs=262145 count=1 2>/dev/null) - firewall_json_size=$(printf "%s" "$firewall_json" | $firewall_wc -c 2>/dev/null || printf 262145) - firewall_json_size=$(printf "%s" "$firewall_json_size" | $firewall_tr_runtime -d "[:space:]" 2>/dev/null || printf 262145) - if test "$firewall_json_size" -gt 0 2>/dev/null && test "$firewall_json_size" -le 262144 2>/dev/null; then - firewall_compact=$(printf "%s" "$firewall_json" | $firewall_tr_runtime -d "[:space:]" 2>/dev/null) - firewall_table=$(printf "%s\n" "$firewall_compact" | $firewall_grep_runtime -Eo "\"table\":\\{[^{{}}]*\\}" 2>/dev/null | $firewall_grep_runtime -E "\"family\":\"inet\"" 2>/dev/null | $firewall_grep_runtime -E "\"name\":\"firewalld\"" 2>/dev/null) - firewall_table_count=$(printf "%s\n" "$firewall_table" | $firewall_grep_runtime -c "\"table\"" 2>/dev/null || true) - if test "$firewall_table_count" = 1 && - printf "%s\n" "$firewall_table" | $firewall_grep_runtime -Eq "\"flags\":\\[[^]]*\"owner\"" 2>/dev/null && - printf "%s\n" "$firewall_table" | $firewall_grep_runtime -Eq "\"flags\":\\[[^]]*\"persist\"" 2>/dev/null; then - firewall_runtime_nft=ready - fi +printf "__VCAP_LEGACY_FIREWALL__fabric_desired_state=%s\n" \ + "$fabric_desired_state_legacy" +vpc_state_legacy=absent +vpc_state_count=0 +for vpc_state_file in /etc/virt-cloud/vpc-dataplane/*.state; do + test -e "$vpc_state_file" || test -L "$vpc_state_file" || continue + vpc_state_count=$((vpc_state_count + 1)) + if test "$vpc_state_count" -gt 256 || test -z "$legacy_stat" || + test -z "$legacy_awk" || ! test -f "$vpc_state_file" || + test -L "$vpc_state_file" || + ! test "$("$legacy_stat" -c "%u:%g:%a" "$vpc_state_file" 2>/dev/null)" = 0:0:600; then + vpc_state_legacy=unknown + break fi -fi -printf "__VCAP_FIREWALL_CONF__runtime_nft=%s\n" "$firewall_runtime_nft" + vpc_state_size=$("$legacy_stat" -c "%s" "$vpc_state_file" 2>/dev/null) + case "$vpc_state_size" in + ""|*[!0-9]*) vpc_state_legacy=unknown; break ;; + esac + if ! test "$vpc_state_size" -le 65536 2>/dev/null; then + vpc_state_legacy=unknown + break + fi + vpc_state_result=$("$legacy_awk" -F"|" " + length(\$0)>4096 {unsafe=1} + \$1==\"firewall_mode\" {firewall++; if (NF!=2 || \$2!=\"native_nftables\") legacy=1; next} + \$1==\"nft_snapshot\" {legacy=1; next} + END { + if (legacy) print \"present\" + else if (unsafe || firewall>1) print \"unknown\" + else print \"absent\" + } + " "$vpc_state_file" 2>/dev/null) || vpc_state_result=unknown + case "$vpc_state_result" in + present) vpc_state_legacy=present; break ;; + absent) : ;; + *) vpc_state_legacy=unknown; break ;; + esac +done +printf "__VCAP_LEGACY_FIREWALL__vpc_state=%s\n" "$vpc_state_legacy" # ifupdown ownership is represented by bounded facts only. The contents of # interfaces files can include addresses, credentials, and arbitrary hooks; @@ -1189,23 +956,6 @@ def _capability_section(output: str, begin: str, end: str) -> str | None: return output[start:finish] -def _capability_status_section( - output: str, - begin: str, - end_prefix: str, -) -> tuple[str | None, str | None]: - """Return one bounded section followed by its exact status marker.""" - if output.count(begin) != 1: - return None, None - start = output.find(begin) + len(begin) - finish = output.find(end_prefix, start) - if (finish < 0 or finish - start > _MAX_CAPABILITY_OUTPUT - or output.find(end_prefix, finish + len(end_prefix)) >= 0): - return None, None - status_line = output[finish:].splitlines()[0] - return output[start:finish], status_line[len(end_prefix):].strip() - - def _capability_lines(output: str, prefix: str) -> dict[str, str]: values: dict[str, str] = {} for line in output.splitlines(): @@ -1361,76 +1111,6 @@ def _parse_local_ipv4_addresses( ) -def _parse_firewall_interface_zones( - section: str | None, - status: str | None, -) -> tuple[bool, dict[str, str]]: - """Parse a bounded ``interface|zone`` inventory without partial use.""" - if section is None or status != "0": - return False, {} - values: dict[str, str] = {} - lines = section.splitlines() - if len(lines) > 4096: - return False, {} - for line in lines: - interface, separator, zone = line.partition("|") - if (not separator or "|" in zone - or _IFACE_NAME.fullmatch(interface) is None - or _FIREWALL_ZONE.fullmatch(zone) is None - or interface in values): - return False, {} - values[interface] = zone - return True, values - - -def _parse_networkmanager_interface_zones( - section: str | None, - status: str | None, -) -> tuple[bool, dict[str, str]]: - """Parse active interface to persistent NM zone (blank means default). - - ``?`` is a per-interface fail-closed sentinel for an active in-memory, - external, ambiguous, or otherwise unverifiable profile. It must not make - an unrelated, fully verified endpoint unusable. - """ - if section is None or status != "0": - return False, {} - values: dict[str, str] = {} - lines = section.splitlines() - if len(lines) > 4096: - return False, {} - for line in lines: - interface, separator, zone = line.partition("|") - if (not separator or "|" in zone - or _IFACE_NAME.fullmatch(interface) is None - or (zone not in {"", "?"} - and _FIREWALL_ZONE.fullmatch(zone) is None)): - return False, {} - if interface in values: - # Multiple active profiles for one device are ambiguous even when - # they currently report the same zone. - values[interface] = "?" - else: - values[interface] = zone - return True, values - - -def _parse_ifcfg_interfaces( - section: str | None, - status: str | None, -) -> tuple[bool, list[str]]: - if section is None or status != "0": - return False, [] - lines = section.splitlines() - if len(lines) > 256: - return False, [] - if any(_IFACE_NAME.fullmatch(line) is None for line in lines): - return False, [] - if len(set(lines)) != len(lines): - return False, [] - return True, sorted(lines) - - def _parse_integer(value: str | None) -> int | None: if value is None or not re.fullmatch(r"[0-9]+", value.strip()): return None @@ -1440,18 +1120,46 @@ def _parse_integer(value: str | None) -> int | None: return None -def _parse_nft_tables(section: str | None, status: str | None) -> bool: - """Confirm that the read-only nft probe returned valid JSON.""" +def _parse_nft_tables( + section: str | None, + status: str | None, +) -> tuple[bool, set[tuple[str, str]]]: + """Parse the bounded native nft table inventory. + + Table names are retained only to reject stale firewalld ownership and + retired emergency tables. The platform never reads or edits them. + """ if section is None or status != "0": - return False + return False, set() try: payload = json.loads(section.strip()) except (TypeError, ValueError): - return False - # ``nft -j list tables`` returns an object in newer nft versions and a - # list in a few vendor builds. Both are evidence that the command ran; - # arbitrary text is not. - return isinstance(payload, (dict, list)) and len(section) <= _MAX_CAPABILITY_OUTPUT + return False, set() + if len(section) > _MAX_CAPABILITY_OUTPUT: + return False, set() + if isinstance(payload, dict): + records = payload.get("nftables") + elif isinstance(payload, list): + records = payload + else: + return False, set() + if not isinstance(records, list): + return False, set() + tables: set[tuple[str, str]] = set() + for record in records: + if not isinstance(record, dict): + return False, set() + table = record.get("table") + if table is None: + continue + if not isinstance(table, dict): + return False, set() + family = table.get("family") + name = table.get("name") + if not isinstance(family, str) or not isinstance(name, str): + return False, set() + tables.add((family, name)) + return True, tables def parse_host_capabilities( @@ -1479,20 +1187,15 @@ def parse_host_capabilities( "__VCAP_IP_BEGIN__bridges\n", "__VCAP_IP_END__bridges=", "__VCAP_IP_BEGIN__wireguard\n", "__VCAP_IP_END__wireguard=", "__VCAP_NFT_BEGIN__\n", "__VCAP_NFT_END__", - "__VCAP_FIREWALL_CONF__backend=", - "__VCAP_FIREWALL_CONF__table_owner=", - "__VCAP_FIREWALL_CONF__source_zones=", - "__VCAP_FIREWALL_CONF__default_runtime=", - "__VCAP_FIREWALL_CONF__default_permanent=", - "__VCAP_FIREWALL_CONF__runtime_nft=", - "__VCAP_FIREWALL_IFACE_BEGIN__runtime\n", - "__VCAP_FIREWALL_IFACE_END__runtime=", - "__VCAP_FIREWALL_IFACE_BEGIN__permanent\n", - "__VCAP_FIREWALL_IFACE_END__permanent=", - "__VCAP_NM_ZONE_BEGIN__\n", "__VCAP_NM_ZONE_END__=", - "__VCAP_IFCFG_ZONE_BEGIN__\n", "__VCAP_IFCFG_ZONE_END__=", "__VCAP_SERVICE_LOAD__firewall=", "__VCAP_SERVICE_LOAD__nftables=", + "__VCAP_LEGACY_FIREWALL__manifest=", + "__VCAP_LEGACY_FIREWALL__fabric_transaction=", + "__VCAP_LEGACY_FIREWALL__emergency_owner=", + "__VCAP_LEGACY_FIREWALL__vpc_transaction=", + "__VCAP_LEGACY_FIREWALL__vpc_snapshot=", + "__VCAP_LEGACY_FIREWALL__fabric_desired_state=", + "__VCAP_LEGACY_FIREWALL__vpc_state=", "__VCAP_END__", ) protocol_ok = all(section in output for section in required_sections) @@ -1507,7 +1210,8 @@ def parse_host_capabilities( services = _capability_lines(output, "__VCAP_SERVICE__") service_enabled = _capability_lines(output, "__VCAP_SERVICE_ENABLED__") service_load = _capability_lines(output, "__VCAP_SERVICE_LOAD__") - firewall_conf = _capability_lines(output, "__VCAP_FIREWALL_CONF__") + legacy_firewall = _capability_lines( + output, "__VCAP_LEGACY_FIREWALL__") files = _capability_lines(output, "__VCAP_FILE__") rp_filter_interfaces = { name: parsed for name, raw in _capability_named_lines( @@ -1551,30 +1255,6 @@ def parse_host_capabilities( _parse_local_ipv4_addresses( ip_sections.get("addresses"), ip_statuses.get("addresses")) ) - firewall_runtime_section, firewall_runtime_status = ( - _capability_status_section( - output, "__VCAP_FIREWALL_IFACE_BEGIN__runtime\n", - "__VCAP_FIREWALL_IFACE_END__runtime=")) - firewall_permanent_section, firewall_permanent_status = ( - _capability_status_section( - output, "__VCAP_FIREWALL_IFACE_BEGIN__permanent\n", - "__VCAP_FIREWALL_IFACE_END__permanent=")) - firewall_runtime_zones_valid, firewall_runtime_interface_zones = ( - _parse_firewall_interface_zones( - firewall_runtime_section, firewall_runtime_status)) - firewall_permanent_zones_valid, firewall_permanent_interface_zones = ( - _parse_firewall_interface_zones( - firewall_permanent_section, firewall_permanent_status)) - nm_zone_section, nm_zone_status = _capability_status_section( - output, "__VCAP_NM_ZONE_BEGIN__\n", "__VCAP_NM_ZONE_END__=") - nm_zones_valid, nm_persistent_interface_zones = ( - _parse_networkmanager_interface_zones( - nm_zone_section, nm_zone_status)) - ifcfg_section, ifcfg_status = _capability_status_section( - output, "__VCAP_IFCFG_ZONE_BEGIN__\n", - "__VCAP_IFCFG_ZONE_END__=") - ifcfg_interfaces_valid, ifcfg_zone_interfaces = _parse_ifcfg_interfaces( - ifcfg_section, ifcfg_status) ip_probe_valid = bool( _valid_json_interfaces( ip_sections.get("links"), ip_statuses.get("links")) @@ -1596,7 +1276,7 @@ def parse_host_capabilities( for name in ( "wg", "wg_quick", "ip", "nft", "nmcli", "networkctl", "ifup", "ifdown", "ifquery", "ifreload", "flock", "systemctl", - "modinfo", "dnf", "apt_get", "firewall_cmd", + "modinfo", "dnf", "apt_get", ) } os_id = os_values.get("ID", "").lower() @@ -1729,28 +1409,6 @@ def parse_host_capabilities( or (services.get("firewall") == "inactive" and service_enabled.get("firewall") == "masked") ) - # nftables.service has no D-Bus activation path. On a firewalld host an - # inactive disabled (or masked/absent) unit follows the documented - # single-manager setup and cannot spontaneously take ownership. - nftables_cannot_take_over = bool( - nftables_load == "not-found" - or (services.get("nftables") == "inactive" - and service_enabled.get("nftables") in {"disabled", "masked"}) - ) - firewalld_backend = firewall_conf.get("backend", "unknown") - firewalld_table_owner = firewall_conf.get("table_owner", "unknown") - firewalld_source_zones = firewall_conf.get("source_zones", "unknown") - firewalld_default_runtime = firewall_conf.get("default_runtime", "unknown") - firewalld_default_permanent = firewall_conf.get( - "default_permanent", "unknown") - firewalld_runtime_nft = firewall_conf.get("runtime_nft", "unknown") - firewalld_interface_inventory_ready = bool( - firewall_runtime_zones_valid and firewall_permanent_zones_valid - and ( - (network_manager == "networkmanager" and nm_zones_valid) - or (network_manager == "ifupdown" and ifcfg_interfaces_valid) - ) - ) firewall_managers = { # An enabled-but-currently-inactive service can take ownership after # a reboot. Treat it as ownership evidence so Fabric never installs @@ -1764,9 +1422,7 @@ def parse_host_capabilities( elif len(active_firewall_managers) > 1: firewall_manager = "conflict" elif (services.get("firewall") == "unknown" - and services.get("nftables") == "unknown" - and not firewalld_cannot_take_over - and not nftables_cannot_take_over): + and services.get("nftables") == "unknown"): firewall_manager = "unknown" else: firewall_manager = "unmanaged" @@ -1798,7 +1454,59 @@ def parse_host_capabilities( # into a safe fabric interface. fabric_interface_present = WG_FABRIC_INTERFACE in all_interfaces fabric_interface_is_wireguard = WG_FABRIC_INTERFACE in wireguard_names - nft_readable = _parse_nft_tables(nft_section, nft_status) + nft_readable, nft_tables = _parse_nft_tables(nft_section, nft_status) + legacy_firewalld_table_present = bool( + ("inet", "firewalld") in nft_tables + or ("inet", "virt_cloud_fabric_emergency") in nft_tables + or ("bridge", "virt_cloud_fabric_emergency") in nft_tables + ) + legacy_manifest_state = legacy_firewall.get("manifest", "unknown") + fabric_transaction_state = legacy_firewall.get( + "fabric_transaction", "unknown") + emergency_owner_state = legacy_firewall.get( + "emergency_owner", "unknown") + vpc_transaction_state = legacy_firewall.get( + "vpc_transaction", "unknown") + vpc_snapshot_state = legacy_firewall.get("vpc_snapshot", "unknown") + fabric_desired_state = legacy_firewall.get( + "fabric_desired_state", "unknown") + vpc_legacy_state = legacy_firewall.get("vpc_state", "unknown") + legacy_firewall_evidence_valid = all( + state in {"absent", "present"} + for state in ( + legacy_manifest_state, + fabric_transaction_state, + emergency_owner_state, + vpc_transaction_state, + vpc_snapshot_state, + fabric_desired_state, + vpc_legacy_state, + ) + ) and all( + sum( + line.startswith( + f"__VCAP_LEGACY_FIREWALL__{name}=") + for line in output.splitlines() + ) == 1 + for name in ( + "manifest", "fabric_transaction", "emergency_owner", + "vpc_transaction", + "vpc_snapshot", + "fabric_desired_state", "vpc_state", + ) + ) + protocol_ok = protocol_ok and legacy_firewall_evidence_valid + legacy_manifest_present = legacy_manifest_state == "present" + legacy_vpc_snapshot_present = vpc_snapshot_state == "present" + legacy_emergency_owner_present = emergency_owner_state == "present" + legacy_desired_state_present = bool( + fabric_desired_state == "present" + or vpc_legacy_state == "present" + ) + firewall_transaction_present = bool( + fabric_transaction_state == "present" + or vpc_transaction_state == "present" + ) # This is only a capability result: a later reconciler still has to create # and own a dedicated private ACL table, then read it back before marking # a VPC revision active. An unknown or unmanaged firewall is rejected so @@ -1811,31 +1519,17 @@ def parse_host_capabilities( and nftables_enabled and nftables_load == "loaded" and firewalld_cannot_take_over + and not legacy_firewalld_table_present + and legacy_firewall_evidence_valid + and not legacy_manifest_present + and not legacy_emergency_owner_present + and not legacy_vpc_snapshot_present + and not legacy_desired_state_present + and not firewall_transaction_present ) - firewalld_nft_ready = bool( - tool_state["nft"]["available"] - and nft_readable - and firewall_manager == "firewalld" - and firewalld_active - and firewalld_enabled - and firewalld_load == "loaded" - and nftables_cannot_take_over - and tool_state["firewall_cmd"]["available"] - and firewalld_backend == "nftables" - and firewalld_table_owner == "yes" - and firewalld_source_zones == "absent" - and firewalld_default_runtime not in {"unknown", "trusted"} - and firewalld_default_runtime == firewalld_default_permanent - and firewalld_runtime_nft == "ready" - and firewalld_interface_inventory_ready - ) - if native_nftables_ready: - firewall_mode = "native_nftables" - elif firewalld_nft_ready: - firewall_mode = "firewalld_nft" - else: - firewall_mode = "unsupported" - nft_ready = native_nftables_ready or firewalld_nft_ready + firewall_mode = ( + "native_nftables" if native_nftables_ready else "unsupported") + nft_ready = native_nftables_ready forwarding_ready = ip_forward == 1 selector = public_network if isinstance(public_network, str) else "" if selector.startswith("bridge:"): @@ -1922,47 +1616,29 @@ def parse_host_capabilities( elif not nftables_enabled: reasons.append("nftables_service_not_enabled") if not firewalld_cannot_take_over: - reasons.append("firewall_competing_service_unknown") + reasons.append("firewalld_unsupported_requires_maintenance") elif firewall_manager == "firewalld": - if not firewalld_active: - reasons.append("firewalld_service_not_active") - if not firewalld_enabled: - reasons.append("firewalld_service_not_enabled") - if not tool_state["firewall_cmd"]["available"]: - reasons.append("firewalld_command_missing") - if firewalld_backend != "nftables": - reasons.append("firewalld_backend_not_nftables") - if firewalld_table_owner != "yes": - reasons.append("firewalld_table_owner_not_enabled") - if firewalld_source_zones != "absent": - reasons.append("firewalld_source_zones_present_or_unknown") - if (firewalld_default_runtime in {"unknown", "trusted"} - or firewalld_default_runtime != firewalld_default_permanent): - reasons.append("firewalld_default_zone_unsafe_or_drifted") - if firewalld_runtime_nft != "ready": - reasons.append("firewalld_runtime_nft_backend_unverified") - if not firewalld_interface_inventory_ready: - reasons.append("firewalld_endpoint_zone_unverified") - if not nftables_cannot_take_over: - reasons.append("firewall_manager_conflict") + reasons.append("firewalld_unsupported_requires_maintenance") elif firewall_manager == "conflict": reasons.append("firewall_manager_conflict") + reasons.append("firewalld_unsupported_requires_maintenance") else: reasons.append("firewall_owner_unsupported") - if firewall_mode == "unsupported" and not any( - reason in reasons for reason in ( - "nftables_service_not_active", "nftables_service_not_enabled", - "firewalld_service_not_active", "firewalld_service_not_enabled", - "firewalld_command_missing", "firewalld_backend_not_nftables", - "firewalld_table_owner_not_enabled", "firewall_manager_conflict", - "firewalld_source_zones_present_or_unknown", - "firewalld_default_zone_unsafe_or_drifted", - "firewalld_runtime_nft_backend_unverified", - "firewalld_endpoint_zone_unverified", - "firewall_competing_service_unknown", - "firewall_owner_unsupported", - ) - ): + if legacy_firewalld_table_present: + reasons.append("legacy_firewalld_state_requires_cleanup") + if legacy_manifest_present: + reasons.append("legacy_firewalld_state_requires_cleanup") + if legacy_emergency_owner_present: + reasons.append("legacy_firewalld_state_requires_cleanup") + if legacy_vpc_snapshot_present: + reasons.append("legacy_firewalld_state_requires_cleanup") + if legacy_desired_state_present: + reasons.append("legacy_firewalld_state_requires_cleanup") + if firewall_transaction_present: + reasons.append("firewall_transaction_requires_recovery_or_cleanup") + if not legacy_firewall_evidence_valid: + reasons.append("legacy_firewall_evidence_invalid") + if firewall_mode == "unsupported": reasons.append("firewall_mode_unsupported") if firewall_mode == "unsupported" and firewall_manager != "nftables": # Preserve the established coarse reason for older callers while the @@ -2029,14 +1705,11 @@ def parse_host_capabilities( "wireguard_kernel_unavailable", "iproute2_missing", "nftables_tool_missing", "nftables_probe_failed", "firewall_owner_not_nftables", "firewall_owner_unsupported", "firewall_mode_unsupported", - "firewall_manager_conflict", "firewalld_service_not_active", - "firewalld_service_not_enabled", "firewalld_command_missing", - "firewalld_backend_not_nftables", - "firewalld_table_owner_not_enabled", - "firewalld_source_zones_present_or_unknown", - "firewalld_runtime_nft_backend_unverified", - "firewalld_endpoint_zone_unverified", - "firewall_competing_service_unknown", + "firewall_manager_conflict", + "firewalld_unsupported_requires_maintenance", + "legacy_firewalld_state_requires_cleanup", + "firewall_transaction_requires_recovery_or_cleanup", + "legacy_firewall_evidence_invalid", "nftables_service_not_active", "nftables_service_not_enabled", "iproute2_probe_failed", "flock_missing", "ipv4_forwarding_disabled_or_unknown", "rp_filter_not_compatible", @@ -2044,7 +1717,7 @@ def parse_host_capabilities( "network_manager_unknown", "network_manager_policy_mismatch", "systemd_networkd_not_supported", "networkmanager_nmcli_missing", "networkmanager_service_not_active", "networkmanager_service_not_enabled", - "ifupdown_tools_missing", "ifupdown_configuration_missing", + "ifupdown_tools_missing", "ifupdown_configuration_missing", "ifupdown_interfaces_d_not_included", "ifupdown_service_not_active_or_enabled", )) @@ -2103,16 +1776,15 @@ def parse_host_capabilities( "firewalld_active": firewalld_active, "firewalld_enabled": firewalld_enabled, "firewalld_load_state": firewalld_load, - "firewalld_backend": firewalld_backend, - "firewalld_table_owner": firewalld_table_owner, - "firewalld_source_zones": firewalld_source_zones, - "firewalld_default_runtime": firewalld_default_runtime, - "firewalld_default_permanent": firewalld_default_permanent, - "firewalld_runtime_nft": firewalld_runtime_nft, - "firewalld_interface_inventory_ready": - firewalld_interface_inventory_ready, - "firewalld_command_available": bool( - tool_state["firewall_cmd"]["available"]), + "legacy_firewalld_table_present": + legacy_firewalld_table_present, + "legacy_manifest_state": legacy_manifest_state, + "fabric_transaction_state": fabric_transaction_state, + "emergency_owner_state": emergency_owner_state, + "vpc_transaction_state": vpc_transaction_state, + "vpc_snapshot_state": vpc_snapshot_state, + "fabric_desired_state": fabric_desired_state, + "vpc_state": vpc_legacy_state, "nftables_active": nftables_active, "nftables_enabled": nftables_enabled, "nftables_load_state": nftables_load, @@ -2121,17 +1793,7 @@ def parse_host_capabilities( "bridges": bridge_names, "interfaces": all_interfaces, "local_ipv4_addresses": local_ipv4_addresses, - # These maps are retained for the coordinator's endpoint-specific - # readiness check. The public projection exposes only the one - # selected interface and its normalized effective zones. "local_ipv4_interfaces": local_ipv4_interfaces, - "firewalld_runtime_interface_zones": - firewall_runtime_interface_zones, - "firewalld_permanent_interface_zones": - firewall_permanent_interface_zones, - "networkmanager_persistent_interface_zones": - nm_persistent_interface_zones, - "ifcfg_zone_interfaces": ifcfg_zone_interfaces, "wireguard_interfaces": wireguard_names, "ip_forward": ip_forward, # ``rp_filter`` is the established name for ``conf/all``. Keep it diff --git a/engine/virsh_panel/vpc_dataplane.py b/engine/virsh_panel/vpc_dataplane.py index cfe4e82e..607437db 100644 --- a/engine/virsh_panel/vpc_dataplane.py +++ b/engine/virsh_panel/vpc_dataplane.py @@ -37,11 +37,9 @@ DATAPLANE_OWNER_MARKER = "virt-cloud-vpc-dataplane-v1" FABRIC_OWNER_MARKER = "virt-cloud-wg-fabric-v1" FABRIC_INTERFACE = "wg-fabric0" FABRIC_LOCK = "/run/virt-cloud/wg-fabric0.lock" -NFT_MANIFEST = "/etc/virt-cloud/nftables.conf" -NFT_MANIFEST_OWNER = "# virt-cloud-nft-manifest-v1" +LEGACY_FIREWALLD_NFT_MANIFEST = "/etc/virt-cloud/nftables.conf" FIREWALL_MODE_NATIVE = "native_nftables" -FIREWALL_MODE_FIREWALLD = "firewalld_nft" -FIREWALL_MODES = frozenset({FIREWALL_MODE_NATIVE, FIREWALL_MODE_FIREWALLD}) +FIREWALL_MODES = frozenset({FIREWALL_MODE_NATIVE}) PRIVATE_INTERFACE = PRIVATE_INTERFACE_ROLE PANEL_NS = "urn:virsh-panel:1" BRIDGE_PREFIX = "vcv-" @@ -55,6 +53,142 @@ _MAC = re.compile(r"[0-9a-fA-F]{2}(?::[0-9a-fA-F]{2}){5}\Z") _KEY = re.compile(r"[A-Za-z0-9+/]{42}==\Z|[A-Za-z0-9+/]{43}=\Z") _DIGEST = re.compile(r"[0-9a-f]{64}\Z") +VPC_NATIVE_TRANSACTION_GUARD_SH = r''' +vc_vpc_firewall_boundary_clean() { + [ -n "${NFT:-}" ] || return 1 + [ ! -e /etc/virt-cloud/nftables.conf ] && + [ ! -L /etc/virt-cloud/nftables.conf ] || return 1 + [ ! -e /etc/virt-cloud/.wg-fabric-txn ] && + [ ! -L /etc/virt-cloud/.wg-fabric-txn ] || return 1 + [ ! -e /run/virt-cloud/wg-fabric-emergency.owner ] && + [ ! -L /run/virt-cloud/wg-fabric-emergency.owner ] || return 1 + vc_vpc_fabric_config=/etc/virt-cloud/wg-fabric0.conf + if [ -e "$vc_vpc_fabric_config" ] || [ -L "$vc_vpc_fabric_config" ]; then + [ -f "$vc_vpc_fabric_config" ] && + [ ! -L "$vc_vpc_fabric_config" ] || return 1 + [ "$($STAT -c '%u:%g:%a' "$vc_vpc_fabric_config" 2>/dev/null)" = 0:0:600 ] || return 1 + vc_vpc_fabric_config_size=$($STAT -c '%s' "$vc_vpc_fabric_config" 2>/dev/null) || return 1 + case "$vc_vpc_fabric_config_size" in ''|*[!0-9]*) return 1 ;; esac + [ "$vc_vpc_fabric_config_size" -le 65536 ] || return 1 + awk -F'|' ' + length($0)>4096 {bad=1} + $1=="firewall" {firewall++; if (NF!=2 || $2!="native_nftables") bad=1; next} + $1=="underlay" || $1=="firewall_rule" {bad=1; next} + END {exit (bad || firewall>1) ? 1 : 0} + ' "$vc_vpc_fabric_config" || return 1 + fi + vc_vpc_transaction_count=0 + for vc_vpc_other_transaction in /etc/virt-cloud/.vpc-dataplane-*.txn; do + [ -e "$vc_vpc_other_transaction" ] || + [ -L "$vc_vpc_other_transaction" ] || continue + vc_vpc_transaction_count=$((vc_vpc_transaction_count + 1)) + [ "$vc_vpc_transaction_count" -le 256 ] || return 1 + [ "$vc_vpc_other_transaction" = "$TXN" ] && continue + return 1 + done + vc_vpc_legacy_snapshot_count=0 + for vc_vpc_legacy_snapshot in \ + /etc/virt-cloud/vpc-dataplane/*.nft.inet.live \ + /etc/virt-cloud/vpc-dataplane/*.nft.bridge.live \ + /etc/virt-cloud/vpc-dataplane/*.nft.arp.live; do + [ -e "$vc_vpc_legacy_snapshot" ] || + [ -L "$vc_vpc_legacy_snapshot" ] || continue + vc_vpc_legacy_snapshot_count=$((vc_vpc_legacy_snapshot_count + 1)) + [ "$vc_vpc_legacy_snapshot_count" -le 256 ] || return 1 + return 1 + done + vc_vpc_state_count=0 + for vc_vpc_state_file in /etc/virt-cloud/vpc-dataplane/*.state; do + [ -e "$vc_vpc_state_file" ] || [ -L "$vc_vpc_state_file" ] || continue + vc_vpc_state_count=$((vc_vpc_state_count + 1)) + [ "$vc_vpc_state_count" -le 256 ] || return 1 + [ -f "$vc_vpc_state_file" ] && [ ! -L "$vc_vpc_state_file" ] || return 1 + [ "$($STAT -c '%u:%g:%a' "$vc_vpc_state_file" 2>/dev/null)" = 0:0:600 ] || return 1 + vc_vpc_state_size=$($STAT -c '%s' "$vc_vpc_state_file" 2>/dev/null) || return 1 + case "$vc_vpc_state_size" in ''|*[!0-9]*) return 1 ;; esac + [ "$vc_vpc_state_size" -le 65536 ] || return 1 + awk -F'|' ' + $1=="firewall_mode" {firewall++; if (NF!=2 || $2!="native_nftables") bad=1; next} + $1=="nft_snapshot" {bad=1; next} + END {exit (bad || firewall>1) ? 1 : 0} + ' "$vc_vpc_state_file" || return 1 + done + vc_vpc_firewall_tables=$($NFT list tables 2>/dev/null) || return 1 + vc_vpc_firewall_tables_size=$(printf '%s' "$vc_vpc_firewall_tables" | + wc -c | tr -d ' ') || return 1 + case "$vc_vpc_firewall_tables_size" in ''|*[!0-9]*) return 1 ;; esac + [ "$vc_vpc_firewall_tables_size" -le 65536 ] || return 1 + ! printf '%s\n' "$vc_vpc_firewall_tables" | + grep -Eq 'table (inet firewalld|(inet|bridge) virt_cloud_fabric_emergency)([[:space:]]|$)' +} +vc_vpc_native_transaction_owned() { + [ -n "${TXN:-}" ] && [ -n "${STAT:-}" ] || return 1 + [ -d "$TXN" ] && [ ! -L "$TXN" ] || return 1 + [ "$($STAT -c '%u:%g:%a' "$TXN" 2>/dev/null)" = 0:0:700 ] || return 1 + [ -f "$TXN/firewall_mode" ] && [ ! -L "$TXN/firewall_mode" ] || return 1 + [ "$($STAT -c '%u:%g:%a' "$TXN/firewall_mode" 2>/dev/null)" = 0:0:600 ] || return 1 + vc_vpc_txn_mode_size=$($STAT -c '%s' "$TXN/firewall_mode" 2>/dev/null) || return 1 + case "$vc_vpc_txn_mode_size" in ''|*[!0-9]*) return 1 ;; esac + [ "$vc_vpc_txn_mode_size" -le 64 ] || return 1 + vc_vpc_txn_mode=$(awk ' + NR==1 && $0=="native_nftables" {ok=1} + {rows++} + END {if (ok && rows==1) print "native_nftables"; else exit 1} + ' "$TXN/firewall_mode") || return 1 + [ "$vc_vpc_txn_mode" = native_nftables ] || return 1 + for vc_vpc_legacy_txn_file in \ + "$TXN"/nft.persist.before.inet \ + "$TXN"/nft.persist.before.bridge \ + "$TXN"/nft.persist.before.arp \ + "$TXN"/reassert.nft \ + "$TXN"/nft.desired.verify.*; do + [ -e "$vc_vpc_legacy_txn_file" ] || [ -L "$vc_vpc_legacy_txn_file" ] || continue + return 1 + done + if [ -e "$TXN/state.before" ] || [ -L "$TXN/state.before" ]; then + [ -f "$TXN/state.before" ] && [ ! -L "$TXN/state.before" ] || return 1 + [ "$($STAT -c '%u:%g:%a' "$TXN/state.before" 2>/dev/null)" = 0:0:600 ] || return 1 + vc_vpc_txn_state_size=$($STAT -c '%s' "$TXN/state.before" 2>/dev/null) || return 1 + case "$vc_vpc_txn_state_size" in ''|*[!0-9]*) return 1 ;; esac + [ "$vc_vpc_txn_state_size" -le 65536 ] || return 1 + awk -F'|' ' + $1=="firewall_mode" {firewall++; if (NF!=2 || $2!="native_nftables") bad=1; next} + $1=="nft_snapshot" {bad=1; next} + END {exit (bad || firewall>1) ? 1 : 0} + ' "$TXN/state.before" || return 1 + fi + return 0 +} +vc_vpc_transaction_file_secure() { + vc_vpc_transaction_file=$1 + vc_vpc_transaction_file_limit=$2 + [ -f "$vc_vpc_transaction_file" ] && + [ ! -L "$vc_vpc_transaction_file" ] || return 1 + [ "$($STAT -c '%u:%g:%a' "$vc_vpc_transaction_file" 2>/dev/null)" = 0:0:600 ] || return 1 + vc_vpc_transaction_file_size=$($STAT -c '%s' "$vc_vpc_transaction_file" 2>/dev/null) || return 1 + case "$vc_vpc_transaction_file_size:$vc_vpc_transaction_file_limit" in + *[!0-9:]*) return 1 ;; + esac + [ "$vc_vpc_transaction_file_size" -le "$vc_vpc_transaction_file_limit" ] || return 1 +} +vc_vpc_transaction_record_owned() { + vc_vpc_transaction_record_file=$1 + vc_vpc_transaction_record_expected=$2 + vc_vpc_transaction_file_secure "$vc_vpc_transaction_record_file" 4096 || return 1 + awk -v wanted="$vc_vpc_transaction_record_expected" ' + {rows++; if (rows!=1 || $0!=wanted) bad=1} + END {exit (bad || rows!=1) ? 1 : 0} + ' "$vc_vpc_transaction_record_file" +} +vc_vpc_write_native_transaction_mode() { + [ -d "$TXN" ] && [ ! -L "$TXN" ] || return 1 + [ ! -e "$TXN/firewall_mode" ] && [ ! -L "$TXN/firewall_mode" ] || return 1 + printf '%s\n' native_nftables > "$TXN/firewall_mode" || return 1 + chmod 600 "$TXN/firewall_mode" || return 1 + vc_vpc_native_transaction_owned +} +'''.strip() + _NFT_INET_REQUIRED_SUFFIXES = ( "anti-spoof-local-ip", "anti-spoof-egress", "anti-spoof-public-to-bridge", "anti-spoof-public-to-fabric", @@ -85,33 +219,6 @@ _NFT_BRIDGE_REQUIRED_SUFFIXES = ( "bridge-input-non-ip-drop", ) -NFT_MANIFEST_VALIDATION_SH = r''' -vc_nft_manifest_validate() { - manifest=$1 - [ -f "$manifest" ] && [ ! -L "$manifest" ] || return 1 - [ "$($STAT -c '%u:%a' "$manifest" 2>/dev/null)" = 0:600 ] || return 1 - awk ' - NR == 1 { - if ($0 != "# virt-cloud-nft-manifest-v1") exit 1 - next - } - $0 == "include \"/etc/nftables.d/virt-cloud-wg-fabric.nft\"" { - if (++fabric != 1 || seen[$0]++) exit 1 - next - } - $0 ~ /^include "\/etc\/nftables[.]d\/virt-cloud-vpc-[0-9a-f]+[.]nft"$/ { - token=$0 - sub(/^include "\/etc\/nftables[.]d\/virt-cloud-vpc-/, "", token) - sub(/[.]nft"$/, "", token) - if (length(token) != 11 || token !~ /^[0-9a-f]+$/ || seen[$0]++) exit 1 - next - } - { exit 1 } - END { if (NR < 2 || fabric != 1) exit 1 } - ' "$manifest" -} -'''.strip() - NFT_LIVE_SNAPSHOT_SH = r''' vc_nft_table_absent() { vc_nft_absent_family=$1 @@ -195,102 +302,6 @@ vc_nft_table_matches_snapshot() { } '''.strip() -FIREWALL_ZONE_VALIDATION_SH = r''' -vc_firewalld_interface_zone() { - zone_scope=$1 - zone_iface=$2 - case "$zone_scope" in - runtime) zone_permanent= ;; - permanent) zone_permanent=--permanent ;; - *) return 1 ;; - esac - zone_names=$("$FIREWALL_CMD" $zone_permanent --get-zones 2>/dev/null) || return 1 - zone_found= - zone_count=0 - for zone_name in $zone_names; do - case "$zone_name" in ''|*[!A-Za-z0-9_-]*) return 1 ;; esac - [ "$(printf '%s' "$zone_name" | awk 'length($0)<=64')" = "$zone_name" ] || return 1 - zone_count=$((zone_count + 1)) - [ "$zone_count" -le 64 ] || return 1 - if "$FIREWALL_CMD" $zone_permanent --zone="$zone_name" \ - --query-interface="$zone_iface" >/dev/null 2>&1; then - [ -z "$zone_found" ] || return 1 - zone_found=$zone_name - else - zone_status=$? - [ "$zone_status" -eq 1 ] || return 1 - fi - done - [ "$zone_count" -gt 0 ] || return 1 - [ -n "$zone_found" ] || zone_found=none - printf '%s\n' "$zone_found" -} -'''.strip() - -FIREWALL_ZONE_OWNERSHIP_SH = r''' -vc_firewalld_default_zone() { - vc_default_zone=$("$FIREWALL_CMD" --get-default-zone 2>/dev/null) || return 1 - printf '%s\n' "$vc_default_zone" | grep -Eq '^[A-Za-z0-9_.-]+$' || return 1 - [ "$vc_default_zone" != trusted ] || return 1 - vc_default_zones=$("$FIREWALL_CMD" --get-zones 2>/dev/null) || return 1 - printf '%s\n' "$vc_default_zones" | tr ' ' '\n' | - grep -Fqx "$vc_default_zone" || return 1 - printf '%s\n' "$vc_default_zone" -} -vc_firewalld_bridge_zone_expect() { - vc_zone_expected=$1 - vc_zone_ownership=$2 - vc_zone_iface=$3 - VC_VPC_ZONE_RUNTIME=$(vc_firewalld_interface_zone runtime "$vc_zone_iface") || return 1 - VC_VPC_ZONE_PERMANENT=$(vc_firewalld_interface_zone permanent "$vc_zone_iface") || return 1 - VC_VPC_ZONE_DEFAULT=$(vc_firewalld_default_zone) || return 1 - [ "$VC_VPC_ZONE_PERMANENT" = none ] || return 1 - case "$vc_zone_expected" in - blocked) - [ "$VC_VPC_ZONE_RUNTIME" = none ] - ;; - nm-blocked) - [ "$VC_VPC_ZONE_RUNTIME" = none ] || - [ "$VC_VPC_ZONE_RUNTIME" = "$VC_VPC_ZONE_DEFAULT" ] - ;; - active) - [ "$VC_VPC_ZONE_RUNTIME" = trusted ] && - "$FIREWALL_CMD" --zone=trusted \ - --query-interface="$vc_zone_iface" >/dev/null 2>&1 - ;; - preapply) - case "$vc_zone_ownership" in - absent) - [ "$VC_VPC_ZONE_RUNTIME" = none ] - ;; - managed) - case "$VC_VPC_ZONE_RUNTIME" in - none|trusted) : ;; - *) return 1 ;; - esac - if [ "$VC_VPC_ZONE_RUNTIME" = trusted ]; then - "$FIREWALL_CMD" --zone=trusted \ - --query-interface="$vc_zone_iface" >/dev/null 2>&1 || return 1 - fi - ;; - managed-nm) - case "$VC_VPC_ZONE_RUNTIME" in - none|trusted) : ;; - *) [ "$VC_VPC_ZONE_RUNTIME" = "$VC_VPC_ZONE_DEFAULT" ] || return 1 ;; - esac - if [ "$VC_VPC_ZONE_RUNTIME" = trusted ]; then - "$FIREWALL_CMD" --zone=trusted \ - --query-interface="$vc_zone_iface" >/dev/null 2>&1 || return 1 - fi - ;; - *) return 1 ;; - esac - ;; - *) return 1 ;; - esac -} -'''.strip() - NETWORKMANAGER_VPC_PROFILE_SH = r''' vc_nm_profile_disk_validate() { vc_nm_disk_file=$1 @@ -441,25 +452,6 @@ vc_nm_link_expect_down() { printf '%s\n' "$vc_nm_link_flags" | grep -Eq '(^|<|,)UP(,|>)' && return 1 || true } -vc_nm_trusted_isolated_retry_ready() { - [ "$FIREWALL_MODE" = firewalld_nft ] || return 1 - vc_nm_profile_base_validate || return 1 - [ "$(vc_nm_profile_field connection.zone)" = trusted ] || return 1 - vc_nm_profile_inactive || return 1 - vc_nm_device_expect_managed no || return 1 - vc_nm_retry_device_raw=$( - "$NMCLI" -t --escape no -g GENERAL.CON-UUID \ - device show "$BRIDGE" 2>/dev/null - ) || return 1 - vc_nm_retry_device_uuid=$(printf '%s\n' "$vc_nm_retry_device_raw" | - sed -n '1p') || return 1 - [ -z "$vc_nm_retry_device_uuid" ] || - [ "$vc_nm_retry_device_uuid" = "$NM_PROFILE_UUID" ] || return 1 - "$IP" link show dev "$BRIDGE" >/dev/null 2>&1 || return 1 - vc_nm_link_expect_down || return 1 - [ "${BRIDGE_TOPOLOGY_CAPTURED:-no}" = yes ] || return 1 - vc_vpc_bridge_topology_matches -} vc_nm_device_quarantine() { "$IP" link show dev "$BRIDGE" >/dev/null 2>&1 || return 1 vc_nm_quarantine_uuid_raw=$( @@ -471,15 +463,8 @@ vc_nm_device_quarantine() { [ -z "$vc_nm_quarantine_uuid" ] || [ "$vc_nm_quarantine_uuid" = "$NM_PROFILE_UUID" ] || return 1 vc_nm_quarantine_zone=$(vc_nm_profile_field connection.zone) || return 1 - case "$vc_nm_quarantine_zone" in - ''|--) : ;; - trusted) - [ "$FIREWALL_MODE" = firewalld_nft ] || return 1 - "$NMCLI" connection modify --temporary uuid "$NM_PROFILE_UUID" \ - connection.zone "" >/dev/null 2>&1 || return 1 - ;; - *) return 1 ;; - esac + [ -z "$vc_nm_quarantine_zone" ] || [ "$vc_nm_quarantine_zone" = -- ] || + return 1 "$NMCLI" device set "$BRIDGE" managed no >/dev/null 2>&1 || return 1 vc_nm_device_expect_managed no || return 1 vc_nm_quarantine_zone=$(vc_nm_profile_field connection.zone) || return 1 @@ -516,34 +501,22 @@ vc_nm_profile_expect() { case "$vc_nm_expected" in blocked) [ -z "$vc_nm_memory_zone" ] || [ "$vc_nm_memory_zone" = -- ] || return 1 - vc_firewalld_bridge_zone_expect blocked managed "$BRIDGE" ;; blocked-active) [ -z "$vc_nm_memory_zone" ] || [ "$vc_nm_memory_zone" = -- ] || return 1 - vc_nm_profile_active || return 1 - vc_firewalld_bridge_zone_expect nm-blocked managed-nm "$BRIDGE" + vc_nm_profile_active ;; active) - [ "$vc_nm_memory_zone" = trusted ] || return 1 + [ -z "$vc_nm_memory_zone" ] || [ "$vc_nm_memory_zone" = -- ] || return 1 vc_nm_device_expect_managed yes || return 1 - vc_nm_profile_active || return 1 - vc_firewalld_bridge_zone_expect active managed "$BRIDGE" + vc_nm_profile_active ;; quarantined) [ -z "$vc_nm_memory_zone" ] || [ "$vc_nm_memory_zone" = -- ] || return 1 - vc_nm_device_expect_managed no || return 1 - vc_firewalld_bridge_zone_expect blocked managed "$BRIDGE" + vc_nm_device_expect_managed no ;; preapply) - vc_firewalld_bridge_zone_expect preapply managed-nm "$BRIDGE" || return 1 - if [ "$VC_VPC_ZONE_RUNTIME" = trusted ]; then - [ "$vc_nm_memory_zone" = trusted ] && vc_nm_profile_active - elif [ "$VC_VPC_ZONE_RUNTIME" = "$VC_VPC_ZONE_DEFAULT" ]; then - { [ -z "$vc_nm_memory_zone" ] || [ "$vc_nm_memory_zone" = -- ]; } && - vc_nm_profile_active - else - [ -z "$vc_nm_memory_zone" ] || [ "$vc_nm_memory_zone" = -- ] - fi + [ -z "$vc_nm_memory_zone" ] || [ "$vc_nm_memory_zone" = -- ] ;; *) return 1 ;; esac @@ -1346,13 +1319,6 @@ def _state( f"digest|{plan.desired_digest}", f"bridge|{r.bridge}", f"table|{r.route_table}", f"priority|{r.rule_priority}", f"firewall_mode|{firewall_mode}"] - if firewall_mode == FIREWALL_MODE_FIREWALLD: - marker = _nft_marker(plan) - for family in ("inet", "bridge", "arp"): - lines.append( - f"nft_snapshot|{family}|/etc/virt-cloud/vpc-dataplane/" - f"{r.token}.nft.{family}.live|{marker}" - ) for route in routes: lines.append("route|{destination}|{device}|{table}|{via}|{metric}".format( destination=route["destination_cidr"], device=route["device"], @@ -1618,64 +1584,6 @@ vc_vpc_nft_matches_snapshot() {{ "$vc_vpc_snapshot_prefix.$vc_vpc_snapshot_family" || return 1 done }} -vc_vpc_nft_snapshots_equal() {{ - vc_vpc_snapshot_left=$1 - vc_vpc_snapshot_right=$2 - for vc_vpc_snapshot_family in {nft_family_words}; do - "$CMP" -s "$vc_vpc_snapshot_left.$vc_vpc_snapshot_family" \ - "$vc_vpc_snapshot_right.$vc_vpc_snapshot_family" || return 1 - done -}} -vc_vpc_persisted_nft_tables_match() {{ - for vc_vpc_snapshot_family in {nft_family_words}; do - vc_vpc_snapshot_file="$NFT_SNAPSHOT_PREFIX.$vc_vpc_snapshot_family.live" - [ -f "$vc_vpc_snapshot_file" ] && [ ! -L "$vc_vpc_snapshot_file" ] || return 1 - [ "$($STAT -c '%u:%a' "$vc_vpc_snapshot_file" 2>/dev/null)" = 0:600 ] || return 1 - vc_nft_table_matches_snapshot "$vc_vpc_snapshot_family" "$TABLE" \ - "$vc_vpc_snapshot_file" || return 1 - done -}} -vc_vpc_persisted_nft_artifacts_ready() {{ - [ -f "$STATE_FILE" ] && [ ! -L "$STATE_FILE" ] || return 1 - for vc_vpc_snapshot_family in {nft_family_words}; do - vc_vpc_snapshot_file="$NFT_SNAPSHOT_PREFIX.$vc_vpc_snapshot_family.live" - [ -f "$vc_vpc_snapshot_file" ] && [ ! -L "$vc_vpc_snapshot_file" ] || return 1 - [ "$($STAT -c '%u:%a' "$vc_vpc_snapshot_file" 2>/dev/null)" = 0:600 ] || return 1 - vc_vpc_snapshot_record=$(awk -F'|' -v family="$vc_vpc_snapshot_family" \ - -v path="$vc_vpc_snapshot_file" ' - $1=="nft_snapshot" && $2==family && $3==path {{count++; marker=$4}} - END {{print (count + 0) "|" marker}} - ' "$STATE_FILE") || return 1 - [ "${{vc_vpc_snapshot_record%%|*}}" = 1 ] || return 1 - vc_vpc_snapshot_marker=${{vc_vpc_snapshot_record#*|}} - printf '%s\n' "$vc_vpc_snapshot_marker" | - grep -Eq '^{DATAPLANE_OWNER_MARKER}:{r.token}:[0-9a-f]{{16}}$' || return 1 - done -}} -vc_vpc_persisted_nft_ready() {{ - vc_vpc_persisted_nft_artifacts_ready || return 1 - vc_vpc_persisted_nft_tables_match -}} -vc_vpc_previous_nft_ready() {{ - vc_previous_inet=$("$NFT" list table inet "$TABLE" 2>/dev/null) || return 1 - printf '%s\\n' "$vc_previous_inet" | grep -Fq '{DATAPLANE_OWNER_MARKER}' || return 1 - printf '%s\\n' "$vc_previous_inet" | grep -Fq ':default-deny"' || return 1 - if [ -f "$TXN/nft.bridge.before" ]; then - vc_previous_bridge=$("$NFT" list table bridge "$TABLE" 2>/dev/null) || return 1 - printf '%s\\n' "$vc_previous_bridge" | grep -Fq '{DATAPLANE_OWNER_MARKER}' || return 1 - printf '%s\\n' "$vc_previous_bridge" | grep -Fq ':bridge-default-deny"' || return 1 - else - "$NFT" list table bridge "$TABLE" >/dev/null 2>&1 && return 1 || true - fi - if [ -f "$TXN/nft.arp.before" ]; then - vc_previous_arp=$("$NFT" list table arp "$TABLE" 2>/dev/null) || return 1 - printf '%s\\n' "$vc_previous_arp" | grep -Fq '{DATAPLANE_OWNER_MARKER}' || return 1 - printf '%s\\n' "$vc_previous_arp" | grep -Fq ':arp-proxy-output"' || return 1 - else - "$NFT" list table arp "$TABLE" >/dev/null 2>&1 && return 1 || true - fi - return 0 -}} vc_vpc_fabric_nft_shape_file() {{ vc_fabric_shape_file=$1 [ -f "$vc_fabric_shape_file" ] && [ ! -L "$vc_fabric_shape_file" ] || return 1 @@ -1713,29 +1621,26 @@ STAT=$(for p in /usr/bin/stat /bin/stat /usr/sbin/stat /sbin/stat; do [ -x "$p" CMP=$(for p in /usr/bin/cmp /bin/cmp /usr/sbin/cmp /sbin/cmp; do [ -x "$p" ] && {{ printf '%s' "$p"; break; }}; done) SYSCTL=$(for p in /usr/sbin/sysctl /usr/bin/sysctl /sbin/sysctl /bin/sysctl; do [ -x "$p" ] && {{ printf '%s' "$p"; break; }}; done) BRIDGE_CMD=$(for p in /usr/sbin/bridge /usr/bin/bridge /sbin/bridge /bin/bridge; do [ -x "$p" ] && {{ printf '%s' "$p"; break; }}; done) -FIREWALL_CMD=$(for p in /usr/bin/firewall-cmd /usr/sbin/firewall-cmd /bin/firewall-cmd /sbin/firewall-cmd; do [ -x "$p" ] && {{ printf '%s' "$p"; break; }}; done) SYSTEMCTL=$(for p in /usr/bin/systemctl /bin/systemctl /usr/sbin/systemctl /sbin/systemctl; do [ -x "$p" ] && {{ printf '%s' "$p"; break; }}; done) {IP_RULE_CONVERGENCE_SH} BRIDGE={_q(r.bridge)}; TABLE={_q(r.nft_table)}; ROUTE_TABLE={r.route_table}; RULE_PRIORITY={r.rule_priority}; WG_RULE_PRIORITY=$((RULE_PRIORITY + 1)) WG_IFACE={_q(FABRIC_INTERFACE)}; OWNER_MARKER={_q(DATAPLANE_OWNER_MARKER)}; FABRIC_OWNER={_q(FABRIC_OWNER_MARKER)} OWNER_FILE={_q(r.owner_file)}; STATE_FILE={_q(r.state_file)}; LOCK={_q(r.lock_file)}; TXN={_q(r.transaction_dir)}; PROXY_FILE={_q(proxy_file)} NFT_SNAPSHOT_PREFIX={_q(f'{state_dir}/{r.token}.nft')} -NFT_FILE={_q(include)}; NFT_INCLUDE={_q(include_line)}; NFT_MANIFEST={_q(NFT_MANIFEST)}; IF_FRAGMENT={_q(fragment)}; SYSCTL_FILE={_q(sysctl_file)}; MODE={_q(deployment_mode)}; FIREWALL_MODE={_q(firewall_mode)}; NFT_MAIN= +NFT_FILE={_q(include)}; NFT_INCLUDE={_q(include_line)}; LEGACY_MANIFEST={_q(LEGACY_FIREWALLD_NFT_MANIFEST)}; IF_FRAGMENT={_q(fragment)}; SYSCTL_FILE={_q(sysctl_file)}; MODE={_q(deployment_mode)}; FIREWALL_MODE={_q(firewall_mode)}; NFT_MAIN= IFUP={_q(ifup_path)}; IFDOWN={_q(ifdown_path)}; IFQUERY={_q(ifquery_path)}; NMCLI=; NM_PROFILE_UUID=; NM_PROFILE_FILE= NM_PROFILE_MARKER={_q(DATAPLANE_OWNER_MARKER + " vpc=" + plan.vpc.vpc_id)} EXPECTED_DIGEST={_q(plan.fabric_plan_digest)} NFT_RULE_MARKER={_q(nft_rule_marker)} -COMMITTED=no; BRIDGE_CREATED=no; BRIDGE_WAS_UP=no; BRIDGE_TOPOLOGY_CAPTURED=no; BRIDGE_PORTS_BEFORE=; NM_CREATED=no; NM_CREATED_UUID=; NM_PROFILE_WAS_PRESENT=no; NM_WAS_ACTIVE=no; NM_DEVICE_QUARANTINED=no; NM_ACTIVATION_ATTEMPTED=no; NM_TRUSTED_ISOLATED_CANDIDATE=no; NM_TRUSTED_ISOLATED_RETRY=no; IF_FRAGMENT_CHANGED=no; SYSCTL_CHANGED=no; SYSCTL_TMP=; RP_FILTER_CHANGED=no; PROXY_ARP_CHANGED=no; PROXY_ARP_PVLAN_CHANGED=no; NFT_FILE_CHANGED=no; NFT_MAIN_CHANGED=no; NFT_FILE_BEFORE=no; NFT_MAIN_BEFORE=no; BRIDGE_NFT_BEFORE=no; ARP_NFT_BEFORE=no; FABRIC_NFT_BEFORE=no; NFT_APPLIED=no; FABRIC_NFT_APPLIED=no; NFT_SNAPSHOT_CHANGED=no; PROXY_ENTRIES_CHANGED=no; PROXY_FILE_CHANGED=no; FIREWALL_RUNTIME_WAS_TRUSTED=no; FIREWALL_HARD_QUARANTINED=no; FIREWALL_PREVIOUS_NFT_SAFE=no; FIREWALL_ZONE_BLOCKED=yes; ZONE_OWNERSHIP=absent; ROLLBACK_FAILED=no -{NFT_MANIFEST_VALIDATION_SH} +COMMITTED=no; BRIDGE_CREATED=no; BRIDGE_WAS_UP=no; BRIDGE_TOPOLOGY_CAPTURED=no; BRIDGE_PORTS_BEFORE=; NM_CREATED=no; NM_CREATED_UUID=; NM_PROFILE_WAS_PRESENT=no; NM_WAS_ACTIVE=no; NM_DEVICE_QUARANTINED=no; NM_ACTIVATION_ATTEMPTED=no; IF_FRAGMENT_CHANGED=no; SYSCTL_CHANGED=no; SYSCTL_TMP=; RP_FILTER_CHANGED=no; PROXY_ARP_CHANGED=no; PROXY_ARP_PVLAN_CHANGED=no; NFT_FILE_CHANGED=no; NFT_MAIN_CHANGED=no; NFT_FILE_BEFORE=no; NFT_MAIN_BEFORE=no; BRIDGE_NFT_BEFORE=no; ARP_NFT_BEFORE=no; FABRIC_NFT_BEFORE=no; NFT_APPLIED=no; FABRIC_NFT_APPLIED=no; PROXY_ENTRIES_CHANGED=no; PROXY_FILE_CHANGED=no; FIREWALL_HARD_QUARANTINED=no; ROLLBACK_FAILED=no {NFT_LIVE_SNAPSHOT_SH} -{FIREWALL_ZONE_VALIDATION_SH} -{FIREWALL_ZONE_OWNERSHIP_SH} {_NMCLI_FILENAME_RESOLVER_SH} {NETWORKMANAGER_VPC_PROFILE_SH} {nft_readback_helper} {_FIREWALL_MODE_GUARD_SH} {_FABRIC_NFT_VALIDATION_SH} {_VPC_SYSCTL_VALIDATION_SH} +{VPC_NATIVE_TRANSACTION_GUARD_SH} nft_plan_owned() {{ nft_plan_text=$1 nft_plan_digest=$2 @@ -1749,15 +1654,15 @@ if [ "$MODE" = networkmanager_nmcli ]; then NMCLI=$(for p in /usr/bin/nmcli /usr/sbin/nmcli /bin/nmcli /sbin/nmcli; do [ -x "$p" ] && {{ printf '%s' "$p"; break; }}; done) [ -n "$NMCLI" ] || exit 70 fi -if [ "$FIREWALL_MODE" = firewalld_nft ]; then - [ -n "$FIREWALL_CMD" ] || exit 70 -fi +[ "$FIREWALL_MODE" = native_nftables ] || exit 71 if [ -L /run/virt-cloud ] || {{ [ -e /run/virt-cloud ] && [ ! -d /run/virt-cloud ]; }}; then exit 71; fi if [ -L /etc/virt-cloud ] || {{ [ -e /etc/virt-cloud ] && [ ! -d /etc/virt-cloud ]; }}; then exit 71; fi if [ -L {state_dir} ] || {{ [ -e {state_dir} ] && [ ! -d {state_dir} ]; }}; then exit 71; fi -mkdir -p /run/virt-cloud {state_dir} -chmod 700 /run/virt-cloud {state_dir} -[ ! -L {state_dir} ] && [ "$($STAT -c '%u:%a' {state_dir} 2>/dev/null)" = 0:700 ] || exit 71 +if [ -e {state_dir} ]; then + [ "$($STAT -c '%u:%a' {state_dir} 2>/dev/null)" = 0:700 ] || exit 71 +fi +mkdir -p /run/virt-cloud +chmod 700 /run/virt-cloud [ ! -L "$LOCK" ] && {{ [ ! -e "$LOCK" ] || [ -f "$LOCK" ]; }} || exit 71 : >> "$LOCK"; chmod 600 "$LOCK" exec 8>>{_q(FABRIC_LOCK)}; "$FLOCK" -x 8 @@ -1845,10 +1750,22 @@ vc_vpc_native_ifupdown_rollback_reopen() {{ fi return 0 }} -# Emergency quarantine deliberately runs before the runtime firewall-mode -# guard. A saved VPC can therefore be hard-blocked even when its firewall -# service/config or the competing service has drifted; raw nft ACLs are the -# only tenant guard in both supported modes. +# Never mutate even the bridge state while an unmarked, legacy, or malformed +# transaction exists. New native transactions carry an exact backend marker. +vc_vpc_firewall_boundary_clean || exit 79 +for vc_vpc_legacy_snapshot in inet bridge arp; do + [ ! -e "$NFT_SNAPSHOT_PREFIX.$vc_vpc_legacy_snapshot.live" ] && + [ ! -L "$NFT_SNAPSHOT_PREFIX.$vc_vpc_legacy_snapshot.live" ] || exit 79 +done +if [ -e "$TXN" ] || [ -L "$TXN" ]; then + vc_vpc_native_transaction_owned || exit 79 +fi +if [ ! -e {state_dir} ]; then + mkdir -m 700 {state_dir} || exit 71 +fi +[ ! -L {state_dir} ] && [ "$($STAT -c '%u:%a' {state_dir} 2>/dev/null)" = 0:700 ] || exit 71 +# Link quarantine deliberately runs before the runtime firewall-mode guard. A +# saved VPC is hard-blocked before its native nft ACL is replaced. PREVIOUS_FIREWALL_MODE=absent if [ -f "$STATE_FILE" ]; then [ ! -L "$STATE_FILE" ] && @@ -1862,7 +1779,6 @@ if [ -f "$STATE_FILE" ]; then state_firewall_quarantine=$(awk -F'|' '$1=="firewall_mode" {{count++; value=$2}} END {{print (count + 0) "|" value}}' "$STATE_FILE") case "$state_firewall_quarantine" in '0|'|'1|native_nftables') PREVIOUS_FIREWALL_MODE=native_nftables ;; - '1|firewalld_nft') PREVIOUS_FIREWALL_MODE=firewalld_nft ;; *) exit 71 ;; esac fi @@ -1884,48 +1800,19 @@ if [ "$PREVIOUS_FIREWALL_MODE" != absent ]; then NM_PROFILE_WAS_PRESENT=yes if vc_nm_profile_active; then NM_WAS_ACTIVE=yes; fi quarantine_memory_zone=$(vc_nm_profile_field connection.zone) || exit 71 - if [ "$PREVIOUS_FIREWALL_MODE" = firewalld_nft ]; then - case "$quarantine_memory_zone" in ''|--|trusted) : ;; *) exit 71 ;; esac - if [ "$quarantine_memory_zone" = trusted ]; then - if [ "$NM_WAS_ACTIVE" = yes ]; then - FIREWALL_RUNTIME_WAS_TRUSTED=yes - else - # An interrupted pre-journal apply may have left the exact - # panel-owned profile trusted but the device already isolated. Do - # not accept that state until the kernel link and bridge topology - # have also been sampled below. - NM_TRUSTED_ISOLATED_CANDIDATE=yes - fi - fi - else - case "$quarantine_memory_zone" in ''|--) : ;; *) exit 71 ;; esac - fi + case "$quarantine_memory_zone" in ''|--) : ;; *) exit 71 ;; esac else [ -f "$IF_FRAGMENT" ] && [ ! -L "$IF_FRAGMENT" ] && [ "$($STAT -c '%u:%a' "$IF_FRAGMENT" 2>/dev/null)" = 0:600 ] || exit 71 - if [ "$PREVIOUS_FIREWALL_MODE" = firewalld_nft ]; then - awk -v owner="# {DATAPLANE_OWNER_MARKER} vpc={plan.vpc.vpc_id} revision=${{state_revision#*|}} digest=${{state_digest#*|}}" \ - -v bridge="$BRIDGE" ' - NR==1 && $0==owner {{next}} - NR==2 && $0=="iface " bridge " inet manual" {{next}} - NR==3 && $0==" bridge_ports none" {{next}} - NR==4 && $0==" bridge-stp off" {{next}} - {{bad=1}} - END {{exit (NR==4 && !bad) ? 0 : 1}} - ' "$IF_FRAGMENT" || exit 71 - [ ! -e "/etc/sysconfig/network-scripts/ifcfg-$BRIDGE" ] && - [ ! -L "/etc/sysconfig/network-scripts/ifcfg-$BRIDGE" ] || exit 71 - else - awk -v owner="# {DATAPLANE_OWNER_MARKER} vpc={plan.vpc.vpc_id} revision=${{state_revision#*|}} digest=${{state_digest#*|}}" \ - -v bridge="$BRIDGE" ' - NR==1 && $0==owner {{next}} - NR==2 && $0=="iface " bridge " inet manual" {{next}} - NR==3 && $0==" bridge_ports none" {{next}} - NR==4 && $0==" bridge-stp off" {{next}} - {{bad=1}} - END {{exit (NR==4 && !bad) ? 0 : 1}} - ' "$IF_FRAGMENT" || exit 71 - fi + awk -v owner="# {DATAPLANE_OWNER_MARKER} vpc={plan.vpc.vpc_id} revision=${{state_revision#*|}} digest=${{state_digest#*|}}" \ + -v bridge="$BRIDGE" ' + NR==1 && $0==owner {{next}} + NR==2 && $0=="iface " bridge " inet manual" {{next}} + NR==3 && $0==" bridge_ports none" {{next}} + NR==4 && $0==" bridge-stp off" {{next}} + {{bad=1}} + END {{exit (NR==4 && !bad) ? 0 : 1}} + ' "$IF_FRAGMENT" || exit 71 fi if "$IP" link show dev "$BRIDGE" >/dev/null 2>&1; then "$IP" -d link show dev "$BRIDGE" 2>/dev/null | @@ -1937,10 +1824,6 @@ if [ "$PREVIOUS_FIREWALL_MODE" != absent ]; then BRIDGE_PORTS_BEFORE=$(vc_vpc_bridge_port_names) || exit 71 BRIDGE_TOPOLOGY_CAPTURED=yes if [ "$MODE" = networkmanager_nmcli ]; then - if [ "$NM_TRUSTED_ISOLATED_CANDIDATE" = yes ]; then - vc_nm_trusted_isolated_retry_ready || exit 71 - NM_TRUSTED_ISOLATED_RETRY=yes - fi vc_nm_device_quarantine || exit 72 NM_DEVICE_QUARANTINED=yes else @@ -1950,98 +1833,23 @@ if [ "$PREVIOUS_FIREWALL_MODE" != absent ]; then fi vc_vpc_bridge_topology_matches || exit 72 fi - [ "$NM_TRUSTED_ISOLATED_CANDIDATE" = no ] || - [ "$NM_TRUSTED_ISOLATED_RETRY" = yes ] || exit 71 FIREWALL_HARD_QUARANTINED=yes fi vc_firewall_mode_ready || exit 71 -if [ "$FIREWALL_MODE" = firewalld_nft ]; then - if [ -f "$STATE_FILE" ]; then - [ "$PREVIOUS_FIREWALL_MODE" = firewalld_nft ] || exit 71 - if [ "$MODE" = networkmanager_nmcli ]; then - ZONE_OWNERSHIP=managed-nm - else - ZONE_OWNERSHIP=managed - fi - vc_firewalld_bridge_zone_expect preapply "$ZONE_OWNERSHIP" "$BRIDGE" || exit 71 - if [ "$VC_VPC_ZONE_RUNTIME" = trusted ] && - [ "$NM_TRUSTED_ISOLATED_RETRY" != yes ]; then - FIREWALL_RUNTIME_WAS_TRUSTED=yes - fi - if [ "$MODE" = networkmanager_nmcli ]; then - if [ "$NM_DEVICE_QUARANTINED" = yes ]; then - vc_nm_profile_base_validate || exit 71 - vc_nm_device_expect_managed no || exit 71 - else - vc_nm_profile_expect preapply || exit 71 - fi - if [ "$VC_VPC_ZONE_RUNTIME" = trusted ]; then - "$NMCLI" connection modify --temporary uuid "$NM_PROFILE_UUID" connection.zone "" >/dev/null || exit 72 - if "$FIREWALL_CMD" --zone=trusted --query-interface="$BRIDGE" >/dev/null 2>&1; then - "$FIREWALL_CMD" --zone=trusted --remove-interface="$BRIDGE" >/dev/null || exit 72 - else - zone_query_status=$? - [ "$zone_query_status" = 1 ] || exit 72 - fi - fi - if [ "$NM_DEVICE_QUARANTINED" = yes ]; then - vc_nm_profile_expect quarantined || exit 72 - elif [ "$NM_WAS_ACTIVE" = yes ]; then - vc_nm_profile_expect blocked-active || exit 72 - else - vc_nm_profile_expect blocked || exit 72 - fi - else - if [ "$FIREWALL_RUNTIME_WAS_TRUSTED" = yes ]; then - "$FIREWALL_CMD" --zone=trusted --remove-interface="$BRIDGE" >/dev/null || exit 72 - fi - vc_firewalld_bridge_zone_expect blocked managed "$BRIDGE" || exit 72 - fi - if "$IP" link show dev "$BRIDGE" >/dev/null 2>&1; then - bridge_quarantine_link_state=$(vc_vpc_bridge_link_state) || exit 72 - [ "$bridge_quarantine_link_state" = down ] || exit 72 - fi - else - ZONE_OWNERSHIP=absent - [ ! -e "$OWNER_FILE" ] && [ ! -L "$OWNER_FILE" ] || exit 71 - [ ! -e "$IF_FRAGMENT" ] && [ ! -L "$IF_FRAGMENT" ] || exit 71 - "$IP" link show dev "$BRIDGE" >/dev/null 2>&1 && exit 71 || true - vc_firewalld_bridge_zone_expect preapply "$ZONE_OWNERSHIP" "$BRIDGE" || exit 71 - if [ "$MODE" = networkmanager_nmcli ]; then - if vc_nm_profile_resolve; then - exit 71 - else - nm_resolve_status=$? - [ "$nm_resolve_status" = 2 ] || exit 71 - fi - fi - fi -fi [ -f /etc/virt-cloud/wg-fabric0.owner ] && [ "$(cat /etc/virt-cloud/wg-fabric0.owner)" = "$FABRIC_OWNER" ] || exit 71 "$WG" show "$WG_IFACE" >/dev/null 2>&1 || exit 71 -if [ "$FIREWALL_MODE" = native_nftables ]; then - NFT_MAIN_COUNT=0 - for nft_candidate in /etc/nftables.conf /etc/sysconfig/nftables.conf; do - if [ -f "$nft_candidate" ] && [ ! -L "$nft_candidate" ] && [ "$("$STAT" -c '%u' "$nft_candidate" 2>/dev/null)" = 0 ]; then - case "$("$STAT" -c '%a' "$nft_candidate" 2>/dev/null)" in - [0-7][0145][0145]) : ;; - *) continue ;; - esac - NFT_MAIN="$nft_candidate"; NFT_MAIN_COUNT=$((NFT_MAIN_COUNT + 1)) - fi - done - [ "$NFT_MAIN_COUNT" -eq 1 ] || exit 71 - [ ! -e "$NFT_MANIFEST" ] && [ ! -L "$NFT_MANIFEST" ] || exit 71 -else - [ "$FIREWALL_MODE" = firewalld_nft ] || exit 71 - vc_nft_manifest_validate "$NFT_MANIFEST" || exit 71 - NFT_MAIN="$NFT_MANIFEST" - for nft_candidate in /etc/nftables.conf /etc/sysconfig/nftables.conf; do - [ -e "$nft_candidate" ] || [ -L "$nft_candidate" ] || continue - [ -f "$nft_candidate" ] && [ ! -L "$nft_candidate" ] || exit 71 - grep -Eq '/etc/nftables[.]d/(virt-cloud-wg-fabric[.]nft|virt-cloud-vpc-[0-9a-f]+[.]nft|[*]([.]nft)?)' "$nft_candidate" && exit 71 || true - done -fi +NFT_MAIN_COUNT=0 +for nft_candidate in /etc/nftables.conf /etc/sysconfig/nftables.conf; do + if [ -f "$nft_candidate" ] && [ ! -L "$nft_candidate" ] && [ "$("$STAT" -c '%u' "$nft_candidate" 2>/dev/null)" = 0 ]; then + case "$("$STAT" -c '%a' "$nft_candidate" 2>/dev/null)" in + [0-7][0145][0145]) : ;; + *) continue ;; + esac + NFT_MAIN="$nft_candidate"; NFT_MAIN_COUNT=$((NFT_MAIN_COUNT + 1)) + fi +done +[ "$NFT_MAIN_COUNT" -eq 1 ] || exit 71 +[ ! -e "$LEGACY_MANIFEST" ] && [ ! -L "$LEGACY_MANIFEST" ] || exit 71 if [ -L /etc/nftables.d ] || {{ [ -e /etc/nftables.d ] && [ ! -d /etc/nftables.d ]; }}; then exit 71; fi if [ -e /etc/nftables.d ]; then [ "$("$STAT" -c '%u:%a' /etc/nftables.d 2>/dev/null)" = 0:755 ] || exit 71 @@ -2076,11 +1884,7 @@ if [ -f "$STATE_FILE" ]; then grep -Fqx 'bridge|{r.bridge}' "$STATE_FILE" || exit 71 grep -Fqx 'table|{r.route_table}' "$STATE_FILE" || exit 71 state_firewall_record=$(awk -F'|' '$1=="firewall_mode" {{count++; value=$2}} END {{print (count + 0) "|" value}}' "$STATE_FILE") - if [ "$FIREWALL_MODE" = native_nftables ]; then - case "$state_firewall_record" in '0|'|'1|native_nftables') : ;; *) exit 71 ;; esac - else - [ "$state_firewall_record" = "1|$FIREWALL_MODE" ] || exit 71 - fi + case "$state_firewall_record" in '0|'|'1|native_nftables') : ;; *) exit 71 ;; esac state_revision=$(awk -F'|' '$1=="revision" {{print $2; exit}}' "$STATE_FILE") state_digest=$(awk -F'|' '$1=="digest" {{print $2; exit}}' "$STATE_FILE") case "$state_revision" in ''|*[!0-9]*) exit 79;; esac @@ -2102,19 +1906,21 @@ if [ -e "$TXN" ] || [ -L "$TXN" ]; then # ownership; preparing/ready/rollback_failed states remain blocked so a # later reconcile cannot guess whether runtime resources were partially # applied. - [ -d "$TXN" ] && [ ! -L "$TXN" ] || exit 79 - [ "$("$STAT" -c '%u:%a' "$TXN" 2>/dev/null)" = 0:700 ] || exit 79 - [ -f "$TXN/inflight" ] && [ ! -L "$TXN/inflight" ] || exit 79 - grep -Fqx '{DATAPLANE_OWNER_MARKER}:{plan.vpc.vpc_id}:{plan.host_id}:{plan.desired_revision}:{plan.desired_digest}' "$TXN/inflight" || exit 79 - [ -f "$TXN/state" ] && [ "$(cat "$TXN/state" 2>/dev/null)" = committed ] || exit 79 + vc_vpc_native_transaction_owned || exit 79 + vc_vpc_transaction_record_owned "$TXN/inflight" \ + '{DATAPLANE_OWNER_MARKER}:{plan.vpc.vpc_id}:{plan.host_id}:{plan.desired_revision}:{plan.desired_digest}' || exit 79 + vc_vpc_transaction_record_owned "$TXN/state" committed || exit 79 rm -rf "$TXN" || exit 79 fi # Fail closed on any foreign rule selecting this VPC CIDR or route table # before installing the destructive rollback trap or mutating host state. vc_ip_rule_scope_check {_q(plan.vpc.cidr)} "$ROUTE_TABLE" "$RULE_PRIORITY" "$WG_IFACE" managed 73 73 || exit $? -# A firewalld bridge was quarantined immediately after the per-VPC lock and -# minimum owner/manager proof, before this RPDB check and all other preflight. +# The bridge was quarantined immediately after the per-VPC lock and minimum +# owner/manager proof, before this RPDB check and all other preflight. mkdir -m 700 "$TXN" +vc_vpc_write_native_transaction_mode || exit 79 +printf '%s\\n' '{DATAPLANE_OWNER_MARKER}:{plan.vpc.vpc_id}:{plan.host_id}:{plan.desired_revision}:{plan.desired_digest}' > "$TXN/inflight" +chmod 600 "$TXN/inflight" printf '%s\\n' preparing > "$TXN/state"; chmod 600 "$TXN/state" rollback() {{ status=$?; [ "$COMMITTED" = yes ] && exit "$status"; trap - EXIT HUP INT TERM @@ -2122,7 +1928,7 @@ rollback() {{ # First rollback action: quarantine this already-proven panel bridge before # touching routes, peers, proxy state, sysctls or raw nft tables. This # closes the failure window where an ACL readback error could otherwise - # leave a trusted interface exposed while unrelated rollback work runs. + # leave an active interface exposed while unrelated rollback work runs. if "$IP" link show dev "$BRIDGE" >/dev/null 2>&1; then if [ "$MODE" = networkmanager_nmcli ] && vc_nm_profile_resolve; then vc_nm_device_quarantine || ROLLBACK_FAILED=yes @@ -2137,54 +1943,6 @@ rollback() {{ vc_vpc_bridge_topology_matches || ROLLBACK_FAILED=yes fi fi - if [ "$FIREWALL_MODE" = firewalld_nft ]; then - rollback_zone_ownership=managed - [ "$MODE" = networkmanager_nmcli ] && rollback_zone_ownership=managed-nm - if vc_firewalld_bridge_zone_expect preapply "$rollback_zone_ownership" "$BRIDGE"; then - if [ "$MODE" = networkmanager_nmcli ]; then - if vc_nm_profile_resolve; then - vc_nm_memory_zone=$(vc_nm_profile_field connection.zone) - if [ "$vc_nm_memory_zone" != trusted ] && - [ -n "$vc_nm_memory_zone" ] && [ "$vc_nm_memory_zone" != -- ]; then - FIREWALL_ZONE_BLOCKED=no - fi - if [ "$FIREWALL_ZONE_BLOCKED" = yes ]; then - "$NMCLI" connection modify --temporary uuid "$NM_PROFILE_UUID" connection.zone "" >/dev/null 2>&1 || FIREWALL_ZONE_BLOCKED=no - fi - if [ "$FIREWALL_ZONE_BLOCKED" = yes ] && - "$FIREWALL_CMD" --zone=trusted --query-interface="$BRIDGE" >/dev/null 2>&1; then - "$FIREWALL_CMD" --zone=trusted --remove-interface="$BRIDGE" >/dev/null 2>&1 || FIREWALL_ZONE_BLOCKED=no - fi - if [ "$FIREWALL_ZONE_BLOCKED" = yes ]; then - if [ "$NM_DEVICE_QUARANTINED" = yes ]; then - vc_nm_profile_expect quarantined || FIREWALL_ZONE_BLOCKED=no - elif vc_nm_profile_active; then - vc_nm_profile_expect blocked-active || FIREWALL_ZONE_BLOCKED=no - else - vc_nm_profile_expect blocked || FIREWALL_ZONE_BLOCKED=no - fi - fi - else - nm_rollback_resolve=$? - if [ "$VC_VPC_ZONE_RUNTIME" = trusted ]; then - FIREWALL_ZONE_BLOCKED=no - elif [ "$NM_PROFILE_WAS_PRESENT" = yes ] || - ! {{ [ "$nm_rollback_resolve" = 2 ] || - {{ [ "$NM_CREATED" = yes ] && [ -n "$NM_PROFILE_UUID" ]; }}; }}; then - ROLLBACK_FAILED=yes - fi - fi - else - if [ "$VC_VPC_ZONE_RUNTIME" = trusted ]; then - "$FIREWALL_CMD" --zone=trusted --remove-interface="$BRIDGE" >/dev/null 2>&1 || FIREWALL_ZONE_BLOCKED=no - fi - vc_firewalld_bridge_zone_expect blocked managed "$BRIDGE" || FIREWALL_ZONE_BLOCKED=no - fi - else - FIREWALL_ZONE_BLOCKED=no - fi - [ "$FIREWALL_ZONE_BLOCKED" = yes ] || ROLLBACK_FAILED=yes - fi while IFS='|' read -r destination device table via metric; do [ -n "$destination" ] || continue actual=$("$IP" -o -4 route show table "$table" "$destination" 2>/dev/null || true) @@ -2306,9 +2064,7 @@ rollback() {{ cat "$TXN/rp_filter.before" > "/proc/sys/net/ipv4/conf/$BRIDGE/rp_filter" || ROLLBACK_FAILED=yes fi fi - # firewalld quarantine ran as the first rollback action above. - if [ "$FIREWALL_ZONE_BLOCKED" = yes ]; then - NFT_ROLLBACK="$TXN/rollback.nft" + NFT_ROLLBACK="$TXN/rollback.nft" NFT_ROLLBACK_SAFE=yes if [ "$NFT_APPLIED" = yes ]; then # A table name or owner comment is not sufficient ownership evidence. @@ -2368,22 +2124,11 @@ rollback() {{ "$TXN/fabric.nft.before" || ROLLBACK_FAILED=yes fi fi - if [ "$ROLLBACK_FAILED" = no ] && [ "$NFT_SNAPSHOT_CHANGED" = yes ]; then - for snapshot_family in {nft_family_words}; do - snapshot_path="$NFT_SNAPSHOT_PREFIX.$snapshot_family.live" - if [ -f "$TXN/nft.persist.before.$snapshot_family" ]; then - cp -p "$TXN/nft.persist.before.$snapshot_family" "$snapshot_path" || ROLLBACK_FAILED=yes - else - rm -f "$snapshot_path" || ROLLBACK_FAILED=yes - fi - done - fi if [ "$NFT_FILE_CHANGED" = yes ]; then if [ -f "$TXN/nft_file.before" ]; then cp -p "$TXN/nft_file.before" "$NFT_FILE" || ROLLBACK_FAILED=yes; else rm -f "$NFT_FILE" || ROLLBACK_FAILED=yes; fi fi - if [ "$NFT_MAIN_CHANGED" = yes ] && [ -f "$TXN/nft_main.before" ]; then - cp -p "$TXN/nft_main.before" "$NFT_MAIN" || ROLLBACK_FAILED=yes - fi + if [ "$NFT_MAIN_CHANGED" = yes ] && [ -f "$TXN/nft_main.before" ]; then + cp -p "$TXN/nft_main.before" "$NFT_MAIN" || ROLLBACK_FAILED=yes fi if [ "$MODE" = networkmanager_nmcli ] && [ -n "$NMCLI" ]; then if [ "$NM_ACTIVATION_ATTEMPTED" = yes ] && [ "$NM_WAS_ACTIVE" = no ]; then @@ -2408,9 +2153,6 @@ rollback() {{ # A same-name delete is ambiguous and could remove an operator profile. vc_nm_created_profile_remove || ROLLBACK_FAILED=yes fi - if [ "$FIREWALL_MODE" = firewalld_nft ]; then - vc_firewalld_bridge_zone_expect blocked managed "$BRIDGE" || ROLLBACK_FAILED=yes - fi fi fi if [ "$IF_FRAGMENT_CHANGED" = yes ]; then @@ -2463,7 +2205,6 @@ rollback() {{ fi fi if [ "$MODE" = ifupdown ] && - [ "$FIREWALL_MODE" = native_nftables ] && [ "$BRIDGE_CREATED" = no ] && [ "$BRIDGE_WAS_UP" = yes ] && [ "$ROLLBACK_FAILED" = no ]; then # The existing bridge was hard-quarantined before preflight. Re-open it @@ -2472,17 +2213,11 @@ rollback() {{ # brought up by rollback. vc_vpc_native_ifupdown_rollback_reopen || ROLLBACK_FAILED=yes fi - # Quarantine is manager-neutral: native nftables and a firewalld profile - # that was not trusted are also made unmanaged/down before preflight. Once - # the original raw ACL has been restored, return an existing profile to its - # exact managed/active state. A previously trusted firewalld profile is - # restored by the stricter ACL+zone branch below. + # Return a quarantined native NetworkManager profile to its exact prior + # managed/active state only after restoring the original nft ACL. if [ "$MODE" = networkmanager_nmcli ] && [ "$NM_PROFILE_WAS_PRESENT" = yes ] && [ "$NM_DEVICE_QUARANTINED" = yes ] && - [ "$NM_TRUSTED_ISOLATED_RETRY" != yes ] && - ! {{ [ "$FIREWALL_MODE" = firewalld_nft ] && - [ "$FIREWALL_RUNTIME_WAS_TRUSTED" = yes ]; }} && [ "$ROLLBACK_FAILED" = no ]; then if vc_nm_profile_resolve && vc_nm_profile_base_validate && vc_nm_device_expect_managed no && @@ -2502,19 +2237,11 @@ rollback() {{ fi [ "$ROLLBACK_FAILED" = no ] && vc_nm_profile_active || ROLLBACK_FAILED=yes - if [ "$ROLLBACK_FAILED" = no ] && - [ "$FIREWALL_MODE" = firewalld_nft ]; then - vc_nm_profile_expect blocked-active || ROLLBACK_FAILED=yes - fi elif [ "$ROLLBACK_FAILED" = no ]; then vc_nm_profile_active && ROLLBACK_FAILED=yes || true "$IP" link set dev "$BRIDGE" down >/dev/null 2>&1 || ROLLBACK_FAILED=yes vc_nm_device_expect_managed yes || ROLLBACK_FAILED=yes - if [ "$FIREWALL_MODE" = firewalld_nft ]; then - vc_firewalld_bridge_zone_expect blocked managed "$BRIDGE" || - ROLLBACK_FAILED=yes - fi fi [ "$ROLLBACK_FAILED" = no ] && vc_vpc_bridge_topology_matches || ROLLBACK_FAILED=yes @@ -2523,78 +2250,6 @@ rollback() {{ ROLLBACK_FAILED=yes fi fi - if [ "$FIREWALL_MODE" = firewalld_nft ] && - [ "$FIREWALL_RUNTIME_WAS_TRUSTED" = yes ] && - [ "$FIREWALL_ZONE_BLOCKED" = yes ] && - [ "$FIREWALL_PREVIOUS_NFT_SAFE" = yes ] && - [ "$ROLLBACK_FAILED" = no ]; then - if vc_vpc_previous_nft_ready && vc_vpc_persisted_nft_tables_match && - vc_vpc_fabric_nft_live_ready && - vc_nft_table_matches_snapshot inet virt_cloud_wg_fabric \ - "$TXN/fabric.nft.before"; then - if [ "$MODE" = networkmanager_nmcli ]; then - if vc_nm_profile_resolve && vc_nm_profile_expect quarantined && - vc_vpc_bridge_topology_matches; then - "$NMCLI" connection modify --temporary uuid "$NM_PROFILE_UUID" connection.zone trusted >/dev/null 2>&1 || ROLLBACK_FAILED=yes - [ "$ROLLBACK_FAILED" = no ] && - vc_nm_device_release || ROLLBACK_FAILED=yes - [ "$ROLLBACK_FAILED" = no ] && - "$NMCLI" connection up uuid "$NM_PROFILE_UUID" >/dev/null 2>&1 || ROLLBACK_FAILED=yes - if [ "$ROLLBACK_FAILED" = no ]; then - if [ "$BRIDGE_WAS_UP" = yes ]; then - "$IP" link set dev "$BRIDGE" up >/dev/null 2>&1 || ROLLBACK_FAILED=yes - else - "$IP" link set dev "$BRIDGE" down >/dev/null 2>&1 || ROLLBACK_FAILED=yes - fi - fi - [ "$ROLLBACK_FAILED" = no ] && - vc_vpc_bridge_topology_matches || ROLLBACK_FAILED=yes - [ "$ROLLBACK_FAILED" = no ] && vc_nm_profile_expect active || ROLLBACK_FAILED=yes - [ "$ROLLBACK_FAILED" = yes ] || NM_DEVICE_QUARANTINED=no - else - ROLLBACK_FAILED=yes - fi - else - if vc_firewalld_bridge_zone_expect blocked managed "$BRIDGE" && - vc_vpc_bridge_topology_matches; then - if [ "$BRIDGE_WAS_UP" = yes ]; then - "$IP" link set dev "$BRIDGE" up >/dev/null 2>&1 || ROLLBACK_FAILED=yes - if [ "$ROLLBACK_FAILED" = no ]; then - rollback_bridge_link_state=$(vc_vpc_bridge_link_state) - if [ "$?" -ne 0 ] || - [ "$rollback_bridge_link_state" != up ]; then - ROLLBACK_FAILED=yes - fi - fi - fi - [ "$ROLLBACK_FAILED" = no ] && - "$FIREWALL_CMD" --zone=trusted --add-interface="$BRIDGE" >/dev/null 2>&1 || ROLLBACK_FAILED=yes - [ "$ROLLBACK_FAILED" = no ] && - vc_vpc_bridge_topology_matches || ROLLBACK_FAILED=yes - else - ROLLBACK_FAILED=yes - fi - fi - if [ "$MODE" = ifupdown ]; then - if [ -e "/etc/sysconfig/network-scripts/ifcfg-$BRIDGE" ] || - [ -L "/etc/sysconfig/network-scripts/ifcfg-$BRIDGE" ]; then - ROLLBACK_FAILED=yes - fi - vc_firewalld_bridge_zone_expect active managed "$BRIDGE" || ROLLBACK_FAILED=yes - fi - else - ROLLBACK_FAILED=yes - fi - fi - if [ "$FIREWALL_MODE" = firewalld_nft ] && - [ "$FIREWALL_RUNTIME_WAS_TRUSTED" = yes ] && - [ "$FIREWALL_ZONE_BLOCKED" = yes ] && - [ "$FIREWALL_PREVIOUS_NFT_SAFE" != yes ]; then - # The old trusted state cannot be reopened without a proven old ACL. - # Keep the transaction journal and report rollback failure instead of - # claiming success while intentionally leaving the bridge quarantined. - ROLLBACK_FAILED=yes - fi if [ -f "$TXN/owner.before" ]; then cp -p "$TXN/owner.before" "$OWNER_FILE" || ROLLBACK_FAILED=yes; else rm -f "$OWNER_FILE"; fi if [ -f "$TXN/state.before" ]; then cp -p "$TXN/state.before" "$STATE_FILE" || ROLLBACK_FAILED=yes; else rm -f "$STATE_FILE"; fi if [ "$ROLLBACK_FAILED" = yes ]; then printf '%s\\n' rollback_failed > "$TXN/state"; chmod 600 "$TXN/state"; exit 79; fi @@ -2610,27 +2265,10 @@ if [ -f "$STATE_FILE" ]; then grep -Fqx 'owner|{DATAPLANE_OWNER_MARKER}' "$STATE_FILE" || exit 71 cp -p "$STATE_FILE" "$TXN/state.before" fi -if [ "$FIREWALL_MODE" = firewalld_nft ]; then - if [ -f "$STATE_FILE" ]; then - # The durable stateless dumps bind the state revision/digest to the full - # live rule semantics. Refuse to update an externally edited table. - vc_vpc_persisted_nft_ready || exit 71 - for snapshot_family in {nft_family_words}; do - cp -p "$NFT_SNAPSHOT_PREFIX.$snapshot_family.live" \ - "$TXN/nft.persist.before.$snapshot_family" || exit 71 - done - else - for snapshot_family in inet bridge arp; do - [ ! -e "$NFT_SNAPSHOT_PREFIX.$snapshot_family.live" ] && - [ ! -L "$NFT_SNAPSHOT_PREFIX.$snapshot_family.live" ] || exit 71 - done - fi -else - for snapshot_family in inet bridge arp; do - [ ! -e "$NFT_SNAPSHOT_PREFIX.$snapshot_family.live" ] && - [ ! -L "$NFT_SNAPSHOT_PREFIX.$snapshot_family.live" ] || exit 71 - done -fi +for snapshot_family in inet bridge arp; do + [ ! -e "$NFT_SNAPSHOT_PREFIX.$snapshot_family.live" ] && + [ ! -L "$NFT_SNAPSHOT_PREFIX.$snapshot_family.live" ] || exit 71 +done "$WG" showconf "$WG_IFACE" > "$TXN/wg.conf"; chmod 600 "$TXN/wg.conf" if "$NFT" -s list table inet virt_cloud_wg_fabric > "$TXN/fabric.nft.before" 2>/dev/null; then vc_vpc_fabric_nft_shape_file "$TXN/fabric.nft.before" || exit 71 @@ -2660,35 +2298,6 @@ cp -p "$NFT_MAIN" "$TXN/nft_main.before"; NFT_MAIN_BEFORE=yes if [ -f "$NFT_FILE" ]; then cp -p "$NFT_FILE" "$TXN/nft_file.before"; NFT_FILE_BEFORE=yes; fi if [ -f "$SYSCTL_FILE" ]; then cp -p "$SYSCTL_FILE" "$TXN/sysctl.before"; fi if [ -f "$PROXY_FILE" ]; then cp -p "$PROXY_FILE" "$TXN/proxy_file.before"; fi -if [ "$FIREWALL_MODE" = firewalld_nft ]; then - if [ "$FIREWALL_RUNTIME_WAS_TRUSTED" = yes ]; then - if vc_vpc_previous_nft_ready && vc_vpc_persisted_nft_ready && - vc_vpc_fabric_nft_live_ready && - vc_nft_table_matches_snapshot inet virt_cloud_wg_fabric \ - "$TXN/fabric.nft.before"; then - FIREWALL_PREVIOUS_NFT_SAFE=yes - else - exit 71 - fi - fi - # The bridge must be back in the non-trusted default boundary before any - # profile, interface, or nft mutation can begin. - if [ "$MODE" = networkmanager_nmcli ]; then - if [ -n "$NM_PROFILE_UUID" ]; then - if [ "$NM_DEVICE_QUARANTINED" = yes ]; then - vc_nm_profile_expect quarantined || exit 72 - elif vc_nm_profile_active; then - vc_nm_profile_expect blocked-active || exit 72 - else - vc_nm_profile_expect blocked || exit 72 - fi - else - vc_firewalld_bridge_zone_expect blocked managed "$BRIDGE" || exit 72 - fi - else - vc_firewalld_bridge_zone_expect blocked managed "$BRIDGE" || exit 72 - fi -fi vc_vpc_install_rules() {{ cat > "$TXN/rules.nft" <<'{nft_marker}' {nft.rstrip()} @@ -2706,18 +2315,10 @@ if ! grep -Fqx "$NFT_INCLUDE" "$NFT_MAIN" && cat "$NFT_MAIN" > "$NFT_MAIN_TMP" printf '%s\\n' "$NFT_INCLUDE" >> "$NFT_MAIN_TMP" chmod "$($STAT -c '%a' "$NFT_MAIN")" "$NFT_MAIN_TMP" - if [ "$FIREWALL_MODE" = native_nftables ]; then - "$NFT" -c -f "$NFT_MAIN_TMP" >/dev/null || exit 72 - else - vc_nft_manifest_validate "$NFT_MAIN_TMP" || exit 72 - fi + "$NFT" -c -f "$NFT_MAIN_TMP" >/dev/null || exit 72 mv -f "$NFT_MAIN_TMP" "$NFT_MAIN"; NFT_MAIN_CHANGED=yes fi -if [ "$FIREWALL_MODE" = native_nftables ]; then - "$NFT" -c -f "$NFT_MAIN" >/dev/null || exit 72 -else - vc_nft_manifest_validate "$NFT_MAIN" || exit 72 -fi +"$NFT" -c -f "$NFT_MAIN" >/dev/null || exit 72 NFT_BATCH="$TXN/apply.nft" : > "$NFT_BATCH" if [ -f "$TXN/nft.before" ]; then printf 'delete table inet %s\\n' "$TABLE" >> "$NFT_BATCH"; fi @@ -2742,34 +2343,6 @@ NFT_APPLIED=yes vc_vpc_desired_nft_ready || exit 72 vc_vpc_capture_nft_snapshot "$TXN/nft.desired" || exit 72 vc_vpc_nft_matches_snapshot "$TXN/nft.desired" || exit 72 -if [ "$FIREWALL_MODE" = firewalld_nft ]; then - # Reassert the root-owned desired source in one atomic batch while the - # bridge remains quarantined. Two independently captured canonical dumps - # must match before trusted can be installed. This rejects a retained - # owner comment whose verdict/expression was changed out of band. - NFT_REASSERT="$TXN/reassert.nft" - : > "$NFT_REASSERT" -{nft_reassert_deletes} - cat "$TXN/rules.nft" >> "$NFT_REASSERT" - chmod 600 "$NFT_REASSERT" - vc_vpc_nft_matches_snapshot "$TXN/nft.desired" || exit 72 - "$NFT" -c -f "$NFT_REASSERT" >/dev/null || exit 72 - vc_vpc_nft_matches_snapshot "$TXN/nft.desired" || exit 71 - "$NFT" -f "$NFT_REASSERT" >/dev/null || exit 72 - vc_vpc_desired_nft_ready || exit 72 - vc_vpc_capture_nft_snapshot "$TXN/nft.desired.verify" || exit 72 - vc_vpc_nft_snapshots_equal "$TXN/nft.desired" \ - "$TXN/nft.desired.verify" || exit 72 - vc_vpc_nft_matches_snapshot "$TXN/nft.desired" || exit 72 - for snapshot_family in {nft_family_words}; do - snapshot_path="$NFT_SNAPSHOT_PREFIX.$snapshot_family.live" - snapshot_tmp=$(mktemp "$snapshot_path.XXXXXX") || exit 72 - cp -p "$TXN/nft.desired.$snapshot_family" "$snapshot_tmp" || exit 72 - chmod 600 "$snapshot_tmp" - mv -f "$snapshot_tmp" "$snapshot_path" || exit 72 - done - NFT_SNAPSHOT_CHANGED=yes -fi }} # Install and exactly read back the raw tenant guard before restoring link # administration in either persistence mode. @@ -2900,45 +2473,16 @@ if [ "$MODE" = networkmanager_nmcli ]; then vc_nm_profile_base_validate || exit 72 NM_CREATED_UUID=$NM_PROFILE_UUID fi - if [ "$FIREWALL_MODE" = native_nftables ]; then - vc_nm_profile_empty connection.zone || exit 71 - fi - if [ "$FIREWALL_MODE" = firewalld_nft ]; then - vc_nm_profile_resolve || exit 71 - if [ "$NM_DEVICE_QUARANTINED" = yes ]; then - vc_nm_profile_expect quarantined || exit 71 - elif [ "$NM_WAS_ACTIVE" = yes ]; then - vc_nm_profile_expect blocked-active || exit 71 - else - vc_nm_profile_expect blocked || exit 71 - fi - # Keep the keyfile and the live connection in the default blocked zone - # until the VPC state has refreshed the shared Fabric tenant set and both - # tables have passed the final semantic/snapshot readback below. - vc_vpc_nft_matches_snapshot "$TXN/nft.desired" || exit 72 - vc_nm_profile_disk_validate "$NM_PROFILE_FILE" "$NM_PROFILE_UUID" || exit 72 - NM_ACTIVATION_ATTEMPTED=yes - if [ "$NM_DEVICE_QUARANTINED" = yes ]; then - vc_nm_device_release || exit 72 - NM_DEVICE_QUARANTINED=no - fi - "$NMCLI" connection up uuid "$NM_PROFILE_UUID" >/dev/null || exit 72 - vc_vpc_rp_filter_set || exit 72 - if [ "$BRIDGE_TOPOLOGY_CAPTURED" = yes ]; then - vc_vpc_bridge_topology_matches || exit 72 - fi - vc_nm_profile_expect blocked-active || exit 72 - else - vc_nm_profile_resolve || exit 71 - vc_nm_profile_base_validate || exit 71 - NM_ACTIVATION_ATTEMPTED=yes - if [ "$NM_DEVICE_QUARANTINED" = yes ]; then - vc_nm_device_release || exit 72 - NM_DEVICE_QUARANTINED=no - fi - "$NMCLI" connection up uuid "$NM_PROFILE_UUID" >/dev/null - vc_vpc_rp_filter_set || exit 72 + vc_nm_profile_empty connection.zone || exit 71 + vc_nm_profile_resolve || exit 71 + vc_nm_profile_base_validate || exit 71 + NM_ACTIVATION_ATTEMPTED=yes + if [ "$NM_DEVICE_QUARANTINED" = yes ]; then + vc_nm_device_release || exit 72 + NM_DEVICE_QUARANTINED=no fi + "$NMCLI" connection up uuid "$NM_PROFILE_UUID" >/dev/null + vc_vpc_rp_filter_set || exit 72 else [ -x "$IFUP" ] && [ -x "$IFQUERY" ] && [ -x "$IFDOWN" ] || exit 70 [ ! -L /etc/network ] && [ ! -L /etc/network/interfaces.d ] || exit 71 @@ -2974,18 +2518,6 @@ else # bridge or restores an empty existing bridge to its sampled DOWN state. VC_FABRIC_APPLYING=1 "$IFUP" --force "$BRIDGE" >/dev/null 2>&1 || exit 72 vc_vpc_rp_filter_set || exit 72 - if [ "$FIREWALL_MODE" = firewalld_nft ]; then - vc_firewalld_bridge_zone_expect blocked managed "$BRIDGE" || exit 71 - vc_vpc_nft_matches_snapshot "$TXN/nft.desired" || exit 72 - if [ "$BRIDGE_TOPOLOGY_CAPTURED" = yes ]; then - vc_vpc_bridge_topology_matches || exit 72 - fi - if [ -e "/etc/sysconfig/network-scripts/ifcfg-$BRIDGE" ] || - [ -L "/etc/sysconfig/network-scripts/ifcfg-$BRIDGE" ]; then - exit 79 - fi - vc_firewalld_bridge_zone_expect blocked managed "$BRIDGE" || exit 72 - fi fi PROXY_ARP_CHANGED=yes printf '%s\\n' 0 > "/proc/sys/net/ipv4/conf/$BRIDGE/proxy_arp" || exit 72 @@ -3116,7 +2648,7 @@ done <<'{peer_marker}' {peer_lines} {peer_marker} # The shared Fabric quarantine table is the last enforcement boundary before -# a VPC bridge may become trusted. Reparse the complete stateless grammar, +# a VPC bridge is admitted. Reparse the complete stateless grammar, # bind every owner comment to the engine-authenticated full Fabric digest, # compare the exact tenant /32 set derived by the state loader, and finally # prove the whole table is still the post-refresh snapshot. A retained @@ -3127,33 +2659,6 @@ vpc_state_nft_set_matches || exit 74 vc_nft_table_matches_snapshot inet virt_cloud_wg_fabric \ "$TXN/fabric.nft.desired" || exit 74 vc_vpc_nft_matches_snapshot "$TXN/nft.desired" || exit 74 -if [ "$FIREWALL_MODE" = firewalld_nft ]; then - if [ "$MODE" = networkmanager_nmcli ]; then - vc_nm_profile_expect blocked-active || exit 72 - "$NMCLI" connection modify --temporary uuid "$NM_PROFILE_UUID" \ - connection.zone trusted >/dev/null || exit 72 - "$NMCLI" device reapply "$BRIDGE" >/dev/null || exit 72 - if [ "$BRIDGE_TOPOLOGY_CAPTURED" = yes ]; then - vc_vpc_bridge_topology_matches || exit 72 - fi - vc_nm_profile_expect active || exit 72 - else - vc_firewalld_bridge_zone_expect blocked managed "$BRIDGE" || exit 72 - if [ "$VC_VPC_ZONE_RUNTIME" = none ]; then - "$FIREWALL_CMD" --zone=trusted --add-interface="$BRIDGE" >/dev/null || exit 72 - else - "$FIREWALL_CMD" --zone=trusted --change-interface="$BRIDGE" >/dev/null || exit 72 - fi - if [ -e "/etc/sysconfig/network-scripts/ifcfg-$BRIDGE" ] || - [ -L "/etc/sysconfig/network-scripts/ifcfg-$BRIDGE" ]; then - exit 79 - fi - if [ "$BRIDGE_TOPOLOGY_CAPTURED" = yes ]; then - vc_vpc_bridge_topology_matches || exit 72 - fi - vc_firewalld_bridge_zone_expect active managed "$BRIDGE" || exit 72 - fi -fi rm -f "$VPC_PEER_FILE" "$VPC_ROUTE_FILE" "$VPC_TENANT_FILE" "$VPC_TENANT_FILE.nft" printf '%s\\n' ready > "$TXN/state"; chmod 600 "$TXN/state" printf '%s\\n' '__VC_VPC_APPLIED__=yes' @@ -3230,25 +2735,22 @@ STAT=$(for p in /usr/bin/stat /bin/stat /usr/sbin/stat /sbin/stat; do [ -x "$p" CMP=$(for p in /usr/bin/cmp /bin/cmp /usr/sbin/cmp /sbin/cmp; do [ -x "$p" ] && {{ printf '%s' "$p"; break; }}; done) MKTEMP=$(for p in /usr/bin/mktemp /bin/mktemp /usr/sbin/mktemp /sbin/mktemp; do [ -x "$p" ] && {{ printf '%s' "$p"; break; }}; done) BRIDGE_CMD=$(for p in /usr/sbin/bridge /usr/bin/bridge /sbin/bridge /bin/bridge; do [ -x "$p" ] && {{ printf '%s' "$p"; break; }}; done) -FIREWALL_CMD=$(for p in /usr/bin/firewall-cmd /usr/sbin/firewall-cmd /bin/firewall-cmd /sbin/firewall-cmd; do [ -x "$p" ] && {{ printf '%s' "$p"; break; }}; done) SYSTEMCTL=$(for p in /usr/bin/systemctl /bin/systemctl /usr/sbin/systemctl /sbin/systemctl; do [ -x "$p" ] && {{ printf '%s' "$p"; break; }}; done) {IP_RULE_CONVERGENCE_SH} BRIDGE={_q(r.bridge)}; TABLE={_q(r.nft_table)}; ROUTE_TABLE={r.route_table}; WG_IFACE={_q(FABRIC_INTERFACE)} OWNER_FILE={_q(r.owner_file)}; STATE_FILE={_q(r.state_file)}; LOCK={_q(r.lock_file)}; TXN={_q(r.transaction_dir)} NFT_SNAPSHOT_PREFIX={_q(f'/etc/virt-cloud/vpc-dataplane/{r.token}.nft')} -NFT_FILE={_q(include)}; NFT_MANIFEST={_q(NFT_MANIFEST)}; IF_FRAGMENT={_q(fragment)}; SYSCTL_FILE={_q(sysctl_file)}; PROXY_FILE={_q(proxy_file)}; STATE_DIR=/etc/virt-cloud/vpc-dataplane +NFT_FILE={_q(include)}; LEGACY_MANIFEST={_q(LEGACY_FIREWALLD_NFT_MANIFEST)}; IF_FRAGMENT={_q(fragment)}; SYSCTL_FILE={_q(sysctl_file)}; PROXY_FILE={_q(proxy_file)}; STATE_DIR=/etc/virt-cloud/vpc-dataplane MODE={_q(deployment_mode)}; FIREWALL_MODE={_q(firewall_mode)}; IFUP={_q(ifup_path)}; IFDOWN={_q(ifdown_path)}; IFQUERY={_q(ifquery_path)}; NMCLI=; NM_PROFILE_UUID=; NM_PROFILE_FILE=; NFT_MAIN=; NFT_INCLUDE={_q(f'include "{include}"')} NM_PROFILE_MARKER={_q(nm_profile_marker)} EXPECTED_DIGEST={_q(plan.fabric_plan_digest)} -{NFT_MANIFEST_VALIDATION_SH} {NFT_LIVE_SNAPSHOT_SH} -{FIREWALL_ZONE_VALIDATION_SH} -{FIREWALL_ZONE_OWNERSHIP_SH} {_NMCLI_FILENAME_RESOLVER_SH} {NETWORKMANAGER_VPC_PROFILE_SH} {_FIREWALL_MODE_GUARD_SH} {_FABRIC_NFT_VALIDATION_SH} {_VPC_SYSCTL_VALIDATION_SH} +{VPC_NATIVE_TRANSACTION_GUARD_SH} nft_plan_owned() {{ nft_plan_text=$1 nft_plan_digest=$2 @@ -3268,9 +2770,7 @@ if [ "$MODE" = networkmanager_nmcli ]; then NMCLI=$(for p in /usr/bin/nmcli /usr/sbin/nmcli /bin/nmcli /sbin/nmcli; do [ -x "$p" ] && {{ printf '%s' "$p"; break; }}; done) [ -n "$NMCLI" ] || exit 70 fi -if [ "$FIREWALL_MODE" = firewalld_nft ]; then - [ -n "$FIREWALL_CMD" ] || exit 70 -fi +[ "$FIREWALL_MODE" = native_nftables ] || exit 71 [ ! -L /run/virt-cloud ] && [ ! -L /etc/virt-cloud ] || exit 71 mkdir -p /run/virt-cloud; chmod 700 /run/virt-cloud if [ -e "$STATE_DIR" ] || [ -L "$STATE_DIR" ]; then @@ -3289,6 +2789,14 @@ fi : >> "$LOCK"; chmod 600 "$LOCK" exec 8>>{_q(FABRIC_LOCK)}; "$FLOCK" -x 8 exec 9>>"$LOCK"; "$FLOCK" -x 9 +vc_vpc_firewall_boundary_clean || exit 79 +for vc_vpc_legacy_snapshot in inet bridge arp; do + [ ! -e "$NFT_SNAPSHOT_PREFIX.$vc_vpc_legacy_snapshot.live" ] && + [ ! -L "$NFT_SNAPSHOT_PREFIX.$vc_vpc_legacy_snapshot.live" ] || exit 79 +done +if [ -e "$TXN" ] || [ -L "$TXN" ]; then + vc_vpc_native_transaction_owned || exit 79 +fi REMOVE_BRIDGE_TOPOLOGY_CAPTURED=no; REMOVE_BRIDGE_PORTS_BEFORE= vc_remove_bridge_port_names() {{ vc_remove_ports_raw=$("$BRIDGE_CMD" link show master "$BRIDGE" 2>/dev/null) || return 1 @@ -3324,7 +2832,6 @@ if [ -f "$STATE_FILE" ]; then state_firewall_quarantine=$(awk -F'|' '$1=="firewall_mode" {{count++; value=$2}} END {{print (count + 0) "|" value}}' "$STATE_FILE") case "$state_firewall_quarantine" in '0|'|'1|native_nftables') PREVIOUS_FIREWALL_MODE=native_nftables ;; - '1|firewalld_nft') PREVIOUS_FIREWALL_MODE=firewalld_nft ;; *) exit 71 ;; esac quarantine_revision=$(awk -F'|' '$1=="revision" {{count++; value=$2}} END {{print (count + 0) "|" value}}' "$STATE_FILE") @@ -3340,37 +2847,19 @@ if [ -f "$STATE_FILE" ]; then vc_nm_profile_resolve || exit 71 vc_nm_profile_base_validate || exit 71 quarantine_memory_zone=$(vc_nm_profile_field connection.zone) || exit 71 - if [ "$PREVIOUS_FIREWALL_MODE" = firewalld_nft ]; then - case "$quarantine_memory_zone" in ''|--|trusted) : ;; *) exit 71 ;; esac - else - case "$quarantine_memory_zone" in ''|--) : ;; *) exit 71 ;; esac - fi + case "$quarantine_memory_zone" in ''|--) : ;; *) exit 71 ;; esac else [ -f "$IF_FRAGMENT" ] && [ ! -L "$IF_FRAGMENT" ] && [ "$($STAT -c '%u:%a' "$IF_FRAGMENT" 2>/dev/null)" = 0:600 ] || exit 71 - if [ "$PREVIOUS_FIREWALL_MODE" = firewalld_nft ]; then - awk -v owner="# {DATAPLANE_OWNER_MARKER} vpc={plan.vpc.vpc_id} revision=${{quarantine_revision#*|}} digest=${{quarantine_digest#*|}}" \ - -v bridge="$BRIDGE" ' - NR==1 && $0==owner {{next}} - NR==2 && $0=="iface " bridge " inet manual" {{next}} - NR==3 && $0==" bridge_ports none" {{next}} - NR==4 && $0==" bridge-stp off" {{next}} - {{bad=1}} - END {{exit (NR==4 && !bad) ? 0 : 1}} - ' "$IF_FRAGMENT" || exit 71 - [ ! -e "/etc/sysconfig/network-scripts/ifcfg-$BRIDGE" ] && - [ ! -L "/etc/sysconfig/network-scripts/ifcfg-$BRIDGE" ] || exit 71 - else - awk -v owner="# {DATAPLANE_OWNER_MARKER} vpc={plan.vpc.vpc_id} revision=${{quarantine_revision#*|}} digest=${{quarantine_digest#*|}}" \ - -v bridge="$BRIDGE" ' - NR==1 && $0==owner {{next}} - NR==2 && $0=="iface " bridge " inet manual" {{next}} - NR==3 && $0==" bridge_ports none" {{next}} - NR==4 && $0==" bridge-stp off" {{next}} - {{bad=1}} - END {{exit (NR==4 && !bad) ? 0 : 1}} - ' "$IF_FRAGMENT" || exit 71 - fi + awk -v owner="# {DATAPLANE_OWNER_MARKER} vpc={plan.vpc.vpc_id} revision=${{quarantine_revision#*|}} digest=${{quarantine_digest#*|}}" \ + -v bridge="$BRIDGE" ' + NR==1 && $0==owner {{next}} + NR==2 && $0=="iface " bridge " inet manual" {{next}} + NR==3 && $0==" bridge_ports none" {{next}} + NR==4 && $0==" bridge-stp off" {{next}} + {{bad=1}} + END {{exit (NR==4 && !bad) ? 0 : 1}} + ' "$IF_FRAGMENT" || exit 71 fi if "$IP" link show dev "$BRIDGE" >/dev/null 2>&1; then "$IP" -d link show dev "$BRIDGE" 2>/dev/null | @@ -3387,52 +2876,15 @@ if [ -f "$STATE_FILE" ]; then fi fi vc_firewall_mode_ready || exit 71 -if [ "$FIREWALL_MODE" = firewalld_nft ] && [ -f "$STATE_FILE" ]; then - [ "$PREVIOUS_FIREWALL_MODE" = firewalld_nft ] || exit 71 - if [ "$MODE" = networkmanager_nmcli ]; then - vc_firewalld_bridge_zone_expect preapply managed-nm "$BRIDGE" || exit 71 - quarantine_runtime=$VC_VPC_ZONE_RUNTIME - vc_nm_profile_base_validate || exit 71 - vc_nm_device_expect_managed no || exit 71 - if [ "$quarantine_runtime" = trusted ]; then - "$NMCLI" connection modify --temporary uuid "$NM_PROFILE_UUID" connection.zone "" >/dev/null || exit 72 - if "$FIREWALL_CMD" --zone=trusted --query-interface="$BRIDGE" >/dev/null 2>&1; then - "$FIREWALL_CMD" --zone=trusted --remove-interface="$BRIDGE" >/dev/null || exit 72 - else - zone_query_status=$? - [ "$zone_query_status" = 1 ] || exit 72 - fi - fi - vc_nm_profile_expect quarantined || exit 72 - else - vc_firewalld_bridge_zone_expect preapply managed "$BRIDGE" || exit 71 - quarantine_runtime=$VC_VPC_ZONE_RUNTIME - if [ "$quarantine_runtime" = trusted ]; then - "$FIREWALL_CMD" --zone=trusted --remove-interface="$BRIDGE" >/dev/null || exit 72 - fi - vc_firewalld_bridge_zone_expect blocked managed "$BRIDGE" || exit 72 - fi - if [ "$REMOVE_BRIDGE_TOPOLOGY_CAPTURED" = yes ]; then - vc_remove_bridge_topology_matches || exit 72 - fi -elif [ "$FIREWALL_MODE" = firewalld_nft ]; then - # No panel state means there is no authority to take over a same-name - # interface. Only an already blocked/absent binding is acceptable. - vc_firewalld_bridge_zone_expect blocked absent "$BRIDGE" || exit 71 -fi [ -f /etc/virt-cloud/wg-fabric0.owner ] && grep -Fqx 'virt-cloud-wg-fabric-v1' /etc/virt-cloud/wg-fabric0.owner || exit 71 "$WG" show "$WG_IFACE" >/dev/null 2>&1 || exit 71 BRIDGE_PRESENT=no if "$IP" link show dev "$BRIDGE" >/dev/null 2>&1; then BRIDGE_PRESENT=yes; fi PHASE=preparing; RESUME=no if [ -e "$TXN" ]; then - [ -d "$TXN" ] && [ ! -L "$TXN" ] || exit 79 - [ "$($STAT -c '%u:%a' "$TXN" 2>/dev/null)" = 0:700 ] || exit 79 - [ -f "$TXN/inflight" ] && [ ! -L "$TXN/inflight" ] || exit 79 - [ "$($STAT -c '%u:%a' "$TXN/inflight" 2>/dev/null)" = 0:600 ] || exit 79 - grep -Fqx '{inflight_marker}' "$TXN/inflight" || exit 79 - [ -f "$TXN/state" ] && [ ! -L "$TXN/state" ] || exit 79 - [ "$($STAT -c '%u:%a' "$TXN/state" 2>/dev/null)" = 0:600 ] || exit 79 + vc_vpc_native_transaction_owned || exit 79 + vc_vpc_transaction_record_owned "$TXN/inflight" '{inflight_marker}' || exit 79 + vc_vpc_transaction_file_secure "$TXN/state" 64 || exit 79 PHASE=$(cat "$TXN/state" 2>/dev/null) || exit 79 case "$PHASE" in preparing|routes_removed|manager_removed|bridge_removed|nft_removed|files_removed|wg_merged|committed) : ;; @@ -3481,29 +2933,6 @@ vc_vpc_artifact_identity_check() {{ fi return 0 }} -vc_vpc_persisted_snapshot_artifacts_ready() {{ - [ -f "$STATE_FILE" ] && [ ! -L "$STATE_FILE" ] || return 1 - for snapshot_family in inet bridge arp; do - snapshot_path="$NFT_SNAPSHOT_PREFIX.$snapshot_family.live" - [ -f "$snapshot_path" ] && [ ! -L "$snapshot_path" ] || return 1 - [ "$($STAT -c '%u:%a' "$snapshot_path" 2>/dev/null)" = 0:600 ] || return 1 - snapshot_record=$(awk -F'|' -v family="$snapshot_family" \ - -v path="$snapshot_path" ' - $1=="nft_snapshot" && $2==family && $3==path {{count++; marker=$4}} - END {{print (count + 0) "|" marker}} - ' "$STATE_FILE") || return 1 - [ "${{snapshot_record%%|*}}" = 1 ] || return 1 - printf '%s\n' "${{snapshot_record#*|}}" | - grep -Eq '^{DATAPLANE_OWNER_MARKER}:{r.token}:[0-9a-f]{{16}}$' || return 1 - done -}} -vc_vpc_persisted_snapshots_match() {{ - vc_vpc_persisted_snapshot_artifacts_ready || return 1 - for snapshot_family in inet bridge arp; do - vc_nft_table_matches_snapshot "$snapshot_family" "$TABLE" \ - "$NFT_SNAPSHOT_PREFIX.$snapshot_family.live" || return 1 - done -}} vc_remove_capture_nft_precommit() {{ vc_remove_precommit_prefix=$1 for vc_remove_precommit_family in inet bridge arp; do @@ -3559,25 +2988,17 @@ if [ -f "$STATE_FILE" ]; then grep -Fqx 'bridge|{r.bridge}' "$STATE_FILE" || exit 71 grep -Fqx 'table|{r.route_table}' "$STATE_FILE" || exit 71 state_firewall_record=$(awk -F'|' '$1=="firewall_mode" {{count++; value=$2}} END {{print (count + 0) "|" value}}' "$STATE_FILE") - if [ "$FIREWALL_MODE" = native_nftables ]; then - case "$state_firewall_record" in '0|'|'1|native_nftables') : ;; *) exit 71 ;; esac - else - [ "$state_firewall_record" = "1|$FIREWALL_MODE" ] || exit 71 - fi + case "$state_firewall_record" in '0|'|'1|native_nftables') : ;; *) exit 71 ;; esac state_revision=$(awk -F'|' '$1=="revision" {{print $2; exit}}' "$STATE_FILE") state_digest=$(awk -F'|' '$1=="digest" {{print $2; exit}}' "$STATE_FILE") case "$state_revision" in ''|*[!0-9]*) exit 79;; esac printf '%s\n' "$state_digest" | grep -Eq '^[0-9a-f]{{64}}$' || exit 79 [ "$state_revision" -le {plan.desired_revision} ] || exit 79 if [ "$state_revision" = {plan.desired_revision} ] && [ "$state_digest" != { _q(plan.desired_digest) } ]; then exit 79; fi - if [ "$FIREWALL_MODE" = firewalld_nft ]; then - vc_vpc_persisted_snapshot_artifacts_ready || exit 71 - else - for snapshot_family in inet bridge arp; do - [ ! -e "$NFT_SNAPSHOT_PREFIX.$snapshot_family.live" ] && - [ ! -L "$NFT_SNAPSHOT_PREFIX.$snapshot_family.live" ] || exit 71 - done - fi + for snapshot_family in inet bridge arp; do + [ ! -e "$NFT_SNAPSHOT_PREFIX.$snapshot_family.live" ] && + [ ! -L "$NFT_SNAPSHOT_PREFIX.$snapshot_family.live" ] || exit 71 + done elif [ -f "$OWNER_FILE" ] && [ "$BRIDGE_PRESENT" = yes ]; then # A live panel bridge without its state cannot be reconciled safely. exit 71 @@ -3634,20 +3055,13 @@ if [ "$MODE" = networkmanager_nmcli ]; then if [ "$PHASE" = preparing ] || [ "$PHASE" = routes_removed ]; then if vc_nm_profile_resolve; then vc_nm_profile_base_validate || exit 71 - if [ "$FIREWALL_MODE" = firewalld_nft ]; then - [ -f "$STATE_FILE" ] || exit 71 - vc_nm_profile_expect preapply || exit 71 - else - vc_nm_profile_empty connection.zone || exit 71 - fi + vc_nm_profile_empty connection.zone || exit 71 else nm_resolve_status=$? [ "$nm_resolve_status" = 2 ] || exit 71 if [ "$BRIDGE_PRESENT" = yes ]; then # An unmanaged live bridge is foreign/ambiguous; do not delete it. exit 71 - elif [ "$FIREWALL_MODE" = firewalld_nft ]; then - vc_firewalld_bridge_zone_expect blocked absent "$BRIDGE" || exit 71 fi fi else @@ -3655,17 +3069,6 @@ if [ "$MODE" = networkmanager_nmcli ]; then fi else if [ "$PHASE" = preparing ] || [ "$PHASE" = routes_removed ]; then - if [ "$FIREWALL_MODE" = firewalld_nft ]; then - if [ -e "/etc/sysconfig/network-scripts/ifcfg-$BRIDGE" ] || - [ -L "/etc/sysconfig/network-scripts/ifcfg-$BRIDGE" ]; then - exit 71 - fi - if [ -f "$STATE_FILE" ] && {{ [ "$BRIDGE_PRESENT" = yes ] || [ -e "$IF_FRAGMENT" ]; }}; then - vc_firewalld_bridge_zone_expect preapply managed "$BRIDGE" || exit 71 - else - vc_firewalld_bridge_zone_expect blocked absent "$BRIDGE" || exit 71 - fi - fi if [ "$BRIDGE_PRESENT" = yes ] || [ -e "$IF_FRAGMENT" ]; then [ -f "$IF_FRAGMENT" ] || exit 71 [ ! -L /etc/network ] && [ ! -L /etc/network/interfaces ] && [ ! -L /etc/network/interfaces.d ] || exit 71 @@ -3677,29 +3080,6 @@ else fi fi fi -if [ "$RESUME" = no ] && [ "$FIREWALL_MODE" = firewalld_nft ]; then - # Static state/snapshot ownership and the exact manager artifact have now - # been proven. Withdraw runtime trusted before the first live ACL compare; - # a missing or semantically altered table must fail with the bridge blocked. - if [ "$MODE" = networkmanager_nmcli ] && [ -n "$NM_PROFILE_UUID" ]; then - nm_remove_zone=$(vc_nm_profile_field connection.zone) || exit 71 - if [ "$nm_remove_zone" = trusted ]; then - vc_nm_profile_active || exit 71 - "$NMCLI" connection modify --temporary uuid "$NM_PROFILE_UUID" connection.zone "" >/dev/null || exit 72 - "$NMCLI" device reapply "$BRIDGE" >/dev/null || exit 72 - elif [ -n "$nm_remove_zone" ] && [ "$nm_remove_zone" != -- ]; then - exit 71 - fi - vc_nm_profile_expect quarantined || exit 72 - else - vc_firewalld_bridge_zone_expect preapply managed "$BRIDGE" || exit 71 - if [ "$VC_VPC_ZONE_RUNTIME" = trusted ]; then - "$FIREWALL_CMD" --zone=trusted --remove-interface="$BRIDGE" >/dev/null || exit 72 - fi - vc_firewalld_bridge_zone_expect blocked managed "$BRIDGE" || exit 72 - fi - vc_vpc_persisted_snapshots_match || exit 71 -fi if [ "$PHASE" = preparing ] || [ "$PHASE" = routes_removed ] || [ "$PHASE" = manager_removed ]; then if "$IP" link show dev "$BRIDGE" >/dev/null 2>&1; then [ -f "$OWNER_FILE" ] || exit 71 @@ -3713,29 +3093,18 @@ if [ "$PHASE" = preparing ] || [ "$PHASE" = routes_removed ] || [ "$PHASE" = man else if "$IP" link show dev "$BRIDGE" >/dev/null 2>&1; then exit 79; fi fi -if [ "$FIREWALL_MODE" = native_nftables ]; then - NFT_MAIN_COUNT=0 - for nft_candidate in /etc/nftables.conf /etc/sysconfig/nftables.conf; do - if [ -f "$nft_candidate" ] && [ ! -L "$nft_candidate" ] && [ "$($STAT -c '%u' "$nft_candidate" 2>/dev/null)" = 0 ]; then - case "$($STAT -c '%a' "$nft_candidate" 2>/dev/null)" in - [0-7][0145][0145]) : ;; - *) continue ;; - esac - NFT_MAIN="$nft_candidate"; NFT_MAIN_COUNT=$((NFT_MAIN_COUNT + 1)) - fi - done - [ "$NFT_MAIN_COUNT" -eq 1 ] || exit 71 - [ ! -e "$NFT_MANIFEST" ] && [ ! -L "$NFT_MANIFEST" ] || exit 71 -else - [ "$FIREWALL_MODE" = firewalld_nft ] || exit 71 - vc_nft_manifest_validate "$NFT_MANIFEST" || exit 71 - NFT_MAIN="$NFT_MANIFEST" - for nft_candidate in /etc/nftables.conf /etc/sysconfig/nftables.conf; do - [ -e "$nft_candidate" ] || [ -L "$nft_candidate" ] || continue - [ -f "$nft_candidate" ] && [ ! -L "$nft_candidate" ] || exit 71 - grep -Eq '/etc/nftables[.]d/(virt-cloud-wg-fabric[.]nft|virt-cloud-vpc-[0-9a-f]+[.]nft|[*]([.]nft)?)' "$nft_candidate" && exit 71 || true - done -fi +NFT_MAIN_COUNT=0 +for nft_candidate in /etc/nftables.conf /etc/sysconfig/nftables.conf; do + if [ -f "$nft_candidate" ] && [ ! -L "$nft_candidate" ] && [ "$($STAT -c '%u' "$nft_candidate" 2>/dev/null)" = 0 ]; then + case "$($STAT -c '%a' "$nft_candidate" 2>/dev/null)" in + [0-7][0145][0145]) : ;; + *) continue ;; + esac + NFT_MAIN="$nft_candidate"; NFT_MAIN_COUNT=$((NFT_MAIN_COUNT + 1)) + fi +done +[ "$NFT_MAIN_COUNT" -eq 1 ] || exit 71 +[ ! -e "$LEGACY_MANIFEST" ] && [ ! -L "$LEGACY_MANIFEST" ] || exit 71 if [ "$PHASE" = preparing ] || [ "$PHASE" = routes_removed ] || [ "$PHASE" = manager_removed ] || [ "$PHASE" = bridge_removed ]; then if [ -f "$NFT_FILE" ]; then nft_tables=$(awk '$1=="table" {{print $2 "|" $3}}' "$NFT_FILE") @@ -3792,6 +3161,7 @@ if [ "$RESUME" = yes ]; then fi if [ "$RESUME" = no ]; then mkdir -m 700 "$TXN" + vc_vpc_write_native_transaction_mode || exit 79 printf '%s\\n' '{inflight_marker}' > "$TXN/inflight"; chmod 600 "$TXN/inflight" set_phase preparing if [ -f "$STATE_FILE" ]; then cp -p "$STATE_FILE" "$TXN/state.before"; fi @@ -3873,27 +3243,14 @@ fi verify_routes_removed || exit 72 set_phase routes_removed if [ "$MODE" = networkmanager_nmcli ]; then - if [ "$FIREWALL_MODE" = firewalld_nft ] && [ -n "$NM_PROFILE_UUID" ]; then - "$NMCLI" connection delete uuid "$NM_PROFILE_UUID" >/dev/null 2>&1 || exit 72 - if "$NMCLI" connection show uuid "$NM_PROFILE_UUID" >/dev/null 2>&1; then exit 72; fi - elif [ "$FIREWALL_MODE" = native_nftables ] && [ -n "$NMCLI" ] && - "$NMCLI" connection show "$BRIDGE" >/dev/null 2>&1; then + if [ -n "$NMCLI" ] && "$NMCLI" connection show "$BRIDGE" >/dev/null 2>&1; then "$NMCLI" connection delete "$BRIDGE" >/dev/null 2>&1 || exit 72 if "$NMCLI" connection show "$BRIDGE" >/dev/null 2>&1; then exit 72; fi fi - if [ "$FIREWALL_MODE" = firewalld_nft ]; then - vc_firewalld_bridge_zone_expect blocked absent "$BRIDGE" || exit 72 - fi else if [ -f "$IF_FRAGMENT" ]; then - if [ "$FIREWALL_MODE" = firewalld_nft ]; then - vc_firewalld_bridge_zone_expect blocked managed "$BRIDGE" || exit 72 - fi "$IFQUERY" "$BRIDGE" >/dev/null 2>&1 || exit 71 "$IFDOWN" --force "$BRIDGE" >/dev/null 2>&1 || exit 72 - if [ "$FIREWALL_MODE" = firewalld_nft ]; then - vc_firewalld_bridge_zone_expect blocked managed "$BRIDGE" || exit 72 - fi fi fi set_phase manager_removed @@ -3923,25 +3280,14 @@ if [ -f "$NFT_FILE" ]; then [ -n "$family" ] || continue case "$family:$name" in inet:*|bridge:*|arp:*) : ;; *) exit 71 ;; esac [ "$name" = "$TABLE" ] || exit 71 - if [ "$FIREWALL_MODE" = firewalld_nft ]; then - vc_nft_table_matches_snapshot "$family" "$name" \ - "$NFT_SNAPSHOT_PREFIX.$family.live" || exit 71 - else - table_dump=$("$NFT" -s list table "$family" "$name" 2>/dev/null) || exit 71 - printf '%s\\n' "$table_dump" | grep -Fq 'virt-cloud-vpc-dataplane-v1' || exit 71 - fi + table_dump=$("$NFT" -s list table "$family" "$name" 2>/dev/null) || exit 71 + printf '%s\\n' "$table_dump" | grep -Fq 'virt-cloud-vpc-dataplane-v1' || exit 71 printf 'delete table %s %s\\n' "$family" "$name" >> "$NFT_REMOVE_BATCH" nft_table_count=$((nft_table_count + 1)) done </dev/null || exit 72 # Check mode has no kernel reservation/CAS primitive. Re-read all complete @@ -3954,11 +3300,7 @@ if grep -Fqx "$NFT_INCLUDE" "$NFT_MAIN"; then NFT_MAIN_TMP=$("$MKTEMP" "$NFT_MAIN.virt-cloud-vpc-remove.XXXXXX") || exit 72 awk -v needle="$NFT_INCLUDE" '$0 != needle' "$NFT_MAIN" > "$NFT_MAIN_TMP" || {{ rm -f "$NFT_MAIN_TMP"; exit 72; }} chmod "$($STAT -c '%a' "$NFT_MAIN")" "$NFT_MAIN_TMP" || {{ rm -f "$NFT_MAIN_TMP"; exit 72; }} - if [ "$FIREWALL_MODE" = native_nftables ]; then - "$NFT" -c -f "$NFT_MAIN_TMP" >/dev/null || {{ rm -f "$NFT_MAIN_TMP"; exit 72; }} - else - vc_nft_manifest_validate "$NFT_MAIN_TMP" || {{ rm -f "$NFT_MAIN_TMP"; exit 72; }} - fi + "$NFT" -c -f "$NFT_MAIN_TMP" >/dev/null || {{ rm -f "$NFT_MAIN_TMP"; exit 72; }} mv -f "$NFT_MAIN_TMP" "$NFT_MAIN" || exit 72 grep -Fqx "$NFT_INCLUDE" "$NFT_MAIN" && exit 72 || true fi @@ -3993,11 +3335,7 @@ if [ -f "$STATE_FILE" ]; then grep -Fqx 'bridge|{r.bridge}' "$STATE_FILE" || exit 71 grep -Fqx 'table|{r.route_table}' "$STATE_FILE" || exit 71 state_firewall_record=$(awk -F'|' '$1=="firewall_mode" {{count++; value=$2}} END {{print (count + 0) "|" value}}' "$STATE_FILE") - if [ "$FIREWALL_MODE" = native_nftables ]; then - case "$state_firewall_record" in '0|'|'1|native_nftables') : ;; *) exit 71 ;; esac - else - [ "$state_firewall_record" = "1|$FIREWALL_MODE" ] || exit 71 - fi + case "$state_firewall_record" in '0|'|'1|native_nftables') : ;; *) exit 71 ;; esac state_revision=$(awk -F'|' '$1=="revision" {{print $2; exit}}' "$STATE_FILE") state_digest=$(awk -F'|' '$1=="digest" {{print $2; exit}}' "$STATE_FILE") case "$state_revision" in ''|*[!0-9]*) exit 79;; esac @@ -4044,10 +3382,6 @@ grep -Fqx "$NFT_INCLUDE" "$NFT_MAIN" && exit 79 || true if [ "$MODE" = networkmanager_nmcli ] && [ -n "$NMCLI" ]; then "$NMCLI" connection show "$BRIDGE" >/dev/null 2>&1 && exit 79 || true fi -if [ "$FIREWALL_MODE" = firewalld_nft ]; then - [ "$(vc_firewalld_interface_zone runtime "$BRIDGE")" = none ] || exit 79 - [ "$(vc_firewalld_interface_zone permanent "$BRIDGE")" = none ] || exit 79 -fi [ ! -e "$STATE_FILE" ] && [ ! -e "$OWNER_FILE" ] && [ ! -e "$NFT_SNAPSHOT_PREFIX.inet.live" ] && [ ! -e "$NFT_SNAPSHOT_PREFIX.bridge.live" ] && @@ -4210,7 +3544,6 @@ NFT=/usr/sbin/nft; [ -x "$NFT" ] || NFT=/usr/bin/nft FLOCK=/usr/bin/flock; [ -x "$FLOCK" ] || FLOCK=/usr/sbin/flock STAT=/usr/bin/stat; [ -x "$STAT" ] || STAT=/bin/stat CMP=/usr/bin/cmp; [ -x "$CMP" ] || CMP=/bin/cmp -FIREWALL_CMD=$(for p in /usr/bin/firewall-cmd /usr/sbin/firewall-cmd /bin/firewall-cmd /sbin/firewall-cmd; do [ -x "$p" ] && {{ printf '%s' "$p"; break; }}; done) SYSTEMCTL=$(for p in /usr/bin/systemctl /bin/systemctl /usr/sbin/systemctl /sbin/systemctl; do [ -x "$p" ] && {{ printf '%s' "$p"; break; }}; done) LOCK=/run/virt-cloud/wg-fabric0.lock if [ -L /run/virt-cloud ] || {{ [ -e /run/virt-cloud ] && [ ! -d /run/virt-cloud ]; }}; then exit 71; fi @@ -4221,20 +3554,18 @@ BRIDGE_CMD= for p in /usr/sbin/bridge /usr/bin/bridge /sbin/bridge /bin/bridge; do if [ -x "$p" ]; then BRIDGE_CMD=$p; break; fi done -BRIDGE={_q(r.bridge)}; TABLE={_q(r.nft_table)}; ROUTE_TABLE={r.route_table}; WG_IFACE={_q(FABRIC_INTERFACE)}; OWNER={_q(r.owner_file)}; STATE={_q(r.state_file)}; PROXY_FILE={_q(proxy_file)} +BRIDGE={_q(r.bridge)}; TABLE={_q(r.nft_table)}; ROUTE_TABLE={r.route_table}; WG_IFACE={_q(FABRIC_INTERFACE)}; OWNER={_q(r.owner_file)}; STATE={_q(r.state_file)}; TXN={_q(r.transaction_dir)}; PROXY_FILE={_q(proxy_file)} NFT_SNAPSHOT_PREFIX={_q(f'/etc/virt-cloud/vpc-dataplane/{r.token}.nft')} -NFT_FILE={_q(nft_file)}; NFT_INCLUDE={_q(nft_include)}; NFT_MANIFEST={_q(NFT_MANIFEST)}; SYSCTL_FILE={_q(sysctl_file)}; IF_FRAGMENT={_q(ifup_fragment)}; NM_PROFILE_MARKER={_q(nm_profile_marker)}; FIREWALL_MODE={_q(firewall_mode)}; NFT_MAIN=; NFT_MAIN_COUNT=0; NM_PROFILE_UUID=; NM_PROFILE_FILE= +NFT_FILE={_q(nft_file)}; NFT_INCLUDE={_q(nft_include)}; LEGACY_MANIFEST={_q(LEGACY_FIREWALLD_NFT_MANIFEST)}; SYSCTL_FILE={_q(sysctl_file)}; IF_FRAGMENT={_q(ifup_fragment)}; NM_PROFILE_MARKER={_q(nm_profile_marker)}; FIREWALL_MODE={_q(firewall_mode)}; NFT_MAIN=; NFT_MAIN_COUNT=0; NM_PROFILE_UUID=; NM_PROFILE_FILE= NFT_MARKER={_q(nft_marker)} EXPECTED_DIGEST={_q(plan.fabric_plan_digest)} -{NFT_MANIFEST_VALIDATION_SH} {NFT_LIVE_SNAPSHOT_SH} -{FIREWALL_ZONE_VALIDATION_SH} -{FIREWALL_ZONE_OWNERSHIP_SH} {_NMCLI_FILENAME_RESOLVER_SH} {NETWORKMANAGER_VPC_PROFILE_SH} {_FIREWALL_MODE_GUARD_SH} {_FABRIC_NFT_VALIDATION_SH} {_VPC_SYSCTL_VALIDATION_SH} +{VPC_NATIVE_TRANSACTION_GUARD_SH} nft_plan_owned() {{ nft_plan_text=$1 nft_plan_digest=$2 @@ -4272,37 +3603,25 @@ vc_vpc_fabric_observe_ready() {{ }} firewall_mode_state=owned vc_firewall_mode_ready || firewall_mode_state=mismatch +[ "$FIREWALL_MODE" = native_nftables ] || firewall_mode_state=mismatch +vc_vpc_firewall_boundary_clean || firewall_mode_state=mismatch +[ ! -e "$TXN" ] && [ ! -L "$TXN" ] || firewall_mode_state=mismatch owner=missing; [ -f "$OWNER" ] && owner=$(cat "$OWNER" 2>/dev/null || printf missing) printf '%s\\n' "__VC_VPC_BRIDGE_OWNER__=$owner" persistence_target=no opposite_clean=yes -if [ "$FIREWALL_MODE" = native_nftables ]; then - for nft_candidate in /etc/nftables.conf /etc/sysconfig/nftables.conf; do - if [ -f "$nft_candidate" ] && [ ! -L "$nft_candidate" ] && [ "$($STAT -c '%u' "$nft_candidate" 2>/dev/null)" = 0 ]; then - case "$($STAT -c '%a' "$nft_candidate" 2>/dev/null)" in - [0-7][0145][0145]) NFT_MAIN="$nft_candidate"; NFT_MAIN_COUNT=$((NFT_MAIN_COUNT + 1)) ;; - *) : ;; - esac - fi - done - if [ -e "$NFT_MANIFEST" ] || [ -L "$NFT_MANIFEST" ]; then opposite_clean=no; fi - if [ "$NFT_MAIN_COUNT" = 1 ] && grep -Fqx "$NFT_INCLUDE" "$NFT_MAIN" && - "$NFT" -c -f "$NFT_MAIN" >/dev/null 2>&1; then - persistence_target=yes - fi -else - if [ "$FIREWALL_MODE" = firewalld_nft ] && vc_nft_manifest_validate "$NFT_MANIFEST"; then - NFT_MAIN="$NFT_MANIFEST"; NFT_MAIN_COUNT=1 - grep -Fqx "$NFT_INCLUDE" "$NFT_MAIN" && persistence_target=yes +for nft_candidate in /etc/nftables.conf /etc/sysconfig/nftables.conf; do + if [ -f "$nft_candidate" ] && [ ! -L "$nft_candidate" ] && [ "$($STAT -c '%u' "$nft_candidate" 2>/dev/null)" = 0 ]; then + case "$($STAT -c '%a' "$nft_candidate" 2>/dev/null)" in + [0-7][0145][0145]) NFT_MAIN="$nft_candidate"; NFT_MAIN_COUNT=$((NFT_MAIN_COUNT + 1)) ;; + *) : ;; + esac fi - for nft_candidate in /etc/nftables.conf /etc/sysconfig/nftables.conf; do - [ -e "$nft_candidate" ] || [ -L "$nft_candidate" ] || continue - if [ ! -f "$nft_candidate" ] || [ -L "$nft_candidate" ]; then - opposite_clean=no - continue - fi - grep -Eq '/etc/nftables[.]d/(virt-cloud-wg-fabric[.]nft|virt-cloud-vpc-[0-9a-f]+[.]nft|[*]([.]nft)?)' "$nft_candidate" && opposite_clean=no || true - done +done +if [ -e "$LEGACY_MANIFEST" ] || [ -L "$LEGACY_MANIFEST" ]; then opposite_clean=no; fi +if [ "$NFT_MAIN_COUNT" = 1 ] && grep -Fqx "$NFT_INCLUDE" "$NFT_MAIN" && + "$NFT" -c -f "$NFT_MAIN" >/dev/null 2>&1; then + persistence_target=yes fi persistence=no nft_file_ok=no @@ -4335,73 +3654,31 @@ if [ -n "$NMCLI" ] && [ "$nm_profile_count" = 1 ]; then nm6_method=$($NMCLI -t --escape no -g ipv6.method connection show "$BRIDGE" 2>/dev/null | sed -n '1p') nm_zone=$($NMCLI -t --escape no -g connection.zone connection show "$BRIDGE" 2>/dev/null | sed -n '1p') if [ "$nm_stable" = "$NM_PROFILE_MARKER" ] && [ "$nm_type" = bridge ] && [ "$nm_iface" = "$BRIDGE" ] && [ "$nm_method" = disabled ] && [ "$nm6_method" = disabled ]; then - if [ "$FIREWALL_MODE" = firewalld_nft ]; then - if vc_nm_profile_resolve && vc_nm_profile_expect active; then nm_owned=yes; fi - else - case "$nm_zone" in ''|--) nm_owned=yes ;; esac - fi + if vc_nm_profile_resolve && vc_nm_profile_expect active; then nm_owned=yes; fi fi fi ifup_owned=no if [ -f "$IF_FRAGMENT" ] && [ ! -L "$IF_FRAGMENT" ] && [ "$(stat -c '%u:%a' "$IF_FRAGMENT" 2>/dev/null)" = 0:600 ] && grep -Fqx '# {DATAPLANE_OWNER_MARKER} vpc={plan.vpc.vpc_id} revision={plan.desired_revision} digest={plan.desired_digest}' "$IF_FRAGMENT" && grep -Fqx 'iface {r.bridge} inet manual' "$IF_FRAGMENT"; then - if ! grep -Eq '^[[:space:]]*(auto|allow-[^[:space:]]+)[[:space:]]+{r.bridge}([[:space:]]|$)' "$IF_FRAGMENT" && - {{ [ "$FIREWALL_MODE" != firewalld_nft ] || - {{ [ ! -e "/etc/sysconfig/network-scripts/ifcfg-$BRIDGE" ] && - [ ! -L "/etc/sysconfig/network-scripts/ifcfg-$BRIDGE" ]; }}; }}; then + if ! grep -Eq '^[[:space:]]*(auto|allow-[^[:space:]]+)[[:space:]]+{r.bridge}([[:space:]]|$)' "$IF_FRAGMENT"; then ifup_owned=yes fi fi -firewall_zone=yes -if [ "$FIREWALL_MODE" = firewalld_nft ]; then - firewall_zone=no - runtime_zone=$(vc_firewalld_interface_zone runtime "$BRIDGE" 2>/dev/null || printf error) - runtime_trusted=no - if [ "$runtime_zone" = trusted ] && - "$FIREWALL_CMD" --zone=trusted --query-interface="$BRIDGE" >/dev/null 2>&1; then - runtime_trusted=yes - fi - permanent_zone=$(vc_firewalld_interface_zone permanent "$BRIDGE" 2>/dev/null || printf error) - if [ "$nm_owned" = yes ] && [ "$runtime_trusted" = yes ] && - [ "$permanent_zone" = none ] && [ "$ifup_owned" = no ]; then - firewall_zone=yes - elif [ "$ifup_owned" = yes ] && [ "$runtime_trusted" = yes ] && - [ "$permanent_zone" = none ] && [ "$nm_present" = no ]; then - firewall_zone=yes - fi -fi printf '%s\\n' "__VC_VPC_FIREWALL_MODE__=$FIREWALL_MODE" printf '%s\\n' "__VC_VPC_FIREWALL_STATE__=$firewall_mode_state" -printf '%s\\n' "__VC_VPC_FIREWALL_ZONE__=$firewall_zone" nft_snapshot_state=yes -if [ "$FIREWALL_MODE" = firewalld_nft ]; then - nft_snapshot_state=no - if [ -n "$CMP" ] && [ -f "$STATE" ] && [ ! -L "$STATE" ]; then - nft_snapshot_state=yes - for snapshot_family in inet bridge arp; do - snapshot_path="$NFT_SNAPSHOT_PREFIX.$snapshot_family.live" - snapshot_record=$(awk -F'|' -v family="$snapshot_family" \ - -v path="$snapshot_path" ' - $1=="nft_snapshot" && $2==family && $3==path {{count++; marker=$4}} - END {{print (count + 0) "|" marker}} - ' "$STATE" 2>/dev/null || printf invalid) - if [ "${{snapshot_record%%|*}}" != 1 ] || - ! printf '%s\\n' "${{snapshot_record#*|}}" | - grep -Eq '^{DATAPLANE_OWNER_MARKER}:{r.token}:[0-9a-f]{{16}}$' || - [ ! -f "$snapshot_path" ] || [ -L "$snapshot_path" ] || - [ "$($STAT -c '%u:%a' "$snapshot_path" 2>/dev/null)" != 0:600 ] || - ! vc_nft_table_matches_snapshot "$snapshot_family" "$TABLE" "$snapshot_path"; then - nft_snapshot_state=no - fi - done +for snapshot_family in inet bridge arp; do + snapshot_path="$NFT_SNAPSHOT_PREFIX.$snapshot_family.live" + if [ -e "$snapshot_path" ] || [ -L "$snapshot_path" ]; then + nft_snapshot_state=no fi -fi +done printf '%s\\n' "__VC_VPC_NFT_SNAPSHOT__=$nft_snapshot_state" if [ "$firewall_mode_state" = owned ] && [ "$NFT_MAIN_COUNT" = 1 ] && [ "$persistence_target" = yes ] && [ "$opposite_clean" = yes ] && [ "$nft_file_ok" = yes ] && [ "$sysctl_file_ok" = yes ] && - [ "$nft_snapshot_state" = yes ] && [ "$firewall_zone" = yes ] && + [ "$nft_snapshot_state" = yes ] && ( [ "$nm_owned" = yes ] && [ "$ifup_owned" = no ] || [ "$nm_owned" = no ] && [ "$nm_present" = no ] && [ "$ifup_owned" = yes ] ); then persistence=yes @@ -4475,14 +3752,10 @@ if [ -f "$STATE" ] && grep -Fqx 'owner|{DATAPLANE_OWNER_MARKER}' "$STATE"; then printf '%s\\n' "__VC_VPC_STATE_DIGEST__=$(awk -F'|' '$1==\"digest\"{{print $2; exit}}' \"$STATE\")" state_firewall_record=$(awk -F'|' '$1=="firewall_mode" {{count++; value=$2}} END {{print (count + 0) "|" value}}' "$STATE") state_firewall_observed=invalid - if [ "$FIREWALL_MODE" = native_nftables ]; then - case "$state_firewall_record" in - '0|') state_firewall_observed= ;; - '1|native_nftables') state_firewall_observed=native_nftables ;; - esac - elif [ "$state_firewall_record" = "1|$FIREWALL_MODE" ]; then - state_firewall_observed=$FIREWALL_MODE - fi + case "$state_firewall_record" in + '0|') state_firewall_observed= ;; + '1|native_nftables') state_firewall_observed=native_nftables ;; + esac printf '%s\\n' "__VC_VPC_STATE_FIREWALL_MODE__=$state_firewall_observed" else printf '%s\\n' '__VC_VPC_STATE_OWNER__=missing' @@ -5146,9 +4419,6 @@ class VpcPrivateHostAdapter: and (parsed.get("state_firewall_mode") == firewall_mode or (firewall_mode == FIREWALL_MODE_NATIVE and parsed.get("state_firewall_mode") in {None, ""})) - and (parsed.get("firewall_zone") == "yes" - or (firewall_mode == FIREWALL_MODE_NATIVE - and parsed.get("firewall_zone") is None)) and parsed.get("persistence") == "yes" and parsed.get("bridge_boundary") == "yes" and parsed.get("proxy_arp") == "0" @@ -5230,4 +4500,4 @@ class VpcPrivateHostAdapter: ).as_dict() -__all__ = ["DATAPLANE_OWNER_MARKER", "FABRIC_INTERFACE", "PRIVATE_INTERFACE", "FIREWALL_MODE_NATIVE", "FIREWALL_MODE_FIREWALLD", "NFT_MANIFEST", "VpcDataplaneError", "VpcFabricPeer", "VpcDataplaneResources", "VpcDataplanePlan", "VpcDataplaneObservation", "derive_private_mac", "private_guest_routes", "private_guest_network_generation", "resource_names", "parse_vpc_dataplane_plan", "render_vpc_nft_rules", "render_apply_script", "render_remove_script", "render_observe_script", "patch_private_interface_xml", "VpcPrivateHostAdapter"] +__all__ = ["DATAPLANE_OWNER_MARKER", "FABRIC_INTERFACE", "PRIVATE_INTERFACE", "FIREWALL_MODE_NATIVE", "VpcDataplaneError", "VpcFabricPeer", "VpcDataplaneResources", "VpcDataplanePlan", "VpcDataplaneObservation", "derive_private_mac", "private_guest_routes", "private_guest_network_generation", "resource_names", "parse_vpc_dataplane_plan", "render_vpc_nft_rules", "render_apply_script", "render_remove_script", "render_observe_script", "patch_private_interface_xml", "VpcPrivateHostAdapter"] diff --git a/web/app/Services/Cloud/CloudAdminFabricService.php b/web/app/Services/Cloud/CloudAdminFabricService.php index 26e18afe..2c209c1e 100644 --- a/web/app/Services/Cloud/CloudAdminFabricService.php +++ b/web/app/Services/Cloud/CloudAdminFabricService.php @@ -67,8 +67,10 @@ final class CloudAdminFabricService 'fabric_endpoint_invalid', 'fabric_endpoint_missing', 'fabric_endpoint_mismatch', 'fabric_endpoint_not_local', 'fabric_endpoint_vpc_conflict', 'fabric_vpc_record_corrupt', - 'firewalld_endpoint_zone_unverified', - 'firewalld_endpoint_zone_unsafe_or_drifted', + 'firewalld_unsupported_requires_maintenance', + 'legacy_firewalld_state_requires_cleanup', + 'firewall_transaction_requires_recovery_or_cleanup', + 'legacy_firewall_evidence_invalid', 'fabric_host_identity_mismatch', 'fabric_host_interface_conflict', 'fabric_host_firewall_mode_conflict', 'fabric_host_key_locked', 'fabric_host_network_manager_conflict', @@ -1693,9 +1695,7 @@ final class CloudAdminFabricService ? $item['capabilities'] : []; foreach (['os_id', 'os_version', 'network_manager', 'network_manager_expected', 'deployment_mode', 'firewall_manager', - 'firewall_mode', 'underlay_interface', - 'underlay_runtime_zone', 'underlay_permanent_zone', - 'public_selector'] as $key) { + 'firewall_mode', 'underlay_interface', 'public_selector'] as $key) { if (is_string($rawCapabilities[$key] ?? null) && strlen($rawCapabilities[$key]) <= 128 && preg_match('/[\x00-\x1f\x7f]/', $rawCapabilities[$key]) !== 1) { @@ -1725,6 +1725,18 @@ final class CloudAdminFabricService $capabilities[$key] = null; } } + $nativeNftables = ($rawCapabilities['firewall_manager'] ?? null) + === 'nftables' + && ($rawCapabilities['firewall_mode'] ?? null) + === 'native_nftables'; + if (!$nativeNftables) { + $reasons[] = (($rawCapabilities['firewall_manager'] ?? null) + === 'firewalld' + || ($rawCapabilities['firewall_mode'] ?? null) + === 'firewalld_nft') + ? 'firewalld_unsupported_requires_maintenance' + : 'firewall_mode_unsupported'; + } $result[$hostId] = [ 'configured' => ($item['configured'] ?? false) === true, 'reachable' => ($item['reachable'] ?? false) === true, @@ -1732,7 +1744,8 @@ final class CloudAdminFabricService 'endpoint_ip' => $endpoint, 'endpoint_source' => $endpointSource, 'probed_at' => max(0, (int) ($item['probed_at'] ?? time())), - 'vpc_ready' => ($item['vpc_ready'] ?? false) === true, + 'vpc_ready' => $nativeNftables + && ($item['vpc_ready'] ?? false) === true, 'reasons' => array_values(array_unique($reasons)), 'capabilities' => $capabilities, ]; diff --git a/web/app/Services/Cloud/CloudEngineClient.php b/web/app/Services/Cloud/CloudEngineClient.php index 93e95486..37e8ea20 100644 --- a/web/app/Services/Cloud/CloudEngineClient.php +++ b/web/app/Services/Cloud/CloudEngineClient.php @@ -308,8 +308,10 @@ final class CloudEngineClient 'fabric_endpoint_rp_filter_unverified' => '无法确认 Fabric 传输接口的 rp_filter', 'fabric_endpoint_vpc_conflict' => 'Fabric 传输地址与现有 VPC CIDR 冲突', 'fabric_vpc_record_corrupt' => 'VPC 资源目录损坏,无法确认 Fabric 地址安全性', - 'firewalld_endpoint_zone_unverified' => '无法确认 Fabric 传输接口的 firewalld 区域', - 'firewalld_endpoint_zone_unsafe_or_drifted' => 'Fabric 传输接口的 firewalld 运行态与持久区域不一致或不安全', + 'firewalld_unsupported_requires_maintenance' => 'firewalld 不再受支持;请在维护窗口清理旧所有权状态并启用原生 nftables', + 'legacy_firewalld_state_requires_cleanup' => '检测到旧 firewalld 规则或事务状态;必须在维护窗口备份并清理后重建原生 nftables 数据面', + 'firewall_transaction_requires_recovery_or_cleanup' => '检测到未收敛的防火墙事务;请先完成恢复或在维护窗口清理', + 'legacy_firewall_evidence_invalid' => '无法安全核验宿主的旧防火墙状态;已禁止 VPC 变更', 'fabric_host_firewall_mode_conflict' => '宿主机防火墙与 Fabric 部署模式冲突', 'fabric_host_network_manager_conflict' => '宿主机网络管理器与 Fabric 部署模式冲突', 'fabric_identity_conflict' => 'Fabric 宿主机公钥身份冲突', diff --git a/web/public/cloud/admin-fabric.js b/web/public/cloud/admin-fabric.js index 58268436..e3c6f70d 100644 --- a/web/public/cloud/admin-fabric.js +++ b/web/public/cloud/admin-fabric.js @@ -202,16 +202,10 @@ const reasonLabels = { firewall_mode_unsupported: '防火墙运行模式不受支持', firewall_manager_conflict: 'firewalld 与 nftables 服务存在所有权冲突', firewall_competing_service_unknown: '无法排除另一防火墙服务在重启后接管', - firewalld_service_not_active: 'firewalld 服务未运行', - firewalld_service_not_enabled: 'firewalld 服务未启用', - firewalld_command_missing: '缺少 firewall-cmd', - firewalld_backend_not_nftables: 'firewalld 后端不是 nftables', - firewalld_table_owner_not_enabled: 'firewalld 未启用 nftables 表所有权保护', - firewalld_runtime_nft_backend_unverified: '无法核验 firewalld 当前 nft 表的所有权与持久属性', - firewalld_default_zone_unsafe_or_drifted: 'firewalld 默认区域不安全,或运行态与持久配置不一致', - firewalld_endpoint_zone_unverified: '无法确认 Fabric 传输地址对应的宿主机接口和 firewalld 区域', - firewalld_endpoint_zone_unsafe_or_drifted: 'Fabric 传输接口的 firewalld 运行态与持久区域不一致或使用了 trusted', - firewalld_source_zones_present_or_unknown: 'firewalld 存在或无法排除按来源地址选区,无法安全定位 underlay 规则', + firewalld_unsupported_requires_maintenance: 'firewalld 已不受支持;请在维护窗口清理旧所有权状态并启用原生 nftables', + legacy_firewalld_state_requires_cleanup: '检测到旧 firewalld 规则或事务状态;请在维护窗口备份并清理后重建', + firewall_transaction_requires_recovery_or_cleanup: '检测到未收敛的防火墙事务;请先完成恢复或在维护窗口清理', + legacy_firewall_evidence_invalid: '无法安全核验宿主的旧防火墙状态;已禁止 VPC 变更', nftables_service_not_active: 'nftables 服务未运行', nftables_service_not_enabled: 'nftables 服务未启用', ipv4_forwarding_disabled_or_unknown: 'IPv4 forwarding 未启用或无法核验', @@ -751,7 +745,6 @@ function renderHosts(region) { '时间:' + date(host.probed_at), ])); const manager = el('td'); - const firewalldNft = capabilities.firewall_mode === 'firewalld_nft'; manager.append(compactLines([ '系统:' + text(capabilities.os_id) + ' ' + text(capabilities.os_version, ''), 'Manager:' + text(capabilities.network_manager), @@ -766,8 +759,8 @@ function renderHosts(region) { 'WireGuard:' + flag(capabilities.wireguard_ready, '就绪', '未就绪'), '接口归属:' + flag(capabilities.wireguard_owned, '平台', '冲突/未归属'), 'nftables:' + flag(capabilities.nftables_ready, '就绪', '未就绪'), - 'nft 服务:' + (firewalldNft ? '由 firewalld 管理' : flag(capabilities.nftables_active)), - '规则恢复:' + (firewalldNft ? '平台 manifest + 修复服务' : flag(capabilities.nftables_persistent)), + 'nft 服务:' + flag(capabilities.nftables_active), + '规则恢复:' + flag(capabilities.nftables_persistent), ])); const publicPath = el('td'); publicPath.append(compactLines([ diff --git a/web/resources/views/venix/layout_assets.tpl b/web/resources/views/venix/layout_assets.tpl index 898cb235..61061322 100644 --- a/web/resources/views/venix/layout_assets.tpl +++ b/web/resources/views/venix/layout_assets.tpl @@ -1,2 +1,2 @@ {* Shared Venix CSS/JS cache key. Increment whenever a shared browser asset changes. *} -{assign var=venix_asset_version value='2026100907' scope='global'} +{assign var=venix_asset_version value='2026101001' scope='global'} diff --git a/web/tests/AdminFabricReadinessWorkflowTest.php b/web/tests/AdminFabricReadinessWorkflowTest.php index 0c3f192b..2dd9b9ed 100644 --- a/web/tests/AdminFabricReadinessWorkflowTest.php +++ b/web/tests/AdminFabricReadinessWorkflowTest.php @@ -631,13 +631,6 @@ CloudRegionFabric::query()->delete(); $probeRows = [ $probeHost(1, [ 'endpoint_source' => 'configured', - 'capabilities' => [ - 'firewall_manager' => 'firewalld', - 'firewall_mode' => 'firewalld_nft', - 'nftables_ready' => true, - 'nftables_active' => false, - 'nftables_persistent' => false, - ], ]), $probeHost(2, [ 'reachable' => false, @@ -646,14 +639,16 @@ $probeRows = [ ]), $probeHost(3, [ 'endpoint_ip' => '10.20.0.10', - 'vpc_ready' => false, - 'reasons' => ['firewall_owner_not_nftables'], + 'vpc_ready' => true, + 'reasons' => [], 'capabilities' => [ 'firewall_manager' => 'firewalld', - 'firewall_mode' => 'unsupported', - 'nftables_ready' => false, + 'firewall_mode' => 'firewalld_nft', + 'nftables_ready' => true, 'nftables_active' => false, 'nftables_persistent' => false, + 'underlay_runtime_zone' => 'public', + 'underlay_permanent_zone' => 'public', 'rp_filter_all' => 3, 'rp_filter_default' => '1', 'rp_filter_fabric_raw' => true, @@ -677,10 +672,10 @@ $assert(count($rows) === 6 && ($hostById($rows, 1)['readiness_state'] ?? null) = $assert(($hostById($rows, 1)['endpoint_source'] ?? null) === 'configured', 'An explicit effective Fabric endpoint source was not projected.'); $assert(($hostById($rows, 1)['capabilities']['firewall_mode'] ?? null) - === 'firewalld_nft' + === 'native_nftables' && ($hostById($rows, 1)['capabilities']['firewall_manager'] ?? null) - === 'firewalld', - 'A supported firewalld+nft host was not preserved as ready.'); + === 'nftables', + 'A native nftables host was not preserved as ready.'); $rpFilterCapabilities = $hostById($rows, 1)['capabilities'] ?? []; $assert(($rpFilterCapabilities['rp_filter'] ?? null) === 0 && ($rpFilterCapabilities['rp_filter_all'] ?? null) === 0 @@ -704,13 +699,18 @@ $assert(is_array($failed) && $failed['probe_checked'] === true 'The isolated failed-host result was dropped or promoted.'); $assert(is_array($firewalld) && ($firewalld['capabilities']['firewall_manager'] ?? null) === 'firewalld' + && ($firewalld['capabilities']['firewall_mode'] ?? null) === 'firewalld_nft' + && ($firewalld['vpc_ready'] ?? true) === false + && !array_key_exists('underlay_runtime_zone', $firewalld['capabilities']) + && !array_key_exists('underlay_permanent_zone', $firewalld['capabilities']) && !array_key_exists('rp_filter_all', $firewalld['capabilities']) && !array_key_exists('rp_filter_default', $firewalld['capabilities']) && !array_key_exists('rp_filter_fabric_raw', $firewalld['capabilities']) - && in_array('firewall_owner_not_nftables', $firewalld['reasons'], true) + && in_array('firewalld_unsupported_requires_maintenance', + $firewalld['reasons'], true) && in_array('fabric_endpoint_vpc_pool_conflict', $firewalld['reasons'], true), - 'A firewall or tenant-pool endpoint blocker was not preserved in readiness.'); + 'A legacy firewalld result was not failed closed with its endpoint blocker.'); $assert(is_array($nonIpv4) && $nonIpv4['endpoint_ip'] === null && $nonIpv4['endpoint_source'] === null && in_array('fabric_endpoint_unavailable', $nonIpv4['reasons'], true), @@ -768,9 +768,10 @@ $blockedWizard = CloudAdminFabricService::probeRegion('1', $actor, $engine([ $response([]), $response(['hosts' => [ $probeHost(1, [ - 'vpc_ready' => false, - 'reasons' => ['firewall_owner_not_nftables'], - 'capabilities' => ['firewall_manager' => 'firewalld'], + 'capabilities' => [ + 'firewall_manager' => 'firewalld', + 'firewall_mode' => 'firewalld_nft', + ], ]), $probeHost(2), ]]), @@ -781,9 +782,9 @@ $assert(($blockedWorkflow['stage'] ?? null) === 'blocked' && ($blockedWorkflow['can_apply'] ?? true) === false && ($blockedWorkflow['can_reconcile'] ?? true) === false && ($blockedWorkflow['action_mode'] ?? null) === 'probe' - && in_array('firewall_owner_not_nftables', + && in_array('firewalld_unsupported_requires_maintenance', $blockedWorkflow['blocking_reasons'] ?? [], true), - 'A failed fresh host probe did not keep the wizard blocked.'); + 'A legacy firewalld probe did not keep the wizard blocked.'); // A fresh successful detection is the only path to ready_to_apply. The // browser receives a bounded server-owned preview, never nodes or endpoints. @@ -942,11 +943,12 @@ $assert($paths === [ $missingEndpointSource = $probeHost(1); unset($missingEndpointSource['endpoint_source']); $blockedProbeSets = [ - 'firewalld' => [ + 'legacy firewalld mode' => [ $probeHost(1, [ - 'vpc_ready' => false, - 'reasons' => ['firewall_owner_not_nftables'], - 'capabilities' => ['firewall_manager' => 'firewalld'], + 'capabilities' => [ + 'firewall_manager' => 'firewalld', + 'firewall_mode' => 'firewalld_nft', + ], ]), $probeHost(2), ], @@ -1506,8 +1508,11 @@ $client = $engine([ $response(['hosts' => [ $probeHost(1, [ 'vpc_ready' => false, - 'reasons' => ['firewall_owner_not_nftables'], - 'capabilities' => ['firewall_manager' => 'firewalld'], + 'reasons' => ['firewalld_unsupported_requires_maintenance'], + 'capabilities' => [ + 'firewall_manager' => 'firewalld', + 'firewall_mode' => 'unsupported', + ], ]), $probeHost(2), ]]), @@ -1570,8 +1575,11 @@ $activeProbeFailure = CloudAdminFabricService::probeRegion('1', $actor, $engine( $probeHost(1, [ 'endpoint_ip' => $overlappingEndpoint, 'vpc_ready' => false, - 'reasons' => ['firewall_owner_not_nftables'], - 'capabilities' => ['firewall_manager' => 'firewalld'], + 'reasons' => ['firewalld_unsupported_requires_maintenance'], + 'capabilities' => [ + 'firewall_manager' => 'firewalld', + 'firewall_mode' => 'unsupported', + ], ]), $probeHost(2), ]]), @@ -1582,7 +1590,7 @@ $assert(($activeBlockedWorkflow['stage'] ?? null) === 'blocked' && ($activeBlockedWorkflow['can_apply'] ?? true) === false && ($activeBlockedWorkflow['can_reconcile'] ?? true) === false && ($activeBlockedWorkflow['action_mode'] ?? null) === 'probe' - && in_array('firewall_owner_not_nftables', + && in_array('firewalld_unsupported_requires_maintenance', $activeBlockedWorkflow['blocking_reasons'] ?? [], true), 'A failed complete fresh probe left active Fabric reconcile enabled.'); diff --git a/web/tests/CloudFirewallUiContractTest.php b/web/tests/CloudFirewallUiContractTest.php index 046d470a..a405e67e 100644 --- a/web/tests/CloudFirewallUiContractTest.php +++ b/web/tests/CloudFirewallUiContractTest.php @@ -18,7 +18,7 @@ $assert(str_contains($template, 'data-instance-setting-tab="firewall"') && str_contains($template, 'id="vm-firewall-rules"'), 'Instance settings must expose the integrated firewall tab and whole-policy editor.'); -$assert(str_contains($assets, "venix_asset_version value='2026100907'"), +$assert(str_contains($assets, "venix_asset_version value='2026101001'"), 'Shared Venix asset cache key must publish the firewall JavaScript and CSS.'); $assert(str_contains($template, '反欺骗保护始终开启') diff --git a/web/tests/CloudHostFabricEndpointContractTest.php b/web/tests/CloudHostFabricEndpointContractTest.php index 59c7b056..2d6987c4 100644 --- a/web/tests/CloudHostFabricEndpointContractTest.php +++ b/web/tests/CloudHostFabricEndpointContractTest.php @@ -48,6 +48,18 @@ $assert(str_contains($engineClient, "'host_fabric_endpoint_invalid'") && str_contains($engineClient, "'fabric_host_firewall_mode_conflict'") && str_contains($engineClient, "'vpc_dataplane_firewall_mode_conflict'"), 'Reviewed endpoint validation failures are not safely mapped for administrators.'); +$assert(str_contains($engineClient, + "'firewalld_unsupported_requires_maintenance'") + && str_contains($engineClient, + "'legacy_firewalld_state_requires_cleanup'") + && str_contains($engineClient, + "'firewall_transaction_requires_recovery_or_cleanup'") + && str_contains($engineClient, + "'legacy_firewall_evidence_invalid'") + && !str_contains($engineClient, "'firewalld_endpoint_zone_unverified'") + && !str_contains($engineClient, + "'firewalld_endpoint_zone_unsafe_or_drifted'"), + 'Retired firewalld compatibility errors remain in the web boundary.'); $assert(str_contains($template, 'id="host-fabric-endpoint-ip"') && preg_match('/id="host-fabric-endpoint-ip"[^>]*maxlength="15"/', $template) === 1 @@ -73,10 +85,16 @@ $assert(str_contains($fabricService, 'private const ENDPOINT_SOURCES') && str_contains($fabricService, 'CloudPrivateCidrAllocator::normalize') && str_contains($fabricService, "'firewall_mode'") && str_contains($fabricService, "'public_selector'") + && str_contains($fabricService, "=== 'nftables'") + && str_contains($fabricService, "=== 'native_nftables'") + && str_contains($fabricService, + "? 'firewalld_unsupported_requires_maintenance'") && str_contains($fabricService, "'endpoint_source' => \$endpointSource") && str_contains($fabricService, "'endpoint_source' => \$probe['endpoint_source'] ?? null") + && !str_contains($fabricService, "'underlay_runtime_zone'") + && !str_contains($fabricService, "'underlay_permanent_zone'") && !str_contains($fabricService, "'private_key'") && !str_contains($fabricService, "'ssh_command'"), 'Fabric readiness does not expose a bounded effective endpoint source.'); @@ -90,14 +108,21 @@ $assert(str_contains($fabricClient, 'function endpointSource(value)') && str_contains($fabricClient, 'fabric_endpoint_source_invalid') && str_contains($fabricClient, 'fabric_endpoint_vpc_pool_conflict') && str_contains($fabricClient, 'fabric_region_conflict') - && str_contains($fabricClient, 'firewalld_backend_not_nftables') - && str_contains($fabricClient, 'firewalld_runtime_nft_backend_unverified') - && str_contains($fabricClient, 'firewalld_default_zone_unsafe_or_drifted') - && str_contains($fabricClient, 'firewalld_source_zones_present_or_unknown') + && str_contains($fabricClient, 'firewalld_unsupported_requires_maintenance') + && str_contains($fabricClient, 'legacy_firewalld_state_requires_cleanup') + && str_contains($fabricClient, + 'firewall_transaction_requires_recovery_or_cleanup') + && str_contains($fabricClient, 'legacy_firewall_evidence_invalid') + && !str_contains($fabricClient, 'firewalld_backend_not_nftables') + && !str_contains($fabricClient, 'firewalld_runtime_nft_backend_unverified') + && !str_contains($fabricClient, 'firewalld_default_zone_unsafe_or_drifted') + && !str_contains($fabricClient, 'firewalld_source_zones_present_or_unknown') + && !str_contains($fabricClient, 'const firewalldNft') + && !str_contains($fabricClient, '由 firewalld 管理') && str_contains($fabricClient, 'firewall_competing_service_unknown') && str_contains($fabricClient, 'firewall_manager_conflict'), 'Fabric readiness UI does not distinguish explicit and SSH-IPv4 fallback endpoints.'); -$assert(str_contains($assets, "venix_asset_version value='2026100907'"), +$assert(str_contains($assets, "venix_asset_version value='2026101001'"), 'Changed host/Fabric browser assets did not advance the shared cache key.'); echo "Cloud host Fabric endpoint contracts passed.\n";