You cannot select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
virt/docs/TRAFFIC_POOL.md

9.2 KiB

Commercial line traffic-pool contract

Scope

The traffic pool is a prepaid accounting projection. It does not replace orders or service periods. Every purchase or renewal still creates one independent cloud_service_periods row, and that period is the only source of traffic entitlement.

A pool is identified by (user_id, traffic_pool_group_id). A traffic-pool group is a stable commercial line identity. Different groups never share quota, even when their hosts or public bridges are technically interchangeable. Orders and service periods freeze the group identity at purchase time.

One commercial line group may bind multiple region-local cloud_network_lines. Region is coverage metadata, not an accounting boundary: neither region_id nor a region code participates in the pool key. Every physical network line belongs to at most one commercial line group, and the user read model exposes each group exactly once with its ordered coverage regions and current instances. Two lines with the same display name or code do not share traffic unless they are explicitly bound to the same group.

The first release does not support moving an active service period between traffic-pool groups. Host maintenance migration is permitted only when the source and target remain in the same group.

Entitlements

Each service period materializes at most one entitlement.

  • A future period is scheduled and contributes no usable bytes.
  • At starts_at, its complete finite quota becomes active exactly once.
  • A paid upgrade may only increase the active entitlement. The positive difference keeps the original ends_at and is idempotent by upgrade source.
  • At ends_at, only that entitlement's unused bytes are forfeited. Previously consumed bytes remain immutable history.
  • Deleting an instance early does not erase its paid entitlement. Other instances in the same pool may consume it until the original ends_at.
  • An unlimited period is excluded from the finite pool: its instance neither contributes infinite quota nor consumes finite entitlements.
  • A maintenance migration or an in-place restore never creates another entitlement. A backup copy receives quota only through a new paid service period.

Usage allocation

Only the Agent's durable cumulative public-interface RX and TX counters are billable. VPC-interface traffic is outside this contract. PHP persists those counters monotonically before traffic-pool settlement.

New usage is allocated across active finite entitlements in strict FEFO order:

ends_at ASC, entitlement_id ASC

The instance that generated the traffic does not own the consumed entitlement. Entitlement origin remains audit metadata only.

All mutations for one pool serialize under a single pool row lock. The lock order is always:

pool -> instance cursor -> entitlements

The accounting transaction advances the instance cursor, updates entitlement and pool projections, and accumulates the exact allocation in a durable UTC daily settlement bucket atomically. A completed UTC day, entitlement exhaustion, expiry, overage settlement or attention transition seals the bucket into one immutable ledger event. Every bucket receives a persisted random 128-bit event identity when it is created. That identity, rather than the reusable auto-increment row id, is the immutable event and compaction identity. The writer and retention verifier derive the same canonical event envelope from one shared module, so a row-id reset cannot collide with prior ledger history and payload drift blocks compaction. The window is fixed by the schema and is not carried as mutable row data. Late observations start a new revision instead of rewriting an already sealed event. This reduces permanent ledger growth while preserving minute-level FEFO allocation and exact RX/TX attribution. It never locks instances, writes network restrictions, or calls an Agent. A later per-instance policy transaction reads the pool projection and owns the separate lock order:

instance -> network policy state -> network restrictions (reason_code ASC)

Traffic collection also maintains a compact cumulative checkpoint tier: one first/last row per logical instance and UTC day. When lifecycle activation selects the first trustworthy observation after a service-period boundary, it freezes that cumulative total in a sparse immutable anchor keyed by the exact service period. The anchor, rather than an indefinitely retained hourly row, is the durable proof used after maintenance migration or later compaction. Hourly checkpoints remain hot for at least 72 hours. Retention removes an old row only when the matching daily first/last counters contain it, every open service-period boundary that could depend on it has a persisted base or valid anchor, and the pool cursor is not in attention. A missing proof retains the hourly row and raises a retention warning; it never guesses a baseline.

The policy state is one aggregate execution projection per instance; reason rows contain facts only. This avoids assigning one NIC mutation to an arbitrary restriction when several reasons are active at the same time.

Repeated observations and repeated lifecycle finalization must have no new effect. A counter regression is an attention condition; it must not refund traffic or charge the observed total from zero.

Sampling overage and expiry boundaries

Polling cannot stop traffic at an exact byte. If an observation exceeds the available pool balance, the remaining entitlements are consumed, the excess is recorded as overage_bytes, and the pool becomes exhausted. A later finite entitlement first settles that debt before exposing usable bytes. Overage is never silently discarded.

At ends_at, an entitlement immediately stops contributing visible or allocatable balance. A bounded closing interval may settle observations whose trusted observed_at is not later than ends_at. Closing must end after at most two normal collection intervals; a missing host cannot keep a paid period or pool permanently locked.

Network policy

Traffic accounting owns only the traffic_pool_exhausted network restriction. Service expiry and administrator policy have independent reasons. Public network access is desired only when no active restriction exists.

When a pool is exhausted, all active finite instances in that user and group receive the traffic restriction. When usable balance returns, accounting removes only its own restriction. It must never clear service_expired, admin_hold, or another module's restriction.

Remote network mutations use the existing durable job and operation-identity contracts. A timeout or malformed receipt has unknown effect: retain the original operation identifier for reconciliation and never automatically replay it.

Read model

User and administrator pages read a pool snapshot produced by the accounting module. They do not sum entitlements themselves.

The user page groups by commercial line identity, never by region. A cross-region group is one line pool with coverage_regions; each instance may also expose its placement region for context. Missing placement metadata or a placement line whose group differs from the service period's frozen identity is an attention state, not permission to infer or rewrite the identity.

The primary user values are:

  • currently active finite quota;
  • consumption from currently active entitlements;
  • usable balance;
  • unsettled sampling overage;
  • the next expiry time and the unused bytes that will be forfeited then;
  • last trusted observation and network-restriction state.

Lifetime or calendar-month usage is a separate statistic. It must not be divided by the currently active quota because expired entitlements legitimately make historical usage greater than the quota still active today.

Conservation and recovery invariants

For every entitlement:

granted_bytes = consumed_bytes + remaining_bytes + forfeited_bytes

For accepted traffic observations:

observed_delta = allocated_consumption + overage_delta

No projection may become negative. Sealed immutable ledger events plus any still-open durable settlement buckets are the accounting audit source; pool, entitlement and cursor projections must be verifiable from that combined state. Projection mismatch fences automatic network restoration for that pool but does not delete the ledger or issue another remote mutation.

Before enforcement, the feature may run in shadow mode. After traffic has been shared across instances, rolling back to per-instance balances is not safe; recovery must pause new network mutations, retain the ledger and repair or rebuild the projection forward.

Changing the presentation from region grouping to commercial line grouping does not migrate accounting data. Never merge existing shadow or enforced groups, rewrite frozen service periods, or manufacture entitlements with SQL. Only a disabled group with no durable references may have its physical-line membership changed through the revision-checked administrator operation.

The executable deployment, readiness and incident procedure is defined in TRAFFIC_POOL_ROLLOUT.md. In particular, php xcat cloudInit is not an incremental upgrade command and old service periods must not be moved into a pool by SQL backfill.