Commit graph

15480 commits

Author SHA1 Message Date
pulse-triage[bot]
da0d19cd60
Release v6.6.0-rc.5 (#3073)
Some checks failed
Build and Test / Provider pair Docker acceptance (push) Has been cancelled
Build and Test / Frontend (push) Has been cancelled
Build and Test / Compile backend race tests (no execution) (push) Has been cancelled
Build and Test / Backend tests (rest-0) (push) Has been cancelled
Build and Test / Backend tests (rest-1) (push) Has been cancelled
Build and Test / Backend tests (api-0) (push) Has been cancelled
Build and Test / Backend tests (api-1) (push) Has been cancelled
Build and Test / Backend tests (api-2) (push) Has been cancelled
Build and Test / Backend tests (api-3) (push) Has been cancelled
Build and Test / Backend tests (api-4) (push) Has been cancelled
Build and Test / Backend tests (api) (push) Has been cancelled
Build and Test / Script smoke tests (shard 0/2) (push) Has been cancelled
Build and Test / Script smoke tests (shard 1/2) (push) Has been cancelled
Build and Test / Script smoke tests & backend build (push) Has been cancelled
Build and Test / Workload chart 90ms SLO (push) Has been cancelled
Build and Test / Benchmarks (push) Has been cancelled
Public docs / check (push) Has been cancelled
Core E2E Tests / Validate E2E tier selection (push) Has been cancelled
Core E2E Tests / Offline Organization provisioning (push) Has been cancelled
Core E2E Tests / Build source-bound E2E images once (push) Has been cancelled
Core E2E Tests / Playwright Core E2E (shard 1/8) (push) Has been cancelled
Core E2E Tests / Playwright Core E2E (shard 2/8) (push) Has been cancelled
Core E2E Tests / Playwright Core E2E (shard 3/8) (push) Has been cancelled
Core E2E Tests / Playwright Core E2E (shard 4/8) (push) Has been cancelled
Core E2E Tests / Playwright Core E2E (shard 5/8) (push) Has been cancelled
Core E2E Tests / Playwright Core E2E (shard 6/8) (push) Has been cancelled
Core E2E Tests / Playwright Core E2E (shard 7/8) (push) Has been cancelled
Core E2E Tests / Playwright Core E2E (shard 8/8) (push) Has been cancelled
Core E2E Tests / Agent registration lifecycle (push) Has been cancelled
Core E2E Tests / E2E verdict (push) Has been cancelled
Release preparation for `v6.6.0-rc.5`. After this merges, the release
workflow runs at this exact head
`e7fffbbf9c`.
2026-10-19 10:16:05 +00:00
pulse-triage[bot]
e7fffbbf9c Prepare v6.6.0-rc.5
Release candidate cut from main at 1c2f23aef6. It carries every change merged since stable v6.5.0.

Contract-Neutral: Release-only v6.6.0-rc.5 version, release notes and distribution metadata; no public install or runtime contract changes.
Change-source: pulse-maintainer
2026-10-19 10:58:43 +01:00
rcourtman
1c2f23aef6
Merge pull request #3072: Record strict chart latency proof
Retain the passing strict two-CPU chart proof, exact source and executable identities, complete capacity receipts and previous adverse results.
2026-10-19 10:24:17 +01:00
rcourtman
208762072b Record source-bound chart latency qualification
Retain the resource projection source checks and strict two-CPU chart
measurement with exact source, control and executable identities, complete
capacity receipts and earlier adverse outcomes. Keep the 90 ms budget and
the broader performance lane residual unchanged.
2026-10-19 10:14:01 +01:00
rcourtman
062a114a04
Merge pull request #3071: Reduce retained query resource decoding
Use request-local resource numbers in raw fleet reads and restore stable series order in the shared reader. Preserve the strict 90 ms chart budget.
2026-10-19 10:05:38 +01:00
rcourtman
7a622c0b76 Reduce repeated resource decoding in retained metrics reads
Raw fleet reads copied resource IDs across SQLite for every observation and
merged tier branches by those repeated strings. Bind the requested scope once,
return statement-local resource numbers and restore each detached series to
stable timestamp order in the shared reader. Preserve indexed tier coverage,
metric filters, chunk limits and ordered streamed display buckets.
2026-10-19 09:37:00 +01:00
rcourtman
5153c4ccda
Cluster retained metrics storage to remove chart row lookups (#3070)
Some checks are pending
Build and Test / Backend tests (api-2) (push) Blocked by required conditions
Build and Test / Backend tests (api-3) (push) Blocked by required conditions
Build and Test / Backend tests (api-4) (push) Blocked by required conditions
Build and Test / Backend tests (api) (push) Blocked by required conditions
Build and Test / Script smoke tests (shard 0/2) (push) Blocked by required conditions
Build and Test / Script smoke tests (shard 1/2) (push) Blocked by required conditions
Build and Test / Script smoke tests & backend build (push) Blocked by required conditions
Build and Test / Workload chart 90ms SLO (push) Blocked by required conditions
Build and Test / Benchmarks (push) Blocked by required conditions
Canonical Governance / governance (push) Waiting to run
Canonical Private Governance / private-governance (push) Waiting to run
Public docs / Documentation recipes (0) (push) Waiting to run
Public docs / Documentation recipes (1) (push) Waiting to run
Public docs / Documentation recipes (2) (push) Waiting to run
Public docs / Documentation recipes (3) (push) Waiting to run
Public docs / check (push) Blocked by required conditions
Core E2E Tests / Detect browser receipt-only changes (push) Waiting to run
Core E2E Tests / Validate E2E tier selection (push) Blocked by required conditions
Core E2E Tests / Offline Organization provisioning (push) Blocked by required conditions
Core E2E Tests / Build source-bound E2E images once (push) Blocked by required conditions
Core E2E Tests / Playwright Core E2E (shard 1/8) (push) Blocked by required conditions
Core E2E Tests / Playwright Core E2E (shard 2/8) (push) Blocked by required conditions
Core E2E Tests / Playwright Core E2E (shard 3/8) (push) Blocked by required conditions
Core E2E Tests / Playwright Core E2E (shard 4/8) (push) Blocked by required conditions
Core E2E Tests / Playwright Core E2E (shard 5/8) (push) Blocked by required conditions
Core E2E Tests / Playwright Core E2E (shard 6/8) (push) Blocked by required conditions
Core E2E Tests / Playwright Core E2E (shard 7/8) (push) Blocked by required conditions
Core E2E Tests / Playwright Core E2E (shard 8/8) (push) Blocked by required conditions
Core E2E Tests / Agent registration lifecycle (push) Blocked by required conditions
Core E2E Tests / E2E verdict (push) Blocked by required conditions
Use clustered range reads and atomic migration for retained metrics, with bounded rollup aggregation and preserved write budgets.
2026-10-19 01:36:33 +01:00
rcourtman
e9dbca7c2b Reuse bounded rollup query parameters
Bind the source tier and time window once for both clustered rollup stages.
Reusing the ordinary aggregate's six SQLite slots removes the duplicated
argument array and boxing that exceeded the small-rollup allocation budget.
Preserve indexed ranges, aggregation results and the legacy query path.
2026-10-19 01:08:37 +01:00
rcourtman
fc58c597ca Read clustered rollups through bounded primary ranges
Use the covering time index to discover resources in each rollup window,
then aggregate their contiguous primary-key ranges. Group on a query-local
resource number to avoid repeating composite identity comparisons for every
point. Preserve the legacy aggregate until its physical migration completes.

Keep nullable extrema, bucket boundaries, resource identities and idempotence
equivalent to the legacy aggregate, and guard both indexed query stages.
2026-10-19 00:31:47 +01:00
rcourtman
6209d03d5a
Keep guest HTTP tests on namespaced fixture identities (#3069)
PR3063 already isolated guest HTTP scenarios through endpoint
namespaces. Remove PR3068’s redundant global-registry resets and helper
move, keeping the existing namespace regression that checks earlier
uncertainty fences remain intact.

The Proxmox files return byte-for-byte to PR3068’s first parent. Its
installer reader-readiness correction stays in place. This is a
test-only composition correction.

Validation: full Proxmox package and normal commit hooks on Linux.
Required source checks remain enforced.

Refs #3063 and #3068.
2026-10-18 23:56:23 +01:00
rcourtman
5c37e620b4 Cluster retained metrics storage to remove chart row lookups
Retained chart reads found each sample in the identity index and then fetched its payload from the rowid table. Store observations in the time-major primary table to remove that second lookup while preserving sample identity and retention.

Migrate installed stores through deferred atomic WAL maintenance, preserving old read snapshots, queued writes, nullable bounds, tier reconciliation and rollup metadata. Share the rewrite deadline with auto-vacuum and retain the installed schema when capacity or migration checks fail.

Keep the existing write ceiling, query-cache bounds, chart sampling and 90 ms SLO unchanged. Qualify migration cancellation and crash recovery, cross-Store cached reads, restart and production checkpoint behavior.
2026-10-18 23:49:08 +01:00
rcourtman
487569d8a1 Keep guest HTTP tests on namespaced fixture identities
PR3063 landed endpoint namespaces while PR3068 was being prepared from an
older release base. Keep that existing fixture identity boundary and its
uncertainty-fence regression, and remove the redundant global-registry
resets and helper move added by PR3068.

The Proxmox files return to the first parent of PR3068. The installer
reader-readiness correction remains in place. This keeps one HTTP fixture
isolation mechanism and preserves earlier uncertainty fences throughout
the tests. Production guest admission and release controls are unchanged.

Refs #3063, #3068
2026-10-18 23:28:44 +01:00
rcourtman
ed4a2c6b54
Stabilize private bootstrap and guest-agent HTTP fixtures (#3068)
Release preparation #3065 failed on two fixture boundaries: the PTY
could signal Bash before its read setup finished, and a guest-agent HTTP
case could inherit a previous fixture’s cooldown after ephemeral port
reuse. Wait for the actual blocked reader and isolate the registry per
serial HTTP scenario, retaining shared admission across clients and
aliases within each scenario.

Controlled regressions fail before the fixes and pass afterward,
covering premature signal delivery and inherited cooldown state. This
changes test fixtures only and preserves the production installer,
guest-agent safeguards and release qualification controls.

Validation:
- Full config API and Proxmox packages passed on Linux.
- Real BusyBox input and blocked-reader regression passed five
repetitions.
- Guest-agent tests passed with the race detector.
- Status and subsystem registry audits passed.

Refs #3065.
2026-10-18 23:23:53 +01:00
rcourtman
d87c98cfab Stabilize private bootstrap and guest-agent HTTP fixtures
The two CI attempts for release preparation #3065 failed in fixture setup:
the PTY signalled a running Bash reader before input setup completed, and
an HTTP fixture inherited an earlier origin's uncertainty cooldown when
an ephemeral port was reused.

Wait for noncanonical, silent terminal input and a blocked Linux reader
before signalling. Isolate the shared guest-agent registry per serial
HTTP scenario while retaining admission sharing across its clients and
aliases. Move the existing isolation helper to the shared fixture file
and update its callers. Production guard and installer code are unchanged.

Both new regressions reproduce the old failures before the fixture fixes.
Retain those failures and the earlier unsuccessful compile proof. Source
test stabilization is separate from release qualification and publication.

Refs #3065
2026-10-18 22:57:44 +01:00
rcourtman
8a590102fe
Register the focused workload chart idle-worker proof (#3066)
Register the private manual workload-chart proof workflow and its
operating guide under the existing L10 performance contract. The focused
route uses an accepted disposable runner, literal public main checkout
with exact-source verification, and the unchanged strict 90 ms proof and
idle-capacity checks.

Its explicit verification policy names the private release latency gate
tests, which exercise the actual workflow guard. Source qualification,
release qualification and the broader open performance follow-up remain
separate. This ownership change does not claim a passing 90 ms
measurement.

Validation: canonical status and registry audits, plus the private
source-guard and workflow trust checks.
2026-10-18 22:39:59 +01:00
rcourtman
d66d6cf62a Register the workload chart disposable-worker proof
Hosted measurements had competing CPU activity and could not establish the
existing 90 ms local SLO. Give the focused disposable-worker route explicit
L10 ownership and source-guard verification without changing qualification
budgets or closing the broader performance follow-up.
2026-10-18 22:32:04 +01:00
rcourtman
5561db2e85
Reduce retained chart read overhead (#3063)
Remove unnecessary retained-read sorting, duplicate extrema decoding and
chart point copies while preserving canonical samples and full extrema APIs.
Isolate guest HTTP test fixtures from earlier endpoint cooldowns.

Keep the idle-worker 90 ms verification and existing headroom follow-up open.
2026-10-18 22:15:55 +01:00
rcourtman
667e98082e Isolate guest HTTP fixtures from retained endpoint cooldowns
Reuse the existing per-case endpoint namespace helper for guest HTTP-status
tests so a recycled loopback address cannot inherit another fixture's
intentional uncertainty fence. Keep diagnostic and cluster clients within
each fixture on the same identity, preserving all runtime cooldown rules.

A shared-listener control reproduces the old bare-endpoint failure and
verifies independent first commands plus unchanged earlier fences. Targeted
and full Proxmox tests pass. Record this separate test-stabilization slice.
2026-10-18 21:45:43 +01:00
rcourtman
03adb23556 Avoid unused extrema and copies in persisted chart reads
Use the canonical timestamp/value model directly from the shared retained
read engine. Preserve the full extrema reader, tier reconciliation, Docker
observation provenance, filters and chart sampling while eliminating unused
column transport and a second series copy.

Verify both projections, detached reads and chart sampling equivalence.
Record the measured memory reduction and retain inconclusive timing and
failed capacity evidence. The idle-worker 90 ms proof remains open.
2026-10-18 21:37:54 +01:00
rcourtman
ea3d71a799 Avoid duplicate extrema decoding in retained metric reads
Keep absent bounds nullable across the SQLite row boundary and apply their
existing value fallback during decoding. Preserve explicit bounds and the
same fallback before SQL and streamed display aggregation.

The full metrics suite passes. Ten alternating diagnostic samples reduce
allocations by 24.59% and bytes by 4.24%; their latency difference remains
inconclusive. Preserve the strict candidate failure and competing-capacity
receipt without claiming a qualified 90 ms result.

Contract-Neutral: Retained chart values, extrema, tier reconciliation, sampling and cache boundaries remain unchanged; only decoding allocation cost and capacity-error reporting change.
2026-10-18 20:52:20 +01:00
rcourtman
0a6eb9b541 Run the absolute chart SLO independently of baseline context
Execute the fixed candidate directly at the existing 90 ms budget. Keep
relative comparisons in the paired benchmark job so an invalid baseline
cannot suppress the candidate check.

Retain per-process CPU and measurement I/O counters. Require quiet I/O
before admission while counting the test's own reads in its latency.
Preserve the rejected hosted measurement and inconclusive tier-pruning
experiment as adverse evidence.

Contract-Neutral: Only verification harness and evidence change; chart payloads, retained reconciliation, latency budgets and required source/release gates are unchanged.
2026-10-18 20:27:01 +01:00
rcourtman
4fe662f309
Record maintenance qualification and remaining recovery evidence (#3062)
Record PR2888's exact source qualification and protected-main landing
evidence in the existing L1 reliability follow-up. Retain the failed CI
attempts, native inventory and authentication repairs, advisory
benchmark evidence and the proof boundaries, then distinguish the
remaining release, installed-workflow and operational outcomes.

[PR2888](https://github.com/rcourtman/Pulse/pull/2888) landed as
`eeda20c0dd` from qualified head
`b6e56c5baa`. All nine required checks,
Core E2E and the six-job native matrix passed. Intel accounted for 532
passes and 13 explicit platform skips with no failures. The advisory
comparison failed on legacy empty history filtering, +10.82%. Its actual
samples, source identities and repeatability/attribution residual remain
in the existing record and follow-up. No required check or release gate
changed.

Reconcile the separately landed notification-label repair from its
owning record. Reliability retains its current priority, completion
states and manual exit decision. The recovery and change investigation
plan stays deferred.

Validation: status, ownership and contract audits, the normal governance
commit checks, and a structural comparison confirming that only L1's
completion summary and its existing follow-up summary change in the
typed roadmap.
2026-10-18 19:42:02 +01:00
rcourtman
2a77054035 Fix filtered chart reads and verify the idle-worker SLO
Stream ungrouped retained reads in the existing time-major index order,
avoiding temporary metric sorts without changing tier reconciliation or
chart samples. Extend query-plan and filtered-result coverage.

Add an explicit strict mode for the existing 90 ms API budget and an
optional fresh-worker CI proof with fixed executable identities and raw
capacity evidence. Keep ordinary hosted budgets and required gates intact.

Contract-Neutral: Chart payloads, retained-tier reconciliation, sampling, index layout and source/release gates are unchanged; the implementation removes an unnecessary sort and adds controlled verification of existing latency budgets.
2026-10-18 19:13:30 +01:00
rcourtman
a4398b56f2 Record maintenance source qualification and remaining recovery
Bind the completed source blocker to its exact checks and protected-main
landing. Keep release inclusion, installed recovery and the existing manual
reliability exit decision explicit without changing priority or lane state.
2026-10-18 18:50:41 +01:00
rcourtman
eeda20c0dd
Keep monitoring readings, trusted connections and recovery guidance reliable (#2888)
Some checks failed
Canonical Governance / governance (push) Waiting to run
Canonical Private Governance / private-governance (push) Waiting to run
Helm CI / Lint and Render Chart (push) Waiting to run
Provider MSP startup / Provider MSP production startup (push) Waiting to run
Public docs / Documentation recipes (0) (push) Waiting to run
Public docs / Documentation recipes (1) (push) Waiting to run
Public docs / Documentation recipes (2) (push) Waiting to run
Public docs / Documentation recipes (3) (push) Waiting to run
Public docs / check (push) Blocked by required conditions
Core E2E Tests / Validate E2E tier selection (push) Blocked by required conditions
Core E2E Tests / Detect browser receipt-only changes (push) Waiting to run
Core E2E Tests / Offline Organization provisioning (push) Blocked by required conditions
Core E2E Tests / Build source-bound E2E images once (push) Blocked by required conditions
Core E2E Tests / Playwright Core E2E (shard 1/8) (push) Blocked by required conditions
Core E2E Tests / Playwright Core E2E (shard 2/8) (push) Blocked by required conditions
Core E2E Tests / Playwright Core E2E (shard 3/8) (push) Blocked by required conditions
Core E2E Tests / Playwright Core E2E (shard 4/8) (push) Blocked by required conditions
Core E2E Tests / Playwright Core E2E (shard 5/8) (push) Blocked by required conditions
Core E2E Tests / Playwright Core E2E (shard 6/8) (push) Blocked by required conditions
Core E2E Tests / Playwright Core E2E (shard 7/8) (push) Blocked by required conditions
Core E2E Tests / Playwright Core E2E (shard 8/8) (push) Blocked by required conditions
Core E2E Tests / Agent registration lifecycle (push) Blocked by required conditions
Core E2E Tests / E2E verdict (push) Blocked by required conditions
Unified Agent Native Verification / Linux ARM64 (push) Has been cancelled
Unified Agent Native Verification / Linux x64 (push) Has been cancelled
Unified Agent Native Verification / Windows x64 (push) Has been cancelled
Unified Agent Native Verification / macOS ARM64 (push) Has been cancelled
Unified Agent Native Verification / macOS Intel (push) Has been cancelled
Demo transaction native acceptance / native (push) Has been cancelled
Unified Agent Native Verification / FreeBSD cross-build contract (push) Has been cancelled
Keep monitoring readings, trusted connections and recovery guidance
reliable when observations are stale, guest responses are incomplete or
agent recovery is interrupted.

Qualified source: `b6e56c5baa`. All nine
[required source
checks](https://github.com/rcourtman/Pulse/pull/2888/checks), the
[native
matrix](https://github.com/rcourtman/Pulse/actions/runs/39148879516) and
[Core E2E](https://github.com/rcourtman/Pulse/actions/runs/39148879470)
passed. Intel accounted for all 545 installer roots, with 532 passes, 13
explicit platform skips and no failures. Its remaining installer phase
passed in 18m48s, followed by native agent build and execution. The
matrix covers Linux x64/ARM64, macOS Intel/ARM64 and Windows x64
execution plus the FreeBSD cross-build contract.

- Preserve reading provenance, measured zero, stored History and VM/LXC
memory interpretation. Selected-agent staleness cannot renew a retained
observation or invalidate independent provider readings.
- Reject incomplete or foreign guest responses and retain saved
certificate trust, scoped access, private installer input, native NAS
teardown and bounded recovery guidance.
- Keep initial Proxmox login within its caller and total client timeout
while accepted-session renewal remains shared. Preserve the PBS owner's
context outcome when cancellation races a response, prevent a cancelled
fallback and reject late session publication.
- Assert troubleshooting workflows and destinations semantically. Run
every discovered Python safety control and native Go root independently
with existing fixture, operation, root and job deadlines. Script-smoke
and native verification share one control suite, including real
compiled-Go failure, exit-zero, timeout and exact-selector controls.
- Integrate current main while retaining selected-agent freshness, guest
memory interpretation, native GNU prerequisites, API concurrency
controls and both parents' subsystem contracts.

Worker proof passed 113 focused frontend cases, API and guest-transport
race checks, all 59 Python fixture controls, production frontend build
and full frontend lint/types. The exhaustive worker installer inventory
accounted for 551 roots: 535 passed and 16 explicitly skipped for
platform reasons. The eight consolidated runner controls passed through
the script-smoke harness and native Go wrapper. Authentication proof
passed all 340 Proxmox tests, 108 PBS tests and the affected monitoring
cancellation controls under the race detector. The new PBS test first
failed all ten JSON/form cases on unchanged runtime, then passed with
the correction. Status, ownership and normal commit checks passed.

Browser proof uses production components and CSS with synthetic
observations and endpoints. Drawer History passed at 1280×900, 1024×768
and 390×844. Private installation passed at 1280×900 and 390×900 in both
themes. It covers stale/current/unknown readings, provider caveats,
stored History and keyboard inspection, wrapped signing keys, token
copy/dismissal and Last seen advancing without another lookup. Final
screenshots were inspected, the source-bound receipt covers the combined
runtime files, and owned browsers and servers were closed.

Retain the [previous Frontend
assertions](https://github.com/rcourtman/Pulse/actions/runs/38660106123/job/116179237804),
[Intel
timeouts](https://github.com/rcourtman/Pulse/actions/runs/39138980077/job/117709536519),
[script-smoke integration
failure](https://github.com/rcourtman/Pulse/actions/runs/39145479921/job/117728549560)
and [PBS cancellation
race](https://github.com/rcourtman/Pulse/actions/runs/39145479921/job/117729442632)
with their diagnosed causes and corrected proof. The final matrix above
qualifies the combined corrections.

The [earlier advisory benchmark
comparison](https://github.com/rcourtman/Pulse/actions/runs/39138980114/job/117709718583)
reported three legacy history-filter regressions. A worker repeat
reproduced one. Longer controls showed wide timing variation even
between byte-identical binaries. The [45a82ea03c
comparison](https://github.com/rcourtman/Pulse/actions/runs/39145479921/job/117728549510)
passed against a newer baseline. The [final
comparison](https://github.com/rcourtman/Pulse/actions/runs/39148879492/job/117738375776)
completed execution but failed on test-only empty/legacy filtering,
2.945 to 3.263ns, +10.82% (p<.001, ten samples). Production empty
filtering improved 16.70%, interleaved filtering rose 2.96%, and history
allocation counts were unchanged. The filter and benchmark sources are
byte-identical across the final comparison. This does not establish a
cause or performance closure. Artifact 12145720151 retains full paired
samples, comparison and metadata. The overall Build and Test workflow is
red from this advisory job while all nine protected required checks
pass. The existing L1 reliability follow-up owns repeatability and
attribution. No benchmark threshold, protected check or release gate
changed.

Main's benchmark/SLO test and documentation changes in PR3053 compose
without conflict. The candidate benchmark artifact records synthetic
merge c21333c676ac004935c9683027490203d8f869ea with tree
fa059ae37b, combining this head and main
52b40fd611. A bounded worker composition
proof also executed all three chart benchmark cases under the race
detector with source hashes unchanged. Its SLO test explicitly skipped
under the race detector, and one-iteration execution supplies no
performance comparison.

This proposal qualifies source delivery. Release inclusion, installed
upgrade/recovery, guest thaw, fresh writes on every backed-up filesystem
and workload liveness remain separate outcomes. Reliability keeps its
existing priority. Recovery/change feature expansion stays deferred.

Related to #1099, #2439, #2619, #2822, #2897, #2757, #2511, #2513,
#1788, #2755 and #2140.
2026-10-18 18:49:19 +01:00
rcourtman
b6e56c5baa Preserve PBS authentication cancellation while reading responses
A cancelled login can consume an empty response before the transport reports
cancellation. Preserve that context outcome for the owner and joined readers,
reject late session publication and prevent a cancelled refusal from retrying
another encoding. Keep live response validation and later recovery intact.
2026-10-18 18:11:01 +01:00
rcourtman
52b40fd611
Measure workload chart generation in performance guards (#3053)
The workload chart benchmark warmed its response cache, then mostly
timed a fresh recorder allocating a roughly 1.6 MiB response buffer. It
did not reliably measure the store-backed computation it claimed to
protect.

Split it into cached handler work, fresh-service computation and fully
drained loopback HTTP transfer. The store fixture waits for startup
maintenance. The adjacent SLO now exercises uncached chart generation at
its existing budgets. Retain the original #3052 failure and document the
new baseline boundary. Runtime chart behavior is unchanged.

Validation: 20 alternating samples on both recorded runtime trees with
the same revised harness and Go 1.26.9 found no significant change in
any of the three cases. The race smoke passed all three cases with
source hashes verified before and after execution. The focused SLO
measured 216 ms p95 and skipped its 90 ms budget under the shared-worker
contention policy. Non-race release qualification retains the 550 ms
hosted budget. Source CI uses race tests and skips latency SLOs. The
investigation record retains the exact scope and evidence.

The original +12.72% remains unresolved historical evidence. This change
makes no product speedup claim.
2026-10-18 18:04:28 +01:00
rcourtman
266144de1f Share native installer execution controls with script smoke tests
Script smoke still asserted the removed grouped runner while native
verification used a separate suite. Share the canonical execution controls
so both entrypoints reject lost checks, false passes and changed deadlines.
2026-10-18 17:55:06 +01:00
rcourtman
c369bf3076 fix: measure workload chart generation in performance guards
The chart benchmark warmed its response cache and then timed a fresh
payload-retaining recorder. That hid store-backed work behind test buffer
allocation and made the regression signal ambiguous.

Measure cached handler work, fresh-service computation and real loopback
HTTP transfer separately. Wait for store startup maintenance, preserve the
original failed comparison and record the paired diagnostic limits. Make
the adjacent SLO exercise uncached chart generation at its existing budget.

Refs #3052
2026-10-18 17:32:34 +01:00
rcourtman
45a82ea03c Merge current notification reliability repairs into maintenance
Preserve the landed L6 source, response-verdict controls and ownership evidence
alongside the maintenance source repairs. Final protected checks and native
qualification must cover this combined source before landing.
2026-10-18 17:21:17 +01:00
rcourtman
79a35fbe2e Give every native installer check its own process budget
Compile and discover one exact test binary per native workflow phase, then
execute every discovered root check independently. This removes inventory
growth from another check's ten-minute Go alarm while preserving fixture,
installer and overall CI job deadlines.

Retain exhaustive phase coverage and reject missing completion, failed checks,
unexpected exit zero and timeouts using real compiled Go execution controls.
Register the runner with deployment installability and retain the failed Intel
and advisory benchmark evidence in the existing reliability record.
2026-10-18 17:17:52 +01:00
rcourtman
6bf9e818e8 Respect construction and shared session login lifetimes
Initial password authentication has one owner and must end with its
constructor. Keep its complete JSON/form exchange within the caller and
client timeout. Retained-session renewal still serves concurrent readers
independently of any one caller's cancellation.

Retain generation-based coalescing and the shared parser, exercise both
initial encodings under the client timeout, and document the ownership
boundary alongside existing startup and cluster cancellation controls.
Retain the failed CI and reproduced baseline in the existing reliability
record while final-source qualification remains open.
2026-10-18 15:57:16 +01:00
rcourtman
0b3261e1aa Bound native fixture controls independently
The NAS suite exhausts its aggregate alarm after fifteen successful
controls on macOS Intel. Run each discovered NAS and updater control
under its existing bound, retaining the native phase's ten-minute alarm
and all installer operation deadlines.

Keep required controls mandatory and reject skipped, absent or failed
execution. Exercise discovery and false-pass rejection with real Python
controls, including a newly added method.
2026-10-18 15:40:55 +01:00
rcourtman
3d6949851f
Fix notification failure diagnosis and response completion (#3052)
Plain connection closes and opaque network-operation errors could be
recorded as unknown, while a disconnect during SMTP authentication could
stop retries as a credential failure. Structural network evidence now
takes precedence over diagnostic prose. Explicit unconfirmed SMTP
acceptance after DATA keeps its existing unknown verdict and transport
retry stop.

HTTP Apprise discarded response-read errors and could mark interrupted
2xx responses sent with successful receipts. It now shares the bounded
webhook response verdict and safe diagnostics. Invalid payload
templates, malformed retained configuration and unsupported retained
types become configuration failures on their first attempt. Enhanced
ntfy uses the canonical renderer, and errors/logs withhold private
template and rendered payload text.

The shared delivery activity label and retention description say retries
stopped. Dead-letter status alone cannot establish budget exhaustion
when a permanent verdict stops the first attempt.

Regression controls use real HTTP and SMTP receivers plus reopened
queue, audit, receipt and telemetry records. They cover transient SMTP
recovery, deterministic failures without outgoing requests, interrupted
and complete responses, authoritative HTTP/SMTP verdicts, genuine
unknowns and confidentiality. Evidence and remaining destination-level
scope are recorded in
docs/release-control/v6/internal/records/notification-unclassified-reliability-2026-10-18.md
under the existing notification-delivery-terminal-failure-diagnosis gap.
Aggregate telemetry cannot attribute fleet impact or establish recovery.

Validation passed: the full notifications package, targeted notification
race controls, 51 frontend tests, and the production Notifications
browser fixture at 1440x1000 and 390x1100. Browser checks cover labels,
safe details, refresh, cancelled retry with focus return and reload.
Staged status, registry, contract, completion and browser-receipt audits
pass.
2026-10-18 15:38:47 +01:00
rcourtman
4b86af9a33 Record browser proof for merged reliability surfaces
Bind the integrated source to drawer History and private installer checks.
Retain installed recovery and release qualification as separate outcomes.
2026-10-18 15:17:16 +01:00
rcourtman
17527049ea Align notification operator checks with retained-failure copy
Terminal failures can stop early and do not prove retry-budget exhaustion.
Update the existing operator qualification expectation to match the shared
retention description already exercised in the browser.
2026-10-18 15:15:53 +01:00
rcourtman
2fc54a5a1f Merge pending maintenance repairs with current main
Preserve selected memory freshness and the VM/LXC interpretation.
Retain both native tool contracts and current API concurrency controls.
Qualify this source separately from release and installed recovery.

Contract-Neutral: Integrate already-governed parents without a new contract change. Incoming verification files already match main. Fresh integration tests and source-bound browser proof cover the combined memory and installer surfaces.
2026-10-18 15:11:28 +01:00
rcourtman
84050a05e2 Fix notification failure diagnosis and response completion
Connection closes and invalid local jobs entered the unknown failure bucket,
and disconnects during SMTP authentication could stop transient retries.
Use structural network evidence and declared configuration verdicts while
preserving unconfirmed SMTP acceptance as unknown.

Propagate interrupted HTTP Apprise responses through the shared bounded
response reader so they cannot create successful queue receipts. Share ntfy
template rendering and withhold private template and payload diagnostics.

Describe retained terminal failures as retries stopped, since a permanent
verdict can stop the first attempt before the budget is exhausted.
2026-10-18 15:03:26 +01:00
courtmanr@gmail.com
92b784bef2 Keep reliability checks tied to workflows and selected tools
Require the existing troubleshooting jobs and their destinations without
making additional help entries break unrelated safety checks. Keep the
opening backup warning scoped to its actual entry point.

Run closed service and middleware models through the containing fixture's
selected Python interpreter without site startup. Retain every safety
scenario and deadline while qualifying the pending maintenance source.

Refs #2888

Contract-Neutral: Test-only navigation assertions and confined model interpreter selection. Production behavior and safety deadlines are unchanged.
2026-10-18 14:28:37 +01:00
rcourtman
98817ebce4
Fix SMTP LOGIN compatibility and authentication negotiation (#3051)
Some checks are pending
Build and Test / Backend tests (api-1) (push) Blocked by required conditions
Build and Test / Backend tests (api-2) (push) Blocked by required conditions
Build and Test / Backend tests (api-3) (push) Blocked by required conditions
Build and Test / Backend tests (api-4) (push) Blocked by required conditions
Build and Test / Backend tests (api) (push) Blocked by required conditions
Build and Test / Script smoke tests (shard 0/2) (push) Blocked by required conditions
Build and Test / Script smoke tests (shard 1/2) (push) Blocked by required conditions
Build and Test / Script smoke tests & backend build (push) Blocked by required conditions
Build and Test / Benchmarks (push) Blocked by required conditions
Canonical Governance / governance (push) Waiting to run
Canonical Private Governance / private-governance (push) Waiting to run
Provider MSP startup / Provider MSP production startup (push) Waiting to run
Public docs / check (push) Blocked by required conditions
Public docs / Documentation recipes (0) (push) Waiting to run
Public docs / Documentation recipes (1) (push) Waiting to run
Public docs / Documentation recipes (2) (push) Waiting to run
Public docs / Documentation recipes (3) (push) Waiting to run
Core E2E Tests / E2E verdict (push) Blocked by required conditions
Core E2E Tests / Playwright Core E2E (shard 7/8) (push) Blocked by required conditions
Core E2E Tests / Playwright Core E2E (shard 8/8) (push) Blocked by required conditions
Core E2E Tests / Agent registration lifecycle (push) Blocked by required conditions
Core E2E Tests / Validate E2E tier selection (push) Waiting to run
Core E2E Tests / Offline Organization provisioning (push) Waiting to run
Core E2E Tests / Build source-bound E2E images once (push) Blocked by required conditions
Core E2E Tests / Playwright Core E2E (shard 1/8) (push) Blocked by required conditions
Core E2E Tests / Playwright Core E2E (shard 2/8) (push) Blocked by required conditions
Core E2E Tests / Playwright Core E2E (shard 3/8) (push) Blocked by required conditions
Core E2E Tests / Playwright Core E2E (shard 4/8) (push) Blocked by required conditions
Core E2E Tests / Playwright Core E2E (shard 5/8) (push) Blocked by required conditions
Core E2E Tests / Playwright Core E2E (shard 6/8) (push) Blocked by required conditions
SMTP LOGIN rejected correct credentials when a receiver used legacy,
empty or localized challenge text. Authentication now follows
username/password exchange order and refuses incomplete or excessive
challenges without copying receiver-controlled prompts into errors. AUTH
selection matches complete mechanism names and reports unsupported lists
as configuration failures before attempting an unadvertised mechanism.

Connection tests and delivery share the implicit TLS rule for port 465,
avoiding a second STARTTLS upgrade inside an established TLS session.
The existing PLAIN preference, legacy probe, TLS trust policy and
structured server verdicts remain covered.

Validation exercises real plain/TLS/STARTTLS receivers, connection
tests, ordinary/threaded/attachment sends, bad credentials, temporary
replies, certificate refusal, and persisted queue/audit/telemetry
verdicts after restart. The full notification package, targeted race
controls, and status, registry and contract audits pass. Evidence is
recorded in
docs/release-control/v6/internal/records/notification-authentication-reliability-2026-10-18.md
under the existing notification-delivery-terminal-failure-diagnosis gap.
Aggregate telemetry cannot identify which fleet failures these defects
explain or establish fleet recovery.
2026-10-18 12:01:37 +01:00
rcourtman
960b5771ec Fix SMTP LOGIN compatibility and authentication negotiation
Literal LOGIN prompt matching rejected valid credentials and substring AUTH
selection could choose an unadvertised mechanism. Use exchange order and
complete mechanism names, preserve server reply classes, and classify
unsupported mechanism lists as configuration failures.

Share the implicit TLS decision between connection tests and delivery so a
port-465 test does not attempt STARTTLS inside an established TLS session.
Add real SMTP controls and persisted queue verdict verification.
2026-10-18 11:35:57 +01:00
rcourtman
82b2b3a78b
Reconcile reliability roadmap and exit evidence (#3050)
The reliability roadmap still described a shipped source-binding repair
as unlanded and the backup-running correction as preview-only. Reconcile
both against stable v6.5.0 and link the remaining work and manual exit
evidence through the existing L1 follow-up and active target.

The linked record distinguishes completed repairs from final-source
maintenance qualification, installed customer recovery, each candidate's
own soak, stable delivery, and normal operational recovery. The
recovery/change plan remains deferred. Active priority, lane states,
release gates and the planned feature sequence are preserved.

Validation: governance audits, exact before/after state and sequence
checks, ancestry and reporter evidence, and normal commit and push
checks on the validation worker.
2026-10-18 11:26:03 +01:00
rcourtman
3f59baa680 Reconcile reliability roadmap and exit evidence
Replace stale unshipped-fix claims with stable ancestry and reporter
results. Link remaining delivery and recovery outcomes to the existing
manual reliability target without changing priority or release gates.
2026-10-18 11:11:04 +01:00
rcourtman
13c99c17c1
Fix notification connectivity fallback and DNS cancellation (#3049)
Pinned HTTP dialing could spend a request's timeout on a stalled first
address even when another approved address was healthy. Both
notification and restricted HTTP clients now use one bounded fallback
dialer. It limits overlapping connections to two, shares the connection
budget, preserves DNS pinning and closes unused successful connections.

Webhook DNS admission and dialing now honor cancellation with a
five-second lookup ceiling. Redirect validation retains the request
context and empty DNS results fail closed. Shared transport ownership
and proof routing are explicit.

Validation covers a real stalled TCP listener beside a healthy HTTP
receiver, cancellation through the production resolver, one queued
delivery and its successful persisted verdict after restart, full
affected package tests and targeted race checks. Aggregate fleet
telemetry identifies connectivity as the largest failure class but
cannot attribute those failures to these defects or establish fleet
recovery.
2026-10-18 09:11:19 +01:00
rcourtman
3c103f1263 Fix pinned notification connectivity and cancelled DNS lookups
Serial pinned-address dialing could spend the request deadline on a stalled
first address while a validated fallback was healthy. Share one bounded
two-worker dialer with restricted HTTP clients and retain cancellation in
webhook DNS admission and dialing.

Add real stalled-TCP and DNS cancellation controls, persistent queue verdict
readback, shared transport ownership and explicit proof routing.
2026-10-18 08:49:56 +01:00
rcourtman
87164fa219
Track recovery and change gaps after reliability (#3048)
The recovery and change investigation audit was saved separately from
the roadmap records used to select future work. Link it from the planned
product target and the existing L15/L19 follow-ups, with an explicit
handoff after reliability yields priority.

The recorded order is backup task outcomes, focused change
investigation, workload protection assessment, safe configuration diffs,
then demand-dependent recovery-test evidence. The active reliability
target, lane states, release gates and demand thresholds stay unchanged.

Validation: control-plane, status, registry and contract audits,
explicit before/after governance invariants, and normal commit and push
checks on the validation worker.
2026-10-18 08:23:14 +01:00
rcourtman
606acc0a2d Track recovery and change gaps after reliability
Link the ordered audit into the planned product target and existing L15
and L19 residuals so future work has an explicit pickup path. Preserve
the active reliability target, lane states and release qualification.
2026-10-18 08:13:40 +01:00
rcourtman
4b69b8c7ea
Fix SMTP provider authentication and isolate delivery envelopes (#3045)
Supported SMTP providers could pass validation with their default
username while delivery skipped authentication and failed with SMTP 530.
Preparing overlapping sends could also overwrite another delivery's
sender and recipients. Resolve provider credentials when creating the
SMTP manager and keep each delivery's configuration independent while
preserving the shared rate limit. SMTP attempt logs also report the
resolved authentication state for provider defaults.

Real SMTP regression controls cover all four provider defaults, accepted
envelopes, concurrent sends, shared rate limits and persisted queue
outcomes after restart. Invalid credentials still produce an
authentication dead letter. The notification contract and investigation
record retain the broader reliability gap because aggregate telemetry
cannot attribute fleet failures to these two defects.

Validation on pulse-dev with Go 1.26.9:

- Full notification package passed.
- Focused SMTP and queue controls passed normally and under the race
detector.
- Release-control status, registry and contract audits passed.
2026-10-18 00:17:50 +01:00
rcourtman
a598e2a19c Report resolved SMTP authentication in delivery logs
Provider-default usernames could leave hasAuth=false in SMTP diagnostics even
though the delivery manager required authentication. Report the resolved
manager state at the SMTP attempt and remove the premature admission-level
credential inference.

Refresh synthetic SMTP, persistent queue and race evidence for the final source.

Contract-Neutral: Diagnostic-only follow-up: report resolved SMTP hasAuth state. Existing real SMTP, persisted queue and race controls were rerun and retained.
2026-10-17 23:54:29 +01:00
rcourtman
0e574d9c91
Document recovery and change investigation gaps (#3046)
Audit Pulse's recovery and change investigation flows against PegaProx
and ProxCenter, then prioritise the gaps worth tackling after
reliability. The plan records existing capabilities, customer demand,
source ownership, dependencies and acceptance criteria. It preserves the
current reliability target and lane states.

Validation includes a current-source build on the development worker,
desktop and narrow browser inspection with mock data, and injected
posture and inventory failures. Real provider recovery and competitor
reliability were outside this audit.
2026-10-17 23:51:38 +01:00