4_Blanton_Script_thermal_safety.ttl brings the DUT up the way Script A
does, then loads it - mlucas-avx2 -cpu 0:15 in the background plus
blanton_tr518.sh start - and samples platform data and both switch
boards' rail power every minute. Power per rail under sustained full
load, over hours, is the whole point of it.
It is standalone: nothing starts it, it starts nothing, and it must not
share a DUT with Script B because both take over the cd ports.
It also has no teardown. When you interrupt it the unit is still in
port cd lb=mac / l2 learn off / test mode nr=yes and mlucas is still
running, and show interfaces status does not reveal any of that. Called
out in the version history, the release notes, CLAUDE.md gotchas and the
ARCHITECTURE future-work list, because whoever uses the box next will not
find out on their own.
Script B's per-round loop drops the margin scan. The loop is meant to be
a quick minute-by-minute sample and margin_status_all.sh walks nine
LTC2980s over I2C, which dominated the round. Script C takes it instead,
once, before show uptime. The trade-off is real: margin drift DURING the
soak is no longer visible, only a start and an end reading. Documented
rather than glossed over, with a suggestion (scan every N rounds) if it
needs to come back.
Script C also sends exit and waits for "Script done" before
hw-test-session finish, so finish runs in the login shell rather than
inside the session it is closing.
Adds docs/Blanton_Test_Flow.drawio: all four scripts side by side, with
the config flags that change each step marked, Script B's one real goto
drawn, and the A->B->C handoffs shown as the manual steps they are.
The new script arrived LF-only while every other .ttl is CRLF and
.gitattributes marks *.ttl as -text, so what is committed is exactly what
Tera Term reads. Converted.
Script A -> V1.1.4.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014RetWKFZFG1ZHcitQAyhwM
86 of the 144 margin channels moved up +0.8% .. +2.5% to the measured
median for this platform (3.3 -> 3.38, 0.9 -> 0.915, 0.8 -> 0.82, ...).
That is not a display change. VNOM is the denominator of margin_status'
deviation %, and it is also the basis margin_set derives high/low/OV/UV
from - so both the number you read and the voltage you apply move with
it. A rail steady at 0.756 V used to read +0.80% against a nominal 0.75
and now reads 0.00%. Margin logs from before and after cannot be
compared: the reference moved, the hardware did not.
comboA/comboB are replaced by combo_SWB_CONN13..16_{high,low}. The
per-channel percentages are positional, so they follow the rails of one
specific board - applying a CONN13 profile to a CONN16 margins the wrong
rails and nothing detects it. SWB0 and SWB1 share a profile per CONN
because their rails are identical.
Fixes two things the deletion broke:
- margin_apply_profile_all.sh still named comboA for all nine boards,
which would now stop at "Profile not found" nine times over. It names
the matching per-CONN profile instead, with CB on the generic
combo_high3, and a note that the low pass is _high -> _low.
- docs/LTC2980_channel_map.csv is generated from settings/*.conf, so
all 86 changed rails were stale in it. Regenerated.
The eight new profiles also still carried comboA's header comment
("alternating high/low"), which describes neither what they contain nor
what they are for.
Script A -> V1.1.3.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014RetWKFZFG1ZHcitQAyhwM
nfc_polling.sh wraps blantons_nfc_validate.py in a background loop and
keeps score: start/stop/status/tail/report/clear/fg. Script A takes a
30s baseline, Script B starts the poll for the soak, Script C stops it
with the other monitors and reports.
report only reads the log, so it can be run mid-soak without disturbing
anything - and it says RUNNING so a snapshot is not read as the final
answer. Each round appends a machine-readable RESULT line, which keeps
the tally independent of the tool's wording.
ERROR is counted apart from FAIL. A round where the tool printed neither
[PASS] nor [FAIL] - crashed, missing, hung past the timeout - measured
nothing, and that is a different fault from a tag that would not read.
Rolling them together hides whichever one you are not looking for.
Two habits from earlier bugs are baked in: tail does not follow (tail -f
never returns and a macro waiting on the prompt hangs there), and the .py
is called through python3 rather than ./ (it arrives mode 100644 and a
USB copy carries no execute bit).
Also fixes Script C copying "nfc_polling.log" to the job dir when the
file is "nfc_poll.log". The cp failed, nothing checked it, and the run
finished looking fine - you would find out at the desk, with a job
archive that had no NFC log in it.
Script A -> V1.1.2.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014RetWKFZFG1ZHcitQAyhwM
After a run that ended badly - Tera Term closed, DUT rebooted mid-soak,
macro stopped by hand - the stress jobs kept running and the test session
stayed open. The next run then started on a box that was already loaded.
Nothing errored; the numbers were just quietly wrong.
Script A now opens with hw-test-session finish, bgctl reset --yes,
bgctl stop --all, then bgctl list. The list is the point of the sequence:
it is the one line in the log that proves the DUT was idle at the start.
Script A -> V1.1.1.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014RetWKFZFG1ZHcitQAyhwM
Three new tools and one fix to an old one.
blanton_ber.sh - PRBS BER across every cabled port, init/start/report/
stop/clear, -u 0|1|all. A lane passes only below 1e-6 (equal to the
threshold fails). report ends with a per-unit summary carrying the worst
lane, so an all-PASS run still shows how much margin there was. init
checks every lane locked, because an unlocked lane still reports a BER
and it is a meaningless one. Script C runs it after the traffic report,
following SWB_UNIT0/SWB_UNIT1 like the traffic block does.
blanton_tr518.sh - the built-in packet test, used to load the rails
before measuring power. start settles 120s so the reading afterwards is
loaded rather than mid-ramp. stop sends port cd lb=none and nothing else,
exactly what the bench procedure does - l2 learning and test mode are
left as start set them, and stop says so instead of quietly leaving the
box in a state nobody asked about.
TH6_SWB{0,1}_power_readback.sh - 19 rails, V x I and total. These read
the two sensor tables ONCE. The previous version read them per rail: 38
invocations of a slow CLI at 38 different instants, so the "total" was a
sum of readings seconds apart - and under load the currents move (TH6_CORE
186 A -> 636 A). A rail missing from the table now warns on stderr instead
of silently contributing 0 W.
blanton_traffic_linespeed.sh - ce0 (VLAN 138) joins the test. It has no
loopback partner and a switch never sends a frame back out its ingress
port, so a tx burst was one round trip and then silence. init now sets an
ingress mirror of ce0 back to itself; the mirrored copy is not
egress-filtered, so packets keep lapping until stop turns the mirror off.
stop is what ends it - removing the VLAN membership does not, because the
copy never consulted the VLAN. TL_CE_MODE also drops to 'self': ps ce
shows one ce port per unit and a unit-0-only burst returned on unit 0.
Read as cross-unit, both links dying would still report PASS (0 == 0).
Fixes a parser bug worth remembering: bcmcmd returns CRLF, and the BER is
the last token on its line, so the CR stayed glued to the number. Every
lane read as NA, and printing the CR sent the cursor to column 1 so the
RESULT column overwrote the start of the row. One stray \r, two symptoms.
Not yet verified on hardware, all called out in the release notes: the
ce0 mirror (destport == srcport may be refused), the 120s settle (tr 518
may be time-limited), and 'test mode nr=no' on stop.
Script A -> V1.1.0.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014RetWKFZFG1ZHcitQAyhwM
FAN_SPEED 30 -> 100. A soak is a thermal test of everything except the
cooling, so the cooling should not be one of the variables in it. At 30%
a long run can throttle partway through, and throttled results read like
a different fault entirely.
Folded into V1.0.11 rather than bumping the version: the tag has not
moved on and this is the same delivery.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014RetWKFZFG1ZHcitQAyhwM
margin_save discarded the result of every write, so a NACK printed
"Change Saved" exactly like a successful store. For the one permanent
operation in this tool that is the worst place to stay quiet. Each store
is now checked, named on failure, and reflected in the exit code.
STORE_USER_ALL also went out as 0x15 followed by a dummy 0x00 - an SMBus
write-byte where the command is a send-byte. The LTC2977 tolerated it,
but tolerance is not correctness. Now i2cset ... 0x15 c on the native
path and a no-data cb_pmbus_write on the FPGA path.
Two new subcommands:
margin_save all - all nine boards in settings/, then re-init
whichever board was loaded beforehand so a
following margin_status still reports the board
you were looking at. Does not stop on the first
failure; a half-saved set is harder to reason
about than a fully-attempted one.
margin_save blanton - this platform exposes the CB FPGA F3 I2C
channels as native Linux i2c buses (Ch6->13,
Ch7->14, Ch8->15, Ch9->16; CB stays on bus 4),
so all 18 LTC2977 can be stored with plain
i2cset: no margin_init, no FPGA channel setup,
no pcimem. 18 commands instead of 42.
The 18 raw commands were verified on the DUT 2026-08-27; the wrapper was
tested against stubbed hardware only. Release notes say so.
Board list is MARGIN_BLANTON_STORE at the top of margin.sh - comment out
what this DUT does not have rather than letting it talk to absent boards.
Missing bus and unresponsive chip are reported as distinct failures.
MARGIN_STORE_SETTLE (0.5s) sits between stores because nothing here polls
the part's busy bit, and blanton fires 18 in a row.
Script A -> V1.0.11.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014RetWKFZFG1ZHcitQAyhwM
Script B's monitoring loop had no pause at all - each round started the
instant the last one ended, so an overnight soak filled the log with
near-identical samples taken seconds apart. The section was even labelled
"Get data every 10mins" while running with no delay whatsoever.
The loop now ends each round with pause 60 and prints both job lists:
bgctl list for the platform job runner, jobs for anything the macro
backgrounded in that shell. A stress job that dies mid-soak shows up as a
shorter list on the next round instead of going unnoticed until the end.
60s is still hard-coded and the Status sheet asks for 10 minutes, so the
Phase 3 item stays open - this makes the loop sane, it does not close it.
Also adds docs/LTC2980_channel_map.csv: the nine settings/*.conf flattened
into one Board,CONN,Ch,NetName,Vnom sheet, generated by
tools/gen_channel_map.sh. The CSV is output, not source. NC channels are
kept as NC/- because the channel number is also the PMBus page - dropping
the gaps would shift every channel after them.
Two things the merged table makes visible: SWB0 and SWB1 carry identical
net names and voltages (only the bus and CB_I2C_CH differ), so those are
two copies that can drift apart silently; and there are exactly 8 NC
channels, all on CONN14/CONN15.
tools/100G_PRBS.txt, TR518.txt and fan_ctrl.txt join the existing
traffic_loopback_*.txt references. TR518 is not scripted yet and cannot
share the hardware with blanton_traffic_linespeed - both drive port
loopback and L2 learning.
Script A -> V1.0.10.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014RetWKFZFG1ZHcitQAyhwM
port_prbs_monitor.sh drives PRBS on both 100G uplinks: lpmode off, arm
the pattern, then poll `phy diag <phy> prbs get` in the background until
stopped, counting a poll as PASS only when the output says "PRBS OK!".
prbsstat Ber is captured beside every poll for the record but does not
decide the verdict. `stop` runs prbsstat STOp and prbs clear before
rendering the report, so the test does not leave PRBS armed.
`start a` / `start b` runs one uplink alone, which is how a genuine port
fault is separated from the DUT not coping with two PRBS streams at
once. A port that was not run reports SKIP rather than FAIL -- deciding
on poll count alone would have made a deliberate single-port run look
like half the hardware was broken.
The port status is deliberately not printed at `start`: with PRBS armed
the link reports DOWN, which reads as a failure to anyone glancing at
the console. It appears in the report instead, after prbs clear and
labelled as the recovered state.
The calls in Script A, B and C are committed but commented out -- PRBS
is still under bring-up.
Fix: the job directory was written as /host/hw-eval/current/jobs in some
places and /host/hw-eval/jobs in others, so what Script B wrote was not
what Script C collected. Unified on /host/hw-eval/jobs.
Fix: Script C copied to /mnt/usb assuming it was mounted. It now finds
the device with usb_target.sh and mounts it first, so the logs actually
leave the DUT.
Script C also reorders its results -- stress logs, then the MGMT ping
report, then 100G status -- and cats the bgctl ssd/usb logs.
Script A -> V1.0.9. Docs and docs/release-notes/v1.0.9.md updated.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014RetWKFZFG1ZHcitQAyhwM
Script A: the baseline ping window is 30 s rather than 45, and the log
is cleared afterwards so Script B starts from nothing.
Script B: `status` after `start`, and Script C: `status` before `stop`
plus a `cat` of the report. The status line shows both pids and the
replies received per NIC, so a dead link is visible while the soak is
still running instead of only at the end.
docs/release-notes/v1.0.8.md written in the house format.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014RetWKFZFG1ZHcitQAyhwM
Script A carried a V1.0.7 history block while its "; Version :" header
still read V1.0.6, so publish.sh built Script_ABC_Blanton_V1.0.6 and the
folder would have disagreed with the tag. Header and date corrected --
reading the version from the header rather than a flag is what surfaced
this.
utils/wait_init.ttl V3.1.1: poll interval 10 s -> 60 s, and the WT_MIN
comments now say "at least" to match the test, which has been
`< WT_MIN` since the threshold was fixed. The stale "more than" wording
described exactly the off-by-one that once hung Script A.
Script C: the BMC USB journalctl dump is commented out.
docs/release-notes/v1.0.7.md written in the house format.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014RetWKFZFG1ZHcitQAyhwM
etwenandClaude Opus 5