86 of the 144 margin channels moved up +0.8% .. +2.5% to the measured
median for this platform (3.3 -> 3.38, 0.9 -> 0.915, 0.8 -> 0.82, ...).
That is not a display change. VNOM is the denominator of margin_status'
deviation %, and it is also the basis margin_set derives high/low/OV/UV
from - so both the number you read and the voltage you apply move with
it. A rail steady at 0.756 V used to read +0.80% against a nominal 0.75
and now reads 0.00%. Margin logs from before and after cannot be
compared: the reference moved, the hardware did not.
comboA/comboB are replaced by combo_SWB_CONN13..16_{high,low}. The
per-channel percentages are positional, so they follow the rails of one
specific board - applying a CONN13 profile to a CONN16 margins the wrong
rails and nothing detects it. SWB0 and SWB1 share a profile per CONN
because their rails are identical.
Fixes two things the deletion broke:
- margin_apply_profile_all.sh still named comboA for all nine boards,
which would now stop at "Profile not found" nine times over. It names
the matching per-CONN profile instead, with CB on the generic
combo_high3, and a note that the low pass is _high -> _low.
- docs/LTC2980_channel_map.csv is generated from settings/*.conf, so
all 86 changed rails were stale in it. Regenerated.
The eight new profiles also still carried comboA's header comment
("alternating high/low"), which describes neither what they contain nor
what they are for.
Script A -> V1.1.3.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014RetWKFZFG1ZHcitQAyhwM
nfc_polling.sh wraps blantons_nfc_validate.py in a background loop and
keeps score: start/stop/status/tail/report/clear/fg. Script A takes a
30s baseline, Script B starts the poll for the soak, Script C stops it
with the other monitors and reports.
report only reads the log, so it can be run mid-soak without disturbing
anything - and it says RUNNING so a snapshot is not read as the final
answer. Each round appends a machine-readable RESULT line, which keeps
the tally independent of the tool's wording.
ERROR is counted apart from FAIL. A round where the tool printed neither
[PASS] nor [FAIL] - crashed, missing, hung past the timeout - measured
nothing, and that is a different fault from a tag that would not read.
Rolling them together hides whichever one you are not looking for.
Two habits from earlier bugs are baked in: tail does not follow (tail -f
never returns and a macro waiting on the prompt hangs there), and the .py
is called through python3 rather than ./ (it arrives mode 100644 and a
USB copy carries no execute bit).
Also fixes Script C copying "nfc_polling.log" to the job dir when the
file is "nfc_poll.log". The cp failed, nothing checked it, and the run
finished looking fine - you would find out at the desk, with a job
archive that had no NFC log in it.
Script A -> V1.1.2.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014RetWKFZFG1ZHcitQAyhwM
After a run that ended badly - Tera Term closed, DUT rebooted mid-soak,
macro stopped by hand - the stress jobs kept running and the test session
stayed open. The next run then started on a box that was already loaded.
Nothing errored; the numbers were just quietly wrong.
Script A now opens with hw-test-session finish, bgctl reset --yes,
bgctl stop --all, then bgctl list. The list is the point of the sequence:
it is the one line in the log that proves the DUT was idle at the start.
Script A -> V1.1.1.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014RetWKFZFG1ZHcitQAyhwM
Three new tools and one fix to an old one.
blanton_ber.sh - PRBS BER across every cabled port, init/start/report/
stop/clear, -u 0|1|all. A lane passes only below 1e-6 (equal to the
threshold fails). report ends with a per-unit summary carrying the worst
lane, so an all-PASS run still shows how much margin there was. init
checks every lane locked, because an unlocked lane still reports a BER
and it is a meaningless one. Script C runs it after the traffic report,
following SWB_UNIT0/SWB_UNIT1 like the traffic block does.
blanton_tr518.sh - the built-in packet test, used to load the rails
before measuring power. start settles 120s so the reading afterwards is
loaded rather than mid-ramp. stop sends port cd lb=none and nothing else,
exactly what the bench procedure does - l2 learning and test mode are
left as start set them, and stop says so instead of quietly leaving the
box in a state nobody asked about.
TH6_SWB{0,1}_power_readback.sh - 19 rails, V x I and total. These read
the two sensor tables ONCE. The previous version read them per rail: 38
invocations of a slow CLI at 38 different instants, so the "total" was a
sum of readings seconds apart - and under load the currents move (TH6_CORE
186 A -> 636 A). A rail missing from the table now warns on stderr instead
of silently contributing 0 W.
blanton_traffic_linespeed.sh - ce0 (VLAN 138) joins the test. It has no
loopback partner and a switch never sends a frame back out its ingress
port, so a tx burst was one round trip and then silence. init now sets an
ingress mirror of ce0 back to itself; the mirrored copy is not
egress-filtered, so packets keep lapping until stop turns the mirror off.
stop is what ends it - removing the VLAN membership does not, because the
copy never consulted the VLAN. TL_CE_MODE also drops to 'self': ps ce
shows one ce port per unit and a unit-0-only burst returned on unit 0.
Read as cross-unit, both links dying would still report PASS (0 == 0).
Fixes a parser bug worth remembering: bcmcmd returns CRLF, and the BER is
the last token on its line, so the CR stayed glued to the number. Every
lane read as NA, and printing the CR sent the cursor to column 1 so the
RESULT column overwrote the start of the row. One stray \r, two symptoms.
Not yet verified on hardware, all called out in the release notes: the
ce0 mirror (destport == srcport may be refused), the 120s settle (tr 518
may be time-limited), and 'test mode nr=no' on stop.
Script A -> V1.1.0.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014RetWKFZFG1ZHcitQAyhwM
FAN_SPEED 30 -> 100. A soak is a thermal test of everything except the
cooling, so the cooling should not be one of the variables in it. At 30%
a long run can throttle partway through, and throttled results read like
a different fault entirely.
Folded into V1.0.11 rather than bumping the version: the tag has not
moved on and this is the same delivery.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014RetWKFZFG1ZHcitQAyhwM
margin_save discarded the result of every write, so a NACK printed
"Change Saved" exactly like a successful store. For the one permanent
operation in this tool that is the worst place to stay quiet. Each store
is now checked, named on failure, and reflected in the exit code.
STORE_USER_ALL also went out as 0x15 followed by a dummy 0x00 - an SMBus
write-byte where the command is a send-byte. The LTC2977 tolerated it,
but tolerance is not correctness. Now i2cset ... 0x15 c on the native
path and a no-data cb_pmbus_write on the FPGA path.
Two new subcommands:
margin_save all - all nine boards in settings/, then re-init
whichever board was loaded beforehand so a
following margin_status still reports the board
you were looking at. Does not stop on the first
failure; a half-saved set is harder to reason
about than a fully-attempted one.
margin_save blanton - this platform exposes the CB FPGA F3 I2C
channels as native Linux i2c buses (Ch6->13,
Ch7->14, Ch8->15, Ch9->16; CB stays on bus 4),
so all 18 LTC2977 can be stored with plain
i2cset: no margin_init, no FPGA channel setup,
no pcimem. 18 commands instead of 42.
The 18 raw commands were verified on the DUT 2026-08-27; the wrapper was
tested against stubbed hardware only. Release notes say so.
Board list is MARGIN_BLANTON_STORE at the top of margin.sh - comment out
what this DUT does not have rather than letting it talk to absent boards.
Missing bus and unresponsive chip are reported as distinct failures.
MARGIN_STORE_SETTLE (0.5s) sits between stores because nothing here polls
the part's busy bit, and blanton fires 18 in a row.
Script A -> V1.0.11.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014RetWKFZFG1ZHcitQAyhwM
Script B's monitoring loop had no pause at all - each round started the
instant the last one ended, so an overnight soak filled the log with
near-identical samples taken seconds apart. The section was even labelled
"Get data every 10mins" while running with no delay whatsoever.
The loop now ends each round with pause 60 and prints both job lists:
bgctl list for the platform job runner, jobs for anything the macro
backgrounded in that shell. A stress job that dies mid-soak shows up as a
shorter list on the next round instead of going unnoticed until the end.
60s is still hard-coded and the Status sheet asks for 10 minutes, so the
Phase 3 item stays open - this makes the loop sane, it does not close it.
Also adds docs/LTC2980_channel_map.csv: the nine settings/*.conf flattened
into one Board,CONN,Ch,NetName,Vnom sheet, generated by
tools/gen_channel_map.sh. The CSV is output, not source. NC channels are
kept as NC/- because the channel number is also the PMBus page - dropping
the gaps would shift every channel after them.
Two things the merged table makes visible: SWB0 and SWB1 carry identical
net names and voltages (only the bus and CB_I2C_CH differ), so those are
two copies that can drift apart silently; and there are exactly 8 NC
channels, all on CONN14/CONN15.
tools/100G_PRBS.txt, TR518.txt and fan_ctrl.txt join the existing
traffic_loopback_*.txt references. TR518 is not scripted yet and cannot
share the hardware with blanton_traffic_linespeed - both drive port
loopback and L2 learning.
Script A -> V1.0.10.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014RetWKFZFG1ZHcitQAyhwM
port_prbs_monitor.sh drives PRBS on both 100G uplinks: lpmode off, arm
the pattern, then poll `phy diag <phy> prbs get` in the background until
stopped, counting a poll as PASS only when the output says "PRBS OK!".
prbsstat Ber is captured beside every poll for the record but does not
decide the verdict. `stop` runs prbsstat STOp and prbs clear before
rendering the report, so the test does not leave PRBS armed.
`start a` / `start b` runs one uplink alone, which is how a genuine port
fault is separated from the DUT not coping with two PRBS streams at
once. A port that was not run reports SKIP rather than FAIL -- deciding
on poll count alone would have made a deliberate single-port run look
like half the hardware was broken.
The port status is deliberately not printed at `start`: with PRBS armed
the link reports DOWN, which reads as a failure to anyone glancing at
the console. It appears in the report instead, after prbs clear and
labelled as the recovered state.
The calls in Script A, B and C are committed but commented out -- PRBS
is still under bring-up.
Fix: the job directory was written as /host/hw-eval/current/jobs in some
places and /host/hw-eval/jobs in others, so what Script B wrote was not
what Script C collected. Unified on /host/hw-eval/jobs.
Fix: Script C copied to /mnt/usb assuming it was mounted. It now finds
the device with usb_target.sh and mounts it first, so the logs actually
leave the DUT.
Script C also reorders its results -- stress logs, then the MGMT ping
report, then 100G status -- and cats the bgctl ssd/usb logs.
Script A -> V1.0.9. Docs and docs/release-notes/v1.0.9.md updated.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014RetWKFZFG1ZHcitQAyhwM
bmc_monitor.sh comes back into the run with a second job beyond the
`free -m` sampling: an I2C pattern test against bus 1, address 0x41,
offset 0x00C0 -- write 55AA55AA and read it back, then write AA55AA55
and read that back. Two complementary patterns catch a stuck bit either
way round, and repeating it through the soak turns an intermittent I2C
fault into something the log shows rather than something the tester has
to catch live.
Script B starts it alongside the other stress; Script C stops it and
cats log/bmc_poll.log.
bmc_monitor_ddr.sh stays disabled -- BMC DDR stress runs through
bgctl + bmc-manager.
Docs: the "disabled" notes I added last time now applied to only half
the pair and read as wrong for bmc_monitor.sh. Split so the monitor is
documented as live with its I2C test, and only the _ddr variant is
marked disabled.
Script A -> V1.0.8.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014RetWKFZFG1ZHcitQAyhwM
config.ttl gains FAN_SPEED (30). Script A rebinds the two max31790
controllers (18-0020, 25-0020) and applies the speed with
fan-speed-control.sh, answering its confirmation prompt, so a run starts
from a known thermal state rather than whatever the last test left.
Script A also clears /host/hw-eval/current/jobs before the run, and
Script C copies that directory to /mnt/usb/jobs-<date>_<time> so the
bgctl job logs leave the DUT with the run they belong to.
Fix: 25-0020 was bound twice in a row -- the second bind can only fail,
since the driver is already attached. Removed; the two controllers now
have a matching unbind/sleep/bind each.
Script A -> V1.0.7, history covering the config.ttl and Script C changes
as well.
Docs: fan control and the job-log archive added to Tech Stack, Key
Features and Data Flow; the config.ttl example refreshed with FAN_SPEED
and the SWB_UNIT flags. Three risks recorded in Future Extensions --
`rm` without -r cannot clear the job subdirectories that Script C copies
with -r, the fan confirmation wait has no timeout and would hang
silently if the prompt ever changes, and /mnt/usb is never checked for
being mounted.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014RetWKFZFG1ZHcitQAyhwM
Script A: wait 45 s between starting and stopping mgmt_ping_monitor.
`start` returns after one second, but a round is roughly 30 s -- without
the pause the log held only the START banner and no RESULT line.
Docs brought up to date with the eight commits since 419b680:
- Tech Stack gains bgctl, systemd-run, iproute2 and hw-test-session;
the stress row moves off the hammer scripts
- Structure and Key Features cover mgmt_ping_monitor.sh, and mark
bmc_monitor{,_ddr}.sh as disabled -- their call sites in B and C are
commented out, though the files remain
- Data Flow rewritten for all three scripts
- Two constraints added: TTL's `timeout` is global and must be restored,
and two management NICs on one subnet cause ARP flux (hence
ARP_STRICT, and nic_tx/nic_rx as the evidence of which NIC sent)
- CLAUDE gotchas add the `ip -s link` column order, that a monitor's
`start` returns in one second rather than after a round, and that
show_dmesg now ends with `dmesg -C`
- Phase 3 ticks the move to bgctl; Phase 5 gains the mgmt-ping, NVMe and
EDAC parsing targets. Future Extensions notes that Script C never cats
mgmt_ping.log, so the soak's ping results stay on the DUT
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014RetWKFZFG1ZHcitQAyhwM
publish/publish.sh packages src/Script_ABC_Blanton into
publish/Script_ABC_Blanton_<Ver>/, with -z for a zip, -f to overwrite,
-n for a dry run.
The version is read from Script A's "; Version :" header rather than
passed in, so the folder name cannot disagree with what the tester sees
on opening the macro. The header is CRLF, so the carriage return is
stripped -- left in, it becomes part of the directory name. Captured
logs are excluded (they carry DUT serials) but the empty logs/ is kept
because logopen writes there. .gitignore now admits the tool while
still ignoring its output.
config.ttl: testcase ENV, margin scan off for this run.
Docs brought up to date with the 14 commits since 8aa5c90 -- SWB
channel/address map, the traffic and BMC tooling, line-ending policy,
and the exec-bit and tail -f traps.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014RetWKFZFG1ZHcitQAyhwM
Bring both docs in line with the five commits pulled from Gitea.
- BDF: pcimem V1.5.0 auto-detects the CB FPGA BDF, so the "edit the four
lines per DUT" guidance was inverted. Documented the detection order
and fpga_rescan, and flagged that utils/show_pcie_error_reg_*.ttl
still hard-codes its 7 BDFs -- the footgun is only half gone
- Traffic: documented blanton_traffic_linespeed.sh (bcmcmd SWB loopback,
per-pair TX/RX cross-check) and tools/bcm_mibpair_report + the three
loopback command tables; noted the route taken differs from the
SONiC-CLI approach Phase 4 originally planned
- Stress: stress_hhmd and stress_pcie were removed on 08-17, so the job
count is 8 not 10; soak loop now pulls full setup_pmon (9 items)
- Constraints: added the bcmcmd exit-0 and stdin-eating gotchas
- Phases: ticked 10 completed items, corrected Phase 3/4 acceptance
criteria, and moved BDF auto-detect out of Future Extensions
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014RetWKFZFG1ZHcitQAyhwM
Bring the Blanton Tera Term TTL test suite under version control and
document how the pieces fit together.
- ARCHITECTURE.md: A/B/C script roles, host-to-DUT-to-FPGA data path,
config/settings/profile data models, PCIe AER + EDAC topology table,
10 key constraints, and 6 development phases derived from the
Status_20260814 spreadsheet's On-Going items
- CLAUDE.md: stack, smoke-test commands, conventions, and the hardware
footguns (per-unit BDF, VSPI vs VI2C timing, repeated START,
STORE_USER_ALL, ARB_LOST false positives)
- secret/: gitignored credential store with README + .example template,
so the DUT login stops living in 1_Blanton_Script_A.ttl
- .gitignore: secret/*, For_AI/, *.log, publish/, *.DSN
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014RetWKFZFG1ZHcitQAyhwM
etwenandClaude Opus 5