Blanton_TTL_Script

V1.1.0 59861cf953

v1.1.0 — BER, power under load, and a 100G port that actually keeps sending Stable

etwen released this 2026-08-28 16:33:43 +08:00 | 24 commits to main since this release

✨ New features

Whole-switch SerDes BER sweep

New blanton_ber.sh runs PRBS across every cabled port and reports the bit error rate lane by lane. init → start → wait → report → stop → clear, with -u 0 / -u 1 for a single switch unit.
A lane passes only when its BER is below 1e-6. Equal to the threshold is a FAIL, not a pass. -c <ber> sets a different bar.

report ends with a summary across both units, and each unit's row carries its worst lane — so a run where everything passes still tells you how much margin there was:

Unit        LANES     PASS     FAIL       NA  WORST BER    WORST LANE RESULT
unit 0        432      432        0        0  1.98e-08     203[0]     PASS
unit 1        432      432        0        0  6.34e-10     366[1]     PASS

init also checks that every lane locked. This matters: an unlocked lane still reports a BER later, and it is a meaningless one. Without the check, a dead lane can report a beautiful number and pass.
A lane whose reading is not a number is NA — counted separately from FAIL, because "not measured" and "measured and bad" are different problems.
Script C runs the sweep after the traffic report, following SWB_UNIT0 / SWB_UNIT1 the same way the traffic block does.

TR518 load test, for measuring power under traffic

New blanton_tr518.sh puts the cd ports in loopback and runs the built-in tr 518 packet test, then waits TR_SETTLE_SEC (120s by default) before handing back — so the power figures you take afterwards are loaded and settled, not caught mid-ramp.
-u 0 / -u 1 for one unit; no flag runs both.
stop sends port cd lb=none and nothing else, which is exactly what the bench procedure does. L2 learning and test mode are left as start set them — the tool says so on every stop rather than quietly leaving the box in a state you did not ask about. TR_RESTORE_LEARN=1 / TR_RESTORE_TESTMODE=1 if you want a true inverse.

Per-rail power readback for both switch boards

TH6_SWB0_power_readback.sh / TH6_SWB1_power_readback.sh report V × I and total power for the 19 rails on each board.

🐛 Bug fixes

The power readback measured every rail at a different moment

It ran show platform voltage and show platform current once per rail — 38 invocations of a slow CLI, and 38 different instants. Under load the currents move a lot (TH6_CORE was seen going from 186 A to 636 A), so the "total" was a sum of readings taken seconds apart. Both tables are now read once, up front. Same output, one moment.
A rail missing from the sensor table used to contribute a silent 0.000 V / 0.00 W to the total. It now says so on stderr; the printed report format is unchanged.

The 100G port stopped sending after one round trip

ce0 has no loopback partner, and a switch never forwards a frame back out the port it arrived on — so a tx burst went out, came back, and was dropped. TX 100, RX 100, silence. That measures the link; it does not exercise it.
init now sets an ingress mirror of ce0 back to itself. The mirrored copy is not egress-filtered, so each packet keeps lapping and the load is set by how much you inject (-tx). stop is what ends it — removing the VLAN membership does not, because the mirrored copy never consulted the VLAN.

The 100G verdict was being decided the wrong way

ce0 was treated as a cross-unit check. The bench says otherwise: there is one ce0 per unit and a burst sent on unit 0 alone came back on unit 0. It is now judged within its own unit. The distinction is invisible while everything works and decisive when it does not — read as cross-unit, both links dying still leaves unit0.TX == unit1.RX (0 == 0) and reports PASS.

Every BER lane reported NA, and the Port column printed NA too

bcmcmd returns CRLF, and the BER is the last token on its line, so the carriage return stayed glued to the number: it failed the numeric check, and printing it sent the cursor back to column 1 so the RESULT column overwrote the start of the row. Both symptoms, one stray \r.

📦 Downloads

File	Contents
`Script_ABC_Blanton_V1.1.0.zip`	The full Tera Term working directory, including `Blanton_Script/` to copy onto the DUT

After copying to the DUT:

chmod +x ~/Blanton_Script/*.sh      # see below
grep -rlU $'\r' ~/Blanton_Script    # expect no output

blanton_ber.sh is invoked by Script C as ./Blanton_Script/blanton_ber.sh, so it joins bmc_monitor.sh and usb_target.sh in needing the execute bit — and neither git nor a Windows/USB copy carries it reliably. Without the chmod the BER section fails with Permission denied and the log simply has no BER in it.

⚠️ Before you run

These three tests cannot share the switch. blanton_ber.sh, blanton_tr518.sh and blanton_traffic_linespeed all take over port loopback and L2 learning. Run one at a time; Script C already orders BER after the traffic report.
PRBS takes the links down. While blanton_ber.sh is armed the ports report DOWN and no traffic passes. That is the test working, not a fault. clear releases them.
ce0 keeps running until you stop it. If a run is interrupted between start and stop, the 100G port stays saturated with nothing in show interfaces status to hint at it. blanton_traffic_linespeed stop (or bcmcmd -n <u> -c 'mirror port ce0 Mode=Off') ends it.
Two numbers must stay in step: Script C's sleep 30 and blanton_ber.sh's BER_INTERVAL. Change one without the other and report reads an interval that has not finished accumulating — with no warning.
FAN_SPEED is 100. This build runs the fans flat out unless you change config.ttl.
Declare the switch population with SWB_UNIT0 / SWB_UNIT1; traffic, the readiness gate and now the BER sweep all follow it.
PRBS on the two 100G uplinks (port_prbs_monitor.sh) is still not wired into the macros. Unchanged from v1.0.9.

🔬 Not yet verified on hardware

Three things in this build were reasoned out rather than measured, and each has a one-command check:

The ce0 ingress mirror. mirror port ce0 Mode=Ingress DestPort=ce0 should make the port circulate, but the hardware may refuse a mirror destination equal to its source. Check show c | grep POK.ce0 twice ten seconds apart — the number must climb. The report cannot tell you: a one-shot burst is also TX == RX and also says PASS. Fallback is tr 518 PortList=268.
The 120s settle in blanton_tr518.sh. If tr 518's traffic is time-limited (its parameters include runtime=1, CheckDataTime=200), waiting 120s could mean measuring after the load has already dropped. Compare a power readback at 0s, 30s and 120s.
test mode nr=no in blanton_tr518.sh is the derived inverse of the nr=yes that start sends; it is not part of the bench-verified sequence, and is off by default.

🔗 Links

ARCHITECTURE.md — test flow, data models, constraints
CLAUDE.md — commands and the bench gotchas

Full changelog: V1.0.11...V1.1.0

Downloads

Script_ABC_Blanton_V1.1.0.zip
145 KiB
2026-08-28 16:34:02 +08:00
Source Code (ZIP)
Source Code (TAR.GZ)

V1.0.11 43314444e9

v1.0.11 — Saving the margin settings to NVM is now one command, and the command is correct Stable

etwen released this 2026-08-28 10:32:12 +08:00 | 25 commits to main since this release

✨ New features

margin_save blanton — all 18 LTC2977 in one go

On this platform the CB FPGA's I2C channels are also exposed as ordinary Linux i2c buses, so every margin controller on the box — CB and both switch boards — can be reached with plain i2cset. margin_save blanton walks all 18 of them from one flat list: no margin_init, no FPGA channel setup, no pcimem.
The channel-to-bus mapping it relies on (F3 Ch6 → bus 13, Ch7 → 14, Ch8 → 15, Ch9 → 16; CB is bus 4 as before) is now written down in ARCHITECTURE.md and the margin README, not just known at the bench.
A partly-populated DUT is a normal build. Comment out the lines you do not have in MARGIN_BLANTON_STORE at the top of margin.sh rather than letting the tool talk to boards that are not there.
Missing bus and unresponsive chip are reported as two different failures. /dev/i2c-16 does not exist means the FPGA's i2c adapters did not enumerate; i2cset failed (NACK/busy) means the board is absent or the address is wrong. Chasing the second when you have the first wastes an afternoon.

margin_save all — every board in settings/

Runs margin_init + margin_save across all nine boards, then puts back whichever board you had loaded, so a following margin_status still reports the board you were looking at.
It does not stop at the first failure: a half-saved set is harder to reason about than a fully-attempted one. The summary names the boards that failed.

Both print a per-chip line and a N saved, M failed tally, and both return a non-zero exit code when anything failed.

The fans now default to full speed

FAN_SPEED in config.ttl is 100 instead of 30. A soak is a thermal test of everything except the cooling, so the cooling should not be a variable in it — and a DUT that throttles halfway through a run produces results that look like a different fault. Lower it deliberately if quiet operation is what you are measuring.

🐛 Bug fixes

A failed NVM write still reported success

margin_save discarded the result of every write, so a NACK printed Change Saved exactly like a successful store. For the one operation in this tool that is permanent, that is the worst place to stay quiet. Each store is now checked and named if it fails.

STORE_USER_ALL was sent with a data byte it should not have

The command went out as 0x15 followed by a dummy 0x00 — an SMBus write-byte, where STORE_USER_ALL is a send-byte. The LTC2977 tolerated it, but tolerance is not correctness. It is now i2cset ... 0x15 c on the native path and a no-data cb_pmbus_write on the FPGA path.

📦 Downloads

File	Contents
`Script_ABC_Blanton_V1.0.11.zip`	The full Tera Term working directory, including `Blanton_Script/` to copy onto the DUT

After copying to the DUT:

chmod +x ~/Blanton_Script/*.sh      # see below
grep -rlU $'\r' ~/Blanton_Script    # expect no output

mgmt_ping_monitor.sh and port_prbs_monitor.sh launch their workers through bash, so those two run without the execute bit. bmc_monitor.sh and usb_target.sh still need it, and neither git (mode 100644) nor a Windows/USB copy carries it. Without the chmod, start fails with Permission denied and the later cat finds nothing: the section ends up empty and nothing reports an error.

⚠️ Before you run

margin_save in any form writes NVM permanently. A power cycle does not undo it. margin_save all and margin_save blanton do it to all 18 controllers at once. Run margin_status_all.sh first and never call any of them during a test run — the macros do not, and should not be made to.
The 18 raw commands were verified on the DUT; the wrapper was not. margin_save blanton issues exactly the sequence that was run by hand at the bench, and the packaging was tested against stubbed hardware, but the function itself has not been run on a DUT yet. Watch the per-chip lines the first time.
Declare the switch population. SWB_UNIT0 / SWB_UNIT1 in config.ttl say which switch units this DUT has; traffic and the readiness gate both follow them.
FAN_SPEED is now 100. It is applied at the start of every run, so this build runs the fans flat out unless you change config.ttl. Expect the noise.
Management connectivity is in use, not preserved. Both management NICs are up and pinging — drive the run from the serial console.
PRBS is still not wired into the macros. Run port_prbs_monitor.sh by hand if you want it; the calls in A/B/C remain commented out.
Two settings persist to config_db.json and survive a reboot: LLDP is left disabled and the 100G uplinks are left configured. Restore them before the DUT moves on.

🔗 Links

ARCHITECTURE.md — test flow, data models, constraints
CLAUDE.md — commands and the bench gotchas
LTC2980_Margin_Script/README.md — the three margin_save variants and what each one sends

Full changelog: V1.0.10...V1.0.11

Downloads

Script_ABC_Blanton_V1.0.11.zip
123 KiB
2026-08-28 10:32:33 +08:00
Source Code (ZIP)
Source Code (TAR.GZ)

V1.0.10 15ea776b82

v1.0.10 — The soak loop stops spinning flat out, and shows you what is still running Stable

etwen released this 2026-08-27 14:29:50 +08:00 | 27 commits to main since this release

✨ New features

Every soak round now reports the background jobs

Script B's monitoring loop prints bgctl list (platform jobs) and jobs (the login shell's own background jobs) on every pass, right after the platform data. A stress job that dies three hours into an overnight soak now shows up as a shorter list on the next round, instead of only being noticed at the end — or not at all.
The two lists answer different questions and are both worth having: bgctl knows about the platform's job runner, jobs knows about anything the macro backgrounded in that shell.

A single table for all 144 margin channels

docs/LTC2980_channel_map.csv flattens the nine settings/*.conf files into one sheet — Board,CONN,Ch,NetName,Vnom — so "which board and channel is PVDD1V5_TSC on?" is one search instead of opening nine files.
tools/gen_channel_map.sh regenerates it. The CSV is output, not source: edit the .conf files and re-run the script, never the other way round.
Unused channels stay in the table as NC / - rather than being dropped, because the channel number is also the PMBus page (ch / 8 picks the chip, ch % 8 picks the page) — filtering the gaps out would shift everything after them.

The bench command lists that these scripts came from

tools/100G_PRBS.txt, tools/TR518.txt and tools/fan_ctrl.txt join the existing traffic_loopback_*.txt set: the raw bcmcmd / sysfs sequences, kept in the form you can paste into a console when a script is misbehaving and you want to drive the hardware directly.

🐛 Bug fixes

The soak loop ran as fast as the DUT could answer

while 1 had no pause at all, so each round started the moment the previous one finished. Over a long soak that means constant console traffic and a log full of near-identical samples taken seconds apart. The loop now ends each round with pause 60.
Note the gap this leaves: the Status sheet asks for a 10-minute sampling interval, and 60 seconds is still hard-coded rather than configurable. This makes the loop sane, it does not finish the job.

A comment promised a cadence the code never had

The loop was labelled Get data every 10mins while running with no delay whatsoever. It now says what it does.

📦 Downloads

File	Contents
`Script_ABC_Blanton_V1.0.10.zip`	The full Tera Term working directory, including `Blanton_Script/` to copy onto the DUT

After copying to the DUT:

chmod +x ~/Blanton_Script/*.sh      # see below
grep -rlU $'\r' ~/Blanton_Script    # expect no output

mgmt_ping_monitor.sh and port_prbs_monitor.sh launch their workers through bash, so those two run without the execute bit. bmc_monitor.sh and usb_target.sh still need it, and neither git (mode 100644) nor a Windows/USB copy carries it. Without the chmod, start fails with Permission denied and the later cat finds nothing: the section ends up empty and nothing reports an error.

⚠️ Before you run

Declare the switch population. SWB_UNIT0 / SWB_UNIT1 in config.ttl say which switch units this DUT has; traffic and the readiness gate both follow them.
Set FAN_SPEED. It is applied at the start of every run.
Management connectivity is in use, not preserved. Both management NICs are up and pinging — drive the run from the serial console.
PRBS is still not wired into the macros. Run port_prbs_monitor.sh by hand if you want it; the calls in A/B/C remain commented out, unchanged from v1.0.9.
TR518 and the loopback traffic test cannot both run. tools/TR518.txt sets port all lb=mac and l2 learn off across the whole unit, which is the same hardware blanton_traffic_linespeed is using. Pick one.
Two settings persist to config_db.json and survive a reboot: LLDP is left disabled and the 100G uplinks are left configured. Restore them before the DUT moves on.

🔗 Links

ARCHITECTURE.md — test flow, data models, constraints
CLAUDE.md — commands and the bench gotchas
LTC2980_channel_map.csv — all 144 margin channels

Full changelog: V1.0.9...V1.0.10

Downloads

Script_ABC_Blanton_V1.0.10.zip
118 KiB
2026-08-27 14:30:06 +08:00
Source Code (ZIP)
Source Code (TAR.GZ)

V1.0.9 03d609cc1f

V1.0.9 — 100G PRBS testing, and the job logs actually reach the USB stick Stable

etwen released this 2026-08-26 15:35:48 +08:00 | 28 commits to main since this release

✨ New features

100G uplink PRBS test

New port_prbs_monitor.sh drives a PRBS test on both 100G uplinks: it clears low-power mode, arms the pattern, then polls phy diag <phy> prbs get in the background until stopped. A poll counts as PASS only when the output says PRBS OK!; prbsstat Ber is captured alongside every poll for the record.
stop runs prbsstat STOp and prbs clear on each port before rendering the report, so the test does not leave PRBS armed behind it.

report gives the tally the bench actually wants:

PORT           POLLS    PASS     FAIL     RESULT
Ethernet513    42       42       0        PASS
Ethernet514    42       40       2        FAIL

One port at a time is a first-class mode. start a or start b runs a single uplink, which is how you separate a genuine port fault from the DUT not coping with two PRBS streams at once. A port that was not run reports SKIP, not FAIL.
Every command and its full output — setup, each poll, teardown — goes to that port's raw log.

⚠️ The script ships in this build and can be run by hand, but the calls in Script A, B and C are commented out: PRBS is still under bring-up on this platform.

USB target detection

usb_target.sh reports an inserted USB device's node, mount point or by-id path. It excludes whatever disk backs / and /host — this platform can boot from a USB DOM, which also reports as USB — refuses to guess when several USB disks are present, and when mounting, verifies the result is writable rather than trusting that mount succeeded.
Script B builds the USB stress command from the detected node instead of a hard-coded /dev/sda1.

🐛 Bug fixes

The job logs were being written to a path nobody collected

Stress output, the monitors' logs and the USB archive disagreed about where the job directory lives. Everything now uses /host/hw-eval/jobs/, so what Script B writes is what Script C copies out.

Script C copied to a USB that was never mounted

The archive step assumed /mnt/usb was ready. Script C now finds the device and mounts it first, so a run's logs leave the DUT instead of being written into an empty mount point.

show interfaces status right after starting PRBS reads as a failure

With PRBS armed the link is out of normal operation and reports DOWN. port_prbs_monitor.sh deliberately does not print the port status at start — it appears in the report instead, after prbs clear, labelled as the recovered state.

📦 Downloads

File	Contents
`Script_ABC_Blanton_V1.0.9.zip`	The full Tera Term working directory, including `Blanton_Script/` to copy onto the DUT

After copying to the DUT:

chmod +x ~/Blanton_Script/*.sh      # see below
grep -rlU $'\r' ~/Blanton_Script    # expect no output

mgmt_ping_monitor.sh and port_prbs_monitor.sh launch their workers through bash, so those two run without the execute bit. bmc_monitor.sh and usb_target.sh still need it, and neither git (mode 100644) nor a Windows/USB copy carries it. Without the chmod, start fails with Permission denied and the later cat finds nothing: the section ends up empty and nothing reports an error.

⚠️ Before you run

Declare the switch population. SWB_UNIT0 / SWB_UNIT1 in config.ttl say which switch units this DUT has; traffic and the readiness gate both follow them.
Set FAN_SPEED. It is applied at the start of every run.
Management connectivity is in use, not preserved. Both management NICs are up and pinging — drive the run from the serial console.
PRBS is not wired into the macros yet. Run port_prbs_monitor.sh by hand if you want it; uncommenting the calls in A/B/C is not supported in this build.
Two settings persist to config_db.json and survive a reboot: LLDP is left disabled and the 100G uplinks are left configured. Restore them before the DUT moves on.

🔗 Links

ARCHITECTURE.md — test flow, data models, constraints
CLAUDE.md — commands and the bench gotchas

Full changelog: V1.0.8...V1.0.9

Downloads

Script_ABC_Blanton_V1.0.9.zip
118 KiB
2026-08-26 15:36:25 +08:00
Source Code (ZIP)
Source Code (TAR.GZ)

V1.0.8 5e1e4a6a8f

V1.0.8 — Both management links tested at once, and ARP stops looking like packet loss Stable

etwen released this 2026-08-26 10:13:34 +08:00 | 29 commits to main since this release

✨ New features

Both management NICs are pinged simultaneously and continuously

mgmt_ping_monitor.sh no longer alternates fixed bursts. start configures both NICs, then pings from each at the same time and keeps accumulating until stopped — so a soak-long run is one continuous measurement rather than a series of snapshots.
stop renders a report into the log: the last 20 entries per NIC, each one's ping statistics and PASS/FAIL, then ip -s link show for both interfaces.
Every line is timestamped, and a request that got no reply prints a marker instead of merely being absent — a drop is visible in the tail, not inferred from a gap in the sequence numbers.
status shows both pids and how many replies each NIC has received so far, which is the quick way to see one side is dead without waiting for the report.

The USB stress target is detected, not assumed

New usb_target.sh reports an inserted USB device's node, mount point or by-id path. Script B asks it for the node and builds the stress command from the answer, so a stick that enumerates as sdb no longer sends a write test at whatever /dev/sda1 happens to be.
It excludes whatever disk backs / and /host — this platform can boot from a USB DOM, which also reports as USB — refuses to guess when several USB disks are present, and when it mounts, verifies the result is actually writable rather than trusting that mount succeeded.

BMC I2C integrity is exercised through the soak

bmc_monitor.sh is back in the run and now writes two complementary patterns to a BMC scratch register and reads each back. One pattern alone cannot catch a bit stuck the same way it was written; repeating the pair through the soak turns an intermittent I2C fault into something the log records rather than something the tester has to witness.

Test artefacts leave the DUT with the run

Script C copies bmc_poll.log and mgmt_ping.log into the job directory before it is archived to USB, so the monitors' output travels with the bgctl job logs.
DDR stress now runs continuously instead of stopping after 100 passes, matching the other soak loads.

🐛 Bug fixes

Three FAILs that were not link faults

A bench run reported FAIL=3 with zero NIC errors, zero drops and sub-millisecond replies. Every failure was missing exactly icmp_seq=1 and nothing else: once the neighbour entry for the target expires, the first echo request is spent resolving ARP and ping counts it as loss. A discarded warm-up ping per NIC now absorbs that, which is what lets the loss threshold stay at zero and still mean something. Raising the threshold instead would have hidden genuine single-packet loss.

Job logs were being cleared before they were collected

Script C ran bgctl reset --yes while killing processes — before the job directory is copied to USB. Moved to after the archive, so a run's own logs are collected before anything clears them.

The fan controller was bound twice

One of the two MAX31790 controllers received a second bind immediately after the first, which could only fail because the driver was already attached.

📦 Downloads

File	Contents
`Script_ABC_Blanton_V1.0.8.zip`	The full Tera Term working directory, including `Blanton_Script/` to copy onto the DUT

After copying to the DUT:

chmod +x ~/Blanton_Script/*.sh      # see below
grep -rlU $'\r' ~/Blanton_Script    # expect no output

mgmt_ping_monitor.sh no longer re-execs itself, so bash mgmt_ping_monitor.sh start works without the execute bit. bmc_monitor.sh and usb_target.sh still need it — and neither git (mode 100644) nor a Windows/USB copy carries it. Without the chmod, start fails with Permission denied and the later cat finds nothing: the section ends up empty and nothing reports an error.

⚠️ Before you run

Declare the switch population. SWB_UNIT0 / SWB_UNIT1 in config.ttl say which switch units this DUT has. Traffic and the readiness gate both follow them; with neither set the traffic stage is skipped rather than failing against hardware that is not there.
Set the fan speed you want. FAN_SPEED in config.ttl is applied at the start of every run.
Management connectivity is in use, not preserved. Both NICs are up and pinging — drive the run from the serial console.
If the two management NICs share a subnet, keep ARP_STRICT=1. Otherwise the target's ARP can be answered by either NIC and a reply may arrive on the one that did not send. Check nic_tx / nic_rx on the RESULT line: they are that interface's own counter delta.
Two settings persist to config_db.json and survive a reboot: LLDP is left disabled and the 100G uplinks are left configured. Restore them before the DUT moves on.

🔗 Links

ARCHITECTURE.md — test flow, data models, constraints
CLAUDE.md — commands and the bench gotchas

Full changelog: V1.0.7...V1.0.8

Downloads

Script_ABC_Blanton_V1.0.8.zip
112 KiB
2026-08-26 10:13:52 +08:00
Source Code (ZIP)
Source Code (TAR.GZ)

V1.0.7 ef7cbdffd4

V1.0.7 — Management-link ping test, and the fans start where you set them Stable

etwen released this 2026-08-25 14:22:22 +08:00 | 35 commits to main since this release

✨ New features

10G / 1G management port ping test

New mgmt_ping_monitor.sh exercises both management NICs in one loop: it brings each up with iproute2, waits for the link to settle, then pings that NIC's own target — eth0 → 192.168.1.30, eth1 → 192.168.1.31.
Runs detached like the other monitors: start / stop / status / fg / summary. start clears the previous log, so one run means one log.
Every leg writes a single greppable line — RESULT 10G eth0 -> 192.168.1.30 tx=10 rx=10 loss=0% rtt_avg=1.743ms nic_tx=10 nic_rx=10 PASS — so a whole soak can be read with summary instead of scrolling.
nic_tx / nic_rx are that interface's own counter delta across the burst. If a ping passes but they stay near zero, the traffic left on the other NIC and the number does not belong to this one.
Script A runs one round as a baseline; Script B leaves it running through the soak.

Fan speed is set at the start of every run

FAN_SPEED in config.ttl (default 30) is applied by Script A, so a run no longer inherits whatever the previous test left the fans doing.
Both MAX31790 controllers are rebound first, which clears a controller left in an odd state.

Stress load moved onto the platform's own job runner

CPU, DDR, SSD and USB stress now go through bgctl, replacing the ~/hammer/tools/stress_*.py scripts. bgctl list shows what is running; Script C stops everything with bgctl stop --all.
BMC DDR stress runs through bmc-manager, and a new BMC USB test discovers the cdc_ncm interface and pings across it for four hours under systemd-run.

More of the DUT on record

Script A now captures the boot image, show version, firmware status, system EEPROM, SSD health, TPM version and NVMe SMART data before the test starts.
PCIe AER now sits next to ras-mc-ctl --summary, and Script C closes with NVMe health so a disk that degraded during the soak is visible.
dmesg capture uses human-readable timestamps and one combined error pattern, then clears the ring buffer — so Script C's dmesg shows only what the soak produced, not everything since boot.
Each run opens with hw-test-session start and closes with finish; Script C copies the job logs to /mnt/usb/jobs-<date>_<time> so they leave the DUT with the run they belong to.

🐛 Bug fixes

Script A no longer hangs waiting for ports that are already up

The readiness gate required more than 216 ports up, but a fully cabled unit reports exactly 216 — 108 loopback pairs times two. Script A sat in the poll loop forever on the very state it was waiting for. It now proceeds at 216.
The comments in wait_init.ttl still described the old "more than" rule; they now match the code, so the off-by-one cannot be reintroduced by reading the wrong line.

The macro no longer runs ahead of the DUT during the soak

Script B set a 15-second wait cap for the cdc_ncm probe and never restored it. In Tera Term that cap is global, so it stayed in force for everything after — including the traffic setup, which walks 108 VLANs per switch unit and takes far longer. A timed-out wait returns without the prompt, leaving every later command issued a step early.

Fan controller is no longer bound twice

One of the two MAX31790 controllers received a second bind immediately after the first, which could only fail because the driver was already attached.

📦 Downloads

File	Contents
`Script_ABC_Blanton_V1.0.7.zip`	The full Tera Term working directory, including `Blanton_Script/` to copy onto the DUT

After copying to the DUT:

chmod +x ~/Blanton_Script/*.sh      # required -- see below
grep -rlU $'\r' ~/Blanton_Script    # expect no output

The chmod +x is not optional. mgmt_ping_monitor.sh is executed rather than sourced and re-execs its own path to daemonise, and neither git (mode 100644) nor a Windows/USB copy carries the execute bit. Without it start fails with Permission denied and the later cat finds nothing — the section ends up empty and nothing reports an error.

⚠️ Before you run

Declare the switch population. SWB_UNIT0 / SWB_UNIT1 in config.ttl say which switch units this DUT has. Traffic and the readiness gate both follow them: a single unit gets -u 0 or -u 1, and with neither set the traffic stage is skipped instead of failing against hardware that is not there.
Both management NICs stay up. If they share a subnet, keep ARP_STRICT=1 — otherwise the target's ARP can be answered by either NIC and a reply may arrive on the one that did not send.
Management connectivity is exercised, not preserved. Drive the run from the serial console.
Two settings persist to config_db.json and survive a reboot: LLDP is left disabled, and the 100G uplinks are left configured. Restore them before the DUT moves on.

🔗 Links

ARCHITECTURE.md — test flow, data models, constraints
CLAUDE.md — commands and the bench gotchas

Full changelog: V1.0.5...V1.0.7

Downloads

Script_ABC_Blanton_V1.0.7.zip
109 KiB
2026-08-25 14:22:41 +08:00
Source Code (ZIP)
Source Code (TAR.GZ)

V1.0.5 9a11c1d5bb

V1.0.5 — Partly populated DUTs and readiness gating Stable

etwen released this 2026-08-21 16:06:54 +08:00 | 39 commits to main since this release

Blanton TTL Script V1.0.5

One test run is Script A (baseline) -> B (soak) -> C (verdict), all captured in a single Tera Term log.

✨ New features

Partly populated DUTs

config.ttl declares the switch population once:

SWB_UNIT0 = 1   ; this DUT has switch unit 0
SWB_UNIT1 = 1   ; this DUT has switch unit 1

The traffic blocks in A, B and C branch on it: no -u when both units are fitted (the tool's own TL_UNITS="0 1" applies), -u 0 or -u 1 for a single one, and the whole stage is skipped when neither is set
utils/wait_init.ttl reads the same two flags, so the wait and the traffic cannot disagree about what is installed

Readiness gate

Script A now waits on the data plane rather than the sensors: it polls bcmcmd -n <u> -c ps | grep -w up | wc -l and proceeds once every fitted unit reports at least WT_MIN (216) ports up — 216 being 108 loopback pairs x 2, i.e. every cabled port

BMC

bmc_monitor.sh (samples free -m) and bmc_monitor_ddr.sh (memtester) run through the soak via bmc-manager run, started by Script B and collected by Script C

Inventory and link state

Script A: boot image, board EEPROM (hpe-eeprom-tlv), BMC version, 100G uplink configuration and status, uptime
Script C: show reboot-cause and uptime, so an unplanned reset during the soak shows up in the closing snapshot
LLDP disabled and transceiver low-power mode cleared before traffic, so the loopback counters only reflect the injected burst

🐛 Bug fixes

Script A never got past the port wait. The threshold was strictly greater than 216, but a fully cabled unit reports exactly 216, so the loop spun forever on the very state it was waiting for. Now "at least 216"
Single-unit traffic paths were unreachable. All nine branch conditions compared SWB_UNIT0 against itself, so a DUT with only unit 0 ran the both-unit commands against an absent unit 1, and a DUT with only unit 1 skipped traffic altogether
config save -y was sent without waiting for the prompt, leaving the macro one step ahead of the DUT from that point on

⚠️ Deployment notes

scp -r src/Script_ABC_Blanton/Blanton_Script admin@<DUT>:~/
chmod +x ~/Blanton_Script/bmc_monitor*.sh      # required -- see below
grep -rlU $'\r' ~/Blanton_Script               # expect no output

The chmod +x is still not optional. bmc_monitor*.sh are executed rather than sourced, do_start re-execs its own path, and neither git (mode 100644) nor a Windows/USB copy carries the execute bit. Without it Script B's start fails with Permission denied and Script C's cat finds nothing — the BMC section ends up empty and nothing reports an error.

Two settings in this build persist to config_db.json and survive a reboot: LLDP is left disabled, and Ethernet513 is left shut down. Restore them before the DUT moves on to other work.

📦 Downloads

File	Contents
`Script_ABC_Blanton_V1.0.5.zip`	The full Tera Term working directory, including `Blanton_Script/` to scp onto the DUT

Downloads

Script_ABC_Blanton_V1.0.5.zip
101 KiB
2026-08-21 16:07:25 +08:00
Source Code (ZIP)
Source Code (TAR.GZ)

V1.0.3 419b680980

V1.0.3 — Traffic, BMC monitoring and SWB margin fixes Stable

etwen released this 2026-08-21 09:33:06 +08:00 | 46 commits to main since this release

Blanton TTL Script V1.0.3

One test run is Script A (baseline) -> B (soak) -> C (verdict), all captured in a single Tera Term log.

✨ New features

Traffic

blanton_traffic_linespeed.sh drives SWB loopback line-speed traffic over bcmcmd, VLAN 30..137 pairing cdN with cdN+32
Per-pair TX/RX cross-check (cdA.TX == cdB.RX) yields PASS/FAIL; awk on the DUT, tools/bcm_mibpair_report_V1.1.0.py off-box
All three scripts now cover both switch units

BMC

bmc_monitor.sh samples free -m; bmc_monitor_ddr.sh runs memtester as a stress load. Both reach the BMC through bmc-manager run from the host, so no interactive SSH to the BMC is needed
Script B starts them, Script C stops them and folds their logs into the master log

Data collection

Script A records show boot, the BMC version, 100G port status and show uptime, and waits for pmon to come up before taking a baseline (utils/wait_init.ttl)
Script C adds show reboot-cause and show uptime, so an unplanned reset during the soak is visible in the closing snapshot
PMON snapshot extended with leak status and leak channels

Packaging

publish/publish.sh builds publish/Script_ABC_Blanton_<Ver>/ (-z for a zip). The version is read from Script A's header, so the folder name cannot disagree with the macro

🐛 Bug fixes

Every counter in the traffic report printed as 2147483647 — the DUT's awk casts to a 32-bit int for %d, and a real line-speed run is around 1e12. Now formatted with %.0f. Verdicts were never affected: the cross-check compares doubles and only the display path was wrong
SWB I2C channels and LTC2977 addresses — corrected against the bench. CONN13/15 and CONN14/16 were paired to the wrong channels, and addresses step by 2, not 1. This also clears the collision where two CONNs both landed on 0x5E on Ch6
CRLF made margin.sh unrunnable on SONiC ($'\r': command not found). .gitattributes now pins *.sh and *.conf to LF. settings/*.conf was the quieter half: they are sourced, so CB_I2C_CH=6 became "6\r"
margin.sh v2.6.0 — the SWB path never found the cb_i2c backend, so TRANSPORT=swb always failed to load. It now searches the parent Blanton_Script/ directory, and the duplicated backend copies were removed
Script C hung — it called a subcommand that runs tail -f and never returns, so the run never reached the stress results or the closing messagebox. Replaced with a direct cat of the logs

⚠️ Deployment notes

scp -r src/Script_ABC_Blanton/Blanton_Script admin@<DUT>:~/
chmod +x ~/Blanton_Script/bmc_monitor*.sh      # required -- see below
grep -rlU $'\r' ~/Blanton_Script               # expect no output

The chmod +x is not optional. bmc_monitor*.sh are the first scripts here meant to be executed rather than sourced, do_start re-execs its own path, and neither git (mode 100644) nor a Windows/USB copy carries the execute bit. Without it Script B's start fails with Permission denied and Script C's cat finds nothing — the BMC section ends up empty and nothing reports an error.

📦 Downloads

File	Contents
`Script_ABC_Blanton_V1.0.3.zip`	The full Tera Term working directory, including `Blanton_Script/` to scp onto the DUT

Downloads

Script_ABC_Blanton_V1.0.3.zip
99 KiB
2026-08-21 09:33:47 +08:00
Source Code (ZIP)
Source Code (TAR.GZ)

18 Releases 18 Tags

v1.1.0 — BER, power under load, and a 100G port that actually keeps sending Stable

✨ New features

🐛 Bug fixes

📦 Downloads

⚠️ Before you run

🔬 Not yet verified on hardware

🔗 Links

v1.0.11 — Saving the margin settings to NVM is now one command, and the command is correct Stable

✨ New features

🐛 Bug fixes

📦 Downloads

⚠️ Before you run

🔗 Links

v1.0.10 — The soak loop stops spinning flat out, and shows you what is still running Stable

✨ New features

🐛 Bug fixes

📦 Downloads

⚠️ Before you run

🔗 Links

V1.0.9 — 100G PRBS testing, and the job logs actually reach the USB stick Stable

✨ New features

🐛 Bug fixes

📦 Downloads

⚠️ Before you run

🔗 Links

V1.0.8 — Both management links tested at once, and ARP stops looking like packet loss Stable

✨ New features

🐛 Bug fixes

📦 Downloads

⚠️ Before you run

🔗 Links

V1.0.7 — Management-link ping test, and the fans start where you set them Stable

✨ New features

🐛 Bug fixes

📦 Downloads

⚠️ Before you run

🔗 Links

V1.0.5 — Partly populated DUTs and readiness gating Stable

Blanton TTL Script V1.0.5

✨ New features

🐛 Bug fixes

⚠️ Deployment notes

📦 Downloads

V1.0.3 — Traffic, BMC monitoring and SWB margin fixes Stable

Blanton TTL Script V1.0.3

✨ New features

🐛 Bug fixes

⚠️ Deployment notes

📦 Downloads