CLAUDE.md: status to V1.2.0 / 2026-09-07; the PRBS paragraph now says the
A/B/C call sites were deleted in V1.1.9 rather than commented out; the
DSC scan and parser get a commands block and a line in the status
summary; the soak loop uses sleep 60; the bcmcmd gotcha records that
bcm_dsc_scan.sh hit the same stdin trap on 2026-09-07 and why the
existing marker check could not see it; and a new gotcha on pause versus
sleep.
ARCHITECTURE.md: bcm_dsc_scan.sh / bcm_dsc_parse.sh in the tree and a
Key Features row of their own; EN_TX_FIR in the config.ttl line; Script A
ref to V1.2.0 and the A / B / C rows updated for the DSC steps and sleep;
constraint 10 extended with the recurrence; new constraint 18 (pause
stops the macro, sleep stops the DUT) with the old 18 renumbered to 19
and EN_TX_FIR added to it; the git tag checkbox to V1.2.0; the PRBS and
bmc_monitor_ddr known issues corrected from "commented out" to "deleted";
and a new known issue for the DSC scan being off by default and sharing
ports with the other traffic tools.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014RetWKFZFG1ZHcitQAyhwM
Every 'pause N' in Scripts A, B and C is now 'sendln "sleep N"' followed
by 'wait prompt_sonic_root'.
pause stops the macro for N seconds without touching the DUT: the shell
sits idle and the macro is guessing how long the previous command needed.
On a loaded DUT the guess runs short, the next command goes out before
the previous one has finished, and the whole run ends up a beat ahead of
the switch. Putting the wait on the DUT means the macro resumes when the
prompt actually returns, so the two cannot drift apart - and the wait
lands in the console log with a timestamp, so a round that ran long is
visible afterwards.
Durations changed with it: the management-ping sample in Script A is 15 s
(was 30) and the settle after each traffic 'clear' is 5 s (was 15). The
traffic run stays 15 s and Script B's monitor loop stays 60 s. Script 4
and Script 5 were not converted; the 'pause 1' at the top of B and C
stays, since it runs before there is a prompt to wait for.
EN_TX_FIR gates the DSC scan in Script A and Script C, and the four cp
lines that carry blanton_dsc_Script{A,C}_u{0,1}.log out with the job
logs. It defaults to 0: a full scan is ~10 min and both scripts do one,
so leaving it off keeps ~20 min out of a routine run.
Script C also stops printing bmc_poll.log and mlucas_amm_log to the
console. Both are still copied to /host/hw-eval/jobs and out to the USB
stick, so nothing is lost - the master log is just shorter.
Script A's header goes to V1.2.0 and its Version History covers all of
the above.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014RetWKFZFG1ZHcitQAyhwM
The scan loop read its port file on stdin. bcmcmd reads stdin too, so the
first call swallowed the rest of the file and the loop ended after a
single port.
Nothing about that looked wrong. The log held one PORT marker and one dsc
block, the marker-to-dump comparison passed because both counts shrank
together, the exit code was 0, and a 445-port scan just finished in
seconds.
The loop now reads on fd 3 (done 3< "$PORTFILE") and bcmcmd is given
< /dev/null. A second check compares the iteration count against the port
list count - the marker/dump comparison cannot see a loop that ends
early, so it needed a number that does not shrink with it. The Saved:
line reports dumps out of TOTAL rather than out of markers.
This is the same bcmcmd trap blanton_traffic_linespeed.sh hit at v0.3.1,
where only the first VLAN was configured.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014RetWKFZFG1ZHcitQAyhwM
Script A scans just before 'show uptime', Script C just before the NVMe
error info, so the pair brackets the soak:
./Blanton_Script/bcm_dsc_scan.sh -f blanton_dsc_ScriptA.log -F
./Blanton_Script/bcm_dsc_scan.sh -f blanton_dsc_ScriptC.log -F
Neither name carries %u, so the scan inserts _u0 / _u1 itself and unit 1
cannot overwrite unit 0 - that is where the four file names Script C
copies to /host/hw-eval/jobs come from. The scan already copies its own
output there; the explicit cp lines are a second pass, so a copy that
fails inside the scan still leaves the data on the USB stick.
Also removed the commented-out 100G PRBS call sites from all three
macros, along with a duplicated commented 100G port status block in
Script A and the stale bmc_monitor_ddr.sh block in Script C. They had
been dead since bring-up. port_prbs_monitor.sh still ships and still
runs by hand.
Script A's header goes to V1.1.9 and its Version History covers all of
the above, including the note that both new scripts are recorded 100644
and need chmod +x on the DUT.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014RetWKFZFG1ZHcitQAyhwM
bcm_dsc_scan.sh runs
bcmcmd -n <unit> -c "dsh -c 'phydiag <port> dsc'"
over the whole port list (one 100G port plus the 200G ranges, ~445 in
total) on both units, writes the raw log under <script_dir>/log, and
copies it to /host/hw-eval/jobs. -f names the log (%u -> u0/u1, %t ->
timestamp), -F overwrites, -u picks one unit, -n dry runs, -C skips the
copy.
Two things it deliberately does not do:
- judge success by bcmcmd's exit code. The diag shell rejects commands
and bcmcmd still exits 0, so the script counts
"SERDES DISPLAY DIAG DATA END" markers instead. Same trap
blanton_traffic_linespeed.sh hit.
- resolve ~. Script A calls it after sudo -i, so paths are derived
from $0 rather than $HOME.
Output is one line per port when stdout is not a tty, so it does not
flood the Tera Term log, and it ends with "### DSC_SCAN_DONE rc=<n>" for
a TTL wait to key on.
bcm_dsc_parse.sh is the host-side half: raw log -> <prefix>_lane.csv
(TXEQ / RXFFE / DFE / EYE / SNR per lane) and <prefix>_pm.csv (per
pm_id). POSIX sh + awk, no Python. Lane rows are read by the order of
their bracket groups rather than by fixed columns, so the SDK re-spacing
its output does not break parsing; a row whose group count is not 9 is
reported as UNPARSED on stderr instead of being parsed wrongly.
A full scan is about 10 minutes.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014RetWKFZFG1ZHcitQAyhwM
The thermal / safety macro started mlucas-avx2 with '&', exactly as
Script B used to, so it carried the same failure: the load ended quietly
when the console went away - in the one test where a long, steady full
load is the entire point of the run.
It now uses the same mlucas-amm transient unit, and 'systemctl is-active
mlucas-amm' joins 'bgctl list' and 'jobs' in the monitor loop.
Script 4 still has no teardown section, so the matching stop stays
manual, and it is now two commands rather than one:
blanton_tr518.sh stop
systemctl stop mlucas-amm
Closing Tera Term no longer stops the CPU load, and 'kill $(jobs -p)'
never could reach a unit. Script A's V1.1.8 history, CLAUDE.md,
ARCHITECTURE.md and the release note all say so now; the version stays
V1.1.8.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014RetWKFZFG1ZHcitQAyhwM
CLAUDE.md: status to V1.1.8 / 2026-09-06; the soak loop now prints
'systemctl is-active mlucas-amm'; hammer/ added to the folder list with
why it is not in git; the --runtime gotcha notes the new 3600 default;
three new gotchas - kill_all_process.ttl cannot reach a systemd unit, a
missing config.ttl variable aborts the macro rather than reading as 0,
and the log timestamp FORMAT lives in TERATERM.INI, not in the script.
ARCHITECTURE.md: hammer/ and the config.ttl knobs in the project tree;
Script A ref V1.1.7 -> V1.1.8; Script B and Script C rows rewritten for
the systemd unit; constraint 1 notes that mlucas-amm now outlives the
session; two new constraints (17 - whoever moves a stress job to systemd
owns the matching stop, and Script 4 still uses '&'; 18 - config.ttl
completeness and the TERATERM.INI timestamp format); the git tag
checkbox to V1.1.8; and three known issues - the bundle no longer ships
hammer/, mlucas-amm's '~' depends on systemd's environment, and the
three stress durations are deliberately different lengths.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014RetWKFZFG1ZHcitQAyhwM
mlucas-avx2 was backgrounded with '&' in the login shell, so it belonged
to that one terminal. Close Tera Term, drop the console, or reconnect on
a different session and the load was gone - and 'jobs' from the new
session showed nothing to say so. A multi-day soak could spend most of
its length with an idle CPU while the thermal numbers looked fine.
Script B now starts it as a transient unit:
systemd-run --unit=mlucas-amm --property=Type=exec \
--property=Restart=always --property=RestartSec=2 ...
The log is truncated once with '>' before the unit starts and appended
with '>>' inside it, so a restart adds to the run instead of erasing it.
'systemctl is-active mlucas-amm' now sits beside 'jobs', both in the
one-shot check and in every pass of the monitor loop. The old '&' line is
kept commented out above the new one to fall back to.
Script C has the matching stop. utils/kill_all_process.ttl is only
"kill $(jobs -p)" and a systemd unit is not a shell job, so without this
the load would have run through the post-test snapshot and into the next
test - every temperature and power reading taken at full load, with
nothing in 'show interfaces status' to reveal it. mlucas_amm_log is
copied to /host/hw-eval/jobs/ with the other stress logs.
Two values move into config.ttl:
EN_timestamp feeds logopen's 5th argument in Script A, so the
per-line timestamp is a knob rather than a literal 1
BMC_USB_RUNTIME feeds Script B's ping unit; it was hard-coded to
-w 14400 (4 h), which no longer matched the soak
lengths the SSD and USB knobs were being set to
SSD_RUNTIME and USB_RUNTIME also go 600 -> 3600, so the shipped default
is a one hour stress rather than the ten minutes V1.1.7 went out with.
--runtime is still the job's overall limit; COUNT does not multiply it.
Script A's Version History carries the V1.1.8 entries for all of this.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014RetWKFZFG1ZHcitQAyhwM
src/Script_ABC_Blanton/hammer/ is ~99 MB of unstripped mlucas ELF
binaries (avx512 / avx2 / avx / sse2 / generic-c). They come with the DUT
image, so this repo has no reason to carry them - and a binary tree that
size is permanent once committed: every clone pays for it forever.
publish.sh excluded them too, because rsync copies whatever happens to be
sitting in src/. Without the exclude the delivery bundle was ~100 MB from
a machine that had the binaries and ~200 KB from a fresh clone.
While there, usage() read a fixed line range of the header, so adding a
Version History entry pushed code into the usage text. It now reads up to
the first line of code.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014RetWKFZFG1ZHcitQAyhwM
Script A's Version History gains the reboot-cycle macro, the Script B timeout
fix, the tmp/ rule and the stress knobs.
CLAUDE.md: current status to V1.1.7, a Script 5 paragraph, tmp/ in the folder
list, and two gotchas rewritten - the timeout one now covers both restore
positions, and --runtime is described as the job's overall limit rather than
"a 24 hour ceiling". Dropped the stale "TR518 not scripted yet" line.
ARCHITECTURE.md: Script 5 in the tree and in Key Features, tmp/ in the tree,
constraint 14 extended with "copy the :label along with the goto", and two
known issues added for the parts not yet run on hardware.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014RetWKFZFG1ZHcitQAyhwM
5_Blanton_Script_Reboot_Cycle.ttl loops: login -> sudo -i -> date -s -> fan
speed -> wait_init -> Check Pt1..Pt13 -> show uptime -> reboot -> wait for the
next boot. The cycle number is printed in the banner of every round.
Like Script 4 it is standalone: it does not chain to A -> B -> C and does not
use hw-test-session. Every cycle is a fresh boot, so there is nothing to carry
over and nothing to tear down.
The cdc_ncm probe was copied out of Script B without its :skip_ping label, so
'goto skip_ping' had no target. Tera Term does not check labels at load time -
it raises the error only when it actually jumps, i.e. only when the NCM
interface is missing, which is exactly the run where something is already
wrong. The label is now there, and timeout is restored BEFORE the branch so
one line covers both paths: the ping block waits on a 'sleep 10', already two
thirds of the 15 s cap on a loaded DUT.
Dmesg_i2c and PCIE Error Counters both announced themselves as Check Pt12;
the second is now Pt13.
Not verified on hardware yet. Two things to confirm on the bench: that
wait "Booting in blind mode" really appears in this build's boot output (it
has no timeout guard, so a changed string waits forever), and that re-running
bmc-first-enroll every cycle is safe.
Also ignore tmp/ - scripts not yet run on a DUT stage there, so an unverified
script cannot reach the publish bundle where the tester cannot tell it apart
from the rest.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014RetWKFZFG1ZHcitQAyhwM
SSD_RUNTIME / SSD_COUNT and USB_RUNTIME / USB_COUNT replace the values that
were hard-coded in Script B, so a site changes them in one place. COUNT is
passed straight through as --passes.
--runtime is the job's OVERALL limit, not a per-pass one: qfx5252-stress-*
stops after RUNTIME seconds whatever --passes says. RUNTIME 600 is a ten
minute stress, not 600 x COUNT. A 24 hour soak needs RUNTIME 86400.
Also restore timeout = 0 on the cdc_ncm success path. It was restored only at
:skip_ping, which the success path reaches several waits later - until then
every wait gave up after 15 s instead of blocking, and a wait that returns
without eating the prompt puts the macro a beat ahead of the DUT.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014RetWKFZFG1ZHcitQAyhwM
blanton_multiphase_margin.sh gains 'normal', which returns all 38 rails
to their nominal set point from tools/Multiphase_Normal.txt. Same rails,
same per-rail reporting, same -u 0|1|all as the margin profiles.
Until now the tool could apply high3/low3/comboA/comboB and commit with
save, but had no way to undo. That left a power cycle as the only exit -
and worse, a save run while still margined would have written the offsets
into NVM, which a power cycle does not undo either.
Emitted commands were diffed against the source file: argv identical.
The nominal labels in the source carry extra text ("0.900V (nominal)",
"0.8V(nominal) , 0.8591V with load line"); the per-rail line prints just
the voltage so the column matches the other profiles. The load-line note
is kept as a comment above the data block, because reading 0.859 V back
from TH6_CORE after 'normal' looks like a failed write and is not one.
Script A -> V1.1.7.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014RetWKFZFG1ZHcitQAyhwM
journalctl now runs with -o short-iso into log/bmc_usb_net.log and is
copied into the job directory with the other monitor logs. rm -f first so
the file covers this run only, and the ISO timestamps line up with
mgmt_ping.log and nfc_poll.log for cross-referencing.
The cost, recorded rather than glossed over: it is no longer echoed to
the console, so the USB copy is the only copy. That cp return value is
not checked - the same shape as the nfc_polling.log filename bug, which
failed silently and was only noticed back at the desk with a job archive
missing a file.
Folded into V1.1.6 rather than bumping: the tag has not moved on and this
is the same delivery.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014RetWKFZFG1ZHcitQAyhwM
Every PDB reading came back NA, and the cause was a base error. The four
ADPM12200 bricks put each measurement on its own PMBus PAGE, written
before the read. The bring-up spreadsheet listed those page numbers in
decimal but with an 0x prefix:
Vin PAGE 9 -> it said 0x00 (0x09)
Iin PAGE 10 -> it said 0x10 (0x0A)
Vout PAGE 2 -> it said 0x02 correct by luck
Iout PAGE 14 -> it said 0x14 (0x0E)
Temp PAGE 18 -> it said 0x18 (0x12)
Only Vout worked, because 2 reads the same in either base. Hence Iin and
Iout answering 0xFFFF, Temp answering 0x0000, and a Vin scale factor
having to be invented to make 50 V appear out of a page-0 register.
pwr_brick.sh reads all five bricks with the right pages and the
datasheet's DIRECT equation, X = (1/m)(Y x 10^-R - b): voltage Y x 8 mV,
current Y x 0.04 A, temperature Y x 0.01 C. Cross-checked rather than
assumed - READ_VOUT 0x05D4 decodes to 11936 mV, exactly what
'show platform voltage' reports for that rail through a separate sensor
path. The datasheet is committed alongside so the numbers are checkable.
Left flagged: Table 3 does not list READ_VIN or READ_IIN. They borrow the
voltage and current coefficients here, which is consistent with a ~50 V
input but is not something the datasheet states.
Each value is re-read until it passes three checks: not 0xFFFF, not
0x0000, and inside a plausibility window. The window is the one that
matters - this bus corrupts the HIGH BYTE only, which turns 11.98 V into
32.00 V. Positive, plausible in magnitude, and invisible to any all-ones
filter. 0x0000 is rejected for the mirror-image reason: on the
temperature register it decodes to a believable 0 C.
Adds comboA/comboB margin profiles per board. comboB is comboA with every
direction flipped, so the pair covers each rail high and low while its
neighbours sit the other way. NC channels stay nominal in both - there is
nothing to margin on an unused rail.
Script C re-enables the BMC USB journalctl dump, which had been commented
out, so that service log now reaches the master log.
Script A -> V1.1.6.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014RetWKFZFG1ZHcitQAyhwM
qfx5252-stress-usb was started with --runtime 60. It finished a minute
into an overnight run and the USB path sat idle for the rest of the
night - nothing failed, nothing said so, and the log looked complete.
SSD had the same shape at --runtime 3600. Both are now 86400, and the
BMC DDR memtester drops its loop count so it runs until stopped.
86400 is 24h, not forever. Past that SSD and USB die quietly and the
only clue is the shorter bgctl list Script B already prints each round.
Said so in the version history, the release notes and ARCHITECTURE.
Adds blanton_multiphase_margin.sh: the multiphase controllers on both
switch boards over the native i2c buses (SWB0 on i2c-13/15, SWB1 on
-14/-16), 19 rails each. high3/low3/comboA/comboB set voltages and print
one line per rail; save commits them and prints Multiphase Save All,
because that list has no per-rail mapping to report against.
Two hazards it carries: save is STORE_USER_ALL and permanent, and these
are NOT the LTC2980s margin.sh drives - the two tools reach the same
rails from opposite sides with no interlock between them.
The emitted commands were diffed against tools/Multiphase_*.txt: argv
identical for all five. The rail-to-netname mapping was cross-checked
against the bus numbers rather than trusted by line position - blocks
1..19 are all bus 13/15 and 20..38 all 14/16, matching the SWB0/SWB1
split of Multiphase_Netname.
blanton_traffic_linespeed v0.7.0 appends PatternRandom=yes to every tx
including ce0, and the defaults become tx 200 length=324. A fixed
pattern can sit on a benign bit sequence for a whole run; a changing one
is what exposes a weak lane. -r no forces fixed, -r "" omits the
argument for an SDK that will not take it. Bytes per burst went 51200 ->
64800, so absolute counter totals do not carry across this build - the
pair cross-check that decides PASS/FAIL does.
Script A -> V1.1.5.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014RetWKFZFG1ZHcitQAyhwM
4_Blanton_Script_thermal_safety.ttl brings the DUT up the way Script A
does, then loads it - mlucas-avx2 -cpu 0:15 in the background plus
blanton_tr518.sh start - and samples platform data and both switch
boards' rail power every minute. Power per rail under sustained full
load, over hours, is the whole point of it.
It is standalone: nothing starts it, it starts nothing, and it must not
share a DUT with Script B because both take over the cd ports.
It also has no teardown. When you interrupt it the unit is still in
port cd lb=mac / l2 learn off / test mode nr=yes and mlucas is still
running, and show interfaces status does not reveal any of that. Called
out in the version history, the release notes, CLAUDE.md gotchas and the
ARCHITECTURE future-work list, because whoever uses the box next will not
find out on their own.
Script B's per-round loop drops the margin scan. The loop is meant to be
a quick minute-by-minute sample and margin_status_all.sh walks nine
LTC2980s over I2C, which dominated the round. Script C takes it instead,
once, before show uptime. The trade-off is real: margin drift DURING the
soak is no longer visible, only a start and an end reading. Documented
rather than glossed over, with a suggestion (scan every N rounds) if it
needs to come back.
Script C also sends exit and waits for "Script done" before
hw-test-session finish, so finish runs in the login shell rather than
inside the session it is closing.
Adds docs/Blanton_Test_Flow.drawio: all four scripts side by side, with
the config flags that change each step marked, Script B's one real goto
drawn, and the A->B->C handoffs shown as the manual steps they are.
The new script arrived LF-only while every other .ttl is CRLF and
.gitattributes marks *.ttl as -text, so what is committed is exactly what
Tera Term reads. Converted.
Script A -> V1.1.4.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014RetWKFZFG1ZHcitQAyhwM
86 of the 144 margin channels moved up +0.8% .. +2.5% to the measured
median for this platform (3.3 -> 3.38, 0.9 -> 0.915, 0.8 -> 0.82, ...).
That is not a display change. VNOM is the denominator of margin_status'
deviation %, and it is also the basis margin_set derives high/low/OV/UV
from - so both the number you read and the voltage you apply move with
it. A rail steady at 0.756 V used to read +0.80% against a nominal 0.75
and now reads 0.00%. Margin logs from before and after cannot be
compared: the reference moved, the hardware did not.
comboA/comboB are replaced by combo_SWB_CONN13..16_{high,low}. The
per-channel percentages are positional, so they follow the rails of one
specific board - applying a CONN13 profile to a CONN16 margins the wrong
rails and nothing detects it. SWB0 and SWB1 share a profile per CONN
because their rails are identical.
Fixes two things the deletion broke:
- margin_apply_profile_all.sh still named comboA for all nine boards,
which would now stop at "Profile not found" nine times over. It names
the matching per-CONN profile instead, with CB on the generic
combo_high3, and a note that the low pass is _high -> _low.
- docs/LTC2980_channel_map.csv is generated from settings/*.conf, so
all 86 changed rails were stale in it. Regenerated.
The eight new profiles also still carried comboA's header comment
("alternating high/low"), which describes neither what they contain nor
what they are for.
Script A -> V1.1.3.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014RetWKFZFG1ZHcitQAyhwM
nfc_polling.sh wraps blantons_nfc_validate.py in a background loop and
keeps score: start/stop/status/tail/report/clear/fg. Script A takes a
30s baseline, Script B starts the poll for the soak, Script C stops it
with the other monitors and reports.
report only reads the log, so it can be run mid-soak without disturbing
anything - and it says RUNNING so a snapshot is not read as the final
answer. Each round appends a machine-readable RESULT line, which keeps
the tally independent of the tool's wording.
ERROR is counted apart from FAIL. A round where the tool printed neither
[PASS] nor [FAIL] - crashed, missing, hung past the timeout - measured
nothing, and that is a different fault from a tag that would not read.
Rolling them together hides whichever one you are not looking for.
Two habits from earlier bugs are baked in: tail does not follow (tail -f
never returns and a macro waiting on the prompt hangs there), and the .py
is called through python3 rather than ./ (it arrives mode 100644 and a
USB copy carries no execute bit).
Also fixes Script C copying "nfc_polling.log" to the job dir when the
file is "nfc_poll.log". The cp failed, nothing checked it, and the run
finished looking fine - you would find out at the desk, with a job
archive that had no NFC log in it.
Script A -> V1.1.2.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014RetWKFZFG1ZHcitQAyhwM
After a run that ended badly - Tera Term closed, DUT rebooted mid-soak,
macro stopped by hand - the stress jobs kept running and the test session
stayed open. The next run then started on a box that was already loaded.
Nothing errored; the numbers were just quietly wrong.
Script A now opens with hw-test-session finish, bgctl reset --yes,
bgctl stop --all, then bgctl list. The list is the point of the sequence:
it is the one line in the log that proves the DUT was idle at the start.
Script A -> V1.1.1.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014RetWKFZFG1ZHcitQAyhwM
Three new tools and one fix to an old one.
blanton_ber.sh - PRBS BER across every cabled port, init/start/report/
stop/clear, -u 0|1|all. A lane passes only below 1e-6 (equal to the
threshold fails). report ends with a per-unit summary carrying the worst
lane, so an all-PASS run still shows how much margin there was. init
checks every lane locked, because an unlocked lane still reports a BER
and it is a meaningless one. Script C runs it after the traffic report,
following SWB_UNIT0/SWB_UNIT1 like the traffic block does.
blanton_tr518.sh - the built-in packet test, used to load the rails
before measuring power. start settles 120s so the reading afterwards is
loaded rather than mid-ramp. stop sends port cd lb=none and nothing else,
exactly what the bench procedure does - l2 learning and test mode are
left as start set them, and stop says so instead of quietly leaving the
box in a state nobody asked about.
TH6_SWB{0,1}_power_readback.sh - 19 rails, V x I and total. These read
the two sensor tables ONCE. The previous version read them per rail: 38
invocations of a slow CLI at 38 different instants, so the "total" was a
sum of readings seconds apart - and under load the currents move (TH6_CORE
186 A -> 636 A). A rail missing from the table now warns on stderr instead
of silently contributing 0 W.
blanton_traffic_linespeed.sh - ce0 (VLAN 138) joins the test. It has no
loopback partner and a switch never sends a frame back out its ingress
port, so a tx burst was one round trip and then silence. init now sets an
ingress mirror of ce0 back to itself; the mirrored copy is not
egress-filtered, so packets keep lapping until stop turns the mirror off.
stop is what ends it - removing the VLAN membership does not, because the
copy never consulted the VLAN. TL_CE_MODE also drops to 'self': ps ce
shows one ce port per unit and a unit-0-only burst returned on unit 0.
Read as cross-unit, both links dying would still report PASS (0 == 0).
Fixes a parser bug worth remembering: bcmcmd returns CRLF, and the BER is
the last token on its line, so the CR stayed glued to the number. Every
lane read as NA, and printing the CR sent the cursor to column 1 so the
RESULT column overwrote the start of the row. One stray \r, two symptoms.
Not yet verified on hardware, all called out in the release notes: the
ce0 mirror (destport == srcport may be refused), the 120s settle (tr 518
may be time-limited), and 'test mode nr=no' on stop.
Script A -> V1.1.0.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014RetWKFZFG1ZHcitQAyhwM
FAN_SPEED 30 -> 100. A soak is a thermal test of everything except the
cooling, so the cooling should not be one of the variables in it. At 30%
a long run can throttle partway through, and throttled results read like
a different fault entirely.
Folded into V1.0.11 rather than bumping the version: the tag has not
moved on and this is the same delivery.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014RetWKFZFG1ZHcitQAyhwM
margin_save discarded the result of every write, so a NACK printed
"Change Saved" exactly like a successful store. For the one permanent
operation in this tool that is the worst place to stay quiet. Each store
is now checked, named on failure, and reflected in the exit code.
STORE_USER_ALL also went out as 0x15 followed by a dummy 0x00 - an SMBus
write-byte where the command is a send-byte. The LTC2977 tolerated it,
but tolerance is not correctness. Now i2cset ... 0x15 c on the native
path and a no-data cb_pmbus_write on the FPGA path.
Two new subcommands:
margin_save all - all nine boards in settings/, then re-init
whichever board was loaded beforehand so a
following margin_status still reports the board
you were looking at. Does not stop on the first
failure; a half-saved set is harder to reason
about than a fully-attempted one.
margin_save blanton - this platform exposes the CB FPGA F3 I2C
channels as native Linux i2c buses (Ch6->13,
Ch7->14, Ch8->15, Ch9->16; CB stays on bus 4),
so all 18 LTC2977 can be stored with plain
i2cset: no margin_init, no FPGA channel setup,
no pcimem. 18 commands instead of 42.
The 18 raw commands were verified on the DUT 2026-08-27; the wrapper was
tested against stubbed hardware only. Release notes say so.
Board list is MARGIN_BLANTON_STORE at the top of margin.sh - comment out
what this DUT does not have rather than letting it talk to absent boards.
Missing bus and unresponsive chip are reported as distinct failures.
MARGIN_STORE_SETTLE (0.5s) sits between stores because nothing here polls
the part's busy bit, and blanton fires 18 in a row.
Script A -> V1.0.11.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014RetWKFZFG1ZHcitQAyhwM
Script B's monitoring loop had no pause at all - each round started the
instant the last one ended, so an overnight soak filled the log with
near-identical samples taken seconds apart. The section was even labelled
"Get data every 10mins" while running with no delay whatsoever.
The loop now ends each round with pause 60 and prints both job lists:
bgctl list for the platform job runner, jobs for anything the macro
backgrounded in that shell. A stress job that dies mid-soak shows up as a
shorter list on the next round instead of going unnoticed until the end.
60s is still hard-coded and the Status sheet asks for 10 minutes, so the
Phase 3 item stays open - this makes the loop sane, it does not close it.
Also adds docs/LTC2980_channel_map.csv: the nine settings/*.conf flattened
into one Board,CONN,Ch,NetName,Vnom sheet, generated by
tools/gen_channel_map.sh. The CSV is output, not source. NC channels are
kept as NC/- because the channel number is also the PMBus page - dropping
the gaps would shift every channel after them.
Two things the merged table makes visible: SWB0 and SWB1 carry identical
net names and voltages (only the bus and CB_I2C_CH differ), so those are
two copies that can drift apart silently; and there are exactly 8 NC
channels, all on CONN14/CONN15.
tools/100G_PRBS.txt, TR518.txt and fan_ctrl.txt join the existing
traffic_loopback_*.txt references. TR518 is not scripted yet and cannot
share the hardware with blanton_traffic_linespeed - both drive port
loopback and L2 learning.
Script A -> V1.0.10.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014RetWKFZFG1ZHcitQAyhwM
port_prbs_monitor.sh drives PRBS on both 100G uplinks: lpmode off, arm
the pattern, then poll `phy diag <phy> prbs get` in the background until
stopped, counting a poll as PASS only when the output says "PRBS OK!".
prbsstat Ber is captured beside every poll for the record but does not
decide the verdict. `stop` runs prbsstat STOp and prbs clear before
rendering the report, so the test does not leave PRBS armed.
`start a` / `start b` runs one uplink alone, which is how a genuine port
fault is separated from the DUT not coping with two PRBS streams at
once. A port that was not run reports SKIP rather than FAIL -- deciding
on poll count alone would have made a deliberate single-port run look
like half the hardware was broken.
The port status is deliberately not printed at `start`: with PRBS armed
the link reports DOWN, which reads as a failure to anyone glancing at
the console. It appears in the report instead, after prbs clear and
labelled as the recovered state.
The calls in Script A, B and C are committed but commented out -- PRBS
is still under bring-up.
Fix: the job directory was written as /host/hw-eval/current/jobs in some
places and /host/hw-eval/jobs in others, so what Script B wrote was not
what Script C collected. Unified on /host/hw-eval/jobs.
Fix: Script C copied to /mnt/usb assuming it was mounted. It now finds
the device with usb_target.sh and mounts it first, so the logs actually
leave the DUT.
Script C also reorders its results -- stress logs, then the MGMT ping
report, then 100G status -- and cats the bgctl ssd/usb logs.
Script A -> V1.0.9. Docs and docs/release-notes/v1.0.9.md updated.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014RetWKFZFG1ZHcitQAyhwM
Script A: the baseline ping window is 30 s rather than 45, and the log
is cleared afterwards so Script B starts from nothing.
Script B: `status` after `start`, and Script C: `status` before `stop`
plus a `cat` of the report. The status line shows both pids and the
replies received per NIC, so a dead link is visible while the soak is
still running instead of only at the end.
docs/release-notes/v1.0.8.md written in the house format.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014RetWKFZFG1ZHcitQAyhwM
The bench run reported FAIL=3 with no NIC errors and no drops. All three
failures were on eth1 and all three were missing exactly icmp_seq=1,
with the other nine replies present and sub-millisecond. That is
neighbour resolution: once the ARP entry for the target expires, the
first echo request is spent resolving it and ping counts it as loss.
Rounds sat about 24 s apart, right on the edge of the default reachable
time, so it happened on some rounds and not others.
mgmt_ping_monitor.sh V3.0.0:
- One discarded ping per NIC before the measured run resolves ARP, so
MAX_LOSS_PCT can stay at 0 and a FAIL means a real drop. Raising the
threshold instead would have hidden genuine single-packet loss, and a
longer settle would not help -- nothing triggers ARP until the ping.
- Both NICs now ping at the same time and keep accumulating until
stopped, rather than alternating fixed bursts. Each writes its own raw
capture.
- `stop` sends SIGINT rather than SIGTERM, because ping prints its
statistics block on interrupt and that block is what the report
parses, then renders into the log: the last TAIL_LINES entries per
NIC, each one's statistics and verdict, and `ip -s link show` for both
interfaces.
- `-D -O` are on by default so every line carries a timestamp and a
request that got no reply is visible in the tail instead of merely
absent. Clear PING_EXTRA_OPTS if the platform's ping rejects them.
Since the script no longer re-execs itself to daemonise, `bash
mgmt_ping_monitor.sh start` works without the execute bit.
.gitignore: the new *.raw / *.pid / *.state runtime files were outside
the *.log rule and would have been committed.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014RetWKFZFG1ZHcitQAyhwM
New Blanton_Script/usb_target.sh reports a USB mass-storage device's
node, mount point or by-id path. It excludes whatever disk backs / and
/host -- this platform can boot from a USB DOM, which also reports
TRAN=usb -- refuses to guess when several USB disks are present, falls
back to the whole device for superfloppy media, and when mounting,
checks the result is actually writable rather than trusting that mount
succeeded.
Script B asks it for the device node and builds the stress command from
the answer, so a stick that enumerates as sdb no longer sends the write
test at whatever /dev/sda1 happens to be.
The device is carried back through an echo marker, split in the shell
literal as "<<D""EV=..." so the command echo cannot satisfy the pattern
the macro waits for -- the same false-match guard used for PORTS0= and
NCMIF=, arranged differently.
Script B: DDR memtester drops its 100-pass limit and runs until stopped,
matching the other soak loads.
Fix: Script C ran `bgctl reset --yes` while killing processes, which is
before the job logs are copied to USB. Moved to after the archive and
after hw-test-session finish, so the run's own logs are collected before
anything clears them.
Script A -> V1.0.8 history updated.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014RetWKFZFG1ZHcitQAyhwM
Script C now copies log/mgmt_ping.log into
/host/hw-eval/current/jobs/ alongside bmc_poll.log, so the management
link results leave the DUT with the run instead of staying behind. Both
monitors' logs are now covered.
Script A history: the two copies are one action in one block, so they
share a single V1.0.8 line rather than reading as separate changes.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014RetWKFZFG1ZHcitQAyhwM
Script C copies log/bmc_poll.log into /host/hw-eval/current/jobs/ before
that directory is archived to USB, so the BMC sampling and I2C
read-back results leave the DUT with the run they belong to rather than
only existing in the Tera Term capture.
The wait moves ahead of the getdate/sprintf2 block; those build strings
without touching the terminal, so the prompt is still consumed exactly
once before the copy is sent.
Script A history updated under the existing V1.0.8 block -- that version
is not tagged yet, so this is the same batch.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014RetWKFZFG1ZHcitQAyhwM
bmc_monitor.sh comes back into the run with a second job beyond the
`free -m` sampling: an I2C pattern test against bus 1, address 0x41,
offset 0x00C0 -- write 55AA55AA and read it back, then write AA55AA55
and read that back. Two complementary patterns catch a stuck bit either
way round, and repeating it through the soak turns an intermittent I2C
fault into something the log shows rather than something the tester has
to catch live.
Script B starts it alongside the other stress; Script C stops it and
cats log/bmc_poll.log.
bmc_monitor_ddr.sh stays disabled -- BMC DDR stress runs through
bgctl + bmc-manager.
Docs: the "disabled" notes I added last time now applied to only half
the pair and read as wrong for bmc_monitor.sh. Split so the monitor is
documented as live with its I2C test, and only the _ddr variant is
marked disabled.
Script A -> V1.0.8.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014RetWKFZFG1ZHcitQAyhwM
Script A carried a V1.0.7 history block while its "; Version :" header
still read V1.0.6, so publish.sh built Script_ABC_Blanton_V1.0.6 and the
folder would have disagreed with the tag. Header and date corrected --
reading the version from the header rather than a flag is what surfaced
this.
utils/wait_init.ttl V3.1.1: poll interval 10 s -> 60 s, and the WT_MIN
comments now say "at least" to match the test, which has been
`< WT_MIN` since the threshold was fixed. The stale "more than" wording
described exactly the off-by-one that once hung Script A.
Script C: the BMC USB journalctl dump is commented out.
docs/release-notes/v1.0.7.md written in the house format.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014RetWKFZFG1ZHcitQAyhwM
config.ttl gains FAN_SPEED (30). Script A rebinds the two max31790
controllers (18-0020, 25-0020) and applies the speed with
fan-speed-control.sh, answering its confirmation prompt, so a run starts
from a known thermal state rather than whatever the last test left.
Script A also clears /host/hw-eval/current/jobs before the run, and
Script C copies that directory to /mnt/usb/jobs-<date>_<time> so the
bgctl job logs leave the DUT with the run they belong to.
Fix: 25-0020 was bound twice in a row -- the second bind can only fail,
since the driver is already attached. Removed; the two controllers now
have a matching unbind/sleep/bind each.
Script A -> V1.0.7, history covering the config.ttl and Script C changes
as well.
Docs: fan control and the job-log archive added to Tech Stack, Key
Features and Data Flow; the config.ttl example refreshed with FAN_SPEED
and the SWB_UNIT flags. Three risks recorded in Future Extensions --
`rm` without -r cannot clear the job subdirectories that Script C copies
with -r, the fan confirmation wait has no timeout and would hang
silently if the prompt ever changes, and /mnt/usb is never checked for
being mounted.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014RetWKFZFG1ZHcitQAyhwM
Script A: wait 45 s between starting and stopping mgmt_ping_monitor.
`start` returns after one second, but a round is roughly 30 s -- without
the pause the log held only the START banner and no RESULT line.
Docs brought up to date with the eight commits since 419b680:
- Tech Stack gains bgctl, systemd-run, iproute2 and hw-test-session;
the stress row moves off the hammer scripts
- Structure and Key Features cover mgmt_ping_monitor.sh, and mark
bmc_monitor{,_ddr}.sh as disabled -- their call sites in B and C are
commented out, though the files remain
- Data Flow rewritten for all three scripts
- Two constraints added: TTL's `timeout` is global and must be restored,
and two management NICs on one subnet cause ARP flux (hence
ARP_STRICT, and nic_tx/nic_rx as the evidence of which NIC sent)
- CLAUDE gotchas add the `ip -s link` column order, that a monitor's
`start` returns in one second rather than after a round, and that
show_dmesg now ends with `dmesg -C`
- Phase 3 ticks the move to bgctl; Phase 5 gains the mgmt-ping, NVMe and
EDAC parsing targets. Future Extensions notes that Script C never cats
mgmt_ping.log, so the soak's ping results stay on the DUT
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014RetWKFZFG1ZHcitQAyhwM
New Blanton_Script/mgmt_ping_monitor.sh: brings both management NICs up
with iproute2 (`ip link set` / `address replace` / `route replace` with
per-NIC metric) and pings each one's own target -- eth0 -> .30, eth1 ->
.31 -- appending to a rotating log. Same sub-commands as the bmc
monitors, plus `summary` for just the per-leg RESULT lines.
Both NICs stay up, which on a shared subnet lets the target's ARP be
answered by either one, so a reply can land on the NIC that did not
send. ARP_STRICT applies arp_ignore/arp_announce to prevent that, and
each leg logs `ip -s link show` with the interface's own TX/RX packet
delta across the burst as direct evidence of which NIC carried the
traffic. Note `ip -s link` orders columns "bytes packets ...", so the
packet count is the second field.
Script A: hw-test-session start/log/status, a fuller DUT inventory
(version, fwutil, syseeprom, ssdhealth, TPM, nvme smart-log, smartctl),
bmc-first-enroll and bmc-manager version/status, ras-mc-ctl summary, and
both 100G uplinks now brought up rather than 513 being left down.
Script B: stress moves to bgctl (memtester, qfx5252-stress-ssd/-usb) and
the BMC DDR load runs through bmc-manager, replacing the hammer scripts
and the bmc_monitor pair. Adds a BMC USB net test that discovers the
cdc_ncm interface and runs a 4-hour ping under systemd-run.
Script C: stops the bgctl jobs, the BMC USB unit and the mgmt ping, then
collects journalctl, per-NIC counters, ras-mc-ctl and NVMe health, and
closes the session with hw-test-session finish.
utils/show_dmesg.ttl: one combined error regex with `dmesg -T`, an
i2c-filtered view, then `dmesg -C` so Script C's capture shows only what
the soak produced. utils/setup_pmon.ttl: add the TPM FRU/read checks.
Fix: Script B set `timeout = 15` for the cdc_ncm probe and never
restored it, leaving the cap in force for everything after -- including
the traffic init, which walks 108 VLANs per unit and takes far longer
than 15 s. A timed-out wait returns without the prompt, so the macro
would have run ahead of the DUT for the rest of the soak. Reset to 0 at
:skip_ping, where both branches meet.
Script A -> V1.0.6.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014RetWKFZFG1ZHcitQAyhwM
`config save -y` was sent straight after the Ethernet514 startup with no
wait in between. The shell buffers the second line and still runs it, so
nothing visibly breaks, but every wait from there on matches the prompt
of the previous command -- leaving the macro permanently one step ahead
and issuing `show interfaces status` while `config save` is still
running. Every other sendln in the file waits first; this one now does
too.
Version left at V1.0.5.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014RetWKFZFG1ZHcitQAyhwM
wait_init.ttl: the comparison was strictly greater than WT_MIN, so a unit
reporting exactly 216 never passed and Script A sat in the poll loop
forever. 216 is not an arbitrary number -- TL_PAIRS holds 108 loopback
pairs, so 108 x 2 = 216 is every cabled port being up, i.e. precisely the
state being waited for. Comparison is now `< WT_MIN`, making the
threshold "at least 216".
Script A: bring the 100G uplinks into a known state before reading their
status -- Ethernet513 on asic0, Ethernet514 on asic1 -- and persist it
with `config save -y`. The stray `wait` after show_dmesg is dropped; the
one opening the new block consumes that prompt instead.
Version left at V1.0.5.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014RetWKFZFG1ZHcitQAyhwM
Rework the traffic blocks in A, B and C as explicit if/elseif branches
holding literal commands, and drop the derived swb_any/swb_opt from
config.ttl -- the branches read straight off the page and there is no
indirection to follow.
All nine conditions compared SWB_UNIT0 against itself, though:
if SWB_UNIT0 = 1 && SWB_UNIT0 = 1 -> SWB_UNIT0 = 1
elseif SWB_UNIT0 = 1 && SWB_UNIT0 = 0 -> never
elseif SWB_UNIT0 = 0 && SWB_UNIT0 = 1 -> never
so the single-unit paths were unreachable. A DUT with only unit 0 would
have run the both-unit commands and hit bcmcmd on an absent unit 1,
while a DUT with only unit 1 fell through to the empty else and skipped
traffic entirely. Second operand is now SWB_UNIT1.
Verified per branch that the -u argument matches the condition guarding
it: both units carry no -u (tool default TL_UNITS="0 1"), unit-0-only
carries -u 0, unit-1-only carries -u 1, and the else stays empty.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014RetWKFZFG1ZHcitQAyhwM
The bench is not always fully populated. config.ttl now declares which
switch units exist:
SWB_UNIT0 = 1 ; this DUT has switch unit 0
SWB_UNIT1 = 1 ; this DUT has switch unit 1
Two values are derived there rather than repeating the same test in
three scripts: swb_any (0 = no unit at all) and swb_opt, the suffix
appended to each blanton_traffic_linespeed call -- "" for both units so
the tool's own TL_UNITS="0 1" applies, " -u 0" or " -u 1" for a single
one.
Script A, B and C build their traffic commands with sprintf2 and the
suffix, wrapped in `if swb_any = 1`. A half-populated DUT no longer
issues bcmcmd against an absent unit, and a DUT with no switch board
skips the traffic stage outright instead of filling the log with
failures.
wait_init.ttl reads the same two flags instead of its own copies, so the
wait and the traffic blocks cannot disagree about what is installed.
Script A -> V1.0.5.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014RetWKFZFG1ZHcitQAyhwM
wait_init.ttl V3.0.0 adds WT_UNIT0 and WT_UNIT1 at the top of the file.
Benches are not always fully populated, and V2.0.0 waited on both units
unconditionally, so a DUT with one switch board sat in the poll loop
forever -- silently, since the loop neither advances nor reports.
both units -> 1 , 1
unit 0 only -> 1 , 0
unit 1 only -> 0 , 1
no unit -> 0 , 0 (bypass, returns immediately)
A disabled unit is skipped rather than polled and ignored: its whole
block sits inside the if, so no bcmcmd is issued for it and no
misleading error reaches the log.
Two independent integer flags rather than a "0 1" string, because
parsing a string in TTL needs strscan and this is meant to be edited by
hand at the bench.
All three exit paths still leave exactly one prompt unconsumed, so
Script A's surrounding waits are unaffected.
Script A history: recorded under the existing V1.0.4 block rather than a
new version, matching the consolidation done there.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014RetWKFZFG1ZHcitQAyhwM
Script A: `hpe-eeprom-tlv show --bus 4 --addr 0x50` after `show boot`, so
the board identity is on record next to the image it booted. The empty
Check History placeholder is dropped.
Script A and B: disable the lldp feature and `config save -y` before the
traffic stage, so the switch stops sourcing its own frames and the
loopback pair counters reflect only the injected burst.
Script B: `sfputil lpmode off` on Ethernet513/514 before reading their
status -- a transceiver left in low-power mode will not link.
Script A -> V1.0.5, with the history covering the Script B changes too,
per this project's convention of keeping one consolidated log in A.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014RetWKFZFG1ZHcitQAyhwM
wait_init.ttl V2.0.0 now polls
bcmcmd -n 0 -c ps | grep -w up | wc -l
bcmcmd -n 1 -c ps | grep -w up | wc -l
and proceeds only when BOTH exceed 216, so the baseline is taken with
the data plane actually up rather than merely with pmon answering.
V1.0.0 could use `wait "Thermal Not detected" prompt` because that was a
string-presence test. Comparing a count needs the value captured, so the
count is wrapped in an echo marker and read with waitregex +
groupmatchstr1 + str2int. The command echo cannot false-match: it reads
"PORTS0=$(bcmcmd ..." and the pattern requires a digit immediately after
the "=". The optional-space allowance covers a wc that pads its output.
The two thresholds are compared in nested ifs rather than with `and`,
which is bitwise in TTL.
WT_MIN and WT_INTERVAL are at the top of the file. The enter/exit prompt
contract is unchanged, so Script A's surrounding waits still line up.
Script A -> V1.0.4.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014RetWKFZFG1ZHcitQAyhwM
publish/publish.sh packages src/Script_ABC_Blanton into
publish/Script_ABC_Blanton_<Ver>/, with -z for a zip, -f to overwrite,
-n for a dry run.
The version is read from Script A's "; Version :" header rather than
passed in, so the folder name cannot disagree with what the tester sees
on opening the macro. The header is CRLF, so the carriage return is
stripped -- left in, it becomes part of the directory name. Captured
logs are excluded (they carry DUT serials) but the empty logs/ is kept
because logopen writes there. .gitignore now admits the tool while
still ignoring its output.
config.ttl: testcase ENV, margin scan off for this run.
Docs brought up to date with the 14 commits since 8aa5c90 -- SWB
channel/address map, the traffic and BMC tooling, line-ending policy,
and the exec-bit and tail -f traps.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014RetWKFZFG1ZHcitQAyhwM
New utils/wait_init.ttl polls `show platform temperature` every 10 s
until it stops reporting "Thermal Not detected", so Script A does not
record a baseline while pmon is still coming up. Script A calls it right
after `show boot` and bumps to V1.0.3.
The include pointed at utils/wait_thermal.ttl while the file is
wait_init.ttl -- as Script A's own V1.0.3 history line says. Corrected;
Tera Term aborts the macro when an include cannot be opened, so this
would have stopped Script A at line 59 on the first run.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014RetWKFZFG1ZHcitQAyhwM
Script A and B: `show interfaces status Ethernet513,Ethernet514` before
the traffic stage, so the uplink state is on record next to the counters
it explains. In B this takes the slot the empty "eye measurement"
placeholder held.
Script B: drop -u 0 from the traffic calls. A and C already defaulted to
TL_UNITS="0 1", so B was pinned to unit 0 while C stopped both -- the
soak only ever loaded one switch board, and C's stop hit a unit that was
never initialised. All three scripts now cover both units.
Script B: post-clear settle 10 s -> 15 s, matching A.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014RetWKFZFG1ZHcitQAyhwM
`bmc_monitor.sh tail` runs `tail -n 50 -f`, which never returns, so
Script C stopped at the first call and never reached CHECK Stress
results, the traffic counters, or the closing messagebox. Stopping the
monitors first made it worse: with the file no longer growing, tail -f
just waits forever.
Read the two logs directly instead, so the whole BMC capture lands in
the master log and the macro carries on.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014RetWKFZFG1ZHcitQAyhwM
Script A: dump the BMC banner via `bmc-manager run 'cat /etc/issue'`,
and add the missing `wait` before `show boot`.
Script B: start bmc_monitor_ddr.sh and bmc_monitor.sh alongside the
host stress load.
Script C: stop both and dump their logs into the master log.
bmc_monitor.sh: INTERVAL_SEC 5 -> 10.
Known blockers, filed here so they are not lost -- this wiring does not
work yet as written:
1. `bmc_monitor.sh tail` runs `tail -n 50 -f`, which never returns.
Script C blocks on the first one and never reaches CHECK Stress
results, the traffic counters, or the final messagebox. It needs a
non-following dump (cat) instead.
2. Both scripts are mode 100644 and are the first .sh here meant to be
executed rather than sourced. do_start re-execs "$SCRIPT_PATH"
__daemon directly, so the execute bit is required even when invoked
through bash -- and a Windows -> USB -> DUT copy cannot carry it.
A chmod +x is needed before the start calls, as Script B already
does for mlucas-avx2.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014RetWKFZFG1ZHcitQAyhwM
Both scripts defaulted to LOG_NAME="bmc_poll.log", and PID_FILE is
derived from it, so the monitor and the DDR stress shared one log and
one pid file -- while the whole point of the split is to run them at
the same time.
The failure was quiet rather than loud: is_running greps the pid's
cmdline for its own SCRIPT_NAME, so the second script did not recognise
the first as running and started anyway. prepare_log_on_start then
truncated the log the first one was writing (START_LOG_MODE=new), both
raced on the pid file, and a later stop could only reap whichever wrote
it last, orphaning the other.
bmc_monitor_ddr.sh now uses bmc_ddr.log, which also moves its pid file
to bmc_ddr.pid. Both remain covered by the *.log ignore rule.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014RetWKFZFG1ZHcitQAyhwM
bmc_monitor.sh was doing two unrelated jobs: sampling `free -m` every
few seconds, and running `memtester 500M 1` as a stress load. They want
different timeouts -- a monitor should give up in 30 s, a memtester
pass legitimately runs for minutes -- and mixing them meant either
cutting memtester short or letting a hung sample stall the loop.
bmc_monitor.sh now polls `free -m` only, keeping CMD_TIMEOUT_SEC=30.
bmc_monitor_ddr.sh is the stress half: memtester only, with
CMD_TIMEOUT_SEC raised to 3600. The two are otherwise identical.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014RetWKFZFG1ZHcitQAyhwM
bmc_monitor.sh (V1.1.0): polls the BMC from the SONiC host via
`bmc-manager run`, appending to a rotating log. Runs detached, so no
interactive SSH session to the BMC is needed and the BMC's busybox
toolchain never comes into play. Subcommands start/stop/status/fg/tail/
clear; the polled command list, interval, per-command timeout and log
rotation are configured at the top of the file. START_LOG_MODE defaults
to "new", so one run means one log. Default COMMANDS are `free -m` and
`memtester 500M 1`, i.e. the BMC is memory-stressed alongside the host
soak.
Script A: `show boot` after the time sync to record the image the DUT
booted, and `show uptime` at the end of the traffic baseline.
Script C: `show reboot-cause` and `show uptime` after the traffic
report, so an unplanned reset during the soak is visible in the
closing snapshot rather than only in dmesg.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014RetWKFZFG1ZHcitQAyhwM
etwenandClaude Opus 5