36 Commits
Author SHA1 Message Date
etwenandClaude Opus 5 15ea776b82 feat(soak): Show the background jobs each round, and stop the loop spinning
Script B's monitoring loop had no pause at all - each round started the
instant the last one ended, so an overnight soak filled the log with
near-identical samples taken seconds apart. The section was even labelled
"Get data every 10mins" while running with no delay whatsoever.

The loop now ends each round with pause 60 and prints both job lists:
bgctl list for the platform job runner, jobs for anything the macro
backgrounded in that shell. A stress job that dies mid-soak shows up as a
shorter list on the next round instead of going unnoticed until the end.

60s is still hard-coded and the Status sheet asks for 10 minutes, so the
Phase 3 item stays open - this makes the loop sane, it does not close it.

Also adds docs/LTC2980_channel_map.csv: the nine settings/*.conf flattened
into one Board,CONN,Ch,NetName,Vnom sheet, generated by
tools/gen_channel_map.sh. The CSV is output, not source. NC channels are
kept as NC/- because the channel number is also the PMBus page - dropping
the gaps would shift every channel after them.

Two things the merged table makes visible: SWB0 and SWB1 carry identical
net names and voltages (only the bus and CB_I2C_CH differ), so those are
two copies that can drift apart silently; and there are exactly 8 NC
channels, all on CONN14/CONN15.

tools/100G_PRBS.txt, TR518.txt and fan_ctrl.txt join the existing
traffic_loopback_*.txt references. TR518 is not scripted yet and cannot
share the hardware with blanton_traffic_linespeed - both drive port
loopback and L2 learning.

Script A -> V1.0.10.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014RetWKFZFG1ZHcitQAyhwM
2026-08-27 14:29:47 +08:00
etwenandClaude Opus 5 03d609cc1f feat(prbs): Add 100G PRBS monitor; fix the job log path and USB mount
port_prbs_monitor.sh drives PRBS on both 100G uplinks: lpmode off, arm
the pattern, then poll `phy diag <phy> prbs get` in the background until
stopped, counting a poll as PASS only when the output says "PRBS OK!".
prbsstat Ber is captured beside every poll for the record but does not
decide the verdict. `stop` runs prbsstat STOp and prbs clear before
rendering the report, so the test does not leave PRBS armed.

`start a` / `start b` runs one uplink alone, which is how a genuine port
fault is separated from the DUT not coping with two PRBS streams at
once. A port that was not run reports SKIP rather than FAIL -- deciding
on poll count alone would have made a deliberate single-port run look
like half the hardware was broken.

The port status is deliberately not printed at `start`: with PRBS armed
the link reports DOWN, which reads as a failure to anyone glancing at
the console. It appears in the report instead, after prbs clear and
labelled as the recovered state.

The calls in Script A, B and C are committed but commented out -- PRBS
is still under bring-up.

Fix: the job directory was written as /host/hw-eval/current/jobs in some
places and /host/hw-eval/jobs in others, so what Script B wrote was not
what Script C collected. Unified on /host/hw-eval/jobs.

Fix: Script C copied to /mnt/usb assuming it was mounted. It now finds
the device with usb_target.sh and mounts it first, so the logs actually
leave the DUT.

Script C also reorders its results -- stress logs, then the MGMT ping
report, then 100G status -- and cats the bgctl ssd/usb logs.

Script A -> V1.0.9. Docs and docs/release-notes/v1.0.9.md updated.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014RetWKFZFG1ZHcitQAyhwM
2026-08-26 15:35:48 +08:00
etwenandClaude Opus 5 5e1e4a6a8f docs(release): Add v1.0.8 notes; wire mgmt ping status into B and C
Script A: the baseline ping window is 30 s rather than 45, and the log
is cleared afterwards so Script B starts from nothing.

Script B: `status` after `start`, and Script C: `status` before `stop`
plus a `cat` of the report. The status line shows both pids and the
replies received per NIC, so a dead link is visible while the soak is
still running instead of only at the end.

docs/release-notes/v1.0.8.md written in the house format.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014RetWKFZFG1ZHcitQAyhwM
2026-08-26 10:13:34 +08:00
etwenandClaude Opus 5 a31c186142 feat(mgmt): Ping both NICs at once, and stop counting ARP as loss
The bench run reported FAIL=3 with no NIC errors and no drops. All three
failures were on eth1 and all three were missing exactly icmp_seq=1,
with the other nine replies present and sub-millisecond. That is
neighbour resolution: once the ARP entry for the target expires, the
first echo request is spent resolving it and ping counts it as loss.
Rounds sat about 24 s apart, right on the edge of the default reachable
time, so it happened on some rounds and not others.

mgmt_ping_monitor.sh V3.0.0:

- One discarded ping per NIC before the measured run resolves ARP, so
  MAX_LOSS_PCT can stay at 0 and a FAIL means a real drop. Raising the
  threshold instead would have hidden genuine single-packet loss, and a
  longer settle would not help -- nothing triggers ARP until the ping.
- Both NICs now ping at the same time and keep accumulating until
  stopped, rather than alternating fixed bursts. Each writes its own raw
  capture.
- `stop` sends SIGINT rather than SIGTERM, because ping prints its
  statistics block on interrupt and that block is what the report
  parses, then renders into the log: the last TAIL_LINES entries per
  NIC, each one's statistics and verdict, and `ip -s link show` for both
  interfaces.
- `-D -O` are on by default so every line carries a timestamp and a
  request that got no reply is visible in the tail instead of merely
  absent. Clear PING_EXTRA_OPTS if the platform's ping rejects them.

Since the script no longer re-execs itself to daemonise, `bash
mgmt_ping_monitor.sh start` works without the execute bit.

.gitignore: the new *.raw / *.pid / *.state runtime files were outside
the *.log rule and would have been committed.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014RetWKFZFG1ZHcitQAyhwM
2026-08-26 09:53:20 +08:00
etwenandClaude Opus 5 1b8d5a8e91 feat(usb): Detect the USB target instead of assuming /dev/sda1
New Blanton_Script/usb_target.sh reports a USB mass-storage device's
node, mount point or by-id path. It excludes whatever disk backs / and
/host -- this platform can boot from a USB DOM, which also reports
TRAN=usb -- refuses to guess when several USB disks are present, falls
back to the whole device for superfloppy media, and when mounting,
checks the result is actually writable rather than trusting that mount
succeeded.

Script B asks it for the device node and builds the stress command from
the answer, so a stick that enumerates as sdb no longer sends the write
test at whatever /dev/sda1 happens to be.

The device is carried back through an echo marker, split in the shell
literal as "<<D""EV=..." so the command echo cannot satisfy the pattern
the macro waits for -- the same false-match guard used for PORTS0= and
NCMIF=, arranged differently.

Script B: DDR memtester drops its 100-pass limit and runs until stopped,
matching the other soak loads.

Fix: Script C ran `bgctl reset --yes` while killing processes, which is
before the job logs are copied to USB. Moved to after the archive and
after hw-test-session finish, so the run's own logs are collected before
anything clears them.

Script A -> V1.0.8 history updated.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014RetWKFZFG1ZHcitQAyhwM
2026-08-25 17:03:36 +08:00
etwenandClaude Opus 5 cac37583e7 feat(ttl): Include mgmt_ping.log in the USB archive
Script C now copies log/mgmt_ping.log into
/host/hw-eval/current/jobs/ alongside bmc_poll.log, so the management
link results leave the DUT with the run instead of staying behind. Both
monitors' logs are now covered.

Script A history: the two copies are one action in one block, so they
share a single V1.0.8 line rather than reading as separate changes.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014RetWKFZFG1ZHcitQAyhwM
2026-08-25 15:57:23 +08:00
etwenandClaude Opus 5 802ee4d467 feat(ttl): Fold bmc_poll.log into the USB archive
Script C copies log/bmc_poll.log into /host/hw-eval/current/jobs/ before
that directory is archived to USB, so the BMC sampling and I2C
read-back results leave the DUT with the run they belong to rather than
only existing in the Tera Term capture.

The wait moves ahead of the getdate/sprintf2 block; those build strings
without touching the terminal, so the prompt is still consumed exactly
once before the copy is sent.

Script A history updated under the existing V1.0.8 block -- that version
is not tagged yet, so this is the same batch.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014RetWKFZFG1ZHcitQAyhwM
2026-08-25 15:55:05 +08:00
etwenandClaude Opus 5 166b53983f feat(bmc): Re-enable bmc_monitor with an I2C write/read-back test
bmc_monitor.sh comes back into the run with a second job beyond the
`free -m` sampling: an I2C pattern test against bus 1, address 0x41,
offset 0x00C0 -- write 55AA55AA and read it back, then write AA55AA55
and read that back. Two complementary patterns catch a stuck bit either
way round, and repeating it through the soak turns an intermittent I2C
fault into something the log shows rather than something the tester has
to catch live.

Script B starts it alongside the other stress; Script C stops it and
cats log/bmc_poll.log.

bmc_monitor_ddr.sh stays disabled -- BMC DDR stress runs through
bgctl + bmc-manager.

Docs: the "disabled" notes I added last time now applied to only half
the pair and read as wrong for bmc_monitor.sh. Split so the monitor is
documented as live with its I2C test, and only the _ddr variant is
marked disabled.

Script A -> V1.0.8.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014RetWKFZFG1ZHcitQAyhwM
2026-08-25 15:50:56 +08:00
etwenandClaude Opus 5 ef7cbdffd4 docs(release): Add v1.0.7 notes; bump Script A header to V1.0.7
Script A carried a V1.0.7 history block while its "; Version :" header
still read V1.0.6, so publish.sh built Script_ABC_Blanton_V1.0.6 and the
folder would have disagreed with the tag. Header and date corrected --
reading the version from the header rather than a flag is what surfaced
this.

utils/wait_init.ttl V3.1.1: poll interval 10 s -> 60 s, and the WT_MIN
comments now say "at least" to match the test, which has been
`< WT_MIN` since the threshold was fixed. The stale "more than" wording
described exactly the off-by-one that once hung Script A.

Script C: the BMC USB journalctl dump is commented out.

docs/release-notes/v1.0.7.md written in the house format.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014RetWKFZFG1ZHcitQAyhwM
2026-08-25 14:22:22 +08:00
etwenandClaude Opus 5 ae8941a844 feat(ttl): Set fan speed in Script A, archive job logs to USB in C
config.ttl gains FAN_SPEED (30). Script A rebinds the two max31790
controllers (18-0020, 25-0020) and applies the speed with
fan-speed-control.sh, answering its confirmation prompt, so a run starts
from a known thermal state rather than whatever the last test left.

Script A also clears /host/hw-eval/current/jobs before the run, and
Script C copies that directory to /mnt/usb/jobs-<date>_<time> so the
bgctl job logs leave the DUT with the run they belong to.

Fix: 25-0020 was bound twice in a row -- the second bind can only fail,
since the driver is already attached. Removed; the two controllers now
have a matching unbind/sleep/bind each.

Script A -> V1.0.7, history covering the config.ttl and Script C changes
as well.

Docs: fan control and the job-log archive added to Tech Stack, Key
Features and Data Flow; the config.ttl example refreshed with FAN_SPEED
and the SWB_UNIT flags. Three risks recorded in Future Extensions --
`rm` without -r cannot clear the job subdirectories that Script C copies
with -r, the fan confirmation wait has no timeout and would hang
silently if the prompt ever changes, and /mnt/usb is never checked for
being mounted.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014RetWKFZFG1ZHcitQAyhwM
2026-08-25 14:12:24 +08:00
etwenandClaude Opus 5 a6a12cdf2c docs: Sync ARCHITECTURE and CLAUDE with the bgctl and mgmt-ping work
Script A: wait 45 s between starting and stopping mgmt_ping_monitor.
`start` returns after one second, but a round is roughly 30 s -- without
the pause the log held only the START banner and no RESULT line.

Docs brought up to date with the eight commits since 419b680:

- Tech Stack gains bgctl, systemd-run, iproute2 and hw-test-session;
  the stress row moves off the hammer scripts
- Structure and Key Features cover mgmt_ping_monitor.sh, and mark
  bmc_monitor{,_ddr}.sh as disabled -- their call sites in B and C are
  commented out, though the files remain
- Data Flow rewritten for all three scripts
- Two constraints added: TTL's `timeout` is global and must be restored,
  and two management NICs on one subnet cause ARP flux (hence
  ARP_STRICT, and nic_tx/nic_rx as the evidence of which NIC sent)
- CLAUDE gotchas add the `ip -s link` column order, that a monitor's
  `start` returns in one second rather than after a round, and that
  show_dmesg now ends with `dmesg -C`
- Phase 3 ticks the move to bgctl; Phase 5 gains the mgmt-ping, NVMe and
  EDAC parsing targets. Future Extensions notes that Script C never cats
  mgmt_ping.log, so the soak's ping results stay on the DUT

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014RetWKFZFG1ZHcitQAyhwM
2026-08-24 23:34:33 +08:00
etwenandClaude Opus 5 e38d24c775 feat: Add mgmt_ping_monitor.sh and move stress onto bgctl
New Blanton_Script/mgmt_ping_monitor.sh: brings both management NICs up
with iproute2 (`ip link set` / `address replace` / `route replace` with
per-NIC metric) and pings each one's own target -- eth0 -> .30, eth1 ->
.31 -- appending to a rotating log. Same sub-commands as the bmc
monitors, plus `summary` for just the per-leg RESULT lines.

Both NICs stay up, which on a shared subnet lets the target's ARP be
answered by either one, so a reply can land on the NIC that did not
send. ARP_STRICT applies arp_ignore/arp_announce to prevent that, and
each leg logs `ip -s link show` with the interface's own TX/RX packet
delta across the burst as direct evidence of which NIC carried the
traffic. Note `ip -s link` orders columns "bytes packets ...", so the
packet count is the second field.

Script A: hw-test-session start/log/status, a fuller DUT inventory
(version, fwutil, syseeprom, ssdhealth, TPM, nvme smart-log, smartctl),
bmc-first-enroll and bmc-manager version/status, ras-mc-ctl summary, and
both 100G uplinks now brought up rather than 513 being left down.

Script B: stress moves to bgctl (memtester, qfx5252-stress-ssd/-usb) and
the BMC DDR load runs through bmc-manager, replacing the hammer scripts
and the bmc_monitor pair. Adds a BMC USB net test that discovers the
cdc_ncm interface and runs a 4-hour ping under systemd-run.

Script C: stops the bgctl jobs, the BMC USB unit and the mgmt ping, then
collects journalctl, per-NIC counters, ras-mc-ctl and NVMe health, and
closes the session with hw-test-session finish.

utils/show_dmesg.ttl: one combined error regex with `dmesg -T`, an
i2c-filtered view, then `dmesg -C` so Script C's capture shows only what
the soak produced. utils/setup_pmon.ttl: add the TPM FRU/read checks.

Fix: Script B set `timeout = 15` for the cdc_ncm probe and never
restored it, leaving the cap in force for everything after -- including
the traffic init, which walks 108 VLANs per unit and takes far longer
than 15 s. A timed-out wait returns without the prompt, so the macro
would have run ahead of the DUT for the rest of the soak. Reset to 0 at
:skip_ping, where both branches meet.

Script A -> V1.0.6.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014RetWKFZFG1ZHcitQAyhwM
2026-08-24 23:27:37 +08:00
etwenandClaude Opus 5 9a11c1d5bb fix(ttl): Wait for the prompt before config save in the uplink block
`config save -y` was sent straight after the Ethernet514 startup with no
wait in between. The shell buffers the second line and still runs it, so
nothing visibly breaks, but every wait from there on matches the prompt
of the previous command -- leaving the macro permanently one step ahead
and issuing `show interfaces status` while `config save` is still
running. Every other sendln in the file waits first; this one now does
too.

Version left at V1.0.5.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014RetWKFZFG1ZHcitQAyhwM
2026-08-21 12:58:06 +08:00
etwenandClaude Opus 5 cbef5865ec fix(ttl): Treat 216 ports up as ready, and configure the 100G uplinks
wait_init.ttl: the comparison was strictly greater than WT_MIN, so a unit
reporting exactly 216 never passed and Script A sat in the poll loop
forever. 216 is not an arbitrary number -- TL_PAIRS holds 108 loopback
pairs, so 108 x 2 = 216 is every cabled port being up, i.e. precisely the
state being waited for. Comparison is now `< WT_MIN`, making the
threshold "at least 216".

Script A: bring the 100G uplinks into a known state before reading their
status -- Ethernet513 on asic0, Ethernet514 on asic1 -- and persist it
with `config save -y`. The stray `wait` after show_dmesg is dropped; the
one opening the new block consumes that prompt instead.

Version left at V1.0.5.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014RetWKFZFG1ZHcitQAyhwM
2026-08-21 12:56:13 +08:00
etwenandClaude Opus 5 8714bbb773 fix(ttl): Compare SWB_UNIT1 in the second operand of the traffic branches
Rework the traffic blocks in A, B and C as explicit if/elseif branches
holding literal commands, and drop the derived swb_any/swb_opt from
config.ttl -- the branches read straight off the page and there is no
indirection to follow.

All nine conditions compared SWB_UNIT0 against itself, though:

    if     SWB_UNIT0 = 1 && SWB_UNIT0 = 1     ->  SWB_UNIT0 = 1
    elseif SWB_UNIT0 = 1 && SWB_UNIT0 = 0     ->  never
    elseif SWB_UNIT0 = 0 && SWB_UNIT0 = 1     ->  never

so the single-unit paths were unreachable. A DUT with only unit 0 would
have run the both-unit commands and hit bcmcmd on an absent unit 1,
while a DUT with only unit 1 fell through to the empty else and skipped
traffic entirely. Second operand is now SWB_UNIT1.

Verified per branch that the -u argument matches the condition guarding
it: both units carry no -u (tool default TL_UNITS="0 1"), unit-0-only
carries -u 0, unit-1-only carries -u 1, and the else stays empty.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014RetWKFZFG1ZHcitQAyhwM
2026-08-21 12:19:18 +08:00
etwenandClaude Opus 5 d8bf1f7a41 feat(ttl): Drive traffic and port wait from SWB_UNIT0/SWB_UNIT1
The bench is not always fully populated. config.ttl now declares which
switch units exist:

    SWB_UNIT0 = 1   ; this DUT has switch unit 0
    SWB_UNIT1 = 1   ; this DUT has switch unit 1

Two values are derived there rather than repeating the same test in
three scripts: swb_any (0 = no unit at all) and swb_opt, the suffix
appended to each blanton_traffic_linespeed call -- "" for both units so
the tool's own TL_UNITS="0 1" applies, " -u 0" or " -u 1" for a single
one.

Script A, B and C build their traffic commands with sprintf2 and the
suffix, wrapped in `if swb_any = 1`. A half-populated DUT no longer
issues bcmcmd against an absent unit, and a DUT with no switch board
skips the traffic stage outright instead of filling the log with
failures.

wait_init.ttl reads the same two flags instead of its own copies, so the
wait and the traffic blocks cannot disagree about what is installed.

Script A -> V1.0.5.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014RetWKFZFG1ZHcitQAyhwM
2026-08-21 12:06:28 +08:00
etwenandClaude Opus 5 da39122c43 feat(ttl): Let wait_init skip switch units the DUT does not have
wait_init.ttl V3.0.0 adds WT_UNIT0 and WT_UNIT1 at the top of the file.
Benches are not always fully populated, and V2.0.0 waited on both units
unconditionally, so a DUT with one switch board sat in the poll loop
forever -- silently, since the loop neither advances nor reports.

    both units  -> 1 , 1
    unit 0 only -> 1 , 0
    unit 1 only -> 0 , 1
    no unit     -> 0 , 0   (bypass, returns immediately)

A disabled unit is skipped rather than polled and ignored: its whole
block sits inside the if, so no bcmcmd is issued for it and no
misleading error reaches the log.

Two independent integer flags rather than a "0 1" string, because
parsing a string in TTL needs strscan and this is meant to be edited by
hand at the bench.

All three exit paths still leave exactly one prompt unconsumed, so
Script A's surrounding waits are unaffected.

Script A history: recorded under the existing V1.0.4 block rather than a
new version, matching the consolidation done there.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014RetWKFZFG1ZHcitQAyhwM
2026-08-21 11:36:38 +08:00
etwenandClaude Opus 5 b66ae47a58 feat(ttl): Read EEPROM, quiet LLDP and clear lpmode before traffic
Script A: `hpe-eeprom-tlv show --bus 4 --addr 0x50` after `show boot`, so
the board identity is on record next to the image it booted. The empty
Check History placeholder is dropped.

Script A and B: disable the lldp feature and `config save -y` before the
traffic stage, so the switch stops sourcing its own frames and the
loopback pair counters reflect only the injected burst.

Script B: `sfputil lpmode off` on Ethernet513/514 before reading their
status -- a transceiver left in low-power mode will not link.

Script A -> V1.0.5, with the history covering the Script B changes too,
per this project's convention of keeping one consolidated log in A.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014RetWKFZFG1ZHcitQAyhwM
2026-08-21 11:28:06 +08:00
etwenandClaude Opus 5 d065db7d71 feat(ttl): Gate Script A on bcmcmd port-up count, not thermal sensors
wait_init.ttl V2.0.0 now polls

    bcmcmd -n 0 -c ps | grep -w up | wc -l
    bcmcmd -n 1 -c ps | grep -w up | wc -l

and proceeds only when BOTH exceed 216, so the baseline is taken with
the data plane actually up rather than merely with pmon answering.

V1.0.0 could use `wait "Thermal Not detected" prompt` because that was a
string-presence test. Comparing a count needs the value captured, so the
count is wrapped in an echo marker and read with waitregex +
groupmatchstr1 + str2int. The command echo cannot false-match: it reads
"PORTS0=$(bcmcmd ..." and the pattern requires a digit immediately after
the "=". The optional-space allowance covers a wc that pads its output.

The two thresholds are compared in nested ifs rather than with `and`,
which is bitwise in TTL.

WT_MIN and WT_INTERVAL are at the top of the file. The enter/exit prompt
contract is unchanged, so Script A's surrounding waits still line up.
Script A -> V1.0.4.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014RetWKFZFG1ZHcitQAyhwM
2026-08-21 10:41:50 +08:00
etwenandClaude Opus 5 419b680980 feat(publish): Add publish.sh and sync docs for V1.0.3
publish/publish.sh packages src/Script_ABC_Blanton into
publish/Script_ABC_Blanton_<Ver>/, with -z for a zip, -f to overwrite,
-n for a dry run.

The version is read from Script A's "; Version :" header rather than
passed in, so the folder name cannot disagree with what the tester sees
on opening the macro. The header is CRLF, so the carriage return is
stripped -- left in, it becomes part of the directory name. Captured
logs are excluded (they carry DUT serials) but the empty logs/ is kept
because logopen writes there. .gitignore now admits the tool while
still ignoring its output.

config.ttl: testcase ENV, margin scan off for this run.

Docs brought up to date with the 14 commits since 8aa5c90 -- SWB
channel/address map, the traffic and BMC tooling, line-ending policy,
and the exec-bit and tail -f traps.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014RetWKFZFG1ZHcitQAyhwM
2026-08-21 09:33:06 +08:00
etwenandClaude Opus 5 4ed283d0ca feat(ttl): Wait for thermal sensors before taking baseline data
New utils/wait_init.ttl polls `show platform temperature` every 10 s
until it stops reporting "Thermal Not detected", so Script A does not
record a baseline while pmon is still coming up. Script A calls it right
after `show boot` and bumps to V1.0.3.

The include pointed at utils/wait_thermal.ttl while the file is
wait_init.ttl -- as Script A's own V1.0.3 history line says. Corrected;
Tera Term aborts the macro when an include cannot be opened, so this
would have stopped Script A at line 59 on the first run.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014RetWKFZFG1ZHcitQAyhwM
2026-08-21 09:17:50 +08:00
etwenandClaude Opus 5 491e807cf8 feat(ttl): Log 100G port status, run traffic on both switch units
Script A and B: `show interfaces status Ethernet513,Ethernet514` before
the traffic stage, so the uplink state is on record next to the counters
it explains. In B this takes the slot the empty "eye measurement"
placeholder held.

Script B: drop -u 0 from the traffic calls. A and C already defaulted to
TL_UNITS="0 1", so B was pinned to unit 0 while C stopped both -- the
soak only ever loaded one switch board, and C's stop hit a unit that was
never initialised. All three scripts now cover both units.

Script B: post-clear settle 10 s -> 15 s, matching A.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014RetWKFZFG1ZHcitQAyhwM
2026-08-20 23:51:28 +08:00
etwenandClaude Opus 5 18ce7a1a73 fix(ttl): Dump BMC logs with cat instead of the blocking tail
`bmc_monitor.sh tail` runs `tail -n 50 -f`, which never returns, so
Script C stopped at the first call and never reached CHECK Stress
results, the traffic counters, or the closing messagebox. Stopping the
monitors first made it worse: with the file no longer growing, tail -f
just waits forever.

Read the two logs directly instead, so the whole BMC capture lands in
the master log and the macro carries on.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014RetWKFZFG1ZHcitQAyhwM
2026-08-20 11:14:34 +08:00
etwenandClaude Opus 5 d1b5cfea0b feat(ttl): Wire BMC monitors into Script B/C, log BMC version in A
Script A: dump the BMC banner via `bmc-manager run 'cat /etc/issue'`,
and add the missing `wait` before `show boot`.

Script B: start bmc_monitor_ddr.sh and bmc_monitor.sh alongside the
host stress load.

Script C: stop both and dump their logs into the master log.

bmc_monitor.sh: INTERVAL_SEC 5 -> 10.

Known blockers, filed here so they are not lost -- this wiring does not
work yet as written:

1. `bmc_monitor.sh tail` runs `tail -n 50 -f`, which never returns.
   Script C blocks on the first one and never reaches CHECK Stress
   results, the traffic counters, or the final messagebox. It needs a
   non-following dump (cat) instead.

2. Both scripts are mode 100644 and are the first .sh here meant to be
   executed rather than sourced. do_start re-execs "$SCRIPT_PATH"
   __daemon directly, so the execute bit is required even when invoked
   through bash -- and a Windows -> USB -> DUT copy cannot carry it.
   A chmod +x is needed before the start calls, as Script B already
   does for mlucas-avx2.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014RetWKFZFG1ZHcitQAyhwM
2026-08-20 11:12:03 +08:00
etwenandClaude Opus 5 5e63e3da6b fix(bmc): Give bmc_monitor_ddr its own log so the two can run together
Both scripts defaulted to LOG_NAME="bmc_poll.log", and PID_FILE is
derived from it, so the monitor and the DDR stress shared one log and
one pid file -- while the whole point of the split is to run them at
the same time.

The failure was quiet rather than loud: is_running greps the pid's
cmdline for its own SCRIPT_NAME, so the second script did not recognise
the first as running and started anyway. prepare_log_on_start then
truncated the log the first one was writing (START_LOG_MODE=new), both
raced on the pid file, and a later stop could only reap whichever wrote
it last, orphaning the other.

bmc_monitor_ddr.sh now uses bmc_ddr.log, which also moves its pid file
to bmc_ddr.pid. Both remain covered by the *.log ignore rule.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014RetWKFZFG1ZHcitQAyhwM
2026-08-20 11:03:44 +08:00
etwenandClaude Opus 5 855b7a6e30 feat(bmc): Split DDR stress out of bmc_monitor into bmc_monitor_ddr.sh
bmc_monitor.sh was doing two unrelated jobs: sampling `free -m` every
few seconds, and running `memtester 500M 1` as a stress load. They want
different timeouts -- a monitor should give up in 30 s, a memtester
pass legitimately runs for minutes -- and mixing them meant either
cutting memtester short or letting a hung sample stall the loop.

bmc_monitor.sh now polls `free -m` only, keeping CMD_TIMEOUT_SEC=30.
bmc_monitor_ddr.sh is the stress half: memtester only, with
CMD_TIMEOUT_SEC raised to 3600. The two are otherwise identical.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014RetWKFZFG1ZHcitQAyhwM
2026-08-20 11:00:04 +08:00
etwenandClaude Opus 5 310f897067 feat(bmc): Add bmc_monitor.sh, log build/uptime/reboot-cause in A and C
bmc_monitor.sh (V1.1.0): polls the BMC from the SONiC host via
`bmc-manager run`, appending to a rotating log. Runs detached, so no
interactive SSH session to the BMC is needed and the BMC's busybox
toolchain never comes into play. Subcommands start/stop/status/fg/tail/
clear; the polled command list, interval, per-command timeout and log
rotation are configured at the top of the file. START_LOG_MODE defaults
to "new", so one run means one log. Default COMMANDS are `free -m` and
`memtester 500M 1`, i.e. the BMC is memory-stressed alongside the host
soak.

Script A: `show boot` after the time sync to record the image the DUT
booted, and `show uptime` at the end of the traffic baseline.

Script C: `show reboot-cause` and `show uptime` after the traffic
report, so an unplanned reset during the soak is visible in the
closing snapshot rather than only in dmesg.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014RetWKFZFG1ZHcitQAyhwM
2026-08-20 10:47:35 +08:00
etwenandClaude Opus 5 fedf697977 docs(margin): Realign channel-pairing comments with bench wiring
The values were corrected in ababb2c but every comment describing the
pairing still said the old one, leaving four confs where the comment
contradicted the line it sat on:

    CB_I2C_CH=8           # CB F3 Ch6 = SWB0 CONN13/CONN14 ...

That reads as a typo waiting to be "fixed" back to 6, which would undo
a bench-verified value. Comments now state the real pairing:

    Ch6 = SWB0 CONN14/CONN16    Ch8 = SWB0 CONN13/CONN15
    Ch7 = SWB1 CONN14/CONN16    Ch9 = SWB1 CONN13/CONN15

Same map corrected in margin.sh's header (two places).

Ref/margin_command_trace.md gets a staleness banner rather than an
edit. Its 1223 expanded cb_pmbus_* commands were generated from the old
values - 487 of them use addresses that no longer exist (0x5D, 0x5F,
0x63, 0x65) - so patching the headings would leave the bodies wrong and
look authoritative. The banner carries the correct table and says the
SWB chapters need regenerating; chapters 1-4 are transport-independent
and remain valid.

Comments only: no CB_I2C_CH value and no CHIPS address was touched.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014RetWKFZFG1ZHcitQAyhwM
2026-08-19 12:19:44 +08:00
etwenandClaude Opus 5 ababb2c3d1 fix(margin): Correct SWB I2C channels and LTC2977 addresses from bench
Verified against the DUT. Two things were wrong in the SWB confs.

Channel assignment: CONN13/CONN15 and CONN14/CONN16 were paired the
wrong way round. The bench wiring is

    SWB0  Ch6 = CONN14/CONN16    Ch8 = CONN13/CONN15
    SWB1  Ch7 = CONN14/CONN16    Ch9 = CONN13/CONN15

LTC2977 addresses: the high half of each LTC2980 was recorded one step
above the low half (0x5C/0x5D, 0x62/0x63). These are 7-bit addresses
derived from 8-bit pairs, so they step by two: 0x5C/0x5E and 0x62/0x64.

This also clears the collision noted in v2.6.0, where SWB0_CONN13 chip2
and SWB0_CONN14 chip1 both landed on 0x5E on Ch6. Each channel now
carries two disjoint address pairs (0x5C/0x5E and 0x62/0x64), verified
collision-free across all eight SWB confs.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014RetWKFZFG1ZHcitQAyhwM
2026-08-19 12:10:39 +08:00
etwenandClaude Opus 5 a9c3dafc22 fix(traffic): Stop report saturating counters at INT32_MAX
Every value in the report printed as 2147483647 because grp() rendered
counters with sprintf("%d", v). The DUT's awk casts to a 32-bit int for
%d, so any counter above 2^31-1 saturates -- and a real line-speed run
is around 7e11, three orders of magnitude past that. Formatting with
%.0f keeps the value in awk's double domain, which is exact to 2^53.

The FAIL delta line had the same defect (%+d): a dead port makes the
delta as large as the counter itself, so the one number needed to
diagnose the failure would have been the one that saturated.

Verdicts were never affected. The cross-check compares the TX[]/RX[]
doubles directly and only the display path goes through grp(), so past
PASS results stand; only the printed numbers were wrong.

Verified against a captured run (secret/Counter.log, 108 pairs): values
now match the raw MIB_TPOK/MIB_RPOK lines exactly, and the loopback
identity holds (cd0.TX == cd32.RX, cd0.RX == cd32.TX).

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014RetWKFZFG1ZHcitQAyhwM
2026-08-19 09:21:16 +08:00
etwenandClaude Opus 5 4088a828c6 feat(ttl): Add traffic baseline to Script A, tune waits and unit args
Script A: the TRAFFIC-SETUP-STAGE placeholder is now a full loopback
round (ps, init, clear, show, start, stop, report) with 15 s settles, so
the run has a traffic baseline before the soak starts.

Script B: widen the post-clear settle from 3 s to 10 s.
Script C: drop -u 0 from stop/report so both switch units are covered.

Note: Script B still pins -u 0 while A and C now default to both units.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014RetWKFZFG1ZHcitQAyhwM
2026-08-19 09:02:41 +08:00
etwenandClaude Opus 5 afb7b1e283 fix(margin): Convert margin.sh and settings to LF, pin line endings
margin.sh and settings/*.conf carried CRLF into the repo, so sourcing
them on the SONiC DUT failed outright:

    -bash: $'\r': command not found
    margin.sh: line 126: syntax error near unexpected token `$'{\r''

settings/*.conf was the quieter half of the same bug: margin_init
sources them, and an unquoted assignment such as CB_I2C_CH=6 became
"6\r", so the SWB transport would have driven I2C with a malformed
channel number.

Content is unchanged - the diff is carriage returns only, and bash -n
passes on every converted file.

Add .gitattributes so this cannot come back: *.sh/*.conf/*.py/*.txt/*.md
are pinned to LF, *.ttl stays CRLF (Tera Term runs on Windows only) and
office formats are marked binary.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014RetWKFZFG1ZHcitQAyhwM
2026-08-18 16:34:55 +08:00
etwenandClaude Opus 5 fd35857d80 refactor(margin): Drop duplicated cb_i2c/pcimem copies
margin.sh v2.6.0 resolves the SWB backend via _swb_find_backend, which
searches the parent Blanton_Script/ dir, so the copies bundled inside
LTC2980_Margin_Script/ are dead weight and would silently drift from
the shared originals.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014RetWKFZFG1ZHcitQAyhwM
2026-08-18 10:38:59 +08:00
etwen cdf38a09ae feat(margin): Wire SWB transport to shared cb_i2c package, bump to v2.6.0
margin.sh (swb path only, CB/i2c path untouched):
- Search blanton_cb_i2c.sh / blanton_fpga_pcimem.sh in the parent
  Blanton_Script/ dir as well; the old code only looked next to margin.sh
  where those files do not exist, so TRANSPORT=swb always failed to load.
- _swb_setup: validate CB_I2C_CH, cb_i2c_init <ch> 0x88 0x00 once per
  session, then CB_I2C_AUTO_INIT=0 to drop 4 pcimem writes per transfer.
- Probe every LTC2977 in CHIPS with cb_pmbus_read <ch> <addr> 0x00 1.
- Check cb_pmbus_read/write return codes; a NACK is now reported instead
  of silently becoming Vout=0.0000.
- Fix DEBUG_MODE collision with blanton_fpga_pcimem.sh: pcimem tracing is
  gated by MARGIN_SWB_PCIMEM_DEBUG via _swb_call.
- Add margin_swb_info / probe / scan / reset / sem wrappers.

settings: fix CB_I2C_CH per bench wiring (CONN14/CONN15 were swapped)
  Ch6 = SWB0 CONN13/14, Ch8 = SWB0 CONN15/16,
  Ch7 = SWB1 CONN13/14, Ch9 = SWB1 CONN15/16

Ref/margin_command_trace.md: new command trace reference - every i2c /
  cb_pmbus command issued by margin_init, margin_status and
  margin_apply_profile combo_high3, expanded per channel for the CB conf
  and all 8 SWB confs, with LINEAR16 values and register meanings.

Known issue documented, not changed: SWB0_CONN13 chip2 (0x5E) collides
with SWB0_CONN14 chip1 (0x5E) now that both sit on Ch6; SWB1 pattern
suggests 0x5D, pending hardware confirmation.
2026-08-18 10:38:59 +08:00
etwenandClaude Opus 5 8aa5c90166 docs(arch): Sync ARCHITECTURE and CLAUDE with 20260817 changes
Bring both docs in line with the five commits pulled from Gitea.

- BDF: pcimem V1.5.0 auto-detects the CB FPGA BDF, so the "edit the four
  lines per DUT" guidance was inverted. Documented the detection order
  and fpga_rescan, and flagged that utils/show_pcie_error_reg_*.ttl
  still hard-codes its 7 BDFs -- the footgun is only half gone
- Traffic: documented blanton_traffic_linespeed.sh (bcmcmd SWB loopback,
  per-pair TX/RX cross-check) and tools/bcm_mibpair_report + the three
  loopback command tables; noted the route taken differs from the
  SONiC-CLI approach Phase 4 originally planned
- Stress: stress_hhmd and stress_pcie were removed on 08-17, so the job
  count is 8 not 10; soak loop now pulls full setup_pmon (9 items)
- Constraints: added the bcmcmd exit-0 and stdin-eating gotchas
- Phases: ticked 10 completed items, corrected Phase 3/4 acceptance
  criteria, and moved BDF auto-detect out of Future Extensions

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014RetWKFZFG1ZHcitQAyhwM
2026-08-17 21:51:17 +08:00
etwenandClaude 224ddbd21c feat(traffic): Report all pairs and add -tx/-length aliases
V0.4.0 -- report now lists ALL 108 pairs, PASS / FAIL / NA alike,
one line per port (two lines per pair). Missing counters render as
'-', so -a becomes the default (the flag is still accepted) and the
long "# NA (no counters): ..." summary line is dropped since every
NA pair is now a visible row. The table printer is generalised to a
column array with per-column alignment, keeping the pad-then-colour
order so ANSI codes do not break the column widths. This is the only
intentional difference from tools/bcm_mibpair_report_V1.1.0.py.

V0.3.3 -- accept -tx / -length as aliases for -c / -l so a tx burst
can be written the way the bcm command reads
("start -u 0 -tx 100 -length 512"); both are validated as positive
integers. Fix --dry-run printing nothing for init/start/stop/report
since V0.3.1, where the new bcm wrapper discarded the "[DRY ]" lines.
Offline report notes now go to stderr so the table on stdout stays
pipe/diff clean.

Co-Authored-By: Claude <noreply@anthropic.com>
2026-08-17 21:20:36 +08:00
40 changed files with 7271 additions and 1845 deletions
+29
View File
@@ -0,0 +1,29 @@
# =============================================================================
# 換行符政策
# -----------------------------------------------------------------------------
# 這個 repo 同時被 WindowsTera Term 端編輯)與 SONiC/LinuxDUT 端執行)使用,
# 所以必須明確指定,不能靠各機器的 core.autocrlf。
#
# 踩過的坑:margin.sh 帶著 CRLF 進 reposcp 到 DUT 後 bash 直接掛:
# -bash: $'\r': command not found
# syntax error near unexpected token `$'{\r''
# settings/*.conf 同樣危險 —— margin_init 是用 source 讀它們的,未加引號的
# 賦值(例如 CB_I2C_CH=6)會變成 "6\r"SWB transport 會拿著壞掉的通道號去打 I2C。
# =============================================================================
# 在 DUT / Linux 上執行或被 source 的,一律 LF
*.sh text eol=lf
*.conf text eol=lf
*.py text eol=lf
*.txt text eol=lf
*.md text eol=lf
# Tera Term macro 只在 Windows 上跑,維持 CRLF;標成 -text 讓 git 原樣保存,
# 避免對既有檔案產生大量純換行符的假 diff
*.ttl -text
# 二進位,不要做任何轉換
*.xlsx binary
*.docx binary
*.pdf binary
*.h5 binary
+7 -1
View File
@@ -12,9 +12,15 @@ For_AI/
!**/logs/.gitkeep
!**/Logs/.gitkeep
# 打包輸出
# monitor 腳本的執行期產物(原始擷取、pid、狀態)
*.raw
*.pid
*.state
# 打包輸出:產出不進 git,打包工具本身要進
publish/*
!publish/.gitkeep
!publish/publish.sh
# Office 暫存檔
~$*
+291 -75
View File
@@ -11,8 +11,8 @@
| Script | 階段 | 做什麼 |
|--------|------|--------|
| `1_Blanton_Script_A.ttl` | **Pre-test / Baseline** | 登入、對時、載入 shell 工具、清 PCIe AER、抓 PCIe tree / margin / PMON / dmesg 基線 |
| `2_Blanton_Script_B.ttl` | **Stress / Soak** | 開 CPU / DDR / SSD / USB / PCIe / I2C 壓力程序,之後進無限迴圈每輪抓 fan / temp / margin |
| `3_Blanton_Script_C.ttl` | **Post-test / Verdict** | 殺掉壓力程序、再抓一次 PMON / dmesg / PCIe AER,比對 stress log 判定 |
| `2_Blanton_Script_B.ttl` | **Stress / Soak** | 開 CPU / DDR / SSD / USB 壓力程序 + 啟動 SWB loopback 線速流量,之後進無限迴圈每輪抓完整 PMON / margin |
| `3_Blanton_Script_C.ttl` | **Post-test / Verdict** | 殺掉壓力程序、停流量並出 per-pair TX/RX 報表、再抓一次 PMON / dmesg / PCIe AER,比對 stress log 判定 |
DUT 端的實際取數能力則來自一組 **bash 工具庫**`Blanton_Script/`),這一層才是真正碰硬體的地方:
CB FPGA 的 PCIe BAR`pcimem`)→ FPGA 內建 I2C / VI2C master → 板上 VRM、溫度感測器、
@@ -36,8 +36,16 @@ LTC2980 margin 控制器。TTL 只負責「按順序 `sendln` 並把畫面收進
| I2C / PMBus 路徑 | CB FPGA F3 OpenCores I2C master、CB FPGA F3 VI2C proxy、CPU 原生 bus`i2cset`/`i2cget` |
| 電源 margin | LTC2980= 2 × LTC2977over PMBusLINEAR16 編碼 |
| 錯誤計數來源 | Linux PCIe AER sysfs`aer_dev_*`)、EDAC sysfs`dimm_ce/ue_count` |
| 平台監控 | SONiC `show platform *`pmon container |
| 壓力測試 | `mlucas-avx2`AMD CPU)、`~/hammer/tools/stress_*.py`DDR / SSD / USB / PCIe / I2C |
| 平台監控 | SONiC `show platform *`pmon container,含 `leak status` / `leak channels` |
| BMC 存取 | host 端 `bmc-manager run "<cmd>"`(不需對 BMC 開 SSH,繞開其 busybox 工具鏈);`bmc-first-enroll` / `bmc-manager version\|status` |
| 背景工作管理 | `bgctl run/list/stop --all/reset`(平台工具);長時間 ping 用 `systemd-run --unit=` 掛成 transient unit |
| 管理網路 | iproute2`ip link set` / `ip address replace` / `ip route replace ... metric`+ `ping -I` |
| 測試 session | `hw-test-session start\|log\|status\|finish`job log 收在 `/host/hw-eval/jobs/` |
| 風扇控制 | max31790 driver rebind`/sys/bus/i2c/drivers/max31790/{bind,unbind}`+ `fan-speed-control.sh <%>` |
| 100G PRBS | `bcmcmd -n <unit> -c 'dsh -c "phy diag <phy> prbs …"'`set / prbsstat STArt / get / Ber / STOp / clear |
| 壓力測試 | `mlucas-avx2`AMD CPU)、`bgctl run` 排程的 `memtester` / `qfx5252-stress-ssd` / `qfx5252-stress-usb` |
| 資料面流量 | `bcmcmd`Broadcom drivshellVLAN loopback + `tx` burstswitch unit 0 / 1 |
| 流量報表 | DUT 上 awk`blanton_traffic_linespeed report`);離線 `python3 tools/bcm_mibpair_report_V1.1.0.py` |
| 版本控制 | GitNAS + Gitea 私有 remote**不推 GitHub,含客戶 NDA 資料** |
> ⚠️ **無編譯步驟。** TTL 與 bash 都是直譯執行,「build」等同「把 `src/Script_ABC_Blanton/` 複製到
@@ -70,16 +78,19 @@ LTC2980 margin 控制器。TTL 只負責「按順序 `sendln` 並把畫面收進
│ ├ blanton_icb_vi2c.sh vi2c_* (ICB/PDB proxy) │
│ ├ blanton_temp_sensor.sh temp_cb / temp_icb / temp_all │
│ ├ blanton_pwr_data.sh pwr_data / pwr_data_pdb │
│ ├ blanton_traffic_linespeed.sh │
│ │ init/clear/show/start/stop/ps/report (bcmcmd) │
│ └ LTC2980_Margin_Script/ margin_init / margin_status ... │
│ │
│ ~/hammer/tools/ 壓力程序(mlucas-avx2, stress_*.py
SONiC CLI show platform fan/temperature/...
bcmcmd -n 0|1 Broadcom drivshellVLAN loopback / tx
│ SONiC CLI show platform fan/temperature/leak/... │
│ Linux sysfs aer_dev_* / edac mc0/rank* │
└───────────────────────┬──────────────────────────────────────┘
│ pcimem /sys/bus/pci/devices/0000:03:00.<fn>/resource0
┌──────────────────────────────────────────────────────────────┐
│ CB FPGA (PCIe EP4 個 Function)
│ CB FPGA (PCIe EP4 個 FunctionBDF 於 source 時自動偵測)
│ F0 local regsRST_CAUSE_REG 0xD90 / SWB 0xD94,0xD98 │
│ F3 I2C master Ch0..Ch17 (base 0x300, stride 0x20) │
│ F3 VI2C→ICB Ch0..Ch4 (base 0x780, stride 0x20) │
@@ -101,16 +112,29 @@ Blanton_TTL_Script/
├── CLAUDE.md # 專案記憶 & 給 Claude 的指令
├── ARCHITECTURE.md # 本架構文件
├── .gitignore # secret/ + For_AI/ + logs 規則
├── .gitignore # secret/ + For_AI/ + logs + publish 產出
├── .gitattributes # 換行符政策:.sh/.conf 釘 LF、.ttl 保持 CRLF
├── docs/ # 規格與狀態文件(客戶提供 / 專案產出)
│ ├── Blantons_FPGA_Registers.md # FPGA 暫存器規格(markdown 正本,查 offset 用)
│ ├── Blantons_FPGA_Registers_draft.docx # 同上,客戶原始 docx
│ ├── Blantons_FPGA_Registers_Map_draft.xlsx
│ ├── Blantons_PCIe_AER_and_DDR_EDAC_checks.pdf # AER / EDAC 檢查點依據
── TTL_Script_Blanton_Status_20260814.xlsx # ⭐ 三支腳本逐項 StatusOK / On-Going + Owner
── LTC2980_channel_map.csv # 9 個 settings/*.conf 合併:Board,CONN,Ch,NetName,Vnom144 列
│ ├── TTL_Script_Blanton_Status_20260814.xlsx
│ └── TTL_Script_Blanton_Status_20260817.xlsx # ⭐ 最新:三支腳本逐項 StatusOK / On-Going + Owner
├── tools/ # Host 端輔助腳本log 解析、報表產生)
├── tools/ # Host / 離線端輔助腳本
│ ├── bcm_mibpair_report_V1.1.0.py # 離線解析 drivshell log → per-pair TX/RX + PASS/FAIL
│ ├── traffic_loopback_vlan_setting.txt # bcmcmd VLAN 30..137 建立 + 成對 pbmcdN <-> cdN+32
│ ├── traffic_loopback_start.txt # bcmcmd tx 100 length=512 VLantag=<vid>
│ ├── traffic_loopback_vlan_remove.txt # bcmcmd vlan remove <vid> pbm=...
│ │ # ↑ 三份為手動貼上用的原始指令表,
│ │ # blanton_traffic_linespeed.sh 是其 bash 版
│ ├── 100G_PRBS.txt # 100G PRBS 原始指令(port_prbs_monitor.sh 的來源)
│ ├── TR518.txt # 內建封包測試 tr 518Scenario=6 Profile=2)— 尚未腳本化
│ ├── fan_ctrl.txt # MAX31790 unbind/bind + fan-speed-control.sh 30
│ └── gen_channel_map.sh # settings/*.conf → docs/LTC2980_channel_map.csv(需 gawk
├── secret/ # 🚫 gitignored — DUT 密碼、per-unit BDF、COM 設定
│ ├── README.md # ✅ committed — 用途索引
@@ -119,7 +143,9 @@ Blanton_TTL_Script/
├── For_AI/ # 🚫 gitignored — AI 協作素材(波形截圖、草稿)
├── publish/ # 可交付的打包輸出(Blanton_Script_ABC_v<ver>_<date>/ + zip
├── publish/
│ ├── publish.sh # ✅ committed — 打包工具(版號讀 Script A 檔頭)
│ └── Script_ABC_Blanton_<Ver>/ # 🚫 產出,gitignored-z 另出同名 .zip
└── src/
└── Script_ABC_Blanton/ # ← Tera Term 的工作目錄(整包給測試員)
@@ -131,10 +157,12 @@ Blanton_TTL_Script/
├── utils/ # 可 include 的取數片段(TTL,無狀態、只 wait+sendln
│ ├── pcie_bus.ttl # lspci -tvvv / -vv
│ ├── setup_pmon.ttl # show platform summary/fan/temperature/psu/voltage/current/ssd
├── show_dmesg.ttl # date + dmesg grep error/fail/warning
│ ├── setup_pmon.ttl # show platform summary/fan/temperature/psustatus/voltage/
│ # current/ssdhealth/leak status/leak channels + TPM11 項)
│ ├── show_dmesg.ttl # dmesg -T 合併 error 正則 + i2c 過濾 + dmesg -C 清空
│ ├── show_margin_status.ttl # source margin_status_all.sh(掃 9 顆 LTC2980
│ ├── kill_all_process.ttl # kill $(jobs -p)
│ ├── wait_init.ttl # 輪詢 show platform temperature 直到 pmon 起來
│ ├── show_pcie_error_reg.ttl # 舊的合併版(已被下列 per-device 版取代)
│ ├── show_pcie_error_reg_TH6_SWB0.ttl # EP 01:00.0 + RP 00:01.1
│ ├── show_pcie_error_reg_TH6_SWB1.ttl # EP 02:00.0 + RP 00:01.2
@@ -152,12 +180,25 @@ Blanton_TTL_Script/
├── blanton_icb_vi2c.sh # CB F3 VI2C proxy → ICB/PDBCh0~Ch4
├── blanton_temp_sensor.sh # TMP75 / TMP432 讀溫(CB Ch10 mux + ICB VI2C Ch2
├── blanton_pwr_data.sh # SWB VRM + PDB brick 的 Vin/Vout/Iout/Temp 表
├── blanton_traffic_linespeed.sh # SWB loopback 線速流量(bcmcmd unit 0/1
│ # init/clear/show/start/stop/ps/report/run
├── mgmt_ping_monitor.sh # 10G/1G 管理網路 ping 監控(iproute2
│ # eth0→TARGET_10G、eth1→TARGET_1G,兩張卡同時持續 ping
├── port_prbs_monitor.sh # 100G uplink PRBS 測試(bcmcmd phy diag
│ # ⚠️ 已納入交付,但 A/B/C 的呼叫仍註解掉(bring-up 中)
├── usb_target.sh # 偵測 USB 裝置節點/掛載點(排除系統碟)
├── bmc_monitor.sh # BMC 監控:free -m + I2C 寫入/讀回圖樣測試
├── bmc_monitor_ddr.sh # ⚠️ 已停用:B/C 內呼叫處已註解,DDR 壓力改走 bgctl + bmc-manager
│ # ↑ 三支都要 chmod +x 才能跑(見 Key Constraints
└── LTC2980_Margin_Script/ # 電壓 margin 子專案(另有獨立 repo
├── margin.sh # margin_init/status/set/apply_profile/save
├── settings/*.conf # 一個 .conf = 一顆 LTC2980CB + SWB0/1 × CONN13~16
├── profiles/*.conf # 16-ch 批次組合(comboA/B、high3/5、low3/5、normal、off
├── script/*_all.sh # 跨 9 顆 LTC2980 的批次 status / apply / save
├── Ref/margin_command_trace.md # 指令追蹤參考(SWB 章節已標過期,見文件內警告)
└── logs/ # margin 操作 log
# 註:v2.6.0 起不再內含 blanton_cb_i2c.sh / blanton_fpga_pcimem.sh 的複本,
# 改由 _swb_find_backend 往上層 Blanton_Script/ 找共用版
```
---
@@ -169,15 +210,20 @@ Blanton_TTL_Script/
### 1. `config.ttl` — 全域測試參數(TTL 變數)
```ini
strTestcase = "Margin" ; 測項名,進 log 檔名
project_name = "Blanton" ; 專案名,進 log 檔名
EN_Margin = 1 ; 1=跑 margin 掃描, 0=跳過
strTestcase = "ENV" ; 測項名(ENV/EMC/Margin...),進 log 檔名
EN_Margin = 0 ; 1=跑 margin 掃描, 0=跳過
EN_log = 1 ; 1=logopen 存檔, 0=不存
FAN_SPEED = 30 ; 傳給 fan-speed-control.sh 的風扇轉速 (%)
SWB_UNIT0 = 1 ; 這台有 switch unit 0(流量與 wait_init 都依此)
SWB_UNIT1 = 1 ; 這台有 switch unit 1
prompt_login = "sonic login:"
prompt_sonic = "admin@sonic:~$"
prompt_sonic_root = "root@sonic:~#"
```
> `SWB_UNIT0` / `SWB_UNIT1` 兩個都是 0 時,`wait_init.ttl` 直接 bypass、A/B/C 的流量段整段跳過。
Log 檔名規則:`<mdir>\Logs\<project_name>_<strTestcase>_<YYYYmmdd-HHMMSS>.log`
### 2. `settings/<board>.conf` — 一顆 LTC298016 channel
@@ -193,6 +239,28 @@ Log 檔名規則:`<mdir>\Logs\<project_name>_<strTestcase>_<YYYYmmdd-HHMMSS>.l
Channel 定址:`ch / 8``CHIPS[]` index`ch % 8` → LTC2977 PMBus PAGE0x00~0x07)。
**SWB 通道與位址對應(2026-08-19 上機驗證,`ababb2c`):**
| FPGA I2C 通道 | 板 / 接頭 | LTC2977 位址(lo / hi |
|---|---|---|
| Ch6 | SWB0 CONN14 / CONN16 | CONN14 `0x5C`,`0x5E` ; CONN16 `0x62`,`0x64` |
| Ch8 | SWB0 CONN13 / CONN15 | CONN13 `0x5C`,`0x5E` ; CONN15 `0x62`,`0x64` |
| Ch7 | SWB1 CONN14 / CONN16 | CONN14 `0x5C`,`0x5E` ; CONN16 `0x62`,`0x64` |
| Ch9 | SWB1 CONN13 / CONN15 | CONN13 `0x5C`,`0x5E` ; CONN15 `0x62`,`0x64` |
> 同一顆 LTC2980 的兩個位址**相差 2**7-bit,由規格書 8-bit 配對除以 2 而來),不是 1。
> 兩張板共用一條通道,所以同通道上的位址必須互斥 —— 上表兩組(`0x5C/0x5E` 與 `0x62/0x64`)不重疊。
**全通道 net / 電壓對照:`docs/LTC2980_channel_map.csv`**9 個 `.conf` × 16ch = 144 列,
欄位 `Board,CONN,Ch,NetName,Vnom`)。查「某個 net 在哪片哪個 channel」比翻 9 個 `.conf` 快。
CSV 是產出物,不是正本 —— 改完 `.conf``./tools/gen_channel_map.sh` 重產,別手改 CSV。
兩件從表上看得出來、改 `.conf` 時要記得的事:
- **SWB0 與 SWB1 的 net / 電壓完全相同**,差別只在 `CHIPS[]` 的 bus`0:` vs `1:`)與 `CB_I2C_CH`
也就是說有兩份會各自漂移的重複資料 —— 改了 SWB0 沒改 SWB1,不會有任何東西擋你。
- **8 個 NC 通道**SWB0/1 CONN14 的 CH10/CH11/CH13、CONN15 的 CH10)在 CSV 裡以
`NC` / `-` 原樣保留,不是漏掉;濾掉它們會讓 page 編號對不上。
### 3. `profiles/*.conf` — 16-ch 批次 margin 組合
```bash
@@ -235,9 +303,16 @@ PROFILE_CHANNELS=( "0:high:8" "1:low:8" ... ) # "channel:operation:change_perc
| I210 1G NIC | `0000:07:00.0` | `0000:00:02.4` |
| NVMe SSD | `0000:06:00.0` | `0000:00:02.1` |
> ⚠️ **BDF 是 per-unit 的**。`blanton_fpga_pcimem.sh` 的 `FUNCT0..3_RES` 已經改過三次
> 04:00.x → 03:00.x → 05:00.x → 03:00.x)。換 DUT 一定要先 `lspci -Dnn | grep -i fpga` 重新確認,
> 同理 `show_pcie_error_reg_*.ttl` 裡的 BDF 也要跟著校。
> ⚠️ **BDF 是 per-unit 的,而且只解決了一半。**
> `blanton_fpga_pcimem.sh`V1.5.0 起)**已改為 source 時自動偵測**,不再需要每台改檔:
> 掃 sysfs 找 `vendor == 0x1590`HPE)且 `.0`~`.3` 四個 function 都有 `resource0` 的裝置 →
> 退回 `lspci -Dnn` 名稱比對 → 再退回 `FPGA_BDF_FALLBACK``0000:03:00`)。
> 偵測結果會印一行 `[INFO] CB FPGA BDF = ...`,並存在 `FPGA_BDF_ACTIVE`。
> 換 DUT 後可用 `fpga_rescan` 重測,或 `FPGA_BDF=0000:05:00 fpga_rescan` 手動釘住。
>
> **但 `utils/show_pcie_error_reg_*.ttl` 裡的 BDF 仍然是寫死的**(上表那 7 組),
> 自動偵測完全沒有涵蓋到它們。換 DUT 時這些 TTL 還是要人工校對,否則 AER 會讀到錯誤的裝置
> ——甚至讀到不存在的路徑而整段 `No such file or directory`。
---
@@ -245,15 +320,22 @@ PROFILE_CHANNELS=( "0:high:8" "1:low:8" ... ) # "channel:operation:change_perc
| 模組 | 進入點 | 功能 |
|------|--------|------|
| **Script APre-test** | `1_Blanton_Script_A.ttl` | root 登入 → `mkdir ./logs``date -s` 對時 → source 五個 bash 工具 → 清/讀 7 組 PCIe AER → `lspci -tvvv/-vv` → margin 全掃 → PMON 七項 → dmesg 三 grep → `messagebox` 提示接 Script B |
| **Script BStress** | `2_Blanton_Script_B.ttl` | `mlucas-avx2 -cpu 0:15` + 5 份 `stress_mem.py` + `stress_hhmd/ssd/pcie/usb.py` 全部背景執行 → `jobs` 確認 → `while 1` 每輪 `show platform fan` / `temperature` / margin |
| **Script CPost-test** | `3_Blanton_Script_C.ttl` | `kill $(jobs -p)` → PMON + dmesg 再抓一次 → 7 組 AER 再抓一次 → `cat` mlucas / stress_ssd log → 完成 messagebox |
| **FPGA 暫存器存取** | `cb_fpga <fn> <addr> [data]` | pcimem 打 `resource0` + BAR 相對 offset32-bit word`pmc/icb/swb0/swb1_fpga` 走 VSPI 窗口 |
| **Script APre-test** | `1_Blanton_Script_A.ttl` V1.0.7 | root 登入 → `date -s` 對時 → 清 job log → `hw-test-session start`**設定風扇轉速****等資料面就緒**`wait_init.ttl`)→ DUT 清單(boot/version/fwutil/syseeprom/ssdhealth/TPM/NVMe)→ BMC enroll + version/status → source bash 工具 → 清/讀 7 組 PCIe AER + `ras-mc-ctl --summary` `lspci` → margin → PMON → dmesg **10G/1G 管理網路 ping** → 100G uplink 設定與狀態 → **一輪流量基線**`show uptime` |
| **Script BStress** | `2_Blanton_Script_B.ttl` | `mlucas-avx2 -cpu 0:15` + `bgctl run``memtester` / SSD / USB 壓力 + BMC DDR`bmc-manager run memtester`)→ BMC USB net test(探測 cdc_ncm 介面,`systemd-run` 掛 4 小時 ping)→ 啟動 mgmt ping 監控 → `jobs` / `bgctl list` → traffic(依 `SWB_UNIT`)→ `while 1` 每輪完整 PMON + margin |
| **Script CPost-test** | `3_Blanton_Script_C.ttl` | `kill $(jobs -p)` + `bgctl stop --all` / `reset` + 停 BMC USB unit 與 mgmt ping → PMON + dmesg → 7 組 AER + `ras-mc-ctl` → BMC USB `journalctl``ip -s link show eth0/eth1``cat` mlucas log → traffic `stop` + `report` → NVMe 健康 → `show reboot-cause` / `uptime`**job log 複製到 USB(帶時戳)**`hw-test-session finish` |
| **SWB loopback 流量** | `blanton_traffic_linespeed <sub> [-u 0\|1\|all]` | `init`VLAN 30..137,成對 `cdN`/`cdN+32`)→ `clear``start``tx 100 length=512`)→ `stop``report`per-pair TX/RX 交叉比對 PASS/FAIL);另有 `ps`link/speed)與 `run`(一條龍) |
| **FPGA 暫存器存取** | `cb_fpga <fn> <addr> [data]``fpga_rescan` | pcimem 打 `resource0` + BAR 相對 offset32-bit wordBDF 於 source 時自動偵測;`pmc/icb/swb0/swb1_fpga` 走 VSPI 窗口 |
| **CB I2C / PMBus** | `cb_i2c_init/scan/read/write``cb_pmbus_read/write/linear16` | FPGA F3 OpenCores master**讀用 Repeated START**SMBus/PMBus 裝置必需) |
| **ICB / PDB 存取** | `vi2c_init/scan/read/write``vi2c_scan_all` | CB F3 VI2C proxy~3.3 ms/reg,取代 VSPI 的 ~18 ms/reg(後者會踩 SMBus 2535 ms timeout |
| **溫度** | `temp_cb` / `temp_icb` / `temp_all` | TMP7512-bit 左靠齊 ×0.0625)、TMP432(含 RANGE bit 的 64 偏移、remote1/2 |
| **電源資料** | `pwr_data` / `pwr_data_pdb` / `pwr_pdb_id` | SWB VRM 全軌 Vin/Vout/Iout/Temp 對齊表;MP29816 解析度**執行期從 `MFR_VOUT_SCALE_LOOP` 讀**,不寫死 |
| **電壓 Margin** | `margin_init/status/set/apply_profile/save` + `*_all.sh` | 9 顆 LTC2980CB CONN13、SWB0/1 CONN13~16)批次 high/low margin 與狀態掃描 |
| **電壓 Margin** | `margin_init/status/set/apply_profile/save` + `*_all.sh` | 9 顆 LTC2980CB CONN13、SWB0/1 CONN13~16)批次 high/low margin 與狀態掃描。v2.6.0 起 SWB 路徑真正接上共用 cb_i2c 後端,並有 `margin_swb_info/probe/scan/reset/sem` 除錯包裝 |
| **管理網路 ping** | `mgmt_ping_monitor.sh {start\|stop\|status\|fg\|tail\|clear\|summary}` | 一輪 = 10G leg + 1G leg,各自 `ip address/route replace``ping -I <if>` 打自己的對端;兩張卡都保持 up。每 leg 記 `ip -s link show` 與該卡的 TX/RX packet delta,另出一行可 grep 的 `RESULT` |
| **BMC 監控** | `bmc_monitor.sh {start\|stop\|status\|fg\|tail\|clear}` | 由 host 端 `bmc-manager run` 週期取樣 `free -m`,另加一組 **I2C 寫入/讀回圖樣測試**bus 1、addr 0x41、offset 0x00C0,寫 `55AA55AA` 讀回、再寫 `AA55AA55` 讀回)。B 啟動、C 停止並 `cat log/bmc_poll.log` |
| **BMC DDR 壓力** | `bmc_monitor_ddr.sh` | ⚠️ **已停用** —— B/C 內呼叫處已註解,改由 `bgctl run bmc-manager run memtester` 取代。檔案保留未刪 |
| **100G PRBS** | `port_prbs_monitor.sh {start [both\|a\|b]\|stop\|status\|report\|clear}` | 兩個 uplink 各一個 worker,可同時或單獨跑。setup`lpmode off``prbs set``prbsstat STArt`)→ 週期 `prbs get`(判定)+ `prbsstat Ber`(只記錄)→ teardown`STOp` + `prbs clear`)。輸出含 `PRBS OK!` 才算 PASS。**⚠️ A/B/C 的呼叫目前註解掉** |
| **USB 目標偵測** | `usb_target.sh [-o dev\|mnt\|both\|id] [-n] [-r sec]` | 找出插入的 USB 儲存裝置,排除 `/``/host` 的底層碟(本平台可能從 USB DOM 開機),多顆時拒絕猜;掛載時會驗證真的可寫 |
| **打包交付** | `publish/publish.sh [-v Ver] [-f] [-z] [-n]` | 版號讀 Script A 檔頭,複製成 `publish/Script_ABC_Blanton_<Ver>/`,排除 `*.log`,可另出 zip |
---
@@ -268,31 +350,54 @@ PROFILE_CHANNELS=( "0:high:8" "1:low:8" ... ) # "channel:operation:change_perc
Script A ── logopen Logs/Blanton_Margin_<ts>.log ──┐ (之後所有畫面都進這個檔)
│ │
├─ 登入 admin → sudo -i → root@sonic:~# │
├─ date -s "<host 時間>" ← 讓 DUT 時戳可對照
├─ source 5 個 bash 工具(函數進 shell) │
├─ date -s "<host 時間>" ← 讓 DUT 時戳可對照
├─ rm /host/hw-eval/jobs/* ← 清掉上一輪的 job log
├─ hw-test-session start / log / status
├─ 風扇:max31790 unbind→sleep→bind18-0020 與 25-0020 各一次)
│ → fan-speed-control.sh <FAN_SPEED> → 回答 y 確認 │
├─ source 6 個 bash 工具(函數進 shell
│ └ 此時 blanton_fpga_pcimem.sh 印出 [INFO] CB FPGA BDF = ...(自動偵測結果)
├─ AER 基線:7 組 EP+RP 的 correctable/nonfatal/fatal
├─ lspci -tvvv / -vv ← 拓樸 + link speed/width
├─ margin_status_all ← 9 顆 LTC2980 × 16ch 的 Vout 與偏差%
├─ show platform × 7 ← fan/temp/psu/voltage/current/ssdhealth
─ dmesg | grep error/fail/warning
├─ show platform × 9 ← fan/temp/psu/voltage/current/ssdhealth/leak status/leak channels
─ dmesg -T(合併 error 正則 + i2c 過濾)→ dmesg -C 清空 ← C 只會看到 soak 期間新產生的
├─ mgmt pingstart → pause 45 → stop → cat log ← 10G/1G 各一輪
├─ 100G uplinklpmode off → 兩埠 startup → config save → show interfaces status
├─ 一輪流量基線:ps → init → clear → show → start → stop → report
└─ show uptime
(最前面還有 wait_init.ttl:輪詢 show platform temperature 直到不再回報
"Thermal Not detected",避免 pmon 還沒起來就抓基線)
[messagebox "Start Script B"] ← 人工確認後手動跑 B
Script B ── 同一個 log 檔續寫(logwrite 分隔線)
├─ 背景程序:mlucas-avx2(CPU) ×1, stress_mem ×5,
stress_hhmd(I2C/LSB/PCIe), stress_ssd, stress_pcie, stress_usb
├─ jobs ← 確認全部起來了
while 1: fan → temperature → margin_status_all soak 期間持續取樣
├─ 背景程序:mlucas-avx2(CPU) ×1, stress_mem ×5, stress_ssd, stress_usb → 共 8 個 job
stress_hhmd / stress_pcie 已於 2026-08-17 移除,見 Status 表 ScriptB #5/#7 "KC Remove"
├─ bgctl run: memtester 1G / qfx5252-stress-ssd / qfx5252-stress-usb
BMC: bmc-manager run memtesterDDR)、cdc_ncm 介面探測 + systemd-run 4hr pingUSB
├─ mgmt_ping_monitor.sh start
├─ jobs / bgctl list ← 確認全部起來了
├─ traffic: ps → init → clear → (pause 15) → show → start unit 0 與 1VLAN 30..137 loopback
└─ while 1: setup_pmon9 項)→ margin_status_all soak 期間持續取樣)
[測試員 Ctrl-C / 停止 macro,接著跑 C]
Script C
├─ kill $(jobs -p) ← 收掉所有背景壓力
├─ show platform × 7 + dmesg ← 收工快照
├─ show platform × 9 + dmesg ← 收工快照
├─ AER 再抓一次 ← 與 Script A 基線相減 = 本輪新增錯誤
cat mlucas_amm_log / log.stress_ssd ← 壓力程式自身的 pass/fail
bgctl stop --all / reset、systemctl stop BMC USB unit、mgmt ping stop
├─ journalctl BMC USB unit、ip -s link show eth0/eth1、ras-mc-ctl --summary
├─ cat mlucas_amm_log ← CPU 壓力程式自身的 pass/fail
├─ nvme smart-log / smartctl -x /dev/nvme0
├─ traffic: stop → report ← per-pair TX/RX 交叉比對,PASS/FAIL/NAexit 2 = 有 FAIL/NA
├─ show reboot-cause / show uptime ← soak 期間有沒有意外重開,一眼看得到
├─ usb_target.sh -o dev → sync → mount <dev> /mnt/usb
├─ cp bmc_poll.log / mgmt_ping.log → jobs/,再 cp -r jobs/ /mnt/usb/jobs-<YYYYmmdd>_<HHMM>
└─ hw-test-session finish
[messagebox "GOOD JOB! Test Case DONE"] → 人工檢查 log
```
@@ -321,8 +426,9 @@ pwr_data
跑 B/C 前若關掉了 Tera Termlog 就斷了。
3. **PCIe AER 的判定是「差值」不是「絕對值」**Script A 抓基線、Script C 抓結果,
兩者相減才是本輪產生的錯誤。EP 與上游 Root Port **兩邊都要看**(錯誤可能只記在 RP)。
4. **BDF 是 per-unit 的**,換 DUT 必須重新確認並改 `blanton_fpga_pcimem.sh` `FUNCT0..3_RES`
`utils/show_pcie_error_reg_*.ttl`(見 Data Models §5 註)。
4. **BDF 是 per-unit 的,bash 端已自動偵測、TTL 端仍寫死**`blanton_fpga_pcimem.sh` V1.5.0 起
於 source 時自掃(vendor `0x1590``lspci` → fallback),但 `utils/show_pcie_error_reg_*.ttl`
的 7 組 BDF 還是硬編碼,換 DUT 仍需人工校對(見 Data Models §5 註)。
5. **ICB / PDB 一律走 VI2C,不走 VSPI**VSPI ~18 ms/reg 會讓 I2C byte 相之間拉到 ~50 ms
踩中 SMBus 2535 ms timeout,裝置會 NACK register pointer。實測全 5 通道掃描 9.4 s vs 34.0 s。
6. **I2C 讀一律用 Repeated START**(不是 STOP + START):PMBus / SMBus 裝置(VRM、EFUSE
@@ -334,9 +440,36 @@ pwr_data
9. **`ARB_LOST` 不等於裝置存在**`vi2c_scan` 只看 `RX_ACK`,仲裁失敗會被誤報成「有裝置」。
`blanton_pwr_data.sh` 因此加了 ARB_LOST-aware retry 與 plausibility window
`0xFFFF` 或超出範圍一律回 `NA`)——**寧可 NA 也不要印出一個看起來像數據的錯值**。
10. **DUT 密碼不得進 git**`1_Blanton_Script_A.ttl` 目前的 `sendln "YourPaSsWoRd"` 是 placeholder
10. **`bcmcmd` 成功退出不代表指令成功**:BCM shell 拒絕指令(`VLAN: ERROR: Entry exists` /
`Failed to execute the diagnostic command`)時 `bcmcmd` 仍 exit 0。
`blanton_traffic_linespeed.sh` 因此改用輸出比對(`$TL_ERR_PAT`),並把
`vlan create``Entry exists`(重跑 init)與 `vlan remove``not found`init 前先 stop
當良性情況分開計數。另外 `bcmcmd` 一律以 `</dev/null` 執行 ——
否則它會吃掉 while 迴圈的 stdin,導致**只有第一個 VLAN 被設定**V0.3.1 修的 bug)。
11. **DUT 密碼不得進 git**`1_Blanton_Script_A.ttl` 目前的 `sendln "YourPaSsWoRd"` 是 placeholder
實際帳密必須放 `secret/config_secret.ttl`gitignored)並由 `config.ttl` include。
11. **客戶 NDA**`docs/` 內含客戶 FPGA 規格與 schematic 衍生資訊,**本 repo 只推 NAS + Gitea
12. **`.sh` / `.conf` 必須是 LF**CRLF 的 shell script 在 SONiC 上直接掛
`-bash: $'\r': command not found` + `syntax error near unexpected token`),
`settings/*.conf` 則因為是被 `source` 的,未加引號的賦值會變成 `"6\r"`(安靜地壞掉)。
`.gitattributes` 已把 `*.sh` `*.conf` `*.py` `*.txt` `*.md` 釘成 LF、`*.ttl` 維持 CRLF。
⚠️ 但 `.gitattributes` **不會回頭重寫既有工作區檔案** —— 早於它 clone 出來的目錄,
`git rm --cached -r . && git reset --hard` 或刪掉目錄重新 checkout 才會生效,
而且 `git status` 乾淨**不代表**磁碟上是 LF`--renormalize` 只在進 index 的路上轉換)。
13. **`bmc_monitor*.sh` / `mgmt_ping_monitor.sh` 需要執行權**:它們是本專案「被執行」而非被 `source` 的腳本,
`do_start` 內部是 `setsid "$SCRIPT_PATH" __daemon`**即使用 `bash x.sh start` 呼叫
也仍需 exec bit**。git 記錄為 `100644`,且 Windows → 隨身碟 → DUT 的路徑無法攜帶權限位元,
所以佈署後要先 `chmod +x ~/Blanton_Script/bmc_monitor*.sh`,否則 Script B 的 start 會
`Permission denied`、Script C 的 `cat` 撲空(BMC 段全空但不會報錯)。
14. **TTL 的 `timeout` 是全域的,設了就要還原**`timeout = N` 之後**每一個** `wait` 都被限制在 N 秒,
逾時的 `wait` 會直接返回而沒吃到 prompt,於是下一個指令在前一個還沒跑完就送出,之後整輪錯開一拍。
`wait_init.ttl` 設 60 後還原 0Script B 的 cdc_ncm 探測設 15,在 `:skip_ping`(兩條路徑的匯流點)還原。
加新的 `timeout` 時務必配一個 `timeout = 0`
15. **兩張管理網卡同網段時會 ARP flux**`mgmt_ping_monitor.sh` 讓 eth0/eth1 同時 up,若兩者在同一個
/24(本專案預設 `192.168.1.99` / `.101`),對端 ARP 可能由任一張卡回應,回包落在沒送封包的那張,
量到的數字不能歸屬。`ARP_STRICT=1` 會設 `arp_ignore=1` / `arp_announce=2` 擋掉;
另外每個 leg 的 `RESULT` 行帶 `nic_tx` / `nic_rx`(該卡自己的 packet delta),
**ping 成功但 `nic_tx` 接近 0 就代表封包從另一張卡出去了**。兩張卡在不同網段時可設 0。
16. **客戶 NDA**`docs/` 內含客戶 FPGA 規格與 schematic 衍生資訊,**本 repo 只推 NAS + Gitea
絕不推 GitHub**。
---
@@ -371,22 +504,29 @@ cp secret/config_secret.ttl.example secret/config_secret.ttl
# ── 2. DUT 端:部署 bash 工具庫 ─────────────────────────────
scp -r src/Script_ABC_Blanton/Blanton_Script admin@<DUT_IP>:~/
# 佈署後務必補執行權(Windows/隨身碟路徑帶不動 exec bit):
ssh admin@<DUT_IP> 'chmod +x ~/Blanton_Script/bmc_monitor*.sh'
# 若曾在舊工作區編輯過,先確認沒有 CRLFgrep -rlU $'\r' ~/Blanton_Script
ssh admin@<DUT_IP> 'sudo -i'
# (現場無網路時改用 USB 隨身碟複製到 ~/Blanton_Script
# ── 3. DUT 端:確認 FPGA BDF(每台不同!)────────────────────
lspci -Dnn | grep -i fpga # 例:0000:03:00.0
ls -d /sys/bus/pci/devices/0000:*:00.3
# 若不是 0000:03:00.x,改 ~/Blanton_Script/blanton_fpga_pcimem.sh 的 FUNCT0..3_RES
# 以及 src/Script_ABC_Blanton/utils/show_pcie_error_reg_*.ttl 內的 BDF
# ── 4. DUT 端:煙霧測試(跑 Script A 之前先手動確認)──────────
# ── 3. DUT 端:煙霧測試(跑 Script A 之前先手動確認)──────────
source ~/Blanton_Script/blanton_fpga_pcimem.sh
# → 會印 [INFO] CB FPGA BDF = 0000:0X:00.0-.3 (resource0) ← BDF 自動偵測,不用改檔
# 偵測錯或多顆候選時:FPGA_BDF=0000:05:00 fpga_rescan
cb_fpga 0 0xD90 # RST_CAUSE_REG,讀得到值 = BAR 通了
source ~/Blanton_Script/blanton_cb_i2c.sh
cb_i2c_init 11 && cb_i2c_scan 11 # Ch11 風扇控制器,應看到 0x20
source ~/Blanton_Script/LTC2980_Margin_Script/margin.sh
margin_init Blanton_CB_CONN13 && margin_status
source ~/Blanton_Script/blanton_traffic_linespeed.sh
blanton_traffic_linespeed ps -u 0 # 對照線的 link/speed
blanton_traffic_linespeed init -u 0 -n # -n = dry-run,先看要下哪些 bcmcmd
# ── 4. DUT 端:校對 AER 的 BDF(自動偵測沒涵蓋這裡!)─────────
lspci -Dnn # 對照 ARCHITECTURE.md 的 PCIe 拓樸表
# 與 src/Script_ABC_Blanton/utils/show_pcie_error_reg_*.ttl 內寫死的 7 組 BDF 比對,
# 不符就改 TTL——否則 AER 會讀到別的裝置或不存在的路徑
```
**HostWindows)端:**
@@ -402,6 +542,19 @@ margin_init Blanton_CB_CONN13 && margin_status
8. 檢查 Logs\Blanton_Margin_<ts>.log
```
**打包交付:**
```bash
./publish/publish.sh # 版號讀 Script A 檔頭 → publish/Script_ABC_Blanton_V1.0.3/
./publish/publish.sh -z # 另外產出同名 .zip
./publish/publish.sh -n # dry run,只印會做什麼
./publish/publish.sh -f # 覆蓋既有輸出資料夾
```
版號**不是參數而是從 Script A 的 `; Version : Vx.y.z` 讀出來的**,所以資料夾名不可能跟
測試員打開 macro 看到的版本不一致。要出新版就先改 Script A 檔頭再跑。
輸出會排除 `*.log`(測試 log 含 DUT 序號,不隨交付外流),但保留空的 `logs/``logopen` 寫入。
> 註:本專案非 .NET / Blazor / EC2 架構,`aws-ec2-deploy` 與 `vbox-run-deploy` 兩份部署指南不適用,已跳過。
---
@@ -416,12 +569,13 @@ margin_init Blanton_CB_CONN13 && margin_status
**目標:** 專案進 git(NAS + Gitea),帳密不再躺在腳本裡,任何人 clone 下來照文件就能跑。
**包含:**
- [ ] `git init` + `.gitignore``secret/*``For_AI/``logs/*.log``~$*``*.DSN`
- [ ]`secret/README.md` + `secret/config_secret.ttl.example``__CHANGE_ME__` placeholder
- [x] `git init` + `.gitignore``secret/*``For_AI/``logs/*.log``~$*``*.DSN`
- [x]`secret/README.md` + `secret/config_secret.ttl.example``__CHANGE_ME__` placeholder
- [ ] `config.ttl` 改為 `include "..\..\secret\config_secret.ttl"`(或現場放同層),
`1_Blanton_Script_A.ttl``sendln "YourPaSsWoRd"` 換成 `sendln dut_password`
- [ ] `src/Script_ABC_Blanton/logs/.gitkeep`,既有 `Blanton_Margin_20260814-164122.log` 移出 repo
- [ ] `CLAUDE.md` + `ARCHITECTURE.md` 進 repopush `nas` + `gitea`**不推 GitHub**
**← 唯一未完成項;密碼目前仍寫在腳本裡**
- [x] `src/Script_ABC_Blanton/logs/.gitkeep`,既有 `Blanton_Margin_20260814-164122.log` 移出 repo
- [x] `CLAUDE.md` + `ARCHITECTURE.md` 進 repopush `nas` + `gitea`**不推 GitHub**
**驗收條件:** 在另一台機器 `git clone` → 填 `secret/config_secret.ttl` → Script A 能完整登入並產生 log
`git log -p` 中搜不到任何實際密碼。
@@ -442,7 +596,8 @@ margin_init Blanton_CB_CONN13 && margin_status
- [ ] 新增 `utils/show_pwr_data.ttl``pwr_data` + `pwr_data_pdb`)與
`utils/show_temp_sensor.ttl``temp_all`),把 `blanton_pwr_data.sh` /
`blanton_temp_sensor.sh` 真正接進流程(目前 Script A 只 source 沒用)
- [ ] Script A 開頭加 BDF 自檢`lspci -Dnn | grep -i fpga`log 裡看得到本台的實際 BDF
- [x] Script A 開頭加 BDF 自檢 —— 由 `blanton_fpga_pcimem.sh` V1.5.0 的 source 時自動偵測達成,
log 內會出現 `[INFO] CB FPGA BDF = ...`(比原本規劃的 `lspci` 一行更完整)
**驗收條件:** 跑一次 Script Alog 內同時含 reset cause 三個暫存器值、9 顆 LTC2980 × 16ch 的
margin status、全部 VRM 軌的 Vin/Vout/Iout/Temp、CB + ICB 溫感讀值,且無 `ERR` / `NA` 以外的異常。
@@ -454,18 +609,26 @@ margin status、全部 VRM 軌的 Vin/Vout/Iout/Temp、CB + ICB 溫感讀值,
**目標:** 壓力程序起得來、起不來看得出來,soak 迴圈有時戳、可控週期。
**包含:**
- [ ] 部署 `~/hammer/tools/` 前置檢查:新增 `utils/check_hammer_tools.ttl`
`ls -l ~/hammer/tools/` + `ls ~/hammer/tools/amd/mlucas-avx2`),缺檔就別往下跑
- [ ] 前置檢查:`~/hammer/tools/amd/mlucas-avx2` 仍是唯一還依賴 hammer 的項目,缺檔就別往下跑
- [ ] 壓力程序 log 加時戳,避免多輪覆蓋:
`mlucas_amm_log``~/logs/mlucas_amm_log_$(date +%Y%m%d-%H%M%S).log`Status 表 ScriptB #2 的既定寫法)
- [ ] `jobs` 之後加驗證:期望 10 個背景 job1 mlucas + 5 mem + hhmd/ssd/pcie/usb),
數量不符時 `messagebox` 提示
- [x] 壓力層改用平台的 `bgctl``memtester` / `qfx5252-stress-ssd` / `qfx5252-stress-usb`),
取代 `~/hammer/tools/stress_*.py`Script B 以 `jobs` + `bgctl list` 兩者並列確認
- [ ] `bgctl list` 之後加數量驗證,不符時 `messagebox` 提示(目前只印出來給人看)
**← V1.0.10 起 `bgctl list` + `jobs` 改為每輪都印,中途死掉的 job 看得到,但仍要靠人眼比對**
- [ ] soak 迴圈改為「每輪先 `date`、跑完 `pause <間隔>`」,
`EN_LoopInterval`(預設 600 秒 = 10 min)放進 `config.ttl`,兌現 Status 表的「Get data every 10mins」
- [ ] soak 每輪補 `show platform psustatus` / `voltage` / `current`(目前只有 fan + temperature
**← V1.0.10 已補上 `pause 60`(迴圈不再全速空轉),但 60 秒是寫死的、也還沒有每輪 `date`
距 Status 表的 10 分鐘仍有差距,故本項未結**
- [x] soak 每輪補 `show platform psustatus` / `voltage` / `current`
—— 迴圈已改為 `include "utils/setup_pmon.ttl"`,九項全收
- [ ]`; ========== CLEAR EVENT ==========`Status 表 ScriptB #1Owner: Alan
- [x] BMC 納入 soak`bmc_monitor.sh``free -m` 取樣)與 `bmc_monitor_ddr.sh``memtester`
由 Script B 啟動、Script C 停止並把 log 併進 master log
- [x] Script A 加開機資訊與就緒等待:`show boot`、BMC `/etc/issue``wait_init.ttl`
- [x] Script C 加 `show reboot-cause` / `show uptime`soak 期間意外重開看得見
**驗收條件:** Script B 啟動後 `jobs` 顯示 10 個 job;連續跑 3 輪,log 內三段各自帶正確時戳且間隔 ≈10 min;
**驗收條件:** Script B 啟動後 `jobs` 顯示 8 個 job;連續跑 3 輪,log 內三段各自帶正確時戳且間隔 ≈10 min;
故意把 `~/hammer` 改名,Script B 會在前置檢查就停下並提示。
---
@@ -474,19 +637,32 @@ margin status、全部 VRM 軌的 Vin/Vout/Iout/Temp、CB + ICB 溫感讀值,
**目標:** 補齊 Status 表中 Owner=Alan 的三個 traffic 項目,讓測試涵蓋交換晶片的資料面。
**包含:**
- [ ] `utils/traffic_setup.ttl`Blanton/SONiC 版的 port 起始設定與 counter 清零
(對照 Carlsbad 的 `clear int stat global`,改用 SONiC `sonic-clear counters`
- [ ] `utils/show_traffic_counters.ttl``show interface counters`
`show interface counters errors``show interface status`
- [ ] `utils/show_interface_detail.ttl``show interface transceiver eeprom/dom`
`show interface link-training`(對照 Carlsbad `sho int transceiver detail`
- [ ] `utils/show_tech.ttl``show techsupport`Script C 收工快照,Status 表 ScriptC #4
- [ ] Script A 的 TRAFFIC-SETUP-STAGE 與 Script C 的 CHECK Traffic counters 接上這些片段
- [ ] `config.ttl``EN_Traffic` 旗標,未接線的站點可整段跳過
> 📌 **實作路線與原規劃不同**:原本設想用 SONiC CLI`sonic-clear counters` / `show interface counters`),
> 實際落地的是**更底層的 `bcmcmd` drivshell 路線** —— SWB 實體 loopback 線 + VLAN 成對配置 + `tx` burst
> 直接讀 ASIC 的 `MIB_TPOK`/`MIB_RPOK`。好處是能做 per-pair 雙向交叉比對(`cdA.TX == cdB.RX`),
> 比 SONiC CLI 的單向 counter 強得多;代價是綁定 Broadcom SDK 與那條 loopback 線。
**驗收條件** `EN_Traffic=1` 時,Script A log 有清零後的 counter 基線、Script C log 有結束時的
counter 與 error-counter,兩者可直接相減;`EN_Traffic=0` 時整段不執行且不報錯。
**包含**
- [x] `blanton_traffic_linespeed.sh`V0.4.0):`init` / `clear` / `show` / `start` / `stop` /
`ps` / `report` / `run`unit 0 與 1VLAN 30..137 對應 `cdN``cdN+32`
(取代原規劃的 `utils/traffic_setup.ttl`
- [x] per-pair TX/RX 報表與 PASS/FAIL 判定:DUT 上 awk`report`+ 離線
`tools/bcm_mibpair_report_V1.1.0.py`,同版面同判定規則
(取代原規劃的 `utils/show_traffic_counters.ttl`
- [x] Script B 接上 `ps → init → clear → show → start`、Script C 接上 `stop → report`
- [x] 原始 bcmcmd 指令表留檔:`tools/traffic_loopback_{vlan_setting,start,vlan_remove}.txt`
- [x] **Script A 的流量基線** —— 原本的空區塊已改成完整一輪
`ps → init → clear → show → start → stop → report`,含 15 秒沉澱)
- [ ] `utils/show_interface_detail.ttl``show interface transceiver eeprom/dom`
`show interface link-training`Status 表 ScriptC #4,仍 On-Going
- [ ] `utils/show_tech.ttl``show techsupport`Script C 收工快照,Status 表 ScriptC #4
- [ ] `config.ttl``EN_Traffic` 旗標,未接 loopback 線的站點可整段跳過
(現在 B/C 是無條件執行,沒接線會整批 bcmcmd 失敗)
- [x] `-u 0` 已從三支腳本移除,全部沿用工具預設 `TL_UNITS="0 1"`,兩塊 switch board 都測到
- [x] 100G uplink 狀態:`show interfaces status Ethernet513,Ethernet514`A 與 B 各一次)
**驗收條件:** `EN_Traffic=1` 時,Script C 的 `report` 印出 108 對的 TX/RX 表且全 PASS
拔掉一條 loopback 線重跑,該對顯示 FAIL 或 NA 且 exit code 為 2`EN_Traffic=0` 時整段不執行且不報錯。
---
@@ -501,7 +677,13 @@ counter 與 error-counter,兩者可直接相減;`EN_Traffic=0` 時整段不
UE > 0 或 fatal/nonfatal > 0 直接標 FAIL
- [ ] Margin 表解析:把 9 顆 × 16ch 的 Vout / 偏差% 收成 CSV,標出超出 ±(profile 設定值 + 容差) 的軌
- [ ] 溫度 / 風扇 / 電源趨勢:soak 期間逐輪取樣 → CSV(可直接畫圖)
- [ ] Stress 判定:`mlucas_amm_log``log.stress_ssd` 的錯誤關鍵字掃描
- [ ] Stress 判定:`mlucas_amm_log``bgctl` 各工作的 log
`/host/hw-eval/.../qfx5252-stress-{ssd,usb}.log`)的錯誤關鍵字掃描
- [ ] Traffic 判定:**直接複用 `tools/bcm_mibpair_report_V1.1.0.py`**(它已能吃 console log 的
`show c` 段落並輸出 PASS/FAIL),不要重寫一套解析
- [ ] 管理網路判定:抓 `mgmt_ping_monitor.sh``RESULT` 行(已是單行可 grep 格式),
並檢查 `nic_tx`/`nic_rx` 不為 0 —— 否則 PASS 也不能歸屬到那張卡
- [ ] NVMe / EDAC`nvme smart-log``smartctl -x``ras-mc-ctl --summary` 的前後差值
- [ ] 產出 `Logs/<run>_summary.md`:一頁 PASS/FAIL + 每個子系統一行結論
**驗收條件:** 拿一份既有 log(如 `Blanton_Margin_20260814-164122.log`)跑解析器,
@@ -514,9 +696,9 @@ counter 與 error-counter,兩者可直接相減;`EN_Traffic=0` 時整段不
**目標:** 產出可直接交給現場測試員(或客戶)的一包,含操作手冊,不需要看原始碼。
**包含:**
- [ ] `tools/publish.sh`:把 `src/Script_ABC_Blanton/` + 空 `Logs/` + `secret/*.example`
打包成 `publish/Blanton_Script_ABC_v<ver>_<YYYYMMDD>/` 與同名 zip(比照
既有的 `Blanton_Script_ABC_v1.0_20251229` 命名)
- [x] `publish/publish.sh`:把 `src/Script_ABC_Blanton/` 打包成
`publish/Script_ABC_Blanton_<Ver>/``-z` 另出 zip)。版號讀 Script A 檔頭,
排除 `*.log` 但保留空的 `logs/`。工具本身進 git,產出被 gitignore
- [ ] `README.md`(現場用):Tera Term 設定、A→B→C 操作順序、常見錯誤(BDF 不符、
margin 掃不到、hammer 工具缺檔)與對應處理
- [ ] `CHANGELOG.md`:延續三支腳本頂端的 Version History,集中成一份
@@ -537,12 +719,46 @@ counter 與 error-counter,兩者可直接相減;`EN_Traffic=0` 時整段不
COM36 才拿到部分結果;`READ_VIN` 已知不是真的輸入電壓(跟隨 `READ_VOUT`)。待拿到 brick 的
`MFR_ID`/`MFR_MODEL` 後對照 datasheet 定案。
- **`show_pcie_error_reg.ttl` 淘汰**:舊的合併版已被 7 支 per-device 版取代,確認無人引用後刪除。
- **BDF 自動偵測**:把 `blanton_fpga_pcimem.sh` 註解中的 auto-detect 片段轉正,
`lspci -Dnn | grep -i fpga` 動態填 `FUNCT0..3_RES`,讓換 DUT 不用改檔。
- **BDF 自動偵測** —— 已於 `blanton_fpga_pcimem.sh` V1.5.02026-08-17)完成。
下一步是把同樣的偵測結果餵給 `utils/show_pcie_error_reg_*.ttl`
讓 AER 那 7 組 BDF 也不用手改(目前仍是硬編碼)。
- **`stress_hhmd` / `stress_pcie` 的替代**:兩者已於 2026-08-17 移除,I2CPCIe 的持續讀取壓力
目前沒有替代品。若仍需覆蓋這塊,可考慮用 `blanton_cb_i2c.sh` / `cb_fpga` 寫一支輕量 loop。
- **SSH / 網路模式**:目前全走 COM console。若 DUT 管理網路可用,可改用 `plink`/`ssh` 執行同一組
bash 工具,速度與可靠度都比 serial 好,TTL 只保留開機階段。
- **多 DUT 併行**:多台 DUT 同時 soak 時,Tera Term 每台開一個 macro window
可考慮改成 Python`pyserial`)統一排程與集中收 log。
- **與 `LTC2980_Margin_Script` 獨立 repo 的同步機制**:目前 `Blanton_Script/LTC2980_Margin_Script/`
是該 repo 的複本(少了 `ARCHITECTURE.md`/`CLAUDE.md`/`Reference/`,多了兩支 `blanton_*` 後端)
待評估改用 git submodule 或明確的 vendored 同步腳本。
- **margin 子專案的重複檔** —— v2.6.0 已移除內含的 `blanton_cb_i2c.sh` /
`blanton_fpga_pcimem.sh` 複本,改由 `_swb_find_backend` 往上層找共用版,漂移風險解除
與獨立 repo `LTC2980_Margin_Script` 的同步機制(submodule 或 vendored 腳本)仍待評估
- **`Ref/margin_command_trace.md` 需重新產生**:SWB 章節的 1223 條 `cb_pmbus_*` 指令是依
修正前的通道/位址展開的(其中 487 條用到已不存在的 `0x5D`/`0x5F`/`0x63`/`0x65`),
文件已加過期警告但內文未動。第 1~4 章不受影響。
- **`wait_init.ttl` 沒有逾時出口**:若 pmon 一直起不來,`result` 既非 1 也非 2,迴圈會無限重試,
Script A 既不往下走也不報錯。可加最大重試次數 + `messagebox` 提示。
- **腳本的執行權**`mgmt_ping_monitor.sh``bmc_monitor.sh`(以及已停用的 `bmc_monitor_ddr.sh`)需人工 `chmod +x`
可在 Script B 起它之前補一行(比照既有的 `chmod +x ~/hammer/tools/amd/mlucas-avx2`),或在 publish 時處理。
- **`bmc_monitor_ddr.sh` 已停用**:B/C 內的呼叫已註解,DDR 壓力改由 `bgctl run bmc-manager run memtester`
取代。檔案還留在 repo,待確認不再需要後刪除,或在檔頭標註停用。
`bmc_monitor.sh` 已於 2026-08-25 重新啟用,並加上 I2C 寫入/讀回圖樣測試。)
- **`rm /host/hw-eval/jobs/*` 清不掉子目錄**:Script A 用的是不帶 `-r``rm`,但 Script C
`cp -r` 那個目錄,代表裡面可能有子目錄。清不乾淨的話,上一輪的 job log 會被當成這輪的一起
複製到 USB。要確認 `bgctl` 的產出結構,必要時改成 `rm -rf .../jobs/*`
- **100G PRBS 尚未接進 A/B/C**`port_prbs_monitor.sh` 已隨包交付、可手動執行,但三支腳本裡的呼叫
都還註解著(bring-up 中)。要啟用時記得 PRBS 期間 link 會顯示 down`stop` 之後才會回來。
- **Script C 的 USB 掛載走 `mount` 而非 `usb_target.sh -o mnt`**:目前是 `-o dev` 取節點再自己
`mount <dev> /mnt/usb`,繞過了工具內建的 `mkdir -p`、exfat/ntfs `modprobe` 與可寫性驗證。
掛載點不存在、檔案系統模組沒載、或媒體唯讀時只會失敗一行,job log 就沒帶出來。
- **風扇確認提示是無限等待**`wait "Set all configured fan channels to"` 之後才 `sendln "y"`
若該版 `fan-speed-control.sh` 不再詢問(或字串改了),這個 `wait` 會永遠等下去,Script A 停在那裡
且不報錯。加個 `timeout` + 分支會比較保險(記得還原 `timeout = 0`)。
- **TR518 內建封包測試尚未腳本化**:`tools/TR518.txt` 記下了 `tr 518 Scenario=6 Profile=2 Count=260
PktSize=324 PortList=1-259,270-565` 這組指令(含 `port all lb=mac` + `l2 learn off` 前置),
目前只能手貼。它與 `blanton_traffic_linespeed` 是兩條互斥的打流路線(都會動 loopback 與 L2 學習),
要腳本化的話得先決定兩者怎麼共存,不能同時跑。
- **`settings/` 裡 SWB0 與 SWB1 是兩份相同的資料**:8 個 `.conf` 中,SWB1 的四個除了 `CHIPS[]` 的
bus 與 `CB_I2C_CH` 之外,net name 與 `VNOM` 與 SWB0 逐欄相同(見 `docs/LTC2980_channel_map.csv`)。
改一邊忘了改另一邊不會有任何警告。可考慮改成「共用 net 定義 + 各自的 transport 覆寫」兩層檔案。
- **Script C 沒有倒出 `mgmt_ping.log`**C 會 `stop` 管理網路監控並印 `ip -s link show`
但沒有 `cat ~/Blanton_Script/log/mgmt_ping.log`,所以整個 soak 期間累積的 per-leg `RESULT`
留在 DUT 上、沒進 master log。Script A 那輪基線有 `cat`。
+145 -15
View File
@@ -9,6 +9,16 @@ DUT 端的取數能力由一組 **bash 工具庫** 提供(CB FPGA `pcimem` →
完整架構見 [ARCHITECTURE.md](ARCHITECTURE.md)。
**目前狀態(2026-08-27**Script A 為 **V1.0.10**(A 集中記錄 A/B/C 三支的變更)。涵蓋
PCIe AER / DDR EDAC / PMON+TPM / 電壓 margin9 顆 LTC2980/ SWB loopback 線速流量 /
10G+1G 管理網路 ping / BMC DDR 與 USB / NVMe 健康,壓力層走平台的 `bgctl`
switch unit 數量由 `config.ttl``SWB_UNIT0` / `SWB_UNIT1` 宣告,風扇轉速由 `FAN_SPEED` 指定。
測試 job log 收在 DUT 的 `/host/hw-eval/jobs/`Script C 結束時掛好 USB 再帶時戳複製到 `/mnt/usb/`
100G PRBS`port_prbs_monitor.sh`)已隨包交付,但 **A/B/C 的呼叫仍註解掉**,還在 bring-up。
Script B 的 soak 迴圈每輪印 `bgctl list` + `jobs`,並 `pause 60`V1.0.10 之前完全沒有 pause)。
最後一次發布是 **V1.0.9**;SWB 的 I2C 通道與位址已於 2026-08-19 上機驗證。
交付用 `./publish/publish.sh` 打包。
## 技術棧
- **Host**Tera Term 5.x TTL macro`.ttl`),Windows
@@ -17,6 +27,12 @@ DUT 端的取數能力由一組 **bash 工具庫** 提供(CB FPGA `pcimem` →
- **I2C/PMBus**CB FPGA F3 I2C masterCh0~Ch17base 0x300 stride 0x20)、
CB F3 VI2C proxy → ICB/PDBCh0~Ch4base 0x780)、CPU 原生 bus`i2cset`/`i2cget`
- **Margin**LTC2980= 2 × LTC2977over PMBusLINEAR16
- **資料面流量**`bcmcmd`Broadcom drivshellSWB loopbackVLAN 30..137 成對 `cdN`/`cdN+32`
`tx 100 length=512`;判定讀 ASIC `MIB_TPOK`/`MIB_RPOK` 做 per-pair 雙向交叉比對
- **BMC**host 端 `bmc-manager run "<cmd>"`,不需對 BMC 開 SSH(繞開其 busybox 工具鏈)
- **背景工作**:平台的 `bgctl run/list/stop --all/reset`;長時間 ping 用 `systemd-run --unit=` 掛 transient unit
- **管理網路**iproute2`ip link set` / `address replace` / `route replace ... metric`+ `ping -I`
- **風扇**max31790 driver rebind + `fan-speed-control.sh <%>`(轉速讀 `config.ttl``FAN_SPEED`
- **判定資料源**PCIe AER sysfs`aer_dev_*`)、EDAC sysfs`dimm_{ce,ue}_count`)、SONiC `show platform *`
- **無編譯步驟**:TTL 與 bash 都直譯執行
@@ -26,17 +42,19 @@ DUT 端的取數能力由一組 **bash 工具庫** 提供(CB FPGA `pcimem` →
# ── 部署 bash 工具庫到 DUT ──────────────────────────
scp -r src/Script_ABC_Blanton/Blanton_Script admin@<DUT_IP>:~/
# ── DUT 上先確認 FPGA BDF(每台不同!)───────────────
lspci -Dnn | grep -i fpga
ls -d /sys/bus/pci/devices/0000:*:00.3
# ── DUT 上載入工具(Script A 也是做這五件事)─────────
source ~/Blanton_Script/blanton_fpga_pcimem.sh
# ── DUT 上載入工具(Script A 也是做這六件事)─────────
source ~/Blanton_Script/blanton_fpga_pcimem.sh # ← 會自己印出偵測到的 CB FPGA BDF
source ~/Blanton_Script/blanton_cb_i2c.sh
source ~/Blanton_Script/blanton_icb_vi2c.sh
source ~/Blanton_Script/blanton_pwr_data.sh
source ~/Blanton_Script/blanton_traffic_linespeed.sh
source ~/Blanton_Script/LTC2980_Margin_Script/margin.sh
# ── FPGA BDF:自動偵測,不用改檔;必要時手動釘 ───────
fpga_rescan # 換 DUT / 重新列舉後重測
FPGA_BDF=0000:05:00 fpga_rescan # 偵測到多顆或選錯時,指定 domain:bus:dev(不含 .fn
echo $FPGA_BDF_ACTIVE # 目前生效的 BDF
# ── 煙霧測試 ────────────────────────────────────────
cb_fpga 0 0xD90 # RST_CAUSE_REG,讀得到 = BAR 通了
cb_i2c_init 11 && cb_i2c_scan 11 # Ch11 風扇控制器,應看到 0x20
@@ -45,11 +63,70 @@ temp_all # CB + ICB 溫感
pwr_data # SWB VRM 全軌
margin_init Blanton_CB_CONN13 && margin_status
# ── SWB loopback 流量(bcmcmdunit 0/1)─────────────
# 不帶 -u 就是預設 TL_UNITS="0 1",兩塊 switch board 都做(TTL 三支腳本即如此)
blanton_traffic_linespeed ps # 對照線的 link/speed
blanton_traffic_linespeed init # VLAN 30..137cdN <-> cdN+32
blanton_traffic_linespeed clear # clear c
blanton_traffic_linespeed start -tx 100 -length 512
blanton_traffic_linespeed stop # 移除 VLAN 成員(--destroy 連 VLAN 一起砍)
blanton_traffic_linespeed report # per-pair TX/RX + PASS/FAILexit 2 = 有 FAIL/NA
blanton_traffic_linespeed run # ps → init → clear → start → report 一條龍
blanton_traffic_linespeed init -u 0 # 只做單一 unit 時才需要 -u
blanton_traffic_linespeed init -n # -n dry-run:只印要下的 bcmcmd,不碰 DUT
blanton_traffic_linespeed report -f <log> # 離線重解一份存下來的 console log
# 離線版報表(開發機上跑,選項更多)
python3 tools/bcm_mibpair_report_V1.1.0.py <drivshell.log> -a
# ── 管理網路 ping 監控(需先 chmod +x!)──────────────
chmod +x ~/Blanton_Script/*.sh # 佈署後必做一次
~/Blanton_Script/mgmt_ping_monitor.sh fg # 前景試一輪(約 30 秒),確認 IP/對端/網卡
~/Blanton_Script/mgmt_ping_monitor.sh start # 背景,log/mgmt_ping.logstart 會先清空)
~/Blanton_Script/mgmt_ping_monitor.sh status
~/Blanton_Script/mgmt_ping_monitor.sh stop
~/Blanton_Script/mgmt_ping_monitor.sh summary # 只看每段的 RESULT 行
# ⚠️ tail 子命令是 tail -f,會卡住不返回;TTL 裡要倒 log 請用 cat
# bmc_monitor.sh start/stop # free -m + BMC I2C 寫入/讀回圖樣測試,log/bmc_poll.log
# ⚠️ bmc_monitor_ddr.sh 已停用(DDR 壓力改走 bgctl + bmc-manager),檔案仍在
# ── 100G PRBS(目前 TTL 未啟用,手動測用)────────────
bash ~/Blanton_Script/port_prbs_monitor.sh start # 兩個 port 同時
bash ~/Blanton_Script/port_prbs_monitor.sh start a # 只跑 Ethernet513
bash ~/Blanton_Script/port_prbs_monitor.sh start b # 只跑 Ethernet514
bash ~/Blanton_Script/port_prbs_monitor.sh report # PASS/FAIL 統計(沒跑的顯示 SKIP
bash ~/Blanton_Script/port_prbs_monitor.sh stop # 收尾會跑 prbsstat STOp + prbs clear
# ⚠️ PRBS 打起來時 link 會顯示 down,這是正常的 —— 所以 start 刻意不印 port status
# ── USB 目標偵測 ────────────────────────────────────
bash ~/Blanton_Script/usb_target.sh -o dev # /dev/sdX1
bash ~/Blanton_Script/usb_target.sh -o mnt # 掛好並回傳掛載點
# ── 背景壓力(平台工具)──────────────────────────────
bgctl run /usr/sbin/memtester 1G 100
bgctl list
bgctl stop --all ; bgctl reset --yes
# ── 通道對照表(開發機上跑,改完 settings/*.conf 一定要重產)──
./tools/gen_channel_map.sh # → docs/LTC2980_channel_map.csv144 列)
./tools/gen_channel_map.sh - # 只印到 stdout
# ⚠️ 需要 gawk(用到 3 參數的 match());mawk 會直接擋下並提示
# ── 各工具的 help ───────────────────────────────────
blanton_fpga_help ; blanton_cb_i2c_help ; vi2c_help
temp_sensor_help ; pwr_data_help
temp_sensor_help ; pwr_data_help ; blanton_traffic_linespeed_help
```
**打包交付(在開發機上):**
```bash
./publish/publish.sh # 版號讀 Script A 檔頭 → publish/Script_ABC_Blanton_V1.0.3/
./publish/publish.sh -z # 另出 zip
./publish/publish.sh -n # dry run
```
要出新版**先改 Script A 檔頭的 `; Version :`**,publish 才會用新版號建資料夾。
**Host 端跑一輪測試:**
```
@@ -69,10 +146,17 @@ Tera Term 連 COM port (115200-8-N-1)
- **新增取數項目** = 新增一支 `utils/show_xxx.ttl`,再在主腳本 `include`,不要往主腳本塞指令
- **bash 命名**:公開函數 `cb_i2c_* / vi2c_* / temp_* / pwr_* / margin_*`,內部 helper 一律底線開頭
`_cbi2c_* / _vr / _ts_* / _pwr_*`
- **版本紀錄**每支 `.ttl``.sh` 檔頭都有 `Version History` 區塊,改動時**在檔頭補一行**
`V1.0.2 YYYY-MM-DD <做了什麼>`),不要只改版號
- **版本紀錄**`.sh` 每支檔頭都有 `Version History` 區塊,改動時**在檔頭補一行**
`V1.0.2 YYYY-MM-DD <做了什麼>`),不要只改版號
TTL 這邊 **2026-08-17 起改為集中制**`1_Blanton_Script_A.ttl` 的 History 記錄 A/B/C 三支的
所有變更,B 與 C 的檔頭版本區塊已移除。⚠️ 這是待確認的慣例變更 ——
若要回到「每支各自記錄」,B/C 的 History 需要補回去
- **Commit**Conventional Commits**英文**。scope 用 `ttl` / `fpga` / `i2c` / `vi2c` /
`margin` / `pwr` / `temp` / `docs`
- **換行符**:由 `.gitattributes` 管理 —— `*.sh` `*.conf` `*.py` `*.txt` `*.md`**LF**
(要在 SONiC 上跑/被 source),`*.ttl` 維持 **CRLF**Tera Term 只在 Windows 跑)。
編輯 `.ttl` 時要保留 CRLF,別讓整檔變成 LF 產生假 diff
- **打包**`./publish/publish.sh`,版號**讀 Script A 檔頭**不是參數 —— 先改檔頭再打包
- **Status 表**`docs/TTL_Script_Blanton_Status_*.xlsx` 是逐項進度的正本(OK / On-Going + Owner),
完成一項就同步更新,別讓文件跟腳本脫節
@@ -81,9 +165,12 @@ Tera Term 連 COM port (115200-8-N-1)
- 🔒 **NDA:本 repo 只推 `nas` + `gitea`,絕不推 GitHub。** `docs/` 內含客戶 FPGA 規格。
- 🔒 **DUT 帳密不進 git**`1_Blanton_Script_A.ttl``sendln "YourPaSsWoRd"` 是 placeholder
實際值放 `secret/config_secret.ttl`gitignored)。**任何 commit 前先 grep 一次確認沒帶到真密碼。**
- ⚠️ **PCIe BDF 是 per-unit 的**`blanton_fpga_pcimem.sh``FUNCT0..3_RES` 已經改過三次
04:00.x → 03:00.x → 05:00.x → 03:00.x)。換 DUT 一定先 `lspci -Dnn | grep -i fpga`
`utils/show_pcie_error_reg_*.ttl` 內的 BDF 也要一起校。
- ⚠️ **PCIe BDF 是 per-unit 的 —— bash 端已自動化,TTL 端還沒**
`blanton_fpga_pcimem.sh` V1.5.02026-08-17)起 source 時自動偵測(sysfs vendor `0x1590`
`lspci``FPGA_BDF_FALLBACK`),**不用再每台改檔**;印一行 `[INFO] CB FPGA BDF = ...`
可用 `fpga_rescan` 重測、`FPGA_BDF=... fpga_rescan` 手動釘住。
**但 `utils/show_pcie_error_reg_*.ttl` 裡的 7 組 BDF 仍是寫死的**,換 DUT 還是要人工校對 ——
不校就會讀到別的裝置,或整段 `No such file or directory`
- ⚠️ **AER 判定看差值不看絕對值**Script A 抓基線、Script C 抓結果,相減才是本輪錯誤。
**Endpoint 與上游 Root Port 兩邊都要讀**,錯誤可能只記在 RP。
- ⚠️ **ICB/PDB 一律走 VI2C`blanton_icb_vi2c.sh`),不要走 VSPI**VSPI ~18 ms/reg 會讓 I2C
@@ -99,12 +186,55 @@ Tera Term 連 COM port (115200-8-N-1)
shelllog 也只有 A 會 `logopen`,B/C 是續寫。中途關掉 window 就得手動收程序、log 也斷了。
- ⚠️ **MP29816 的 Vout 解析度不是固定值**,由 `MFR_VOUT_SCALE_LOOP(0x29)` bit[12:10] 決定,
`blanton_pwr_data.sh` 執行期從晶片讀;xlsx 上寫的 `/5mV` 與 MP2985B 的 `+49` offset **都是錯的**
- ⚠️ **`bcmcmd` 就算失敗也 exit 0**`VLAN: ERROR: Entry exists` /
`Failed to execute the diagnostic command` 都照樣回 0),所以 `blanton_traffic_linespeed.sh`
是**比對輸出**而非看 exit code。另外 `bcmcmd` 一律要 `</dev/null` ——
否則它會吃掉 while 迴圈的 stdin**只有第一個 VLAN 被設定**(V0.3.1 修過的 bug,很難看出來)。
- ⚠️ **`stress_hhmd.py` / `stress_pcie.py` 已於 2026-08-17 由 KC 移除**Status 表 ScriptB #5/#7)。
Script B 現在是 **8 個背景 job**1 mlucas + 5 mem + ssd + usb),不是 10 個。
I2C/PCIe 的持續讀取壓力目前沒有替代品。
- ⚠️ **soak 迴圈沒有 `pause`**:取樣間隔不是 Status 表寫的 10 分鐘,而是取決於指令執行時間。
`-u 0` 已於 `491e807` 移除,A/B/C 現在都跑兩個 switch unit。)
- 🔒 **`.sh` / `.conf` 一定要 LF**。CRLF 的 shell script 在 SONiC 上直接掛(`$'\r': command not found`);
`settings/*.conf` 更陰險 —— 它們是被 `source` 的,`CB_I2C_CH=6` 會變成 `"6\r"`,安靜地壞掉。
`.gitattributes` 已擋住 git 這條路徑,但**不會回頭重寫既有工作區檔案**:早於它 clone 的目錄要
`git rm --cached -r . && git reset --hard` 才會生效,而且 **`git status` 乾淨不代表磁碟上是 LF**
`--renormalize` 只在進 index 的路上轉換)。驗證要看 byte:`grep -rlU $'\r' <dir>`
- ⚠️ **TTL 的 `timeout` 是全域的**`timeout = N` 之後**每一個** `wait` 都被限制在 N 秒。逾時的 `wait`
直接返回而沒吃到 prompt,下一個指令就在前一個還沒跑完時送出,之後整輪錯開一拍。**設了一定要
配一個 `timeout = 0` 還原**`wait_init.ttl` 60→0Script B 的 cdc_ncm 探測 15→0 在 `:skip_ping`
- ⚠️ **兩張管理網卡同網段會 ARP flux**`mgmt_ping_monitor.sh` 讓 eth0/eth1 同時 up,同一個 /24 下
對端 ARP 可能由任一張卡回應,回包落在沒送封包的那張。`ARP_STRICT=1` 擋掉;判讀時看 `RESULT` 行的
`nic_tx`/`nic_rx`(該卡自己的 packet delta)—— **ping 成功但 `nic_tx` 接近 0 = 封包從另一張卡出去的**
- ⚠️ **PRBS 跑起來時 `show interfaces status` 會顯示 DOWN**,那是 link 離開正常運作模式,不是故障。
`port_prbs_monitor.sh``start` 因此刻意不印 port status(避免測試員誤判),只在 `prbs clear` 之後的報表裡印
- ⚠️ **`ip -s link` 的欄位順序是 `bytes packets errors ...`**,要 packet 數取的是**第 2 欄**不是第 1 欄
- ⚠️ **`mgmt_ping_monitor.sh` / `bmc_monitor*.sh` 需要 exec bit**,且**不是改用 `bash x.sh start` 就能繞過** ——
`do_start` 內部是 `setsid "$SCRIPT_PATH" __daemon`,直接執行該路徑。git 記錄是 `100644`
Windows → 隨身碟 → DUT 也帶不動權限位元,所以佈署後要 `chmod +x ~/Blanton_Script/bmc_monitor*.sh`
沒做的話 start 會 Permission denied、後面的 `cat` 撲空,而且**不會報錯**,只會得到一份全空的 log
- ⚠️ **這幾支 monitor 的 `tail` 子命令是 `tail -n 50 -f`,永遠不返回**。TTL 裡絕對不要呼叫它
Script C 曾因此卡死,`18ce7a1` 改成直接 `cat log/*.log`
- ⚠️ **背景 monitor 的 `start` 只等 1 秒就返回**,不是跑完一輪。Script A 要做一輪基線的話,
`start``stop` 中間必須 `pause`(目前是 45 秒,一輪約 30 秒)—— 少了它 log 只會有 START banner
- ⚠️ **風扇那段設完會 `wait "Set all configured fan channels to"` 等確認提示再送 `y`**。該提示沒出現
(版本改了、字串換了)就會永遠卡住且不報錯 —— 現場看起來像 Script A 當掉
- ⚠️ **`show_dmesg.ttl` 結尾會 `dmesg -C` 清空 kernel ring buffer**。這是刻意的(讓 Script C 只看到
soak 期間新產生的訊息),但代表事後在 DUT 上 `dmesg` 撈不到舊訊息 —— 內容只存在 master log 裡
- ⚠️ **DUT 的 awk 對 `%d` 會夾到 INT32**(2147483647)。線速流量的計數器是 ~10¹² 量級,
所以 awk 裡輸出大數一律用 `%.0f`double 域,精確到 2⁵³)。`blanton_traffic_linespeed.sh`
V0.4.1 已修;寫新的 awk 報表時要記得同一件事
## 資料夾說明
- `src/Script_ABC_Blanton/` — 可執行內容(Tera Term 工作目錄;內含要 scp 到 DUT 的 `Blanton_Script/`
- `docs/` — 客戶 FPGA 規格、AER/EDAC 檢查依據、逐項 Status 表
- `tools/` — Host 端輔助腳本(log 解析、報表產生、打包)
- `publish/` — 交付用打包輸出(`Blanton_Script_ABC_v<ver>_<date>/` + zip
- `docs/` — 客戶 FPGA 規格、AER/EDAC 檢查依據、逐項 Status 表(最新為 `*_20260817.xlsx`)、
`LTC2980_channel_map.csv`9 個 `settings/*.conf` 合併成 Board,CONN,Ch,NetName,Vnom 144 列,
查 net 對應哪片哪個 channel 用;**改 `.conf` 後要重跑產生**
- `tools/` — Host / 離線端:`bcm_mibpair_report_V1.1.0.py`drivshell log → per-pair 報表)、
`traffic_loopback_*.txt`bcmcmd 原始指令表,`blanton_traffic_linespeed.sh` 是其 bash 版,
**改 loopback 線路時兩邊要同步**)、`100G_PRBS.txt` / `TR518.txt` / `fan_ctrl.txt`
(上機手貼用的原始指令;PRBS 已有 `.sh`TR518 尚未腳本化)
- `publish/``publish.sh`(打包工具,**進 git**+ `Script_ABC_Blanton_<Ver>/` 產出(**gitignored**
- `secret/` — 🚫 gitignored(除 `README.md``*.example`):DUT 帳密、per-unit BDF 覆寫、COM 設定
- `For_AI/` — 🚫 gitignored:AI 協作素材(波形截圖、草稿筆記)
+145
View File
@@ -0,0 +1,145 @@
Board,CONN,Ch,NetName,Vnom
CB,CONN13,CH0,V5P0_ALW,5.0
CB,CONN13,CH1,V3P3_ALW,3.3
CB,CONN13,CH2,PWR_VDD_MISC_ALW,0.75
CB,CONN13,CH3,V1P8_ALW,1.8
CB,CONN13,CH4,PWR_APU_VDDIO_SUS,1.1
CB,CONN13,CH5,PWR_VDD_MISC_RUN,0.75
CB,CONN13,CH6,PWR_APU_VDD_MEM_RUN,0.78
CB,CONN13,CH7,V0P8_PHY2_DVDD,0.8
CB,CONN13,CH8,V0P8_AVDD,0.8
CB,CONN13,CH9,V0P8_PHY,0.8
CB,CONN13,CH10,V1P8_FPGA,1.8
CB,CONN13,CH11,V2P5_FPGA,2.5
CB,CONN13,CH12,V1P1_FPGA,1.1
CB,CONN13,CH13,P5V_STBY,5.0
CB,CONN13,CH14,P3V3_STBY,3.3
CB,CONN13,CH15,V0P8_PHY2_AVDD,0.8
SWB0,CONN13,CH0,P0V75_DVDD_47,0.75
SWB0,CONN13,CH1,P0V75_AVDD_47,0.75
SWB0,CONN13,CH2,P1V5_AVDD_47,1.5
SWB0,CONN13,CH3,P0V9_AVDD_47,0.9
SWB0,CONN13,CH4,P1V8_1,1.8
SWB0,CONN13,CH5,PVDD3_3V_SYNTH_LDO_A3,3.3
SWB0,CONN13,CH6,PVDD3_3V_SYNTH_LDO_B1,3.3
SWB0,CONN13,CH7,P1V8_3,1.8
SWB0,CONN13,CH8,P1V8_DVDDIO_3,1.8
SWB0,CONN13,CH9,P0V75_DVDD_39,0.75
SWB0,CONN13,CH10,P0V75_AVDD_39,0.75
SWB0,CONN13,CH11,P1V5_AVDD_39,1.5
SWB0,CONN13,CH12,P0V9_AVDD_39,0.9
SWB0,CONN13,CH13,PVDD1V5_2,1.5
SWB0,CONN13,CH14,PVDD1V8_DUT,1.8
SWB0,CONN13,CH15,PVDD_MDIO,1.2
SWB0,CONN14,CH0,P0V75_AVDD_7,0.75
SWB0,CONN14,CH1,P1V5_AVDD_7,1.5
SWB0,CONN14,CH2,P0V9_AVDD_7,0.9
SWB0,CONN14,CH3,P1V8_5,1.8
SWB0,CONN14,CH4,P1V8_2,1.8
SWB0,CONN14,CH5,PVDD3_3V_SYNTH_LDO_B2,3.3
SWB0,CONN14,CH6,PVDD3_3V_SYNTH_LDO_A2,3.3
SWB0,CONN14,CH7,PVDD3_3V_SYNTH_LDO_A1,3.3
SWB0,CONN14,CH8,PVDD1V5_TSC,1.5
SWB0,CONN14,CH9,PVDD1V5_ANLG,1.5
SWB0,CONN14,CH10,NC,-
SWB0,CONN14,CH11,NC,-
SWB0,CONN14,CH12,PVDD1V5_1,1.5
SWB0,CONN14,CH13,NC,-
SWB0,CONN14,CH14,P1V8_DVDDIO_1,1.8
SWB0,CONN14,CH15,P0V75_DVDD_7,0.75
SWB0,CONN15,CH0,P1V5_AVDD_55,1.5
SWB0,CONN15,CH1,P0V9_AVDD_55,0.9
SWB0,CONN15,CH2,P0V75_DVDD_63,0.75
SWB0,CONN15,CH3,P0V75_AVDD_63,0.75
SWB0,CONN15,CH4,P1V5_AVDD_63,1.5
SWB0,CONN15,CH5,P0V9_AVDD_63,0.9
SWB0,CONN15,CH6,PVDD3_3V_SYNTH_LDO_B3,3.3
SWB0,CONN15,CH7,PVDD1V5_3,1.5
SWB0,CONN15,CH8,P0V85_STBY,0.85
SWB0,CONN15,CH9,P1V8_STBY,1.8
SWB0,CONN15,CH10,NC,-
SWB0,CONN15,CH11,P3V3_STBY,3.3
SWB0,CONN15,CH12,PVDD0V9_2,0.9
SWB0,CONN15,CH13,P0V75_DVDD_55,0.75
SWB0,CONN15,CH14,P0V75_AVDD_55,0.75
SWB0,CONN15,CH15,P1V8_DVDDIO_4,1.8
SWB0,CONN16,CH0,PVDD1V5_0,1.5
SWB0,CONN16,CH1,P1V8_4,1.8
SWB0,CONN16,CH2,PVDD3_3V_SYNTH_LDO_B5,3.3
SWB0,CONN16,CH3,PVDD3_3V_SYNTH_LDO_A5,3.3
SWB0,CONN16,CH4,PVDD3_3V_SYNTH_LDO_B4,3.3
SWB0,CONN16,CH5,PVDD3_3V_SYNTH_LDO_A4,3.3
SWB0,CONN16,CH6,P3V3,3.3
SWB0,CONN16,CH7,PVDD0_8V_T3,0.8
SWB0,CONN16,CH8,P0V75_DVDD_1,0.75
SWB0,CONN16,CH9,P0V75_AVDD_1,0.75
SWB0,CONN16,CH10,P1V5_AVDD_1,1.5
SWB0,CONN16,CH11,P0V9_AVDD_1,0.9
SWB0,CONN16,CH12,PVDD0V9_0,0.9
SWB0,CONN16,CH13,PVDD0V9_1,0.9
SWB0,CONN16,CH14,PVDD0V9_3,0.9
SWB0,CONN16,CH15,PVDD1V2_T3,1.2
SWB1,CONN13,CH0,P0V75_DVDD_47,0.75
SWB1,CONN13,CH1,P0V75_AVDD_47,0.75
SWB1,CONN13,CH2,P1V5_AVDD_47,1.5
SWB1,CONN13,CH3,P0V9_AVDD_47,0.9
SWB1,CONN13,CH4,P1V8_1,1.8
SWB1,CONN13,CH5,PVDD3_3V_SYNTH_LDO_A3,3.3
SWB1,CONN13,CH6,PVDD3_3V_SYNTH_LDO_B1,3.3
SWB1,CONN13,CH7,P1V8_3,1.8
SWB1,CONN13,CH8,P1V8_DVDDIO_3,1.8
SWB1,CONN13,CH9,P0V75_DVDD_39,0.75
SWB1,CONN13,CH10,P0V75_AVDD_39,0.75
SWB1,CONN13,CH11,P1V5_AVDD_39,1.5
SWB1,CONN13,CH12,P0V9_AVDD_39,0.9
SWB1,CONN13,CH13,PVDD1V5_2,1.5
SWB1,CONN13,CH14,PVDD1V8_DUT,1.8
SWB1,CONN13,CH15,PVDD_MDIO,1.2
SWB1,CONN14,CH0,P0V75_AVDD_7,0.75
SWB1,CONN14,CH1,P1V5_AVDD_7,1.5
SWB1,CONN14,CH2,P0V9_AVDD_7,0.9
SWB1,CONN14,CH3,P1V8_5,1.8
SWB1,CONN14,CH4,P1V8_2,1.8
SWB1,CONN14,CH5,PVDD3_3V_SYNTH_LDO_B2,3.3
SWB1,CONN14,CH6,PVDD3_3V_SYNTH_LDO_A2,3.3
SWB1,CONN14,CH7,PVDD3_3V_SYNTH_LDO_A1,3.3
SWB1,CONN14,CH8,PVDD1V5_TSC,1.5
SWB1,CONN14,CH9,PVDD1V5_ANLG,1.5
SWB1,CONN14,CH10,NC,-
SWB1,CONN14,CH11,NC,-
SWB1,CONN14,CH12,PVDD1V5_1,1.5
SWB1,CONN14,CH13,NC,-
SWB1,CONN14,CH14,P1V8_DVDDIO_1,1.8
SWB1,CONN14,CH15,P0V75_DVDD_7,0.75
SWB1,CONN15,CH0,P1V5_AVDD_55,1.5
SWB1,CONN15,CH1,P0V9_AVDD_55,0.9
SWB1,CONN15,CH2,P0V75_DVDD_63,0.75
SWB1,CONN15,CH3,P0V75_AVDD_63,0.75
SWB1,CONN15,CH4,P1V5_AVDD_63,1.5
SWB1,CONN15,CH5,P0V9_AVDD_63,0.9
SWB1,CONN15,CH6,PVDD3_3V_SYNTH_LDO_B3,3.3
SWB1,CONN15,CH7,PVDD1V5_3,1.5
SWB1,CONN15,CH8,P0V85_STBY,0.85
SWB1,CONN15,CH9,P1V8_STBY,1.8
SWB1,CONN15,CH10,NC,-
SWB1,CONN15,CH11,P3V3_STBY,3.3
SWB1,CONN15,CH12,PVDD0V9_2,0.9
SWB1,CONN15,CH13,P0V75_DVDD_55,0.75
SWB1,CONN15,CH14,P0V75_AVDD_55,0.75
SWB1,CONN15,CH15,P1V8_DVDDIO_4,1.8
SWB1,CONN16,CH0,PVDD1V5_0,1.5
SWB1,CONN16,CH1,P1V8_4,1.8
SWB1,CONN16,CH2,PVDD3_3V_SYNTH_LDO_B5,3.3
SWB1,CONN16,CH3,PVDD3_3V_SYNTH_LDO_A5,3.3
SWB1,CONN16,CH4,PVDD3_3V_SYNTH_LDO_B4,3.3
SWB1,CONN16,CH5,PVDD3_3V_SYNTH_LDO_A4,3.3
SWB1,CONN16,CH6,P3V3,3.3
SWB1,CONN16,CH7,PVDD0_8V_T3,0.8
SWB1,CONN16,CH8,P0V75_DVDD_1,0.75
SWB1,CONN16,CH9,P0V75_AVDD_1,0.75
SWB1,CONN16,CH10,P1V5_AVDD_1,1.5
SWB1,CONN16,CH11,P0V9_AVDD_1,0.9
SWB1,CONN16,CH12,PVDD0V9_0,0.9
SWB1,CONN16,CH13,PVDD0V9_1,0.9
SWB1,CONN16,CH14,PVDD0V9_3,0.9
SWB1,CONN16,CH15,PVDD1V2_T3,1.2
1 Board CONN Ch NetName Vnom
2 CB CONN13 CH0 V5P0_ALW 5.0
3 CB CONN13 CH1 V3P3_ALW 3.3
4 CB CONN13 CH2 PWR_VDD_MISC_ALW 0.75
5 CB CONN13 CH3 V1P8_ALW 1.8
6 CB CONN13 CH4 PWR_APU_VDDIO_SUS 1.1
7 CB CONN13 CH5 PWR_VDD_MISC_RUN 0.75
8 CB CONN13 CH6 PWR_APU_VDD_MEM_RUN 0.78
9 CB CONN13 CH7 V0P8_PHY2_DVDD 0.8
10 CB CONN13 CH8 V0P8_AVDD 0.8
11 CB CONN13 CH9 V0P8_PHY 0.8
12 CB CONN13 CH10 V1P8_FPGA 1.8
13 CB CONN13 CH11 V2P5_FPGA 2.5
14 CB CONN13 CH12 V1P1_FPGA 1.1
15 CB CONN13 CH13 P5V_STBY 5.0
16 CB CONN13 CH14 P3V3_STBY 3.3
17 CB CONN13 CH15 V0P8_PHY2_AVDD 0.8
18 SWB0 CONN13 CH0 P0V75_DVDD_47 0.75
19 SWB0 CONN13 CH1 P0V75_AVDD_47 0.75
20 SWB0 CONN13 CH2 P1V5_AVDD_47 1.5
21 SWB0 CONN13 CH3 P0V9_AVDD_47 0.9
22 SWB0 CONN13 CH4 P1V8_1 1.8
23 SWB0 CONN13 CH5 PVDD3_3V_SYNTH_LDO_A3 3.3
24 SWB0 CONN13 CH6 PVDD3_3V_SYNTH_LDO_B1 3.3
25 SWB0 CONN13 CH7 P1V8_3 1.8
26 SWB0 CONN13 CH8 P1V8_DVDDIO_3 1.8
27 SWB0 CONN13 CH9 P0V75_DVDD_39 0.75
28 SWB0 CONN13 CH10 P0V75_AVDD_39 0.75
29 SWB0 CONN13 CH11 P1V5_AVDD_39 1.5
30 SWB0 CONN13 CH12 P0V9_AVDD_39 0.9
31 SWB0 CONN13 CH13 PVDD1V5_2 1.5
32 SWB0 CONN13 CH14 PVDD1V8_DUT 1.8
33 SWB0 CONN13 CH15 PVDD_MDIO 1.2
34 SWB0 CONN14 CH0 P0V75_AVDD_7 0.75
35 SWB0 CONN14 CH1 P1V5_AVDD_7 1.5
36 SWB0 CONN14 CH2 P0V9_AVDD_7 0.9
37 SWB0 CONN14 CH3 P1V8_5 1.8
38 SWB0 CONN14 CH4 P1V8_2 1.8
39 SWB0 CONN14 CH5 PVDD3_3V_SYNTH_LDO_B2 3.3
40 SWB0 CONN14 CH6 PVDD3_3V_SYNTH_LDO_A2 3.3
41 SWB0 CONN14 CH7 PVDD3_3V_SYNTH_LDO_A1 3.3
42 SWB0 CONN14 CH8 PVDD1V5_TSC 1.5
43 SWB0 CONN14 CH9 PVDD1V5_ANLG 1.5
44 SWB0 CONN14 CH10 NC -
45 SWB0 CONN14 CH11 NC -
46 SWB0 CONN14 CH12 PVDD1V5_1 1.5
47 SWB0 CONN14 CH13 NC -
48 SWB0 CONN14 CH14 P1V8_DVDDIO_1 1.8
49 SWB0 CONN14 CH15 P0V75_DVDD_7 0.75
50 SWB0 CONN15 CH0 P1V5_AVDD_55 1.5
51 SWB0 CONN15 CH1 P0V9_AVDD_55 0.9
52 SWB0 CONN15 CH2 P0V75_DVDD_63 0.75
53 SWB0 CONN15 CH3 P0V75_AVDD_63 0.75
54 SWB0 CONN15 CH4 P1V5_AVDD_63 1.5
55 SWB0 CONN15 CH5 P0V9_AVDD_63 0.9
56 SWB0 CONN15 CH6 PVDD3_3V_SYNTH_LDO_B3 3.3
57 SWB0 CONN15 CH7 PVDD1V5_3 1.5
58 SWB0 CONN15 CH8 P0V85_STBY 0.85
59 SWB0 CONN15 CH9 P1V8_STBY 1.8
60 SWB0 CONN15 CH10 NC -
61 SWB0 CONN15 CH11 P3V3_STBY 3.3
62 SWB0 CONN15 CH12 PVDD0V9_2 0.9
63 SWB0 CONN15 CH13 P0V75_DVDD_55 0.75
64 SWB0 CONN15 CH14 P0V75_AVDD_55 0.75
65 SWB0 CONN15 CH15 P1V8_DVDDIO_4 1.8
66 SWB0 CONN16 CH0 PVDD1V5_0 1.5
67 SWB0 CONN16 CH1 P1V8_4 1.8
68 SWB0 CONN16 CH2 PVDD3_3V_SYNTH_LDO_B5 3.3
69 SWB0 CONN16 CH3 PVDD3_3V_SYNTH_LDO_A5 3.3
70 SWB0 CONN16 CH4 PVDD3_3V_SYNTH_LDO_B4 3.3
71 SWB0 CONN16 CH5 PVDD3_3V_SYNTH_LDO_A4 3.3
72 SWB0 CONN16 CH6 P3V3 3.3
73 SWB0 CONN16 CH7 PVDD0_8V_T3 0.8
74 SWB0 CONN16 CH8 P0V75_DVDD_1 0.75
75 SWB0 CONN16 CH9 P0V75_AVDD_1 0.75
76 SWB0 CONN16 CH10 P1V5_AVDD_1 1.5
77 SWB0 CONN16 CH11 P0V9_AVDD_1 0.9
78 SWB0 CONN16 CH12 PVDD0V9_0 0.9
79 SWB0 CONN16 CH13 PVDD0V9_1 0.9
80 SWB0 CONN16 CH14 PVDD0V9_3 0.9
81 SWB0 CONN16 CH15 PVDD1V2_T3 1.2
82 SWB1 CONN13 CH0 P0V75_DVDD_47 0.75
83 SWB1 CONN13 CH1 P0V75_AVDD_47 0.75
84 SWB1 CONN13 CH2 P1V5_AVDD_47 1.5
85 SWB1 CONN13 CH3 P0V9_AVDD_47 0.9
86 SWB1 CONN13 CH4 P1V8_1 1.8
87 SWB1 CONN13 CH5 PVDD3_3V_SYNTH_LDO_A3 3.3
88 SWB1 CONN13 CH6 PVDD3_3V_SYNTH_LDO_B1 3.3
89 SWB1 CONN13 CH7 P1V8_3 1.8
90 SWB1 CONN13 CH8 P1V8_DVDDIO_3 1.8
91 SWB1 CONN13 CH9 P0V75_DVDD_39 0.75
92 SWB1 CONN13 CH10 P0V75_AVDD_39 0.75
93 SWB1 CONN13 CH11 P1V5_AVDD_39 1.5
94 SWB1 CONN13 CH12 P0V9_AVDD_39 0.9
95 SWB1 CONN13 CH13 PVDD1V5_2 1.5
96 SWB1 CONN13 CH14 PVDD1V8_DUT 1.8
97 SWB1 CONN13 CH15 PVDD_MDIO 1.2
98 SWB1 CONN14 CH0 P0V75_AVDD_7 0.75
99 SWB1 CONN14 CH1 P1V5_AVDD_7 1.5
100 SWB1 CONN14 CH2 P0V9_AVDD_7 0.9
101 SWB1 CONN14 CH3 P1V8_5 1.8
102 SWB1 CONN14 CH4 P1V8_2 1.8
103 SWB1 CONN14 CH5 PVDD3_3V_SYNTH_LDO_B2 3.3
104 SWB1 CONN14 CH6 PVDD3_3V_SYNTH_LDO_A2 3.3
105 SWB1 CONN14 CH7 PVDD3_3V_SYNTH_LDO_A1 3.3
106 SWB1 CONN14 CH8 PVDD1V5_TSC 1.5
107 SWB1 CONN14 CH9 PVDD1V5_ANLG 1.5
108 SWB1 CONN14 CH10 NC -
109 SWB1 CONN14 CH11 NC -
110 SWB1 CONN14 CH12 PVDD1V5_1 1.5
111 SWB1 CONN14 CH13 NC -
112 SWB1 CONN14 CH14 P1V8_DVDDIO_1 1.8
113 SWB1 CONN14 CH15 P0V75_DVDD_7 0.75
114 SWB1 CONN15 CH0 P1V5_AVDD_55 1.5
115 SWB1 CONN15 CH1 P0V9_AVDD_55 0.9
116 SWB1 CONN15 CH2 P0V75_DVDD_63 0.75
117 SWB1 CONN15 CH3 P0V75_AVDD_63 0.75
118 SWB1 CONN15 CH4 P1V5_AVDD_63 1.5
119 SWB1 CONN15 CH5 P0V9_AVDD_63 0.9
120 SWB1 CONN15 CH6 PVDD3_3V_SYNTH_LDO_B3 3.3
121 SWB1 CONN15 CH7 PVDD1V5_3 1.5
122 SWB1 CONN15 CH8 P0V85_STBY 0.85
123 SWB1 CONN15 CH9 P1V8_STBY 1.8
124 SWB1 CONN15 CH10 NC -
125 SWB1 CONN15 CH11 P3V3_STBY 3.3
126 SWB1 CONN15 CH12 PVDD0V9_2 0.9
127 SWB1 CONN15 CH13 P0V75_DVDD_55 0.75
128 SWB1 CONN15 CH14 P0V75_AVDD_55 0.75
129 SWB1 CONN15 CH15 P1V8_DVDDIO_4 1.8
130 SWB1 CONN16 CH0 PVDD1V5_0 1.5
131 SWB1 CONN16 CH1 P1V8_4 1.8
132 SWB1 CONN16 CH2 PVDD3_3V_SYNTH_LDO_B5 3.3
133 SWB1 CONN16 CH3 PVDD3_3V_SYNTH_LDO_A5 3.3
134 SWB1 CONN16 CH4 PVDD3_3V_SYNTH_LDO_B4 3.3
135 SWB1 CONN16 CH5 PVDD3_3V_SYNTH_LDO_A4 3.3
136 SWB1 CONN16 CH6 P3V3 3.3
137 SWB1 CONN16 CH7 PVDD0_8V_T3 0.8
138 SWB1 CONN16 CH8 P0V75_DVDD_1 0.75
139 SWB1 CONN16 CH9 P0V75_AVDD_1 0.75
140 SWB1 CONN16 CH10 P1V5_AVDD_1 1.5
141 SWB1 CONN16 CH11 P0V9_AVDD_1 0.9
142 SWB1 CONN16 CH12 PVDD0V9_0 0.9
143 SWB1 CONN16 CH13 PVDD0V9_1 0.9
144 SWB1 CONN16 CH14 PVDD0V9_3 0.9
145 SWB1 CONN16 CH15 PVDD1V2_T3 1.2
+56
View File
@@ -0,0 +1,56 @@
# v1.0.10 — The soak loop stops spinning flat out, and shows you what is still running
## ✨ New features
**Every soak round now reports the background jobs**
* Script B's monitoring loop prints `bgctl list` (platform jobs) and `jobs` (the login shell's own background jobs) on every pass, right after the platform data. A stress job that dies three hours into an overnight soak now shows up as a shorter list on the next round, instead of only being noticed at the end — or not at all.
* The two lists answer different questions and are both worth having: `bgctl` knows about the platform's job runner, `jobs` knows about anything the macro backgrounded in that shell.
**A single table for all 144 margin channels**
* `docs/LTC2980_channel_map.csv` flattens the nine `settings/*.conf` files into one sheet — `Board,CONN,Ch,NetName,Vnom` — so "which board and channel is `PVDD1V5_TSC` on?" is one search instead of opening nine files.
* `tools/gen_channel_map.sh` regenerates it. **The CSV is output, not source**: edit the `.conf` files and re-run the script, never the other way round.
* Unused channels stay in the table as `NC` / `-` rather than being dropped, because the channel number is also the PMBus page (`ch / 8` picks the chip, `ch % 8` picks the page) — filtering the gaps out would shift everything after them.
**The bench command lists that these scripts came from**
* `tools/100G_PRBS.txt`, `tools/TR518.txt` and `tools/fan_ctrl.txt` join the existing `traffic_loopback_*.txt` set: the raw `bcmcmd` / sysfs sequences, kept in the form you can paste into a console when a script is misbehaving and you want to drive the hardware directly.
## 🐛 Bug fixes
**The soak loop ran as fast as the DUT could answer**
* `while 1` had no pause at all, so each round started the moment the previous one finished. Over a long soak that means constant console traffic and a log full of near-identical samples taken seconds apart. The loop now ends each round with `pause 60`.
* Note the gap this leaves: the Status sheet asks for a 10-minute sampling interval, and 60 seconds is still hard-coded rather than configurable. This makes the loop sane, it does not finish the job.
**A comment promised a cadence the code never had**
* The loop was labelled `Get data every 10mins` while running with no delay whatsoever. It now says what it does.
## 📦 Downloads
| File | Contents |
|---|---|
| `Script_ABC_Blanton_V1.0.10.zip` | The full Tera Term working directory, including `Blanton_Script/` to copy onto the DUT |
**After copying to the DUT:**
```bash
chmod +x ~/Blanton_Script/*.sh # see below
grep -rlU $'\r' ~/Blanton_Script # expect no output
```
`mgmt_ping_monitor.sh` and `port_prbs_monitor.sh` launch their workers through `bash`, so those two run without the execute bit. `bmc_monitor.sh` and `usb_target.sh` still need it, and neither git (mode 100644) nor a Windows/USB copy carries it. Without the `chmod`, `start` fails with `Permission denied` and the later `cat` finds nothing: **the section ends up empty and nothing reports an error.**
## ⚠️ Before you run
* **Declare the switch population.** `SWB_UNIT0` / `SWB_UNIT1` in `config.ttl` say which switch units this DUT has; traffic and the readiness gate both follow them.
* **Set `FAN_SPEED`.** It is applied at the start of every run.
* **Management connectivity is in use, not preserved.** Both management NICs are up and pinging — drive the run from the serial console.
* **PRBS is still not wired into the macros.** Run `port_prbs_monitor.sh` by hand if you want it; the calls in A/B/C remain commented out, unchanged from v1.0.9.
* **TR518 and the loopback traffic test cannot both run.** `tools/TR518.txt` sets `port all lb=mac` and `l2 learn off` across the whole unit, which is the same hardware `blanton_traffic_linespeed` is using. Pick one.
* **Two settings persist to `config_db.json`** and survive a reboot: LLDP is left disabled and the 100G uplinks are left configured. Restore them before the DUT moves on.
## 🔗 Links
* [ARCHITECTURE.md](https://etgit.et-wen.com/etwen/Blanton_TTL_Script/src/branch/main/ARCHITECTURE.md) — test flow, data models, constraints
* [CLAUDE.md](https://etgit.et-wen.com/etwen/Blanton_TTL_Script/src/branch/main/CLAUDE.md) — commands and the bench gotchas
* [LTC2980_channel_map.csv](https://etgit.et-wen.com/etwen/Blanton_TTL_Script/src/branch/main/docs/LTC2980_channel_map.csv) — all 144 margin channels
**Full changelog:** [V1.0.9...V1.0.10](https://etgit.et-wen.com/etwen/Blanton_TTL_Script/compare/V1.0.9...V1.0.10)
+65
View File
@@ -0,0 +1,65 @@
# v1.0.7 — Management-link ping test, and the fans start where you set them
## ✨ New features
**10G / 1G management port ping test**
* New `mgmt_ping_monitor.sh` exercises both management NICs in one loop: it brings each up with iproute2, waits for the link to settle, then pings that NIC's own target — `eth0``192.168.1.30`, `eth1``192.168.1.31`.
* Runs detached like the other monitors: `start` / `stop` / `status` / `fg` / `summary`. `start` clears the previous log, so one run means one log.
* Every leg writes a single greppable line — `RESULT 10G eth0 -> 192.168.1.30 tx=10 rx=10 loss=0% rtt_avg=1.743ms nic_tx=10 nic_rx=10 PASS` — so a whole soak can be read with `summary` instead of scrolling.
* `nic_tx` / `nic_rx` are that interface's own counter delta across the burst. If a ping passes but they stay near zero, the traffic left on the *other* NIC and the number does not belong to this one.
* Script A runs one round as a baseline; Script B leaves it running through the soak.
**Fan speed is set at the start of every run**
* `FAN_SPEED` in `config.ttl` (default 30) is applied by Script A, so a run no longer inherits whatever the previous test left the fans doing.
* Both MAX31790 controllers are rebound first, which clears a controller left in an odd state.
**Stress load moved onto the platform's own job runner**
* CPU, DDR, SSD and USB stress now go through `bgctl`, replacing the `~/hammer/tools/stress_*.py` scripts. `bgctl list` shows what is running; Script C stops everything with `bgctl stop --all`.
* BMC DDR stress runs through `bmc-manager`, and a new BMC USB test discovers the `cdc_ncm` interface and pings across it for four hours under `systemd-run`.
**More of the DUT on record**
* Script A now captures the boot image, `show version`, firmware status, system EEPROM, SSD health, TPM version and NVMe SMART data before the test starts.
* PCIe AER now sits next to `ras-mc-ctl --summary`, and Script C closes with NVMe health so a disk that degraded during the soak is visible.
* `dmesg` capture uses human-readable timestamps and one combined error pattern, then clears the ring buffer — so Script C's dmesg shows only what the soak produced, not everything since boot.
* Each run opens with `hw-test-session start` and closes with `finish`; Script C copies the job logs to `/mnt/usb/jobs-<date>_<time>` so they leave the DUT with the run they belong to.
## 🐛 Bug fixes
**Script A no longer hangs waiting for ports that are already up**
* The readiness gate required *more than* 216 ports up, but a fully cabled unit reports exactly 216 — 108 loopback pairs times two. Script A sat in the poll loop forever on the very state it was waiting for. It now proceeds at 216.
* The comments in `wait_init.ttl` still described the old "more than" rule; they now match the code, so the off-by-one cannot be reintroduced by reading the wrong line.
**The macro no longer runs ahead of the DUT during the soak**
* Script B set a 15-second wait cap for the `cdc_ncm` probe and never restored it. In Tera Term that cap is global, so it stayed in force for everything after — including the traffic setup, which walks 108 VLANs per switch unit and takes far longer. A timed-out wait returns without the prompt, leaving every later command issued a step early.
**Fan controller is no longer bound twice**
* One of the two MAX31790 controllers received a second `bind` immediately after the first, which could only fail because the driver was already attached.
## 📦 Downloads
| File | Contents |
|---|---|
| `Script_ABC_Blanton_V1.0.7.zip` | The full Tera Term working directory, including `Blanton_Script/` to copy onto the DUT |
**After copying to the DUT:**
```bash
chmod +x ~/Blanton_Script/*.sh # required -- see below
grep -rlU $'\r' ~/Blanton_Script # expect no output
```
The `chmod +x` is not optional. `mgmt_ping_monitor.sh` is executed rather than sourced and re-execs its own path to daemonise, and neither git (mode 100644) nor a Windows/USB copy carries the execute bit. Without it `start` fails with `Permission denied` and the later `cat` finds nothing — **the section ends up empty and nothing reports an error**.
## ⚠️ Before you run
* **Declare the switch population.** `SWB_UNIT0` / `SWB_UNIT1` in `config.ttl` say which switch units this DUT has. Traffic and the readiness gate both follow them: a single unit gets `-u 0` or `-u 1`, and with neither set the traffic stage is skipped instead of failing against hardware that is not there.
* **Both management NICs stay up.** If they share a subnet, keep `ARP_STRICT=1` — otherwise the target's ARP can be answered by either NIC and a reply may arrive on the one that did not send.
* **Management connectivity is exercised, not preserved.** Drive the run from the serial console.
* **Two settings persist to `config_db.json`** and survive a reboot: LLDP is left disabled, and the 100G uplinks are left configured. Restore them before the DUT moves on.
## 🔗 Links
* [ARCHITECTURE.md](https://etgit.et-wen.com/etwen/Blanton_TTL_Script/src/branch/main/ARCHITECTURE.md) — test flow, data models, constraints
* [CLAUDE.md](https://etgit.et-wen.com/etwen/Blanton_TTL_Script/src/branch/main/CLAUDE.md) — commands and the bench gotchas
**Full changelog:** [V1.0.5...V1.0.7](https://etgit.et-wen.com/etwen/Blanton_TTL_Script/compare/V1.0.5...V1.0.7)
+61
View File
@@ -0,0 +1,61 @@
# v1.0.8 — Both management links tested at once, and ARP stops looking like packet loss
## ✨ New features
**Both management NICs are pinged simultaneously and continuously**
* `mgmt_ping_monitor.sh` no longer alternates fixed bursts. `start` configures both NICs, then pings from each at the same time and keeps accumulating until stopped — so a soak-long run is one continuous measurement rather than a series of snapshots.
* `stop` renders a report into the log: the last 20 entries per NIC, each one's ping statistics and PASS/FAIL, then `ip -s link show` for both interfaces.
* Every line is timestamped, and a request that got no reply prints a marker instead of merely being absent — a drop is visible in the tail, not inferred from a gap in the sequence numbers.
* `status` shows both pids and how many replies each NIC has received so far, which is the quick way to see one side is dead without waiting for the report.
**The USB stress target is detected, not assumed**
* New `usb_target.sh` reports an inserted USB device's node, mount point or by-id path. Script B asks it for the node and builds the stress command from the answer, so a stick that enumerates as `sdb` no longer sends a write test at whatever `/dev/sda1` happens to be.
* It excludes whatever disk backs `/` and `/host` — this platform can boot from a USB DOM, which also reports as USB — refuses to guess when several USB disks are present, and when it mounts, verifies the result is actually writable rather than trusting that `mount` succeeded.
**BMC I2C integrity is exercised through the soak**
* `bmc_monitor.sh` is back in the run and now writes two complementary patterns to a BMC scratch register and reads each back. One pattern alone cannot catch a bit stuck the same way it was written; repeating the pair through the soak turns an intermittent I2C fault into something the log records rather than something the tester has to witness.
**Test artefacts leave the DUT with the run**
* Script C copies `bmc_poll.log` and `mgmt_ping.log` into the job directory before it is archived to USB, so the monitors' output travels with the `bgctl` job logs.
* DDR stress now runs continuously instead of stopping after 100 passes, matching the other soak loads.
## 🐛 Bug fixes
**Three FAILs that were not link faults**
* A bench run reported `FAIL=3` with zero NIC errors, zero drops and sub-millisecond replies. Every failure was missing exactly `icmp_seq=1` and nothing else: once the neighbour entry for the target expires, the first echo request is spent resolving ARP and `ping` counts it as loss. A discarded warm-up ping per NIC now absorbs that, which is what lets the loss threshold stay at zero and still mean something. Raising the threshold instead would have hidden genuine single-packet loss.
**Job logs were being cleared before they were collected**
* Script C ran `bgctl reset --yes` while killing processes — before the job directory is copied to USB. Moved to after the archive, so a run's own logs are collected before anything clears them.
**The fan controller was bound twice**
* One of the two MAX31790 controllers received a second `bind` immediately after the first, which could only fail because the driver was already attached.
## 📦 Downloads
| File | Contents |
|---|---|
| `Script_ABC_Blanton_V1.0.8.zip` | The full Tera Term working directory, including `Blanton_Script/` to copy onto the DUT |
**After copying to the DUT:**
```bash
chmod +x ~/Blanton_Script/*.sh # see below
grep -rlU $'\r' ~/Blanton_Script # expect no output
```
`mgmt_ping_monitor.sh` no longer re-execs itself, so `bash mgmt_ping_monitor.sh start` works without the execute bit. `bmc_monitor.sh` and `usb_target.sh` still need it — and neither git (mode 100644) nor a Windows/USB copy carries it. Without the `chmod`, `start` fails with `Permission denied` and the later `cat` finds nothing: **the section ends up empty and nothing reports an error.**
## ⚠️ Before you run
* **Declare the switch population.** `SWB_UNIT0` / `SWB_UNIT1` in `config.ttl` say which switch units this DUT has. Traffic and the readiness gate both follow them; with neither set the traffic stage is skipped rather than failing against hardware that is not there.
* **Set the fan speed you want.** `FAN_SPEED` in `config.ttl` is applied at the start of every run.
* **Management connectivity is in use, not preserved.** Both NICs are up and pinging — drive the run from the serial console.
* **If the two management NICs share a subnet, keep `ARP_STRICT=1`.** Otherwise the target's ARP can be answered by either NIC and a reply may arrive on the one that did not send. Check `nic_tx` / `nic_rx` on the RESULT line: they are that interface's own counter delta.
* **Two settings persist to `config_db.json`** and survive a reboot: LLDP is left disabled and the 100G uplinks are left configured. Restore them before the DUT moves on.
## 🔗 Links
* [ARCHITECTURE.md](https://etgit.et-wen.com/etwen/Blanton_TTL_Script/src/branch/main/ARCHITECTURE.md) — test flow, data models, constraints
* [CLAUDE.md](https://etgit.et-wen.com/etwen/Blanton_TTL_Script/src/branch/main/CLAUDE.md) — commands and the bench gotchas
**Full changelog:** [V1.0.7...V1.0.8](https://etgit.et-wen.com/etwen/Blanton_TTL_Script/compare/V1.0.7...V1.0.8)
+63
View File
@@ -0,0 +1,63 @@
# v1.0.9 — 100G PRBS testing, and the job logs actually reach the USB stick
## ✨ New features
**100G uplink PRBS test**
* New `port_prbs_monitor.sh` drives a PRBS test on both 100G uplinks: it clears low-power mode, arms the pattern, then polls `phy diag <phy> prbs get` in the background until stopped. A poll counts as PASS only when the output says `PRBS OK!`; `prbsstat Ber` is captured alongside every poll for the record.
* `stop` runs `prbsstat STOp` and `prbs clear` on each port before rendering the report, so the test does not leave PRBS armed behind it.
* `report` gives the tally the bench actually wants:
```
PORT POLLS PASS FAIL RESULT
Ethernet513 42 42 0 PASS
Ethernet514 42 40 2 FAIL
```
* **One port at a time is a first-class mode.** `start a` or `start b` runs a single uplink, which is how you separate a genuine port fault from the DUT not coping with two PRBS streams at once. A port that was not run reports `SKIP`, not `FAIL`.
* Every command and its full output — setup, each poll, teardown — goes to that port's raw log.
> ⚠️ The script ships in this build and can be run by hand, but **the calls in Script A, B and C are commented out**: PRBS is still under bring-up on this platform.
**USB target detection**
* `usb_target.sh` reports an inserted USB device's node, mount point or by-id path. It excludes whatever disk backs `/` and `/host` — this platform can boot from a USB DOM, which also reports as USB — refuses to guess when several USB disks are present, and when mounting, verifies the result is writable rather than trusting that `mount` succeeded.
* Script B builds the USB stress command from the detected node instead of a hard-coded `/dev/sda1`.
## 🐛 Bug fixes
**The job logs were being written to a path nobody collected**
* Stress output, the monitors' logs and the USB archive disagreed about where the job directory lives. Everything now uses `/host/hw-eval/jobs/`, so what Script B writes is what Script C copies out.
**Script C copied to a USB that was never mounted**
* The archive step assumed `/mnt/usb` was ready. Script C now finds the device and mounts it first, so a run's logs leave the DUT instead of being written into an empty mount point.
**`show interfaces status` right after starting PRBS reads as a failure**
* With PRBS armed the link is out of normal operation and reports DOWN. `port_prbs_monitor.sh` deliberately does not print the port status at `start` — it appears in the report instead, after `prbs clear`, labelled as the recovered state.
## 📦 Downloads
| File | Contents |
|---|---|
| `Script_ABC_Blanton_V1.0.9.zip` | The full Tera Term working directory, including `Blanton_Script/` to copy onto the DUT |
**After copying to the DUT:**
```bash
chmod +x ~/Blanton_Script/*.sh # see below
grep -rlU $'\r' ~/Blanton_Script # expect no output
```
`mgmt_ping_monitor.sh` and `port_prbs_monitor.sh` launch their workers through `bash`, so those two run without the execute bit. `bmc_monitor.sh` and `usb_target.sh` still need it, and neither git (mode 100644) nor a Windows/USB copy carries it. Without the `chmod`, `start` fails with `Permission denied` and the later `cat` finds nothing: **the section ends up empty and nothing reports an error.**
## ⚠️ Before you run
* **Declare the switch population.** `SWB_UNIT0` / `SWB_UNIT1` in `config.ttl` say which switch units this DUT has; traffic and the readiness gate both follow them.
* **Set `FAN_SPEED`.** It is applied at the start of every run.
* **Management connectivity is in use, not preserved.** Both management NICs are up and pinging — drive the run from the serial console.
* **PRBS is not wired into the macros yet.** Run `port_prbs_monitor.sh` by hand if you want it; uncommenting the calls in A/B/C is not supported in this build.
* **Two settings persist to `config_db.json`** and survive a reboot: LLDP is left disabled and the 100G uplinks are left configured. Restore them before the DUT moves on.
## 🔗 Links
* [ARCHITECTURE.md](https://etgit.et-wen.com/etwen/Blanton_TTL_Script/src/branch/main/ARCHITECTURE.md) — test flow, data models, constraints
* [CLAUDE.md](https://etgit.et-wen.com/etwen/Blanton_TTL_Script/src/branch/main/CLAUDE.md) — commands and the bench gotchas
**Full changelog:** [V1.0.8...V1.0.9](https://etgit.et-wen.com/etwen/Blanton_TTL_Script/compare/V1.0.8...V1.0.9)
+108
View File
@@ -0,0 +1,108 @@
#!/usr/bin/env bash
# =============================================================================
# publish.sh -- package src/Script_ABC_Blanton for delivery
# =============================================================================
# Version History:
# V1.0.0 2026-08-21 Initial. Copies src/Script_ABC_Blanton to
# publish/Script_ABC_Blanton_<Version>, taking <Version>
# from the "; Version : Vx.y.z" header of
# 1_Blanton_Script_A.ttl.
# =============================================================================
#
# The version is READ FROM Script A rather than passed in, so the folder name
# can never disagree with the header the tester sees when they open the macro.
# Bump Script A's header first, then run this.
#
# Usage:
# ./publish/publish.sh # publish at Script A's version
# ./publish/publish.sh -v V1.0.4 # override the version
# ./publish/publish.sh -f # overwrite an existing output folder
# ./publish/publish.sh -z # also produce a .zip next to it
# ./publish/publish.sh -n # dry run: print what would happen
# =============================================================================
set -u
SCRIPT_DIR="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd)"
REPO_ROOT="$(cd -- "${SCRIPT_DIR}/.." && pwd)"
SRC_DIR="${REPO_ROOT}/src/Script_ABC_Blanton"
OUT_ROOT="${SCRIPT_DIR}"
NAME="Script_ABC_Blanton"
VER=""; FORCE=0; ZIP=0; DRY=0
info() { printf '[\033[34mINFO\033[0m] %s\n' "$*"; }
ok() { printf '[\033[32m OK \033[0m] %s\n' "$*"; }
die() { printf '[\033[31mERR \033[0m] %s\n' "$*" >&2; exit 1; }
usage() { sed -n '2,26p' "${BASH_SOURCE[0]}" | sed 's/^# \{0,1\}//'; exit 0; }
while [ $# -gt 0 ]; do
case "$1" in
-v|--version) VER="${2:-}"; shift 2 ;;
-f|--force) FORCE=1; shift ;;
-z|--zip) ZIP=1; shift ;;
-n|--dry-run) DRY=1; shift ;;
-h|--help) usage ;;
*) die "unknown option '$1' (try -h)" ;;
esac
done
[ -d "${SRC_DIR}" ] || die "source not found: ${SRC_DIR}"
# --- version from Script A's header ------------------------------------------
# The .ttl files are CRLF, so strip the carriage return or it lands in the
# folder name and produces a directory no one can cd into.
if [ -z "${VER}" ]; then
A_TTL="${SRC_DIR}/1_Blanton_Script_A.ttl"
[ -f "${A_TTL}" ] || die "cannot read version: ${A_TTL} missing"
VER=$(grep -m1 -E '^;[[:space:]]*Version[[:space:]]*:' "${A_TTL}" \
| tr -d '\r' | sed -E 's/.*:[[:space:]]*//; s/[[:space:]]+$//')
[ -n "${VER}" ] || die "no '; Version : Vx.y.z' header in $(basename "${A_TTL}")"
info "version from $(basename "${A_TTL}") : ${VER}"
else
info "version (overridden) : ${VER}"
fi
case "${VER}" in
V[0-9]*) ;;
*) die "version '${VER}' does not look like Vx.y.z" ;;
esac
DEST="${OUT_ROOT}/${NAME}_${VER}"
info "source : ${SRC_DIR#${REPO_ROOT}/}"
info "destination : ${DEST#${REPO_ROOT}/}"
if [ -e "${DEST}" ]; then
[ "${FORCE}" = "1" ] || die "${DEST##*/} already exists (use -f to overwrite)"
[ "${DRY}" = "1" ] || rm -rf -- "${DEST}"
info "existing folder removed (-f)"
fi
if [ "${DRY}" = "1" ]; then
info "dry run: nothing copied"
exit 0
fi
# --- copy ---------------------------------------------------------------------
# Captured console logs are per-run and often carry DUT serials, so they are
# never shipped; the empty logs/ directory itself is kept because Script A's
# logopen writes into it.
mkdir -p -- "${DEST}" || die "cannot create ${DEST}"
if command -v rsync >/dev/null 2>&1; then
rsync -a --exclude='*.log' "${SRC_DIR}/" "${DEST}/" || die "rsync failed"
else
cp -a -- "${SRC_DIR}/." "${DEST}/" || die "cp failed"
find "${DEST}" -type f -name '*.log' -delete
fi
# Tera Term's logopen target, in case the source tree never had one
mkdir -p -- "${DEST}/logs"
n_files=$(find "${DEST}" -type f | wc -l)
ok "${n_files} files -> ${DEST##*/}"
if [ "${ZIP}" = "1" ]; then
command -v zip >/dev/null 2>&1 || die "zip not installed"
( cd -- "${OUT_ROOT}" && rm -f -- "${NAME}_${VER}.zip" \
&& zip -qr "${NAME}_${VER}.zip" "${NAME}_${VER}" ) || die "zip failed"
ok "$(du -h "${OUT_ROOT}/${NAME}_${VER}.zip" | cut -f1) -> ${NAME}_${VER}.zip"
fi
+258 -11
View File
@@ -1,7 +1,7 @@
; =============================================================================
; Script A for Blanton
; Version : V1.0.2
; Date : 2026-08-17
; Version : V1.0.10
; Date : 2026-08-27
; Author : ETWen
; =============================================================================
; Version History:
@@ -14,7 +14,66 @@
; ScriptB Remove stress_hhmd, stress_pcie
; Blanton_Script Add blanton_traffic_linespeed.sh
; ScriptB/C Add Traffic Test
;
; V1.0.3 2026-08-21 utils/wait_init.ttl Created
; ScriptA Add Wait DUT Ready
; V1.0.4 2026-08-21 utils/wait_init.ttl Wait on bcmcmd port-up, count (both units > 216)
; utils/wait_init.ttl Add WT_UNIT0/WT_UNIT1, skip absent units
; ScriptA Add EEPROM info (hpe-eeprom-tlv --bus 4 --addr 0x50)
; ScriptA Remove empty Check History block
; ScriptA/B Disable lldp + config save before traffic (silent MAC)
; ScriptB Add sfputil lpmode off Ethernet513/514
; V1.0.5 2026-08-21 config.ttl Add SWB_UNIT0/SWB_UNIT1
; utils/wait_init.ttl Follow config.ttl SWB_UNIT0/SWB_UNIT1
; ScriptA/B/C Traffic follows SWB_UNIT: -u 0 / -u 1 / skip
; V1.0.6 2026-08-24 utils/show_dmesg.ttl Edited dmesg grep i2c & clear event
; ScriptA HW Test Session
; ScriptC HW Test Session Finish
; ScriptA bmc-first-enroll, bmc version/status
; ScriptB Stress test DDR, SSD, USB, BMC DDR, BMC USB Stress test
; ScriptC BMC USB Stress test result
; ScriptA DUT Info
; ScriptAC 10G/1G MGMT ping
; ScriptA Edited 100G Port Status
; ScriptA, utils/setup_pmon.ttl Add TPM
; ScriptAC NVME Error Info
; ScriptAC Add PCIe Summary
; V1.0.7 2026-08-25 config.ttl Add FAN_SPEED
; ScriptA Add Fan Ctrl (max31790 rebind + fan-speed-control.sh)
; ScriptA Clear /host/hw-eval/current/jobs before the run
; ScriptC Copy stress jobs log to USB, timestamped
; utils/wait_init.ttl WT_INTERVAL 10 -> 60, fix WT_MIN comments
; ScriptC Disable BMC USB journalctl dump
; V1.0.8 2026-08-25 Blanton_Script/bmc_monitor.sh Add BMC I2C write/read-back pattern test
; ScriptB Re-enable bmc_monitor.sh start
; ScriptC Re-enable bmc_monitor.sh stop + cat bmc_poll.log
; ScriptC Copy bmc_poll.log + mgmt_ping.log into jobs/ before the USB archive
; Blanton_Script/usb_target.sh Created - auto-detect USB device node / mount point
; ScriptB USB stress uses the detected node, not a hard-coded /dev/sda1
; ScriptB DDR memtester runs continuously (drop the 100-pass limit)
; ScriptC Move bgctl reset --yes after the USB archive
; Blanton_Script/mgmt_ping_monitor.sh V3.0.0 - both NICs ping simultaneously, accumulating
; + ARP warm-up so a stale neighbour entry is not counted as loss
; + stop renders last-N per NIC, stats, and both ip -s link show
; .gitignore Ignore monitor *.raw / *.pid / *.state
; ScriptA MGMT ping baseline 45s -> 30s, clear the log afterwards
; ScriptB Show mgmt_ping status after start
; ScriptC Show mgmt_ping status before stop, cat the report after
; V1.0.9 2026-08-26 Blanton_Script/port_prbs_monitor.sh Add 100G Port PRBS test
; start [both|a|b] - single port isolates a port fault from
; the DUT not coping with two PRBS streams at once
; !! TTL calls are commented out in A/B/C - still under bring-up
; ScriptA/B/C Job log path /host/hw-eval/current/jobs -> /host/hw-eval/jobs
; ScriptC Mount the USB (usb_target.sh) before copying the job logs
; ScriptC Reorder results: stress logs, then MGMT ping, then 100G status
; V1.0.10 2026-08-27 ScriptB Monitor loop lists the background jobs every round
; + bgctl list - platform jobs
; + jobs - shell jobs of the login shell
; + pause 60 - the loop used to run with no pause at all
; docs/LTC2980_channel_map.csv Channel / net / Vnom map built from settings/*.conf
; tools/gen_channel_map.sh Regenerates that CSV - the CSV is output, not source
; tools/100G_PRBS.txt Bench command references, kept for hand-run debug
; tools/TR518.txt TR518 = built-in packet test (swutil / bcmcmd tr 518)
; tools/fan_ctrl.txt MAX31790 rebind + fan-speed-control.sh
; =============================================================================
include "config.ttl"
@@ -49,6 +108,76 @@ gettime timestr
sprintf2 cmd 'sudo date -s "%s %s"' datestr timestr
sendln cmd
;Stress log clear
wait prompt_sonic_root
sendln "rm /host/hw-eval/jobs/*"
; ========== HW Test Session ==========
wait prompt_sonic_root
sendln "hw-test-session start"
wait prompt_sonic_root
sendln "hw-test-session log"
wait prompt_sonic_root
sendln "hw-test-session status"
; ========== FAN SPEED ==========
wait prompt_sonic_root
sendln "echo 18-0020 > /sys/bus/i2c/drivers/max31790/unbind"
wait prompt_sonic_root
sendln "sleep .5"
wait prompt_sonic_root
sendln "echo 18-0020 > /sys/bus/i2c/drivers/max31790/bind"
wait prompt_sonic_root
sendln "echo 25-0020 > /sys/bus/i2c/drivers/max31790/unbind"
wait prompt_sonic_root
sendln "sleep .5"
wait prompt_sonic_root
sendln "echo 25-0020 > /sys/bus/i2c/drivers/max31790/bind"
wait prompt_sonic_root
sprintf2 fan_ctrl_cmd 'fan-speed-control.sh %d' FAN_SPEED
sendln fan_ctrl_cmd
wait "Set all configured fan channels to"
sendln "y"
; ========== Wait DUT Init ==========
include "utils/wait_init.ttl"
; ========== DUT Info ==========
wait prompt_sonic_root
sendln "show boot"
wait prompt_sonic_root
sendln "show version"
wait prompt_sonic_root
sendln "fwutil show status"
wait prompt_sonic_root
sendln "show system-memory"
wait prompt_sonic_root
sendln "show service"
wait prompt_sonic_root
sendln "show platform syseeprom"
wait prompt_sonic_root
sendln "show platform ssdhealth"
wait prompt_sonic_root
sendln "qfx5252-tpm-version"
wait prompt_sonic_root
sendln "/usr/sbin/nvme smart-log -H /dev/nvme0"
wait prompt_sonic_root
sendln "/usr/sbin/smartctl -x /dev/nvme0"
; ========== BMC version ==========
;wait prompt_sonic_root
;sendln "bmc-manager run 'cat /etc/issue'"
wait prompt_sonic_root
sendln "bmc-first-enroll"
wait prompt_sonic_root
sendln "bmc-manager version"
wait prompt_sonic_root
sendln "bmc-manager status"
; ========== Load Shell Script ==========
wait prompt_sonic_root
sendln "source ~/Blanton_Script/blanton_fpga_pcimem.sh"
@@ -72,14 +201,13 @@ include "utils/show_pcie_error_reg_DDR.ttl"
include "utils/show_pcie_error_reg_I210.ttl"
include "utils/show_pcie_error_reg_SSD.ttl"
wait prompt_sonic_root
sendln "/usr/sbin/ras-mc-ctl --summary"
; ========== Check PCIE tree + PCIE link status + GET PCIE bandwidth Test ==========
include "utils/pcie_bus.ttl"
; ========== Check History ==========
;wait prompt_sonic_root
;sendln "command"
; ========== Check Margin ==========
if EN_Margin = 1 then
include "utils/show_margin_status.ttl"
@@ -88,16 +216,135 @@ endif
; ========== TAKE DATA ==========
include "utils/setup_pmon.ttl"
include "utils/show_dmesg.ttl"
wait prompt_sonic_root
;include "utils/xxx.ttl"
;wait prompt_sonic_root
;flushrecv
; ========== TRAFFIC-SETUP-STAGE ==========
;sendln "command"
; ========== 10G/1G MGMT Ping Test ==========
wait prompt_sonic_root
sendln "./Blanton_Script/mgmt_ping_monitor.sh start"
wait prompt_sonic_root
pause 30
sendln "~/Blanton_Script/mgmt_ping_monitor.sh stop"
wait prompt_sonic_root
sendln "cat ~/Blanton_Script/log/mgmt_ping.log"
wait prompt_sonic_root
sendln "~/Blanton_Script/mgmt_ping_monitor.sh clear"
; ========== 100G Port Status ==========
wait prompt_sonic_root
sendln "sudo sfputil lpmode off Ethernet513"
wait prompt_sonic_root
sendln "sudo sfputil lpmode off Ethernet514"
wait prompt_sonic_root
sendln "config interface -n asic0 startup Ethernet513"
wait prompt_sonic_root
sendln "config interface -n asic1 startup Ethernet514"
wait prompt_sonic_root
sendln "config save -y"
wait prompt_sonic_root
sendln "show interfaces status Ethernet513,Ethernet514"
;=============================================================================================================
; ========== 100G PRBS ==========
;wait prompt_sonic_root
;sendln "~/Blanton_Script/port_prbs_monitor.sh start"
;wait prompt_sonic_root
;sendln "sleep 10"
;wait prompt_sonic_root
;sendln "~/Blanton_Script/port_prbs_monitor.sh status"
;wait prompt_sonic_root
;sendln "~/Blanton_Script/port_prbs_monitor.sh stop"
;wait prompt_sonic_root
;sendln "~/Blanton_Script/port_prbs_monitor.sh report"
;wait prompt_sonic_root
;sendln "~/Blanton_Script/port_prbs_monitor.sh clear"
; ========== 100G Port Status ==========
;wait prompt_sonic_root
;sendln "sudo sfputil lpmode off Ethernet513"
;wait prompt_sonic_root
;sendln "sudo sfputil lpmode off Ethernet514"
;wait prompt_sonic_root
;sendln "config interface -n asic0 startup Ethernet513"
;wait prompt_sonic_root
;sendln "config interface -n asic1 startup Ethernet514"
;wait prompt_sonic_root
;sendln "config save -y"
;wait prompt_sonic_root
;sendln "show interfaces status Ethernet513,Ethernet514"
;=============================================================================================================
; ========== First Traffic Test ==========
; Silent MAC
wait prompt_sonic_root
sendln "config feature state lldp disabled"
wait prompt_sonic_root
sendln "config save -y"
if SWB_UNIT0 = 1 && SWB_UNIT1 = 1 then
wait prompt_sonic_root
sendln "blanton_traffic_linespeed ps"
wait prompt_sonic_root
sendln "blanton_traffic_linespeed init"
wait prompt_sonic_root
sendln "blanton_traffic_linespeed clear"
wait prompt_sonic_root
pause 15
sendln "blanton_traffic_linespeed show"
wait prompt_sonic_root
sendln "blanton_traffic_linespeed start"
wait prompt_sonic_root
pause 15
sendln "blanton_traffic_linespeed stop"
wait prompt_sonic_root
sendln "blanton_traffic_linespeed report"
elseif SWB_UNIT0 = 1 && SWB_UNIT1 = 0 then
wait prompt_sonic_root
sendln "blanton_traffic_linespeed ps -u 0"
wait prompt_sonic_root
sendln "blanton_traffic_linespeed init -u 0"
wait prompt_sonic_root
sendln "blanton_traffic_linespeed clear -u 0"
wait prompt_sonic_root
pause 15
sendln "blanton_traffic_linespeed show -u 0"
wait prompt_sonic_root
sendln "blanton_traffic_linespeed start -u 0"
wait prompt_sonic_root
pause 15
sendln "blanton_traffic_linespeed stop -u 0"
wait prompt_sonic_root
sendln "blanton_traffic_linespeed report -u 0"
elseif SWB_UNIT0 = 0 && SWB_UNIT1 = 1 then
wait prompt_sonic_root
sendln "blanton_traffic_linespeed ps -u 1"
wait prompt_sonic_root
sendln "blanton_traffic_linespeed init -u 1"
wait prompt_sonic_root
sendln "blanton_traffic_linespeed clear -u 1"
wait prompt_sonic_root
pause 15
sendln "blanton_traffic_linespeed show -u 1"
wait prompt_sonic_root
sendln "blanton_traffic_linespeed start -u 1"
wait prompt_sonic_root
pause 15
sendln "blanton_traffic_linespeed stop -u 1"
wait prompt_sonic_root
sendln "blanton_traffic_linespeed report -u 1"
else
endif
; ========== Show Power on uptime ==========
wait prompt_sonic_root
sendln "show uptime"
sendln ""
wait prompt_sonic_root
messagebox 'Start Script B' 'Script A DONE'
+124 -19
View File
@@ -20,52 +20,157 @@ sendln 'chmod +x ~/hammer/tools/amd/mlucas-avx2'
wait prompt_sonic_root
sendln "~/hammer/tools/amd/mlucas-avx2 -s l -cpu 0:15 > ~/hammer/tools/amd/mlucas_amm_log 2>&1 &"
; ========== BMC DDR Stress Test ==========
;wait prompt_sonic_root
;sendln "./Blanton_Script/bmc_monitor_ddr.sh start"
;wait prompt_sonic_root
;sendln "./Blanton_Script/bmc_monitor.sh start"
wait prompt_sonic_root
sendln "bgctl run bmc-manager run '/usr/bin/memtester 64M 1'"
; ========== BMC Monitor Test ==========
wait prompt_sonic_root
sendln "./Blanton_Script/bmc_monitor.sh start"
; ========== DDR MEMORY STRESS TEST ==========
;wait prompt_sonic_root
;sendln "bgctl run /usr/sbin/memtester 1G 100"
wait prompt_sonic_root
sendln "python ~/hammer/tools/stress_mem.py &"
wait prompt_sonic_root
sendln "python ~/hammer/tools/stress_mem.py &"
wait prompt_sonic_root
sendln "python ~/hammer/tools/stress_mem.py &"
wait prompt_sonic_root
sendln "python ~/hammer/tools/stress_mem.py &"
wait prompt_sonic_root
sendln "python ~/hammer/tools/stress_mem.py &"
sendln "bgctl run /usr/sbin/memtester 1G" ; Continue Execture
; ========== SSD read/write ==========
wait prompt_sonic_root
sendln "python ~/hammer/tools/stress_ssd.py &"
sendln "bgctl run qfx5252-stress-ssd 1G --runtime 3600 --passes 5 --write --force --log /host/hw-eval/jobs/qfx5252-stress-ssd.log"
; ========== USB read/write ==========
; ---- Get USB partition node -> usb_dev ----
usb_dev = ''
wait prompt_sonic_root
sendln "python ~/hammer/tools/stress_usb.py &"
sendln 'D=$(bash ~/Blanton_Script/usb_target.sh -o dev 2>/dev/null); echo "<<D""EV=${D:-NONE}>>"'
waitregex '<<DEV=([^>]*)>>'
usb_dev = groupmatchstr1
wait prompt_sonic_root
sprintf2 cmd "bgctl run qfx5252-stress-usb %s 256M --runtime 60 --passes 1 --write --log /host/hw-eval/jobs/qfx5252-stress-usb.log" usb_dev
sendln cmd
;wait prompt_sonic_root
;sendln "bgctl run qfx5252-stress-usb /dev/sda1 256M --runtime 60 --passes 1 --write --log /host/hw-eval/qfx5252-stress-usb.log"
; ========== SHOW Background job ==========
wait prompt_sonic_root
sendln "jobs"
wait prompt_sonic_root
sendln "bgctl list"
; ========== Take one round of eye measurement on uplink ports ==========
; ========== BMC USB Test ==========
wait prompt_sonic_root
sendln 'echo "NCMIF=$(ls -d /sys/bus/usb/drivers/cdc_ncm/*/net/* 2>/dev/null | head -n 1 | xargs -r basename)"'
timeout = 15
waitregex 'NCMIF=[A-Za-z0-9_-]+'
if result = 0 then
messagebox 'cdc_ncm interface not found. Ping test skipped.' 'ERROR'
goto skip_ping
endif
strlen matchstr
cut_len = result - 6 ; 'NCMIF=' = 6 chars
strcopy matchstr 7 cut_len ncm_if ; -> ncm_if = "eth2" / "eth3"
; ========== Traffic START ==========
sprintf2 cmd 'ip -br link show %s' ncm_if
wait prompt_sonic_root
sendln "blanton_traffic_linespeed ps -u 0"
sendln cmd
wait prompt_sonic_root
sendln "blanton_traffic_linespeed init -u 0"
sendln 'systemctl stop qfx5252-bmc-usb-net-test.service 2>/dev/null || true'
wait prompt_sonic_root
sendln "blanton_traffic_linespeed clear -u 0"
sendln 'systemctl reset-failed qfx5252-bmc-usb-net-test.service 2>/dev/null || true'
sprintf2 cmd 'systemd-run --unit=qfx5252-bmc-usb-net-test --property=Type=exec /usr/bin/ping -I %s -i 1 -W 2 -w 14400 192.168.200.200' ncm_if
wait prompt_sonic_root
sendln cmd
:skip_ping
; restore the default (no cap): the traffic init below walks 108 VLANs per
; unit and takes far longer than the 15 s set for the NCM probe above.
timeout = 0
; ========== 10G/1G MGMT Ping Test ==========
wait prompt_sonic_root
sendln "./Blanton_Script/mgmt_ping_monitor.sh start"
wait prompt_sonic_root
pause 3
sendln "blanton_traffic_linespeed show -u 0"
sendln "./Blanton_Script/mgmt_ping_monitor.sh status"
; ========== 100G Port ==========
wait prompt_sonic_root
sendln "blanton_traffic_linespeed start -u 0"
sendln "show interfaces status Ethernet513,Ethernet514"
;=============================================================================================================
; ========== 100G PRBS ==========
;wait prompt_sonic_root
;sendln "~/Blanton_Script/port_prbs_monitor.sh start"
;=============================================================================================================
; ========== Traffic START ==========
; Silent MAC
wait prompt_sonic_root
sendln "config feature state lldp disabled"
wait prompt_sonic_root
sendln "config save -y"
if SWB_UNIT0 = 1 && SWB_UNIT1 = 1 then
wait prompt_sonic_root
sendln "blanton_traffic_linespeed ps"
wait prompt_sonic_root
sendln "blanton_traffic_linespeed init"
wait prompt_sonic_root
sendln "blanton_traffic_linespeed clear"
wait prompt_sonic_root
pause 15
sendln "blanton_traffic_linespeed show"
wait prompt_sonic_root
sendln "blanton_traffic_linespeed start"
elseif SWB_UNIT0 = 1 && SWB_UNIT1 = 0 then
wait prompt_sonic_root
sendln "blanton_traffic_linespeed ps -u 0"
wait prompt_sonic_root
sendln "blanton_traffic_linespeed init -u 0"
wait prompt_sonic_root
sendln "blanton_traffic_linespeed clear -u 0"
wait prompt_sonic_root
pause 15
sendln "blanton_traffic_linespeed show -u 0"
wait prompt_sonic_root
sendln "blanton_traffic_linespeed start -u 0"
elseif SWB_UNIT0 = 0 && SWB_UNIT1 = 1 then
wait prompt_sonic_root
sendln "blanton_traffic_linespeed ps -u 1"
wait prompt_sonic_root
sendln "blanton_traffic_linespeed init -u 1"
wait prompt_sonic_root
sendln "blanton_traffic_linespeed clear -u 1"
wait prompt_sonic_root
pause 15
sendln "blanton_traffic_linespeed show -u 1"
wait prompt_sonic_root
sendln "blanton_traffic_linespeed start -u 1"
else
endif
; ========== Get data every 10mins ==========
; ========== Monitor loop: platform data + background jobs (pause 60s per round) ==========
while 1
include "utils/setup_pmon.ttl"
wait prompt_sonic_root
sendln "bgctl list"
wait prompt_sonic_root
sendln "jobs"
; ========== Check Margin ==========
if EN_Margin = 1 then
include "utils/show_margin_status.ttl"
endif
pause 60
endwhile
+134 -5
View File
@@ -12,6 +12,37 @@ pause 1
; ========== Kill Process ==========
include "utils/kill_all_process.ttl"
; bgctrl all stop
wait prompt_sonic_root
sendln "bgctl stop --all"
wait prompt_sonic_root
sendln "bgctl stop --all"
; BMC USB Test STOP
wait prompt_sonic_root
sendln "systemctl stop qfx5252-bmc-usb-net-test.service"
; 10G/1G MGMT Test STOP
wait prompt_sonic_root
sendln "~/Blanton_Script/mgmt_ping_monitor.sh status"
wait prompt_sonic_root
sendln "~/Blanton_Script/mgmt_ping_monitor.sh stop"
;=============================================================================================================
; 100G PRBS STOP
;wait prompt_sonic_root
;sendln "~/Blanton_Script/port_prbs_monitor.sh stop"
;wait prompt_sonic_root
;sendln "sudo sfputil lpmode off Ethernet513"
;wait prompt_sonic_root
;sendln "sudo sfputil lpmode off Ethernet514"
;wait prompt_sonic_root
;sendln "config interface -n asic0 startup Ethernet513"
;wait prompt_sonic_root
;sendln "config interface -n asic1 startup Ethernet514"
;=============================================================================================================
; ========== TAKE DATA ==========
include "utils/setup_pmon.ttl"
include "utils/show_dmesg.ttl"
@@ -25,17 +56,115 @@ include "utils/show_pcie_error_reg_DDR.ttl"
include "utils/show_pcie_error_reg_I210.ttl"
include "utils/show_pcie_error_reg_SSD.ttl"
wait prompt_sonic_root
sendln "/usr/sbin/ras-mc-ctl --summary"
; ========== BMC DDR TAKE DATA ==========
;wait prompt_sonic_root
;sendln "./Blanton_Script/bmc_monitor_ddr.sh stop"
;wait prompt_sonic_root
;sendln "./Blanton_Script/bmc_monitor.sh stop"
;wait prompt_sonic_root
;sendln "cat ./Blanton_Script/log/bmc_poll.log"
;wait prompt_sonic_root
;sendln "cat ./Blanton_Script/log/bmc_ddr.log"
; ========== BMC Monitor TAKE DATA ==========
wait prompt_sonic_root
sendln "./Blanton_Script/bmc_monitor.sh stop"
wait prompt_sonic_root
sendln "cat ./Blanton_Script/log/bmc_poll.log"
; ========== BMC USB Test result ==========
;wait prompt_sonic_root
;sendln "journalctl -u qfx5252-bmc-usb-net-test.service --no-pager"
; ========== CHECK Stress results ==========
wait prompt_sonic_root
sendln "cat ~/hammer/tools/amd/mlucas_amm_log"
wait prompt_sonic_root
sendln "cat ~/hammer/tools/log.stress_ssd"
sendln "cat /host/hw-eval/jobs/qfx5252-stress-ssd.log"
wait prompt_sonic_root
sendln "cat /host/hw-eval/jobs/qfx5252-stress-usb.log"
; ========== 10G/1G MGMT Test result ==========
wait prompt_sonic_root
sendln "cat ~/Blanton_Script/log/mgmt_ping.log"
; ========== 100G Port ==========
wait prompt_sonic_root
sendln "show interfaces status Ethernet513,Ethernet514"
;=============================================================================================================
;wait prompt_sonic_root
;sendln "cat ~/Blanton_Script/log/port_prbs.log "
;=============================================================================================================
; ========== CHECK Traffic counters ==========
wait prompt_sonic_root
sendln "blanton_traffic_linespeed stop -u 0"
wait prompt_sonic_root
sendln "blanton_traffic_linespeed report -u 0"
if SWB_UNIT0 = 1 && SWB_UNIT1 = 1 then
wait prompt_sonic_root
sendln "blanton_traffic_linespeed stop"
wait prompt_sonic_root
sendln "blanton_traffic_linespeed report"
elseif SWB_UNIT0 = 1 && SWB_UNIT1 = 0 then
wait prompt_sonic_root
sendln "blanton_traffic_linespeed stop -u 0"
wait prompt_sonic_root
sendln "blanton_traffic_linespeed report -u 0"
elseif SWB_UNIT0 = 0 && SWB_UNIT1 = 1 then
wait prompt_sonic_root
sendln "blanton_traffic_linespeed stop -u 1"
wait prompt_sonic_root
sendln "blanton_traffic_linespeed report -u 1"
else
endif
; ========== NVME Err Info ==========
wait prompt_sonic_root
sendln "/usr/sbin/nvme smart-log -H /dev/nvme0"
wait prompt_sonic_root
sendln "/usr/sbin/smartctl -x /dev/nvme0"
wait prompt_sonic_root
sendln "show reboot-cause"
; ========== Show Power on uptime ==========
wait prompt_sonic_root
sendln "show uptime"
; ---- Get USB partition node -> usb_dev ----
usb_dev = ''
wait prompt_sonic_root
sendln 'D=$(bash ~/Blanton_Script/usb_target.sh -o dev 2>/dev/null); echo "<<D""EV=${D:-NONE}>>"'
waitregex '<<DEV=([^>]*)>>'
usb_dev = groupmatchstr1
wait prompt_sonic_root
sendln "sync"
wait prompt_sonic_root
sprintf2 cmd "mount %s /mnt/usb" usb_dev
sendln cmd
; ========== Copy Stress Log to USB ==========
wait prompt_sonic_root
sendln "cp ./Blanton_Script/log/bmc_poll.log /host/hw-eval/jobs/"
wait prompt_sonic_root
sendln "cp ./Blanton_Script/log/mgmt_ping.log /host/hw-eval/jobs/"
wait prompt_sonic_root
getdate ts_date "%Y%m%d"
gettime ts_time "%H%M"
sprintf2 usb_dst "/mnt/usb/jobs-%s_%s" ts_date ts_time
sprintf2 cmd "cp -r /host/hw-eval/jobs/ %s" usb_dst
sendln cmd
; ========== HW Test Session ==========
wait prompt_sonic_root
sendln "hw-test-session finish"
wait prompt_sonic_root
sendln "bgctl reset --yes"
messagebox 'GOOD JOB! Test Case DONE' 'teraterm'
@@ -1,444 +0,0 @@
#!/usr/bin/env bash
# =============================================================================
# blanton_cb_i2c.sh -- CB FPGA (F3) I2C / PMBus over the OpenCores I2C master
# =============================================================================
# Version History:
# V0.1.0 2026-06-16 CB sibling of blanton_icb_i2c.sh. Same OpenCores I2C
# handshake, but the I2C block lives in CB FPGA *Function 3*
# register space (base 0x300, stride 0x20, Ch0..Ch17), and
# is reached via cb_fpga 3 <off> (direct PCIe BAR -> .3/resource0).
# =============================================================================
#
# IMPORTANT (vs the hand-written cb_fpga.md pcimem commands):
# * The CB I2C channels are on PCI **Function 3**, so the sysfs file is
# .../0000:02:00.3/resource0 (NOT .0). cb_fpga 3 uses FUNCT3_RES for this.
# * This tool uses **Repeated START** for reads (write reg pointer with WR only,
# no STOP, then Sr + slave|R) so it works for the SMBus/PMBus parts (VRM,
# EFUSE) as well as plain register parts. STOP+START only works for the latter.
# * Slave addresses are 7-bit here. The spec lists 8-bit pairs (e.g. MAX31790
# "0x40/0x41" -> 7-bit 0x20; TCA9546 "0xE8/0xE9" -> 7-bit 0x74). Divide by 2.
#
# CB FPGA F3 I2C channels (base 0x300, stride 0x20) -- 7-bit addrs in (...):
# Ch0 0x300 SWB_0: TCA9546 mux (0x74), M0.C0 temp sensor, ...
# Ch1 0x320 SWB_1: TCA9546 mux (0x74), ...
# Ch2 0x340 SWB_0: SWB_FPGA (0x40), IO Exp PCA9555 (0x24), ...
# Ch3 0x360 SWB_1: SWB_FPGA (0x40), PCA9555 (0x24), ...
# Ch4 0x380 SWB_0: M1 TCA9546 mux (0x74), Clk Gen SI5..., ...
# Ch5 0x3A0 SWB_1: M1 TCA9546 mux (0x74), ...
# Ch6 0x3C0 SWB_0: VRM M29816 (0x20), VRM M2985B (0x21), ... [PMBus]
# Ch7 0x3E0 SWB_1: VRM M29816 (0x20), VRM M2985B (0x21), ... [PMBus]
# Ch8 0x400 SWB_0: VRM M2985B (0x25), M2985B (0x26), ... [PMBus]
# Ch9 0x420 SWB_1: VRM M2985B (0x25), M2985B (0x26), ... [PMBus]
# Ch10 0x440 M0 TCA9543 mux (0x70), M0.C0 Clk Buf 9DBL0452, ...
# Ch11 0x460 Fan Ctrl MAX31790 (0x20)
# Ch12 0x480 ADC leakage ADC128D818 (0x1F)
# Ch13 0x4A0 NFC SM-MFAD4-C02 (0x28)
# Ch14 0x4C0 M1 TCA9543 mux (0x70), M1.C0 QSFP28.P1 xcvr, ...
# Ch15 0x4E0 Fan1/2 EFUSE STEF48H28 (0x10/0x11), ... [PMBus-ish]
# Ch16 0x500 1G PHY I210 Ethernet ctrlr (0x49)
# Ch17 0x520 Chassis EEPROM (0x28..0x29), Diag EEPROM (0x2B), ...
# (slave 7-bit addrs derived from the 8-bit pairs in the register map; verify
# against the device datasheet / your board before driving writes.)
#
# Per-channel register layout (offset from channel base):
# +0x00 I2C_PRSCL_LO +0x04 I2C_PRSCL_HI +0x08 I2C_CTRL (bit7 MOD_EN)
# +0x0C I2C_TX(W)/I2C_RX(R) +0x10 I2C_CMD(W)/I2C_STAT(R)
# +0x14 I2C_MUX_SEL +0x18 I2C_RST (0xD) +0x1C SEM
# CMD : STA=0x80 STO=0x40 RD=0x20 WR=0x10 ACK=0x08(1=NACK) IACK=0x01
# STAT: RX_ACK=0x80(1=NACK) BUSY=0x40 ARB_LOST=0x20 TIP=0x02 INT_FLAG=0x01
# Prescale = (in_clk/(5*scl))-1 @75MHz: 0x88=100kHz, 0x24=400kHz
#
# Usage:
# source blanton_fpga_pcimem.sh # provides cb_fpga / fpga_debug
# source blanton_cb_i2c.sh
# cb_i2c_init 11 # Ch11 (Fan Ctrl) @100kHz
# cb_i2c_scan 11
# cb_i2c_read 11 0x20 0x01 1 # MAX31790 (7-bit 0x20) reg 0x01
# cb_pmbus_read 6 0x20 0x8B 2 # Ch6 VRM READ_VOUT
# =============================================================================
# --- Pull in the register-access backend (cb_fpga) if not present -----------
if ! declare -F cb_fpga >/dev/null 2>&1; then
_CBI2C_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" 2>/dev/null && pwd)"
if [ -n "$_CBI2C_DIR" ] && [ -f "$_CBI2C_DIR/blanton_fpga_pcimem.sh" ]; then
# shellcheck source=/dev/null
source "$_CBI2C_DIR/blanton_fpga_pcimem.sh"
else
echo -e "[\033[33mWARN\033[0m] blanton_cb_i2c.sh: cb_fpga() not found." >&2
echo " source blanton_fpga_pcimem.sh first (provides cb_fpga)." >&2
fi
fi
# === CB I2C geometry =========================================================
CBI2C_FN=3 # CB PCI Function hosting the I2C block
CBI2C_BASE=0x300 # channel 0 base offset (relative to F3 resource0)
CBI2C_CH_STRIDE=0x20
CBI2C_CH_MAX=17
# Per-channel register offsets
_CBI2C_PRSCL_LO=0x00
_CBI2C_PRSCL_HI=0x04
_CBI2C_CTRL=0x08
_CBI2C_TX=0x0C
_CBI2C_RX=0x0C
_CBI2C_CMD=0x10
_CBI2C_STAT=0x10
_CBI2C_MUX_SEL=0x14
_CBI2C_RST=0x18
_CBI2C_SEM=0x1C
# I2C_CTRL_REG / CMD / STAT bits
_CBI2C_EN=0x80
_CBI2C_STA=0x80
_CBI2C_STO=0x40
_CBI2C_RD=0x20
_CBI2C_WR=0x10
_CBI2C_NACK=0x08
_CBI2C_RXACK=0x80
_CBI2C_TIP=0x02
_CBI2C_RST_VAL=0xD
# Defaults
CB_I2C_PRESCALE_LO="${CB_I2C_PRESCALE_LO:-0x88}" # 100 kHz @ 75 MHz
CB_I2C_PRESCALE_HI="${CB_I2C_PRESCALE_HI:-0x00}"
CB_I2C_AUTO_INIT="${CB_I2C_AUTO_INIT:-1}"
CB_I2C_TIP_POLL_MAX="${CB_I2C_TIP_POLL_MAX:-100}"
# --- External I2C-mux RESET control (verified on COM34, 2026-06-16) ----------
# The TCA9543 muxes on CH10 (M0) and CH14 (M1) have an FPGA-driven, active-low
# RESET line. On power-up / FPGA reload the FPGA holds them in reset, so 0x70
# does NOT ACK on the root bus and all downstream legs are unreachable.
#
# The control is a GLOBAL register in CB FPGA *Function 0*, offset 0x7D4
# (I2C_MUX_RST_REG) -- NOT the per-channel I2C_MUX_SEL_REG (+0x14) which is a
# device-select and does nothing to the RESET pin. Per bit: 1 = release,
# 0 = hold-in-reset. Observed default 0xDD (bit1=CH10 mux, bit5=CH14 mux held).
# Writing 0xFF releases all external muxes (normal operating state). This value
# is volatile and is lost on the next power cycle / FPGA reload.
CBI2C_MUX_RST_FN="${CBI2C_MUX_RST_FN:-0}"
CBI2C_MUX_RST_OFF="${CBI2C_MUX_RST_OFF:-0x7D4}"
# Channels whose root bus carries an external mux that hangs off the RESET reg.
# CH10/CH14 = TCA9543; CH0/1/4/5 = TCA9546 (SWB/M1) per the channel map above.
CBI2C_MUX_CHANNELS="${CBI2C_MUX_CHANNELS:-0 1 4 5 10 14}"
# Auto-release external mux RESET before scanning / mux-writing a mux channel.
CB_I2C_AUTO_MUX_RST="${CB_I2C_AUTO_MUX_RST:-1}"
# --- Single FPGA register read/write via cb_fpga <fn> <off> [data] ----------
# _cbi2c_reg <abs_offset> [data] data present = write, absent = read (echo hex)
_cbi2c_reg() {
local off="$1" data="${2:-}"
if [ -n "$data" ]; then cb_fpga "$CBI2C_FN" "$off" "$data"; else cb_fpga "$CBI2C_FN" "$off"; fi
}
# Channel base absolute offset
_cbi2c_chan_base() {
local ch="$1"
if (( ch < 0 || ch > CBI2C_CH_MAX )); then
echo "blanton_cb_i2c: ch must be 0..$CBI2C_CH_MAX" >&2; return 1
fi
printf '0x%X' $(( CBI2C_BASE + ch * CBI2C_CH_STRIDE ))
}
# Normalize a register read-back to a number string ("0x...." -> 0x....)
_cbi2c_num() { local v="$1"; [[ "$v" =~ (0[xX][0-9a-fA-F]+) ]] && echo "${BASH_REMATCH[1]}" || echo "0"; }
# Write CMD then poll STAT (same offset) until TIP clears; echo final status hex
_cbi2c_cmd_wait() {
local cmd_off="$1" cmd_val="$2" i status
_cbi2c_reg "$cmd_off" "$cmd_val" >/dev/null
for (( i=0; i<CB_I2C_TIP_POLL_MAX; i++ )); do
status=$(_cbi2c_num "$(_cbi2c_reg "$cmd_off")")
if (( (status & _CBI2C_TIP) == 0 )); then
printf '0x%X' "$status"; return 0
fi
done
echo -e "[\033[31mERR\033[0m] CB I2C TIP timeout (status=$status)" >&2
printf '0x%X' "${status:-0}"; return 1
}
# Check RX_ACK in status (bit7). 0 = ACK, 1 = NACK -> return 1
_cbi2c_check_ack() {
local status; status=$(_cbi2c_num "$1")
(( (status & _CBI2C_RXACK) != 0 )) && return 1 || return 0
}
# === Public: init / reset / semaphore / mux-sel =============================
# cb_i2c_init <ch> [prescale_lo] [prescale_hi]
cb_i2c_init() {
[[ $# -lt 1 ]] && { echo "Usage: cb_i2c_init <ch> [prescale_lo] [prescale_hi]"; return 1; }
local ch="$1" lo="${2:-$CB_I2C_PRESCALE_LO}" hi="${3:-$CB_I2C_PRESCALE_HI}" base
base=$(_cbi2c_chan_base "$ch") || return 1
_cbi2c_reg "$(printf '0x%X' $((base + _CBI2C_CTRL)))" 0x00 >/dev/null # disable
_cbi2c_reg "$(printf '0x%X' $((base + _CBI2C_PRSCL_LO)))" "$lo" >/dev/null
_cbi2c_reg "$(printf '0x%X' $((base + _CBI2C_PRSCL_HI)))" "$hi" >/dev/null
_cbi2c_reg "$(printf '0x%X' $((base + _CBI2C_CTRL)))" "$_CBI2C_EN" >/dev/null # enable
return 0
}
# cb_i2c_reset <ch>
cb_i2c_reset() {
[[ $# -lt 1 ]] && { echo "Usage: cb_i2c_reset <ch>"; return 1; }
local ch="$1" base; base=$(_cbi2c_chan_base "$ch") || return 1
_cbi2c_reg "$(printf '0x%X' $((base + _CBI2C_RST)))" "$_CBI2C_RST_VAL" >/dev/null
echo " [RST] ch=$ch local I2C controller reset (0xD)"
}
# cb_i2c_sem <ch> [status|acquire [val]|release]
cb_i2c_sem() {
[[ $# -lt 1 ]] && { echo "Usage: cb_i2c_sem <ch> [status|acquire [val]|release]"; return 1; }
local ch="$1" act="${2:-status}" base off cur
base=$(_cbi2c_chan_base "$ch") || return 1
off=$(printf '0x%X' $((base + _CBI2C_SEM)))
case "$act" in
status)
cur=$(_cbi2c_num "$(_cbi2c_reg "$off")")
(( (cur & 0xFF) == 0 )) && echo " [SEM] ch=$ch free (0x00)" \
|| printf ' [SEM] ch=%s held (0x%02X)\n' "$ch" $((cur & 0xFF)) ;;
acquire)
local val="${3:-0x01}"
cur=$(_cbi2c_num "$(_cbi2c_reg "$off")")
if (( (cur & 0xFF) != 0 )); then
printf ' [SEM] ch=%s already held (0x%02X) -- not acquired\n' "$ch" $((cur & 0xFF)); return 1
fi
_cbi2c_reg "$off" "$val" >/dev/null
cur=$(_cbi2c_num "$(_cbi2c_reg "$off")")
(( (cur & 0xFF) != 0 )) && printf ' [SEM] ch=%s acquired (0x%02X)\n' "$ch" $((cur & 0xFF)) \
|| { echo " [SEM] ch=$ch acquire failed"; return 1; } ;;
release)
_cbi2c_reg "$off" 0x00 >/dev/null; echo " [SEM] ch=$ch released (0x00)" ;;
*) echo "Usage: cb_i2c_sem <ch> [status|acquire [val]|release]"; return 1 ;;
esac
}
# cb_i2c_muxsel <ch> <dev_sel> -- write I2C_MUX_SEL_REG (FPGA drives ext mux pins)
cb_i2c_muxsel() {
[[ $# -lt 2 ]] && { echo "Usage: cb_i2c_muxsel <ch> <dev_sel>"; return 1; }
local ch="$1" sel="$2" base; base=$(_cbi2c_chan_base "$ch") || return 1
_cbi2c_reg "$(printf '0x%X' $((base + _CBI2C_MUX_SEL)))" "$sel" >/dev/null
printf ' [MUXSEL] ch=%s DEV_SEL <= 0x%02X\n' "$ch" $((sel & 0xFF))
}
# cb_i2c_mux_reset_release -- de-assert ALL external mux RESET lines (F0 0x7D4 <= 0xFF)
# Global, not per-channel. Volatile: lost on power cycle / FPGA reload.
cb_i2c_mux_reset_release() {
cb_fpga "$CBI2C_MUX_RST_FN" "$CBI2C_MUX_RST_OFF" 0xFF >/dev/null
local rb; rb=$(_cbi2c_num "$(cb_fpga "$CBI2C_MUX_RST_FN" "$CBI2C_MUX_RST_OFF")")
printf ' [MUX-RST] I2C_MUX_RST_REG (F%s %s) <= 0xFF (all external muxes released, readback %s)\n' \
"$CBI2C_MUX_RST_FN" "$CBI2C_MUX_RST_OFF" "$rb"
}
# _cbi2c_is_mux_channel <ch> -- 0 (true) if ch is in CBI2C_MUX_CHANNELS
_cbi2c_is_mux_channel() {
local ch="$1" c
for c in $CBI2C_MUX_CHANNELS; do [[ "$c" == "$ch" ]] && return 0; done
return 1
}
# _cbi2c_auto_mux_rst <ch> -- release external mux RESET if ch is a mux channel
_cbi2c_auto_mux_rst() {
[[ "$CB_I2C_AUTO_MUX_RST" == "1" ]] || return 0
_cbi2c_is_mux_channel "$1" || return 0
cb_i2c_mux_reset_release
}
# === Core transfers (Repeated START, SMBus/PMBus compatible) =================
# _cbi2c_xfer_read <ch> <slave7> <reg> <nbytes> echoes space-separated hex bytes
_cbi2c_xfer_read() {
local ch="$1" slave="$2" reg="$3" n="$4" base tx cmd rx status i last cmdv val out=""
base=$(_cbi2c_chan_base "$ch") || return 1
tx=$(printf '0x%X' $((base + _CBI2C_TX)))
cmd=$(printf '0x%X' $((base + _CBI2C_CMD)))
rx=$(printf '0x%X' $((base + _CBI2C_RX)))
[[ "$CB_I2C_AUTO_INIT" == "1" ]] && cb_i2c_init "$ch" >/dev/null
_cbi2c_reg "$tx" "$(printf '0x%X' $(( (slave << 1) & 0xFF )))" >/dev/null
status=$(_cbi2c_cmd_wait "$cmd" "$(printf '0x%X' $((_CBI2C_STA | _CBI2C_WR)))")
_cbi2c_check_ack "$status" || { echo "ERR: NACK on slave addr (W) 0x$(printf %02X "$slave")" >&2; return 1; }
_cbi2c_reg "$tx" "$(printf '0x%X' $((reg & 0xFF)))" >/dev/null
status=$(_cbi2c_cmd_wait "$cmd" "$(printf '0x%X' $((_CBI2C_WR)))") # WR only, no STOP
_cbi2c_check_ack "$status" || { echo "ERR: NACK on reg 0x$(printf %02X "$reg")" >&2; return 1; }
_cbi2c_reg "$tx" "$(printf '0x%X' $(( ((slave << 1) | 1) & 0xFF )))" >/dev/null
status=$(_cbi2c_cmd_wait "$cmd" "$(printf '0x%X' $((_CBI2C_STA | _CBI2C_WR)))") # Repeated START
_cbi2c_check_ack "$status" || { echo "ERR: NACK on slave addr (R) 0x$(printf %02X "$slave")" >&2; return 1; }
for (( i=0; i<n; i++ )); do
last=$(( i == n-1 ))
if (( last )); then cmdv=$(( _CBI2C_RD | _CBI2C_NACK | _CBI2C_STO )); else cmdv=$(( _CBI2C_RD )); fi
_cbi2c_cmd_wait "$cmd" "$(printf '0x%X' "$cmdv")" >/dev/null
val=$(_cbi2c_num "$(_cbi2c_reg "$rx")")
out+="$(printf '0x%02X ' $((val & 0xFF)))"
done
echo "${out% }"
}
# _cbi2c_xfer_write <ch> <slave7> <reg> <byte> [byte...]
_cbi2c_xfer_write() {
local ch="$1" slave="$2" reg="$3"; shift 3
local base tx cmd status b last cmdv n=$# idx=0
[[ $n -lt 1 ]] && { echo "ERR: no data bytes" >&2; return 1; }
base=$(_cbi2c_chan_base "$ch") || return 1
tx=$(printf '0x%X' $((base + _CBI2C_TX)))
cmd=$(printf '0x%X' $((base + _CBI2C_CMD)))
[[ "$CB_I2C_AUTO_INIT" == "1" ]] && cb_i2c_init "$ch" >/dev/null
_cbi2c_reg "$tx" "$(printf '0x%X' $(( (slave << 1) & 0xFF )))" >/dev/null
status=$(_cbi2c_cmd_wait "$cmd" "$(printf '0x%X' $((_CBI2C_STA | _CBI2C_WR)))")
_cbi2c_check_ack "$status" || { echo "ERR: NACK on slave addr (W) 0x$(printf %02X "$slave")" >&2; return 1; }
_cbi2c_reg "$tx" "$(printf '0x%X' $((reg & 0xFF)))" >/dev/null
status=$(_cbi2c_cmd_wait "$cmd" "$(printf '0x%X' $((_CBI2C_WR)))")
_cbi2c_check_ack "$status" || { echo "ERR: NACK on reg 0x$(printf %02X "$reg")" >&2; return 1; }
for b in "$@"; do
idx=$((idx+1)); last=$(( idx == n ))
_cbi2c_reg "$tx" "$(printf '0x%X' $(( $((b)) & 0xFF )))" >/dev/null
if (( last )); then cmdv=$(( _CBI2C_WR | _CBI2C_STO )); else cmdv=$(( _CBI2C_WR )); fi
status=$(_cbi2c_cmd_wait "$cmd" "$(printf '0x%X' "$cmdv")")
_cbi2c_check_ack "$status" || { echo "ERR: NACK on data byte #$idx" >&2; return 1; }
done
return 0
}
# _cbi2c_xfer_cmd_only <ch> <slave7> <cmd> -- START+W, cmd byte, STOP (no data)
_cbi2c_xfer_cmd_only() {
local ch="$1" slave="$2" cc="$3" base tx cmd status
base=$(_cbi2c_chan_base "$ch") || return 1
tx=$(printf '0x%X' $((base + _CBI2C_TX)))
cmd=$(printf '0x%X' $((base + _CBI2C_CMD)))
[[ "$CB_I2C_AUTO_INIT" == "1" ]] && cb_i2c_init "$ch" >/dev/null
_cbi2c_reg "$tx" "$(printf '0x%X' $(( (slave << 1) & 0xFF )))" >/dev/null
status=$(_cbi2c_cmd_wait "$cmd" "$(printf '0x%X' $((_CBI2C_STA | _CBI2C_WR)))")
_cbi2c_check_ack "$status" || { echo "ERR: NACK on slave addr (W) 0x$(printf %02X "$slave")" >&2; return 1; }
_cbi2c_reg "$tx" "$(printf '0x%X' $((cc & 0xFF)))" >/dev/null
status=$(_cbi2c_cmd_wait "$cmd" "$(printf '0x%X' $((_CBI2C_WR | _CBI2C_STO)))")
_cbi2c_check_ack "$status" || { echo "ERR: NACK on cmd 0x$(printf %02X "$cc")" >&2; return 1; }
return 0
}
# === Public: generic I2C =====================================================
# cb_i2c_read <ch> <slave> <reg> [nbytes]
cb_i2c_read() {
[[ $# -lt 3 ]] && { echo "Usage: cb_i2c_read <ch> <slave> <reg> [nbytes]"; return 1; }
local ch="$1" slave=$(( $2 )) reg=$(( $3 )) n="${4:-1}" bytes
bytes=$(_cbi2c_xfer_read "$ch" "$slave" "$reg" "$n") || return 1
printf ' [I2C-R] ch=%s slave=0x%02X reg=0x%02X => [%s]\n' "$ch" "$slave" "$reg" "$bytes"
}
# cb_i2c_write <ch> <slave> <reg> <byte> [byte...]
cb_i2c_write() {
[[ $# -lt 4 ]] && { echo "Usage: cb_i2c_write <ch> <slave> <reg> <byte> [byte...]"; return 1; }
local ch="$1" slave=$(( $2 )) reg=$(( $3 )); shift 3
_cbi2c_xfer_write "$ch" "$slave" "$reg" "$@" || return 1
printf ' [I2C-W] ch=%s slave=0x%02X reg=0x%02X <= [%s]\n' "$ch" "$slave" "$reg" "$*"
}
# cb_i2c_muxwrite <ch> <mux_slave> <ctrl_byte> -- raw 1-byte write, no reg (TCA954x)
cb_i2c_muxwrite() {
[[ $# -lt 3 ]] && { echo "Usage: cb_i2c_muxwrite <ch> <mux_slave> <ctrl_byte>"; return 1; }
local ch="$1" slave=$(( $2 )) val=$(( $3 )) base tx cmd status
base=$(_cbi2c_chan_base "$ch") || return 1
tx=$(printf '0x%X' $((base + _CBI2C_TX)))
cmd=$(printf '0x%X' $((base + _CBI2C_CMD)))
[[ "$CB_I2C_AUTO_INIT" == "1" ]] && cb_i2c_init "$ch" >/dev/null
_cbi2c_auto_mux_rst "$ch"
_cbi2c_reg "$tx" "$(printf '0x%X' $(( (slave << 1) & 0xFF )))" >/dev/null
status=$(_cbi2c_cmd_wait "$cmd" "$(printf '0x%X' $((_CBI2C_STA | _CBI2C_WR)))")
_cbi2c_check_ack "$status" || { echo "ERR: NACK on mux addr 0x$(printf %02X "$slave")" >&2; return 1; }
_cbi2c_reg "$tx" "$(printf '0x%X' $((val & 0xFF)))" >/dev/null
status=$(_cbi2c_cmd_wait "$cmd" "$(printf '0x%X' $((_CBI2C_WR | _CBI2C_STO)))")
_cbi2c_check_ack "$status" || { echo "ERR: NACK on mux ctrl byte" >&2; return 1; }
printf ' [MUX-W] ch=%s mux=0x%02X <= 0x%02X\n' "$ch" "$slave" "$val"
}
# cb_i2c_scan <ch> -- probe 0x08..0x77, report slaves that ACK
cb_i2c_scan() {
[[ $# -lt 1 ]] && { echo "Usage: cb_i2c_scan <ch>"; return 1; }
local ch="$1" base tx cmd a status found=""
base=$(_cbi2c_chan_base "$ch") || return 1
tx=$(printf '0x%X' $((base + _CBI2C_TX)))
cmd=$(printf '0x%X' $((base + _CBI2C_CMD)))
cb_i2c_init "$ch" >/dev/null
_cbi2c_auto_mux_rst "$ch"
for (( a=0x08; a<=0x77; a++ )); do
_cbi2c_reg "$tx" "$(printf '0x%X' $(( (a << 1) & 0xFF )))" >/dev/null
status=$(_cbi2c_cmd_wait "$cmd" "$(printf '0x%X' $((_CBI2C_STA | _CBI2C_WR | _CBI2C_STO)))")
_cbi2c_check_ack "$status" && found+="$(printf '0x%02X ' "$a")"
done
[[ -n "$found" ]] && echo " [SCAN] ch=$ch found: ${found% }" || echo " [SCAN] ch=$ch no devices"
}
# === Public: PMBus (== SMBus; command code + Repeated START read) ============
# cb_pmbus_read <ch> <slave> <cmd> [nbytes] (default 2 bytes, LSB first)
cb_pmbus_read() {
[[ $# -lt 3 ]] && { echo "Usage: cb_pmbus_read <ch> <slave> <cmd> [nbytes]"; return 1; }
local ch="$1" slave=$(( $2 )) cc=$(( $3 )) n="${4:-2}" bytes
bytes=$(_cbi2c_xfer_read "$ch" "$slave" "$cc" "$n") || return 1
if [[ "$n" == "2" ]]; then
local b0 b1 word; b0=$(( $(echo "$bytes" | awk '{print $1}') )); b1=$(( $(echo "$bytes" | awk '{print $2}') ))
word=$(( b0 | (b1 << 8) ))
printf ' [PMB-R] ch=%s slave=0x%02X cmd=0x%02X => 0x%04X raw=[%s] linear16=%s\n' \
"$ch" "$slave" "$cc" "$word" "$bytes" "$(cb_pmbus_linear16 "$word")"
else
printf ' [PMB-R] ch=%s slave=0x%02X cmd=0x%02X => [%s]\n' "$ch" "$slave" "$cc" "$bytes"
fi
}
# cb_pmbus_write <ch> <slave> <cmd> [data byte...] (no byte = send-byte)
cb_pmbus_write() {
[[ $# -lt 3 ]] && { echo "Usage: cb_pmbus_write <ch> <slave> <cmd> [data byte...]"; return 1; }
local ch="$1" slave=$(( $2 )) cc=$(( $3 )); shift 3
if [[ $# -eq 0 ]]; then
_cbi2c_xfer_cmd_only "$ch" "$slave" "$cc" || return 1
printf ' [PMB-W] ch=%s slave=0x%02X cmd=0x%02X <= (send-byte)\n' "$ch" "$slave" "$cc"; return 0
fi
_cbi2c_xfer_write "$ch" "$slave" "$cc" "$@" || return 1
printf ' [PMB-W] ch=%s slave=0x%02X cmd=0x%02X <= [%s]\n' "$ch" "$slave" "$cc" "$*"
}
# cb_pmbus_linear16 <word>
cb_pmbus_linear16() {
local word=$(( $1 & 0xFFFF ))
awk -v w="$word" 'BEGIN{
e=int(w/2048)%32; if(e>15)e-=32;
m=w%2048; if(m>1023)m-=2048;
printf "%.4f", m * (2.0 ^ e);
}'
}
# === Help ====================================================================
blanton_cb_i2c_help() {
echo -e "\033[1mblanton_cb_i2c.sh - CB FPGA (F3) I2C / PMBus over OpenCores I2C master\033[0m"
echo ""
echo -e " Backend: \033[36mcb_fpga $CBI2C_FN\033[0m (direct PCIe BAR -> .3/resource0), base 0x300, Ch0..$CBI2C_CH_MAX"
echo ""
echo -e "\033[1mSetup\033[0m"
echo -e " \033[33mcb_i2c_init\033[0m <ch> [pre_lo] [pre_hi] prescale + enable (def 0x88/0x00 = 100kHz)"
echo -e " \033[33mcb_i2c_reset\033[0m <ch> LCL_RST=0xD"
echo -e " \033[33mcb_i2c_sem\033[0m <ch> [status|acquire [v]|release]"
echo -e " \033[33mcb_i2c_muxsel\033[0m <ch> <dev_sel> write I2C_MUX_SEL_REG"
echo -e " \033[33mcb_i2c_mux_reset_release\033[0m de-assert ALL ext mux RESET (F0 0x7D4<=0xFF)"
echo ""
echo -e "\033[1mGeneric I2C\033[0m"
echo -e " \033[33mcb_i2c_scan\033[0m <ch>"
echo -e " \033[33mcb_i2c_read\033[0m <ch> <slave> <reg> [nbytes]"
echo -e " \033[33mcb_i2c_write\033[0m <ch> <slave> <reg> <byte> [byte...]"
echo -e " \033[33mcb_i2c_muxwrite\033[0m <ch> <mux_slave> <ctrl_byte> raw 1-byte (TCA954x)"
echo ""
echo -e "\033[1mPMBus / SMBus\033[0m"
echo -e " \033[33mcb_pmbus_read\033[0m <ch> <slave> <cmd> [nbytes] (def 2, LE word + linear16)"
echo -e " \033[33mcb_pmbus_write\033[0m <ch> <slave> <cmd> [byte...]"
echo -e " \033[33mcb_pmbus_linear16\033[0m <word>"
echo ""
echo -e " Slave = 7-bit (tool shifts). Spec lists 8-bit pairs -> divide by 2."
echo -e " \033[33mfpga_debug on\033[0m to see every underlying pcimem command."
}
blanton_cb_i2c_help
@@ -1,270 +0,0 @@
#!/usr/bin/env bash
# =============================================================================
# blanton_fpga_pcimem.sh -- pcimem backend, sysfs resource file + relative offset
# =============================================================================
# Version History:
# V1.3.0 2026-06-12 pcimem variant (SONiC hardware). memtool variant: blanton_fpga_memtool.sh
# Version number kept in sync with blanton_fpga_memtool.sh
# V1.4.0 2026-06-15 Fix: VSPI window bases pointed at the VSPI-Flash block
# (0x340/0x380/0x3C0/0x400, config-flash channel) so register
# reads returned 0; corrected to the VSPI register block
# (PMC 0x640 / ICB 0x680 / SWB0 0x6C0 / SWB1 0x700)
# Add: _vspi_fnwarn guard - warns when PMC/ICB/SWB are called with
# cb_fpga-style "<fn> <addr>" (a read turning into a write)
# Change: suppress pcimem write stdout noise (mmap/Written lines)
# Docs: comments/help translated to English (ASCII-only for SONiC console)
# V1.4.1 2026-06-23 Fix: BDF in example auto-detect was wrong (02:00 vs 04:00); corrected to match the lspci example.
# =============================================================================
#
# SONiC hardware has no memtool / devmem, but it has pcimem. pcimem mmaps the PCI
# sysfs resource file directly and takes "resource file + offset relative to BAR
# start", not an absolute physical address.
#
# pcimem { sysfile } { offset } [ type*count [ data ] ]
# sysfile : sysfs file for the pci resource (e.g. .../resource0)
# offset : offset into pci memory region
# type : [b]yte, [h]alfword, [w]ord, [d]ouble-word
# *count : number of items (w*100 dump 100 words)
# data : data to be written
#
# This tool always accesses as word (w, 32-bit).
#
# After sourcing this file the following functions are available:
# cb_fpga <fn:0-3> <addr> [data]
# pmc_fpga <addr> [data]
# icb_fpga <addr> [data]
# swb0_fpga <addr> [data]
# swb1_fpga <addr> [data]
#
# data present = write, absent = read.
# Usage: source blanton_fpga_pcimem.sh
# === CB FPGA PCIe resource file (sysfs) ===
# Use lspci -D to find the BDF (domain:bus:dev.fn); resource0 is the sysfs mapping of BAR0.
# e.g. lspci -Dnn | grep -i fpga -> 0000:02:00.0 ...
# memtool variant fills in the BAR value; pcimem variant fills in the resource file
# path instead (offset becomes relative to BAR).
FUNCT0_RES=/sys/bus/pci/devices/0000:04:00.0/resource0
FUNCT1_RES=/sys/bus/pci/devices/0000:04:00.1/resource0
FUNCT2_RES=/sys/bus/pci/devices/0000:04:00.2/resource0
FUNCT3_RES=/sys/bus/pci/devices/0000:04:00.3/resource0
# Auto-detect example (uncomment to use; needs the FPGA's BDF prefix):
# _BDF=0000:02:00
# FUNCT0_RES=/sys/bus/pci/devices/${_BDF}.0/resource0
# FUNCT1_RES=/sys/bus/pci/devices/${_BDF}.1/resource0
# FUNCT2_RES=/sys/bus/pci/devices/${_BDF}.2/resource0
# FUNCT3_RES=/sys/bus/pci/devices/${_BDF}.3/resource0
# === VSPI window base offset (relative to CB Function 2 resource0 start) ===
# NOTE: CB.F2 has TWO near-identical VSPI register blocks (see spec register map):
# - VSPI-Flash-* @ 0x340/0x380/0x3C0/0x400 -> accesses the remote FPGA's CONFIG FLASH
# - VSPI-* @ 0x640/0x680/0x6C0/0x700 -> accesses the remote FPGA's REGISTERS
# This tool reads/writes remote registers, so it uses the VSPI-* block (0x640+).
# The VSPI-Flash bases are kept below (commented) in case flash access is needed later.
VSPI_PMC_BASE=0x640
VSPI_ICB_BASE=0x680
VSPI_SWB0_BASE=0x6C0
VSPI_SWB1_BASE=0x700
# VSPI-Flash window bases (config-flash access, not register access):
# VSPI_FLASH_PMC_BASE=0x340
# VSPI_FLASH_ICB_BASE=0x380
# VSPI_FLASH_SWB0_BASE=0x3C0
# VSPI_FLASH_SWB1_BASE=0x400
# === VSPI protocol constants ===
_VSPI_CMD_STAT=0x00
_VSPI_ADDR=0x04
_VSPI_WR_DATA=0x08
_VSPI_RD_DATA=0x0C
_VSPI_NEW_CMD=0x01
_VSPI_READ_W=0x02
_VSPI_WRITE_W=0x04
_VSPI_READY_MASK=0x100
_VSPI_POLL_MAX=200
# === pcimem command ===
PCIMEM_CMD="${PCIMEM_CMD:-pcimem}"
# === Debug mode (1=print commands AND execute, 0=execute only) ===
DEBUG_MODE=${DEBUG_MODE:-0}
fpga_debug() {
if [ "${1:-}" = "on" ]; then DEBUG_MODE=1
elif [ "${1:-}" = "off" ]; then DEBUG_MODE=0
else [ "$DEBUG_MODE" = "0" ] && DEBUG_MODE=1 || DEBUG_MODE=0; fi
echo -e "[\033[34mINFO\033[0m] DEBUG_MODE=$DEBUG_MODE"
}
# Wrapper: debug mode prints command (to stderr) AND executes, normal mode executes only.
# On read, returns the parsed hex value (stripping pcimem's mmap noise).
# Usage: _pcimem <sysfile> <offset> [data] (data present = write, absent = read)
_pcimem() {
local sysfile="$1" offset="$2" data="${3:-}"
if [[ -n "$data" ]]; then
[ "$DEBUG_MODE" = "1" ] && \
echo -e "\033[90m[DEBG] $PCIMEM_CMD $sysfile $offset w $data\033[0m" >&2
# Write output (opened / Target offset / mmap / Written...readback) is never
# parsed -> drop stdout noise; keep stderr so real errors still surface.
$PCIMEM_CMD "$sysfile" "$offset" w "$data" >/dev/null
else
[ "$DEBUG_MODE" = "1" ] && \
echo -e "\033[90m[DEBG] $PCIMEM_CMD $sysfile $offset w\033[0m" >&2
# pcimem read prints mmap info + "Value at offset ...: 0xXXXX";
# take the last 0x... token as the actual read-back value.
$PCIMEM_CMD "$sysfile" "$offset" w | grep -oiE '0x[0-9a-f]+' | tail -1
fi
}
# --- Internal VSPI helpers (all hang off CB Function 2's resource0) ---
_vspi_read() {
local win_off="$1" remote_offset="$2"
local cmd_off add_off rd_off addr_val status i
cmd_off=$(printf '0x%X' $(( win_off + _VSPI_CMD_STAT )))
add_off=$(printf '0x%X' $(( win_off + _VSPI_ADDR )))
rd_off=$(printf '0x%X' $(( win_off + _VSPI_RD_DATA )))
addr_val=$(printf '0x%08X' $(( (0x0 << 20) | (remote_offset & 0xFFFFF) )))
_pcimem "$FUNCT2_RES" "$add_off" "$addr_val"
_pcimem "$FUNCT2_RES" "$cmd_off" $(printf '0x%X' $(( _VSPI_NEW_CMD | _VSPI_READ_W )))
for (( i=0; i<_VSPI_POLL_MAX; i++ )); do
status=$(_pcimem "$FUNCT2_RES" "$cmd_off")
if (( ( ${status:-0} & _VSPI_READY_MASK) != 0 )); then
_pcimem "$FUNCT2_RES" "$rd_off"
return 0
fi
done
echo "ERROR: VSPI read timeout @ offset $remote_offset" >&2
return 1
}
_vspi_write() {
local win_off="$1" remote_offset="$2" data="$3"
local cmd_off add_off wr_off addr_val status i
cmd_off=$(printf '0x%X' $(( win_off + _VSPI_CMD_STAT )))
add_off=$(printf '0x%X' $(( win_off + _VSPI_ADDR )))
wr_off=$(printf '0x%X' $(( win_off + _VSPI_WR_DATA )))
addr_val=$(printf '0x%08X' $(( (0x0 << 20) | (remote_offset & 0xFFFFF) )))
_pcimem "$FUNCT2_RES" "$wr_off" "$data"
_pcimem "$FUNCT2_RES" "$add_off" "$addr_val"
_pcimem "$FUNCT2_RES" "$cmd_off" $(printf '0x%X' $(( _VSPI_NEW_CMD | _VSPI_WRITE_W )))
for (( i=0; i<_VSPI_POLL_MAX; i++ )); do
status=$(_pcimem "$FUNCT2_RES" "$cmd_off")
if (( ( ${status:-0} & _VSPI_READY_MASK) != 0 )); then
return 0
fi
done
echo "ERROR: VSPI write timeout @ offset $remote_offset" >&2
return 1
}
# Guard: PMC/ICB/SWB are single-function; signature is <addr> [data], NO fn arg.
# Common pitfall: using cb_fpga's "<fn> <addr>" form -> 2nd arg becomes data, a read
# turns into a write. If addr is 0~3 (looks like fn) AND data is given, warn and
# confirm; on a non-tty (script) just warn, don't block.
# Usage: _vspi_fnwarn <name> <addr> <data> non-zero return = cancel
_vspi_fnwarn() {
local name="$1" addr="$2" data="$3"
[[ -z "$data" ]] && return 0 # read, nothing to warn
case "$addr" in
0|1|2|3|0x0|0x1|0x2|0x3|0x00|0x01|0x02|0x03) ;; # looks like fn, warn below
*) return 0 ;; # normal offset, allow
esac
echo -e "[\033[33mWARN\033[0m] $name has no fn arg; signature is <addr> [data] (NOT cb_fpga's <fn> <addr>)." >&2
echo -e " This will WRITE $data to PMC/ICB/SWB offset $addr." >&2
echo -e " To READ offset $addr, pass a single arg: $name $addr" >&2
if [ -t 0 ]; then
local ans
read -r -p " Proceed with write anyway? [y/N] " ans
case "$ans" in
y|Y|yes|YES) return 0 ;;
*) echo " Cancelled." >&2; return 1 ;;
esac
fi
return 0
}
# === Public functions ===
cb_fpga() {
if [[ $# -lt 2 ]]; then
echo "Usage: cb_fpga <fn:0-3> <addr> [data]"; return 1
fi
local fn="$1" addr="$2" data="${3:-}" res
case "$fn" in
0) res=$FUNCT0_RES ;; 1) res=$FUNCT1_RES ;;
2) res=$FUNCT2_RES ;; 3) res=$FUNCT3_RES ;;
*) echo "cb_fpga: fn must be 0-3" >&2; return 1 ;;
esac
# pcimem offset is relative to BAR start; do not add BAR.
if [[ -z "$data" ]]; then
_pcimem "$res" "$addr"
else
_pcimem "$res" "$addr" "$data"
fi
}
pmc_fpga() {
if [[ $# -lt 1 ]]; then
echo "Usage: pmc_fpga <addr> [data]"; return 1
fi
local addr="$1" data="${2:-}"
_vspi_fnwarn pmc_fpga "$addr" "$data" || return 1
if [[ -z "$data" ]]; then _vspi_read "$VSPI_PMC_BASE" "$addr"
else _vspi_write "$VSPI_PMC_BASE" "$addr" "$data"; fi
}
icb_fpga() {
if [[ $# -lt 1 ]]; then
echo "Usage: icb_fpga <addr> [data]"; return 1
fi
local addr="$1" data="${2:-}"
_vspi_fnwarn icb_fpga "$addr" "$data" || return 1
if [[ -z "$data" ]]; then _vspi_read "$VSPI_ICB_BASE" "$addr"
else _vspi_write "$VSPI_ICB_BASE" "$addr" "$data"; fi
}
swb0_fpga() {
if [[ $# -lt 1 ]]; then
echo "Usage: swb0_fpga <addr> [data]"; return 1
fi
local addr="$1" data="${2:-}"
_vspi_fnwarn swb0_fpga "$addr" "$data" || return 1
if [[ -z "$data" ]]; then _vspi_read "$VSPI_SWB0_BASE" "$addr"
else _vspi_write "$VSPI_SWB0_BASE" "$addr" "$data"; fi
}
swb1_fpga() {
if [[ $# -lt 1 ]]; then
echo "Usage: swb1_fpga <addr> [data]"; return 1
fi
local addr="$1" data="${2:-}"
_vspi_fnwarn swb1_fpga "$addr" "$data" || return 1
if [[ -z "$data" ]]; then _vspi_read "$VSPI_SWB1_BASE" "$addr"
else _vspi_write "$VSPI_SWB1_BASE" "$addr" "$data"; fi
}
blanton_fpga_help() {
echo -e "\033[1mblanton_fpga (pcimem) - FPGA Register Access Tool\033[0m"
echo ""
echo -e "\033[1mUsage:\033[0m"
echo -e " \033[33mcb_fpga\033[0m <fn:0-3> <addr> [data] CB FPGA direct (sysfs resource0)"
echo -e " \033[33mpmc_fpga\033[0m <addr> [data] PMC FPGA via VSPI"
echo -e " \033[33micb_fpga\033[0m <addr> [data] ICB FPGA via VSPI"
echo -e " \033[33mswb0_fpga\033[0m <addr> [data] SWB0 FPGA via VSPI"
echo -e " \033[33mswb1_fpga\033[0m <addr> [data] SWB1 FPGA via VSPI"
echo -e " \033[33mfpga_debug\033[0m on|off Toggle debug mode (print commands and execute)"
echo ""
echo -e " Omit [data] to \033[32mread\033[0m, provide [data] to \033[31mwrite\033[0m."
echo -e " Backend: \033[36mpcimem\033[0m (sysfs resource file + relative offset)."
echo -e " Run \033[33mblanton_fpga_help\033[0m to show this message again."
}
blanton_fpga_help
@@ -5,6 +5,20 @@
# =============================================================================
# Version Control
# -----------------------------------------------------------------------------
# v2.6.0 SWB path actually wired to the shared CB I2C package. blanton_cb_i2c.sh
# and blanton_fpga_pcimem.sh live in the PARENT dir (Blanton_Script/), not
# next to margin.sh, so the backend is now searched in both places.
# Adds, for TRANSPORT=swb only (CB/i2c path untouched):
# * cb_i2c_init <ch> once in margin_init (prescale 0x88 = 100kHz) and
# CB_I2C_AUTO_INIT=0 afterwards -> 4 fewer pcimem writes per transfer
# * cb_pmbus_read presence probe of every LTC2977 in CHIPS
# * return-code checking on every cb_pmbus_read/write (NACK is reported
# instead of silently reading 0x0000)
# * DEBUG_MODE collision fix: blanton_fpga_pcimem.sh uses the same var
# name, so pcimem tracing is gated by MARGIN_SWB_PCIMEM_DEBUG
# * margin_swb_* wrappers (info/probe/scan/reset/sem) for debugging
# Physical map: CB F3 Ch6=SWB0 CONN14/16, Ch8=SWB0 CONN13/15,
# Ch7=SWB1 CONN14/16, Ch9=SWB1 CONN13/15
# v2.5.0 SWB margin support via a per-conf TRANSPORT layer. The SWB margin
# boards are ALSO LTC2980 (2x LTC2977) -- same chip as CB -- just reached
# through the CB FPGA F3 I2C master instead of a native bus. So the whole
@@ -55,6 +69,37 @@ LOG_MODE=1
TRANSPORT="i2c"
CB_I2C_CH=""
# --- SWB transport backend (blanton_cb_i2c.sh + blanton_fpga_pcimem.sh) -------
# The SWB margin boards' LTC2980s hang off the CB FPGA Function 3 I2C master.
# Physical mapping (per bench wiring):
# CB F3 Ch6 -> SWB0 CONN14 / CONN16
# CB F3 Ch8 -> SWB0 CONN13 / CONN15
# CB F3 Ch7 -> SWB1 CONN14 / CONN16
# CB F3 Ch9 -> SWB1 CONN13 / CONN15
# Two CONNs share one FPGA I2C channel, so the two boards on a channel MUST use
# different LTC2977 slave addresses (the CHIPS array in each .conf).
MARGIN_SWB_CHANNELS="${MARGIN_SWB_CHANNELS:-6 7 8 9}"
# SCL prescale for cb_i2c_init: 0x88 = 100kHz, 0x24 = 400kHz (@75MHz core clk).
MARGIN_SWB_PRESCALE_LO="${MARGIN_SWB_PRESCALE_LO:-0x88}"
MARGIN_SWB_PRESCALE_HI="${MARGIN_SWB_PRESCALE_HI:-0x00}"
# 0 = cb_i2c_init once in margin_init, then CB_I2C_AUTO_INIT=0 so each
# cb_pmbus_* transfer skips the 4-register prescale/enable dance (much faster:
# margin_status all = 32 transfers). Set 1 to keep the package default (re-init
# on every transfer) if something else on the box reprograms the channel.
MARGIN_SWB_AUTO_INIT="${MARGIN_SWB_AUTO_INIT:-0}"
# 1 = grab the per-channel FPGA semaphore (cb_i2c_sem acquire) in margin_init and
# hold it for the whole session; margin_swb_sem_release frees it. Default off so
# margin never fails just because someone else holds it.
MARGIN_SWB_USE_SEM="${MARGIN_SWB_USE_SEM:-0}"
# 1 = probe every LTC2977 in CHIPS (PMBus PAGE read) at margin_init.
MARGIN_SWB_PROBE="${MARGIN_SWB_PROBE:-1}"
# 1 = also let blanton_fpga_pcimem.sh print every underlying pcimem command.
# Needed because that script's debug flag is ALSO called DEBUG_MODE; margin_debug
# would otherwise flood the console with pcimem lines.
MARGIN_SWB_PCIMEM_DEBUG="${MARGIN_SWB_PCIMEM_DEBUG:-0}"
_SWB_READY=0
_SWB_SEM_HELD=0
# Channel enable for margining. Rails the LTC2980 does not sequence (externally
# enabled, e.g. an always-on MPQ8625 buck) sit with STATUS_WORD OFF=1 and won't
# margin, because the servo DAC never soft-connects. Writing ON_OFF_CONFIG (0x02)
@@ -131,6 +176,233 @@ margin_debug() {
echo -e "[\033[34mINFO\033[0m] DEBUG_MODE=$DEBUG_MODE"
}
# =============================================================================
# SWB transport backend (TRANSPORT=swb only -- the CB/i2c path never gets here)
# -----------------------------------------------------------------------------
# Wire ops come from the shared CB I2C package:
# blanton_fpga_pcimem.sh -> cb_fpga <fn> <off> [data] (PCIe BAR via pcimem)
# blanton_cb_i2c.sh -> cb_i2c_init / cb_i2c_reset / cb_i2c_sem /
# cb_i2c_scan / cb_pmbus_read / cb_pmbus_write
# Both live in the parent Blanton_Script/ directory (a copy next to margin.sh is
# also honoured if you ever bundle one).
# =============================================================================
# Locate a backend script: next to margin.sh first, then the parent dir.
_swb_find_backend() {
local name="$1" p
for p in "$SCRIPT_DIR/$name" "$SCRIPT_DIR/../$name"; do
[ -f "$p" ] && { echo "$p"; return 0; }
done
return 1
}
# Source blanton_fpga_pcimem.sh (cb_fpga) + blanton_cb_i2c.sh (cb_pmbus_*).
_swb_backend_load() {
local f
if ! declare -F cb_fpga >/dev/null 2>&1; then
if f=$(_swb_find_backend "blanton_fpga_pcimem.sh"); then
# shellcheck source=/dev/null
source "$f"
echo -e "[\033[34mINFO\033[0m] SWB backend: sourced $f"
else
echo -e "[\033[31mERRO\033[0m] blanton_fpga_pcimem.sh not found in $SCRIPT_DIR or its parent"
return 1
fi
fi
if ! declare -F cb_pmbus_read >/dev/null 2>&1; then
if f=$(_swb_find_backend "blanton_cb_i2c.sh"); then
# shellcheck source=/dev/null
source "$f" >/dev/null
echo -e "[\033[34mINFO\033[0m] SWB backend: sourced $f"
else
echo -e "[\033[31mERRO\033[0m] blanton_cb_i2c.sh not found in $SCRIPT_DIR or its parent"
return 1
fi
fi
declare -F cb_pmbus_read >/dev/null 2>&1 || {
echo -e "[\033[31mERRO\033[0m] cb_pmbus_read still undefined after sourcing the backend"; return 1; }
return 0
}
# Run a cb_* command with pcimem tracing suppressed (shared DEBUG_MODE var).
# Echoes the command's stdout, preserves its return code.
_swb_call() {
local _saved="$DEBUG_MODE" _rc
[ "$MARGIN_SWB_PCIMEM_DEBUG" = "1" ] || DEBUG_MODE=0
"$@"
_rc=$?
DEBUG_MODE="$_saved"
return $_rc
}
# Probe one LTC2977: 1-byte PMBus read of PAGE (0x00). Returns cb_pmbus_read's rc.
_swb_probe_addr() {
_swb_call cb_pmbus_read "$CB_I2C_CH" "$1" 0x00 1 >/dev/null 2>&1
}
# --- SWB wire ops used by the transport switch below ------------------------
# Both use _ADDR (LTC2977 7-bit slave) set by _get_chip and CB_I2C_CH from the
# .conf. Unlike i2cset/i2cget these report failures: a NACK on the FPGA I2C
# master used to silently turn into Vout=0.0000.
# _swb_write <reg> <byte> [byte...] -> cb_pmbus_write (LSB first for LINEAR16)
_swb_write() {
local reg=$1; shift
[ "$DEBUG_MODE" = "1" ] && echo -e "\033[90m[DEBG] cb_pmbus_write $CB_I2C_CH $_ADDR $reg $*\033[0m"
if ! _swb_call cb_pmbus_write "$CB_I2C_CH" "$_ADDR" "$reg" "$@" >/dev/null 2>&1; then
echo -e "[\033[31mERRO\033[0m] cb_pmbus_write ch=$CB_I2C_CH $_ADDR reg=$reg failed (NACK/timeout)" >&2
_log_detail "ERRO cb_pmbus_write ch=$CB_I2C_CH $_ADDR $reg $*"
return 1
fi
return 0
}
# _swb_read_word <reg> -> echoes "0xXXXX" parsed out of cb_pmbus_read's "=> 0xXXXX"
_swb_read_word() {
local reg=$1 out word
[ "$DEBUG_MODE" = "1" ] && echo -e "\033[90m[DEBG] cb_pmbus_read $CB_I2C_CH $_ADDR $reg 2\033[0m" >&2
if ! out=$(_swb_call cb_pmbus_read "$CB_I2C_CH" "$_ADDR" "$reg" 2 2>/dev/null); then
echo -e "[\033[31mERRO\033[0m] cb_pmbus_read ch=$CB_I2C_CH $_ADDR reg=$reg failed (NACK/timeout)" >&2
_log_detail "ERRO cb_pmbus_read ch=$CB_I2C_CH $_ADDR $reg"
echo "0x0000"
return 1
fi
# cb_pmbus_read prints " [PMB-R] ... => 0xXXXX raw=[...] linear16=..."
word=$(printf '%s' "$out" | awk -F'=> ' 'NF>1{print $2}' | awk '{print $1}')
[[ "$word" =~ ^0[xX][0-9a-fA-F]+$ ]] || word="0x0000"
echo "$word"
}
# Bring up the FPGA I2C channel for this .conf: validate CB_I2C_CH, prescale +
# enable it once (cb_i2c_init), optionally take the semaphore, then probe CHIPS.
_swb_setup() {
_SWB_READY=0
if [ -z "$CB_I2C_CH" ]; then
echo -e "[\033[31mERRO\033[0m] TRANSPORT=swb but CB_I2C_CH is empty in the .conf"
return 1
fi
if ! [[ "$CB_I2C_CH" =~ ^[0-9]+$ ]] || [ "$CB_I2C_CH" -gt 17 ]; then
echo -e "[\033[31mERRO\033[0m] CB_I2C_CH='$CB_I2C_CH' invalid (CB F3 I2C channels are 0..17)"
return 1
fi
case " $MARGIN_SWB_CHANNELS " in
*" $CB_I2C_CH "*) ;;
*) echo -e "[\033[33mWARN\033[0m] CB_I2C_CH=$CB_I2C_CH is not a known SWB VRM channel ($MARGIN_SWB_CHANNELS)" ;;
esac
# Prescale + MOD_EN on this channel.
if ! _swb_call cb_i2c_init "$CB_I2C_CH" "$MARGIN_SWB_PRESCALE_LO" "$MARGIN_SWB_PRESCALE_HI"; then
echo -e "[\033[31mERRO\033[0m] cb_i2c_init $CB_I2C_CH failed (FPGA/pcimem access?)"
return 1
fi
CB_I2C_AUTO_INIT="$MARGIN_SWB_AUTO_INIT"
local _khz="?"; [ "$MARGIN_SWB_PRESCALE_LO" = "0x88" ] && _khz="100"
[ "$MARGIN_SWB_PRESCALE_LO" = "0x24" ] && _khz="400"
echo -e "[\033[34mINFO\033[0m] cb_i2c_init ch=$CB_I2C_CH prescale=$MARGIN_SWB_PRESCALE_LO/$MARGIN_SWB_PRESCALE_HI (~${_khz}kHz), CB_I2C_AUTO_INIT=$CB_I2C_AUTO_INIT"
_log "swb: cb_i2c_init ch=$CB_I2C_CH prescale=$MARGIN_SWB_PRESCALE_LO auto_init=$CB_I2C_AUTO_INIT"
if [ "$MARGIN_SWB_USE_SEM" = "1" ]; then
if _swb_call cb_i2c_sem "$CB_I2C_CH" acquire; then
_SWB_SEM_HELD=1
else
echo -e "[\033[33mWARN\033[0m] semaphore on ch=$CB_I2C_CH not acquired; continuing without it"
fi
fi
_SWB_READY=1
if [ "$MARGIN_SWB_PROBE" = "1" ]; then
local idx=0 chip addr bad=0
for chip in "${CHIPS[@]}"; do
IFS=':' read -r _ addr <<< "$chip"
if _swb_probe_addr "$addr"; then
echo -e "[\033[34mINFO\033[0m] probe ch=$CB_I2C_CH LTC2977[$idx] $addr : \033[32mACK\033[0m"
else
echo -e "[\033[33mWARN\033[0m] probe ch=$CB_I2C_CH LTC2977[$idx] $addr : \033[31mNACK\033[0m (wrong CB_I2C_CH / addr, board absent, or channel stuck -> margin_swb_scan / margin_swb_reset)"
bad=1
fi
idx=$((idx+1))
done
_log "swb: probe ch=$CB_I2C_CH chips=${CHIPS[*]} $( [ "$bad" = "0" ] && echo all-ACK || echo has-NACK )"
fi
return 0
}
# --- SWB debug / maintenance wrappers (thin shells over blanton_cb_i2c.sh) ---
_swb_guard() {
if [ "$TRANSPORT" != "swb" ]; then
echo -e "[\033[33mWARN\033[0m] current setting is TRANSPORT=$TRANSPORT (not swb); nothing to do"
return 1
fi
declare -F cb_pmbus_read >/dev/null 2>&1 || { echo -e "[\033[31mERRO\033[0m] SWB backend not loaded; run margin_init first"; return 1; }
return 0
}
# margin_swb_info -- show the transport wiring of the loaded setting
margin_swb_info() {
echo "-------------"
echo "[$PART_NUMBER] transport"
echo "-------------"
echo " TRANSPORT : $TRANSPORT"
if [ "$TRANSPORT" = "swb" ]; then
echo " CB_I2C_CH : $CB_I2C_CH (CB FPGA F3, base $(printf '0x%X' $(( 0x300 + CB_I2C_CH * 0x20 ))))"
echo " LTC2977 : ${CHIPS[*]} (bus field unused on swb)"
echo " prescale : $MARGIN_SWB_PRESCALE_LO/$MARGIN_SWB_PRESCALE_HI auto_init=$CB_I2C_AUTO_INIT sem_held=$_SWB_SEM_HELD"
echo " wire ops : cb_pmbus_write $CB_I2C_CH <addr> 0x00 <page> / cb_pmbus_read $CB_I2C_CH <addr> 0x8b 2"
else
echo " bus:addr : ${CHIPS[*]} (i2cset/i2cget)"
fi
}
# margin_swb_probe -- re-run the PAGE-read presence check on every LTC2977
margin_swb_probe() {
_swb_guard || return 1
local idx=0 chip addr
for chip in "${CHIPS[@]}"; do
IFS=':' read -r _ addr <<< "$chip"
if _swb_probe_addr "$addr"; then
echo -e " ch=$CB_I2C_CH $addr : \033[32mACK\033[0m"
else
echo -e " ch=$CB_I2C_CH $addr : \033[31mNACK\033[0m"
fi
idx=$((idx+1))
done
}
# margin_swb_scan [ch] -- cb_i2c_scan on this conf's channel (0x08..0x77)
margin_swb_scan() {
_swb_guard || return 1
_swb_call cb_i2c_scan "${1:-$CB_I2C_CH}"
}
# margin_swb_reset -- local I2C controller reset (0xD) + re-init, for a stuck bus
margin_swb_reset() {
_swb_guard || return 1
_swb_call cb_i2c_reset "$CB_I2C_CH"
_swb_call cb_i2c_init "$CB_I2C_CH" "$MARGIN_SWB_PRESCALE_LO" "$MARGIN_SWB_PRESCALE_HI"
echo -e "[\033[34mINFO\033[0m] ch=$CB_I2C_CH reset + re-init done"
_log "swb: cb_i2c_reset + cb_i2c_init ch=$CB_I2C_CH"
}
# margin_swb_sem [status|acquire|release]
margin_swb_sem() {
_swb_guard || return 1
local act="${1:-status}"
_swb_call cb_i2c_sem "$CB_I2C_CH" "$act"
case "$act" in
acquire) _SWB_SEM_HELD=1 ;;
release) _SWB_SEM_HELD=0 ;;
esac
}
# margin_swb_sem_release -- release the session semaphore if margin_init took it
margin_swb_sem_release() {
[ "$_SWB_SEM_HELD" = "1" ] || { echo -e "[\033[34mINFO\033[0m] no semaphore held by this session"; return 0; }
_swb_call cb_i2c_sem "$CB_I2C_CH" release
_SWB_SEM_HELD=0
}
# --- Init: load setting by name ---
margin_init() {
if [ -z "$1" ]; then
@@ -150,15 +422,13 @@ margin_init() {
TRANSPORT="${TRANSPORT:-i2c}" # a blank TRANSPORT="" in a .conf means i2c
echo -e "[\033[34mINFO\033[0m] Loaded setting: $setting_file (transport=$TRANSPORT)"
# SWB rails need the cb_pmbus_* backend; source it on demand.
if [ "$TRANSPORT" = "swb" ] && ! declare -F cb_pmbus_read >/dev/null 2>&1; then
if [ -f "$SCRIPT_DIR/blanton_cb_i2c.sh" ]; then
# shellcheck source=/dev/null
source "$SCRIPT_DIR/blanton_cb_i2c.sh"
else
echo -e "[\033[31mERRO\033[0m] TRANSPORT=swb but blanton_cb_i2c.sh not found in $SCRIPT_DIR"
return 1
fi
# SWB rails need the cb_pmbus_* backend (blanton_cb_i2c.sh + blanton_fpga_pcimem.sh,
# normally in the parent Blanton_Script/ dir); load it and bring the FPGA I2C
# channel up once. CB/i2c settings skip this entirely.
_SWB_READY=0; _SWB_SEM_HELD=0
if [ "$TRANSPORT" = "swb" ]; then
_swb_backend_load || return 1
_swb_setup || return 1
fi
_log_start_session
}
@@ -221,8 +491,7 @@ _get_page() {
# PAGE is written the same way on both (PMBus command 0x00).
_open_page() {
if [ "$TRANSPORT" = "swb" ]; then
[ "$DEBUG_MODE" = "1" ] && echo -e "\033[90m[DEBG] cb_pmbus_write $CB_I2C_CH $_ADDR 0x00 $1\033[0m"
cb_pmbus_write "$CB_I2C_CH" "$_ADDR" 0x00 "$1" >/dev/null 2>&1
_swb_write 0x00 "$1"
return
fi
[ "$DEBUG_MODE" = "1" ] && echo -e "\033[90m[DEBG] i2cset -y $_BUS $_ADDR 0x00 $1 b\033[0m"
@@ -233,8 +502,7 @@ _open_page() {
_i2c_write_word() {
local reg=$1 lo=$2 hi=$3
if [ "$TRANSPORT" = "swb" ]; then
[ "$DEBUG_MODE" = "1" ] && echo -e "\033[90m[DEBG] cb_pmbus_write $CB_I2C_CH $_ADDR $reg $lo $hi\033[0m"
cb_pmbus_write "$CB_I2C_CH" "$_ADDR" "$reg" "$lo" "$hi" >/dev/null 2>&1
_swb_write "$reg" "$lo" "$hi"
return
fi
local word=$(( (hi << 8) | lo ))
@@ -245,8 +513,7 @@ _i2c_write_word() {
_i2c_write_byte() {
if [ "$TRANSPORT" = "swb" ]; then
[ "$DEBUG_MODE" = "1" ] && echo -e "\033[90m[DEBG] cb_pmbus_write $CB_I2C_CH $_ADDR $1 $2\033[0m"
cb_pmbus_write "$CB_I2C_CH" "$_ADDR" "$1" "$2" >/dev/null 2>&1
_swb_write "$1" "$2"
return
fi
[ "$DEBUG_MODE" = "1" ] && echo -e "\033[90m[DEBG] i2cset -y $_BUS $_ADDR $1 $2 b\033[0m"
@@ -256,13 +523,7 @@ _i2c_write_byte() {
# Read a 16-bit word; echoes "0xXXXX". For swb, parse cb_pmbus_read's "=> 0xXXXX".
_i2c_read_word() {
if [ "$TRANSPORT" = "swb" ]; then
[ "$DEBUG_MODE" = "1" ] && echo -e "\033[90m[DEBG] cb_pmbus_read $CB_I2C_CH $_ADDR $1 2\033[0m" >&2
local out word
out=$(cb_pmbus_read "$CB_I2C_CH" "$_ADDR" "$1" 2 2>/dev/null)
# cb_pmbus_read prints "... => 0xXXXX raw=[...] ..."; take the word after "=>".
word=$(printf '%s' "$out" | awk -F'=> ' 'NF>1{print $2}' | awk '{print $1}')
[[ "$word" =~ ^0[xX][0-9a-fA-F]+$ ]] || word="0x0000"
echo "$word"
_swb_read_word "$1"
return
fi
[ "$DEBUG_MODE" = "1" ] && echo -e "\033[90m[DEBG] i2cget -y $_BUS $_ADDR $1 w\033[0m" >&2
@@ -9,7 +9,7 @@
# --- SWB transport (added): this LTC2980 sits behind the CB FPGA F3 I2C; reach
# it with cb_pmbus, not i2cset. CB_I2C_CH = FPGA I2C channel for this CONN. ---
TRANSPORT="swb"
CB_I2C_CH=6
CB_I2C_CH=8 # CB F3 Ch8 = SWB0 CONN13/CONN15 (shared with CONN15, addr must differ)
# --- I2C Configuration ---
@@ -9,15 +9,15 @@
# --- SWB transport (added): this LTC2980 sits behind the CB FPGA F3 I2C; reach
# it with cb_pmbus, not i2cset. CB_I2C_CH = FPGA I2C channel for this CONN. ---
TRANSPORT="swb"
CB_I2C_CH=8
CB_I2C_CH=6 # CB F3 Ch6 = SWB0 CONN14/CONN16 (shared with CONN16, addr must differ)
# --- I2C Configuration ---
# Format: "bus:address" (8 channels per entry / per LTC2977)
CHIPS=(
# chip1: SWB0 (CONN14)
"0:0x5E" # CH0 ~ CH7
"0:0x5F" # CH8 ~ CH15
"0:0x5C" # CH0 ~ CH7
"0:0x5E" # CH8 ~ CH15
)
# Pre-commands to run before accessing each chip (empty = none)
@@ -9,7 +9,7 @@
# --- SWB transport (added): this LTC2980 sits behind the CB FPGA F3 I2C; reach
# it with cb_pmbus, not i2cset. CB_I2C_CH = FPGA I2C channel for this CONN. ---
TRANSPORT="swb"
CB_I2C_CH=6
CB_I2C_CH=8 # CB F3 Ch8 = SWB0 CONN13/CONN15 (shared with CONN13, addr must differ)
# --- I2C Configuration ---
@@ -17,7 +17,7 @@ CB_I2C_CH=6
CHIPS=(
# chip1: SWB0 (CONN15)
"0:0x62" # CH0 ~ CH7
"0:0x63" # CH8 ~ CH15
"0:0x64" # CH8 ~ CH15
)
# Pre-commands to run before accessing each chip (empty = none)
@@ -9,15 +9,15 @@
# --- SWB transport (added): this LTC2980 sits behind the CB FPGA F3 I2C; reach
# it with cb_pmbus, not i2cset. CB_I2C_CH = FPGA I2C channel for this CONN. ---
TRANSPORT="swb"
CB_I2C_CH=8
CB_I2C_CH=6 # CB F3 Ch6 = SWB0 CONN14/CONN16 (shared with CONN14, addr must differ)
# --- I2C Configuration ---
# Format: "bus:address" (8 channels per entry / per LTC2977)
CHIPS=(
# chip1: SWB0 (CONN16)
"0:0x64" # CH0 ~ CH7
"0:0x65" # CH8 ~ CH15
"0:0x62" # CH0 ~ CH7
"0:0x64" # CH8 ~ CH15
)
# Pre-commands to run before accessing each chip (empty = none)
@@ -9,7 +9,7 @@
# --- SWB transport (added): this LTC2980 sits behind the CB FPGA F3 I2C; reach
# it with cb_pmbus, not i2cset. CB_I2C_CH = FPGA I2C channel for this CONN. ---
TRANSPORT="swb"
CB_I2C_CH=7
CB_I2C_CH=9 # CB F3 Ch9 = SWB1 CONN13/CONN15 (shared with CONN15, addr must differ)
# --- I2C Configuration ---
@@ -17,7 +17,7 @@ CB_I2C_CH=7
CHIPS=(
# chip1: SWB1 (CONN13)
"1:0x5C" # CH0 ~ CH7
"1:0x5D" # CH8 ~ CH15
"1:0x5E" # CH8 ~ CH15
)
# Pre-commands to run before accessing each chip (empty = none)
@@ -9,15 +9,15 @@
# --- SWB transport (added): this LTC2980 sits behind the CB FPGA F3 I2C; reach
# it with cb_pmbus, not i2cset. CB_I2C_CH = FPGA I2C channel for this CONN. ---
TRANSPORT="swb"
CB_I2C_CH=9
CB_I2C_CH=7 # CB F3 Ch7 = SWB1 CONN14/CONN16 (shared with CONN16, addr must differ)
# --- I2C Configuration ---
# Format: "bus:address" (8 channels per entry / per LTC2977)
CHIPS=(
# chip1: SWB1 (CONN14)
"1:0x5E" # CH0 ~ CH7
"1:0x5F" # CH8 ~ CH15
"1:0x5C" # CH0 ~ CH7
"1:0x5E" # CH8 ~ CH15
)
# Pre-commands to run before accessing each chip (empty = none)
@@ -9,7 +9,7 @@
# --- SWB transport (added): this LTC2980 sits behind the CB FPGA F3 I2C; reach
# it with cb_pmbus, not i2cset. CB_I2C_CH = FPGA I2C channel for this CONN. ---
TRANSPORT="swb"
CB_I2C_CH=7
CB_I2C_CH=9 # CB F3 Ch9 = SWB1 CONN13/CONN15 (shared with CONN13, addr must differ)
# --- I2C Configuration ---
@@ -17,7 +17,7 @@ CB_I2C_CH=7
CHIPS=(
# chip1: SWB1 (CONN15)
"1:0x62" # CH0 ~ CH7
"1:0x63" # CH8 ~ CH15
"1:0x64" # CH8 ~ CH15
)
# Pre-commands to run before accessing each chip (empty = none)
@@ -9,15 +9,15 @@
# --- SWB transport (added): this LTC2980 sits behind the CB FPGA F3 I2C; reach
# it with cb_pmbus, not i2cset. CB_I2C_CH = FPGA I2C channel for this CONN. ---
TRANSPORT="swb"
CB_I2C_CH=9
CB_I2C_CH=7 # CB F3 Ch7 = SWB1 CONN14/CONN16 (shared with CONN14, addr must differ)
# --- I2C Configuration ---
# Format: "bus:address" (8 channels per entry / per LTC2977)
CHIPS=(
# chip1: SWB1 (CONN16)
"1:0x64" # CH0 ~ CH7
"1:0x65" # CH8 ~ CH15
"1:0x62" # CH0 ~ CH7
"1:0x64" # CH8 ~ CH15
)
# Pre-commands to run before accessing each chip (empty = none)
@@ -40,6 +40,40 @@
# to the plain text before colouring so the columns stay
# aligned. Auto-disabled when stdout is not a terminal or
# with --tsv; --color / --no-color / TL_COLOR force it.
# V0.3.3 2026-08-17 Add: -tx / -length aliases for -c / -l so the tx burst can
# be written the way the bcm command reads
# ("start -u 0 -tx 100 -length 512"). Values are validated
# as positive integers; omitted = defaults 100 / 512.
# Fix: --dry-run printed nothing for init/start/stop/report
# since V0.3.1 (the new wrapper discarded the "[DRY ]"
# lines); the commands are listed again.
# V0.4.0 2026-08-17 Change: report now lists ALL 108 pairs, PASS / FAIL / NA
# alike, keeping one line per port (two lines per pair):
# Port TX RX PASS/FAIL
# cd0 4 4 PASS
# cd32 4 4 PASS
# cd204 - - NA
# cd236 - - NA
# Missing counters show '-'. -a is therefore the default
# (the flag is still accepted), and the long
# "# NA (no counters): ..." summary line is dropped because
# every NA pair is now a visible row. This is the only
# intentional difference from
# tools/bcm_mibpair_report_V1.1.0.py, which hides NA pairs
# unless -a is given.
# V0.4.1 2026-08-19 Fix: report printed every counter as 2147483647 (INT32_MAX).
# grp() rendered values with sprintf("%d", v); the DUT's awk
# casts to a 32-bit int for %d, so any counter above 2^31-1
# -- i.e. every real line-speed run -- saturated. Values are
# now formatted with %.0f, which stays in awk's double domain
# (exact to 2^53, far above any MIB counter).
# NOTE: the PASS/FAIL verdicts were NEVER affected -- the
# cross-check compares TX[]/RX[] doubles directly and only
# the display path went through grp(). Past reports that said
# PASS were right; only the numbers shown were wrong.
# Same fix applied to the "# FAIL ..." delta line (%+d ->
# %+.0f), which would otherwise saturate when a dead port
# makes the delta the size of the whole counter.
# =============================================================================
#
# The VLAN/port table below is copied verbatim from
@@ -67,11 +101,14 @@
# Options (any order, after the subcommand):
# -u|--unit <0|1|all|0,1> bcmcmd unit(s). Default: $TL_UNITS ("0 1")
# -c|--count <n> tx packet count (default $TL_TX_COUNT)
# aliases: -tx | --tx
# -l|--length <n> tx packet length (default $TL_TX_LENGTH)
# aliases: -length | --len
# -p|--ports <list|all> ps: port list ('all' = plain 'ps', every port)
# -T|--tolerance <n> report: allowed |delta| in packets (default 0;
# -T 1 absorbs the normal snapshot skew)
# -a|--all report: also list pairs that have no counters
# -a|--all report: accepted for compatibility -- every pair is
# listed by default, NA included
# -g|--grouped report: comma-group the values (1,234,567)
# -P|--with-port report: render cd1(2) instead of cd1
# -q|--quiet report: drop the trailing summary comments
@@ -283,7 +320,8 @@ TL_BENIGN_REMOVE="${TL_BENIGN_REMOVE:-Entry not found|not found|does not exist|n
_tl_bcm_run() {
local unit="$1" benign="$2"; shift 2
local cmd="$*" out rc
if [ "$TL_DRY_RUN" = "1" ]; then _tl_bcm "$unit" "$cmd" >/dev/null; return 0; fi
# dry-run: let _tl_bcm print the "[DRY ] bcmcmd ..." line, run nothing
if [ "$TL_DRY_RUN" = "1" ]; then _tl_bcm "$unit" "$cmd"; return 0; fi
_tl_dbg "$BCMCMD_CMD -n $unit -c '$cmd'"
out=$("$BCMCMD_CMD" -n "$unit" -c "$cmd" </dev/null 2>&1); rc=$?
[ "$DEBUG_MODE" = "1" ] && [ -n "$out" ] && echo "$out"
@@ -319,6 +357,9 @@ _tl_units_check() {
_tl_pair_count() { echo "$TL_PAIRS" | grep -c . ; }
# positive integer? (tx count / length come straight from the command line)
_tl_is_uint() { case "$1" in ''|*[!0-9]*) return 1 ;; 0) return 1 ;; *) return 0 ;; esac; }
# === Step 1: VLAN loopback setup (traffic_loopback_vlan_setting.ttl) =========
traffic_linespeed_init() {
_tl_units_check || return 1
@@ -369,6 +410,8 @@ traffic_linespeed_show() {
# === Step 4: start traffic (traffic_loopback_start.ttl) =====================
traffic_linespeed_start() {
_tl_units_check || return 1
_tl_is_uint "$TL_TX_COUNT" || { _tl_err "tx count '$TL_TX_COUNT' is not a positive integer (-tx|-c <n>)"; return 1; }
_tl_is_uint "$TL_TX_LENGTH" || { _tl_err "tx length '$TL_TX_LENGTH' is not a positive integer (-length|-l <n>)"; return 1; }
local unit vid pa pb n fail
for unit in $TL_UNITS; do
n=0; fail=0
@@ -456,7 +499,7 @@ _tl_pair_report() {
-v TSV="$TL_REPORT_TSV" \
-v COLOR="$color" '
function grp(v, s,out,l,i) {
s = sprintf("%d", v)
s = sprintf("%.0f", v)
if (!GROUPED) return s
out = ""; l = length(s)
for (i = 1; i <= l; i++) {
@@ -480,6 +523,12 @@ _tl_pair_report() {
NP = split(PAIRS, L, "\n")
for (i = 1; i <= NP; i++) { split(L[i], f, " "); VID[i] = f[1]; A[i] = f[2]; B[i] = f[3] }
MISSING = "-"
NC = 4
HDR[1] = "Port"; AL[1] = 0
HDR[2] = "TX"; AL[2] = 1
HDR[3] = "RX"; AL[3] = 1
HDR[4] = "PASS/FAIL"; AL[4] = 1
VCOL = NC # the verdict is always the last column
}
{
idx = index($0, "MIB_")
@@ -512,44 +561,52 @@ _tl_pair_report() {
oka = (a in HTX) && (a in HRX); okb = (b in HTX) && (b in HRX)
if (!(oka && okb)) {
v = "NA"; nna++
miss = miss (miss == "" ? "" : " ") a "/" b
if (!ALL) continue
ta = oka ? grp(TX[a]) : MISSING; ra = oka ? grp(RX[a]) : MISSING
tb = okb ? grp(TX[b]) : MISSING; rb = okb ? grp(RX[b]) : MISSING
# every pair is listed, NA included; absent counters show '-'
ta = (a in HTX) ? grp(TX[a]) : MISSING; ra = (a in HRX) ? grp(RX[a]) : MISSING
tb = (b in HTX) ? grp(TX[b]) : MISSING; rb = (b in HRX) ? grp(RX[b]) : MISSING
} else {
dtx = TX[a] - RX[b]; drx = RX[a] - TX[b]
d = absd(dtx); if (absd(drx) > d) d = absd(drx)
if (d <= TOL + 0) { v = "PASS"; npass++ }
else {
v = "FAIL"; nfail++
FL[++nf] = sprintf("# FAIL %s/%s: %s.TX-%s.RX=%+d %s.RX-%s.TX=%+d", \
FL[++nf] = sprintf("# FAIL %s/%s: %s.TX-%s.RX=%+.0f %s.RX-%s.TX=%+.0f", \
a, b, a, b, dtx, a, b, drx)
}
ta = grp(TX[a]); ra = grp(RX[a]); tb = grp(TX[b]); rb = grp(RX[b])
}
nrep++
nrow++; C0[nrow] = pname(a); C1[nrow] = ta; C2[nrow] = ra; C3[nrow] = v
nrow++; C0[nrow] = pname(b); C1[nrow] = tb; C2[nrow] = rb; C3[nrow] = v
nrow++; CELL[nrow,1] = pname(a); CELL[nrow,2] = ta; CELL[nrow,3] = ra; CELL[nrow,4] = v
nrow++; CELL[nrow,1] = pname(b); CELL[nrow,2] = tb; CELL[nrow,3] = rb; CELL[nrow,4] = v
}
H0 = "Port"; H1 = "TX"; H2 = "RX"; H3 = "PASS/FAIL"
if (TSV) {
print H0 "\t" H1 "\t" H2 "\t" H3
for (r = 1; r <= nrow; r++) print C0[r] "\t" C1[r] "\t" C2[r] "\t" C3[r]
} else {
w0 = length(H0); w1 = length(H1); w2 = length(H2); w3 = length(H3)
line = HDR[1]
for (c = 2; c <= NC; c++) line = line "\t" HDR[c]
print line
for (r = 1; r <= nrow; r++) {
if (length(C0[r]) > w0) w0 = length(C0[r])
if (length(C1[r]) > w1) w1 = length(C1[r])
if (length(C2[r]) > w2) w2 = length(C2[r])
if (length(C3[r]) > w3) w3 = length(C3[r])
line = CELL[r, 1]
for (c = 2; c <= NC; c++) line = line "\t" CELL[r, c]
print line
}
w0 += 2; w1 += 2; w2 += 2; w3 += 2
printf "%-*s%*s%*s%*s\n", w0, H0, w1, H1, w2, H2, w3, H3
} else {
for (c = 1; c <= NC; c++) {
w[c] = length(HDR[c])
for (r = 1; r <= nrow; r++) if (length(CELL[r, c]) > w[c]) w[c] = length(CELL[r, c])
w[c] += 2
}
for (c = 1; c <= NC; c++) printf(AL[c] ? "%*s" : "%-*s", w[c], HDR[c])
printf "\n"
for (r = 1; r <= nrow; r++) {
# pad on the plain text, then colour, so the columns stay aligned
printf "%-*s%*s%*s", w0, C0[r], w1, C1[r], w2, C2[r]
printf "%*s%s\n", w3 - length(C3[r]), "", vc(C3[r])
for (c = 1; c <= NC; c++) {
if (c == VCOL) {
# pad on the plain text, then colour, so columns stay aligned
printf "%*s%s", w[c] - length(CELL[r, c]), "", vc(CELL[r, c])
} else {
printf(AL[c] ? "%*s" : "%-*s", w[c], CELL[r, c])
}
}
printf "\n"
}
}
@@ -559,7 +616,7 @@ _tl_pair_report() {
printf "# pairs reported : %d\n", nrep
printf "# %s %d %s %d %s %d (tolerance %d)\n", \
vc("PASS"), npass, vc("FAIL"), nfail, vc("NA"), nna, TOL + 0
if (miss != "") printf "# %s (no counters): %s\n", vc("NA"), miss
# every NA pair is a visible row -> no long "no counters" list needed
for (i = 1; i <= nf; i++) print redl(FL[i])
}
exit (nfail || nna) ? 2 : 0
@@ -572,13 +629,14 @@ traffic_linespeed_report() {
# -f <log>: offline mode, no DUT access, no unit loop
if [ -n "$TL_REPORT_FILE" ]; then
[ -r "$TL_REPORT_FILE" ] || { _tl_err "cannot read log '$TL_REPORT_FILE'"; return 1; }
_tl_info "report from log: $TL_REPORT_FILE (tolerance $TL_TOLERANCE)"
# note on stderr so the table on stdout stays pipe/diff clean
_tl_info "report from log: $TL_REPORT_FILE (tolerance $TL_TOLERANCE)" >&2
_tl_pair_report "$TL_REPORT_FILE"; return $?
fi
_tl_units_check || return 1
if [ "$TL_DRY_RUN" = "1" ]; then
for unit in $TL_UNITS; do _tl_bcm "$unit" "show c" >/dev/null; done
for unit in $TL_UNITS; do _tl_bcm "$unit" "show c"; done
return 0
fi
@@ -627,17 +685,17 @@ blanton_traffic_linespeed_help() {
echo -e " \033[33mstart\033[0m step 4 tx <count> length=<len> VLantag=<vid> per VLAN"
echo -e " \033[33mstop\033[0m step 5 vlan remove <vid> pbm=<pair> [--destroy also destroys]"
echo -e " \033[33mps\033[0m ps <portlist> port status (link/speed) of the cabled ports"
echo -e " \033[33mreport\033[0m show c -> per-pair TX/RX table + PASS/FAIL cross-check"
echo -e " \033[33mreport\033[0m show c -> per-port TX/RX per pair + PASS/FAIL/NA cross-check"
echo -e " \033[33mrun\033[0m ps -> init -> clear -> start -> report"
echo -e " \033[33mhelp\033[0m"
echo ""
echo -e "\033[1mOptions\033[0m"
echo -e " \033[33m-u|--unit\033[0m <0|1|all> bcmcmd unit(s); 'all' = 0 1 (default: $TL_UNITS)"
echo -e " \033[33m-c|--count\033[0m <n> tx packet count (default $TL_TX_COUNT)"
echo -e " \033[33m-l|--length\033[0m <n> tx packet length (default $TL_TX_LENGTH)"
echo -e " \033[33m-c|--count|-tx\033[0m <n> tx packet count (default $TL_TX_COUNT)"
echo -e " \033[33m-l|--length|-length\033[0m <n> tx packet length (default $TL_TX_LENGTH)"
echo -e " \033[33m-p|--ports\033[0m <list> ps port list, or 'all' for every port"
echo -e " \033[33m-T|--tolerance\033[0m <n> report: allowed |delta| pkts (default $TL_TOLERANCE; -T 1 = snapshot skew)"
echo -e " \033[33m-a|--all\033[0m report: also list pairs with no counters"
echo -e " \033[33m-a|--all\033[0m report: kept for compatibility (all pairs are listed)"
echo -e " \033[33m-g|--grouped\033[0m report: comma-group values"
echo -e " \033[33m-P|--with-port\033[0m report: render cd1(2) instead of cd1"
echo -e " \033[33m-q|--quiet\033[0m report: no trailing summary"
@@ -658,7 +716,8 @@ blanton_traffic_linespeed_help() {
echo -e " blanton_traffic_linespeed report -u 1 -q | grep -v PASS"
echo -e " blanton_traffic_linespeed report -f /tmp/showc_unit1_*.log -a"
echo -e " blanton_traffic_linespeed show -u 1 | grep -iE 'error|discard|drop'"
echo -e " blanton_traffic_linespeed start -c 1000 -l 9216 -d"
echo -e " blanton_traffic_linespeed start -u 0 -tx 100 -length 512"
echo -e " blanton_traffic_linespeed start -tx 100000 -length 9216 -d"
echo -e " blanton_traffic_linespeed stop --destroy"
echo ""
echo -e " report exit code: 0 = all pairs PASS, 2 = any FAIL/NA, 1 = parse error"
@@ -679,8 +738,8 @@ blanton_traffic_linespeed() {
*) TL_UNITS="$2" ;;
esac
shift 2 ;;
-c|--count) TL_TX_COUNT="$2"; shift 2 ;;
-l|--length) TL_TX_LENGTH="$2"; shift 2 ;;
-c|--count|-tx|--tx) TL_TX_COUNT="$2"; shift 2 ;;
-l|-length|--length|--len) TL_TX_LENGTH="$2"; shift 2 ;;
-p|--ports) TL_PS_PORTS="$2"; shift 2 ;;
-T|--tolerance) TL_TOLERANCE="$2"; shift 2 ;;
-f|--file) TL_REPORT_FILE="$2"; shift 2 ;;
@@ -0,0 +1,344 @@
#!/bin/bash
###############################################################################
# bmc_monitor.sh
#
# Version : V1.1.0
# Author : ETWen
# Date : 20260820
#
# Purpose : Periodically execute a list of commands on the BMC through
# `bmc-manager run "<cmd>"` from the SONiC host side, and append all
# output to a rotating log file. Designed to run detached in the
# background so no interactive SSH session to the BMC is required.
#
# Notes : - Everything runs on the HOST. Nothing is deployed to the BMC, so
# the BMC busybox toolchain (no base64 / no setsid ...) is a
# non-issue.
# - `bmc-manager run` prints the remote command output on stdout and
# its own banner on stderr; the two streams are captured separately
# so the log stays clean.
#
# Version History
# V1.0.0 20260820 Initial Version
# V1.1.0 20260820 `start` clears the previous log by default
# (START_LOG_MODE: new / archive / append)
###############################################################################
set -u
###############################################################################
# User Configurable Section -- normally this is the only part you need to edit
###############################################################################
# Commands executed on the BMC, one per line, in this order.
# Add a line to add a command; comment it out to disable it.
COMMANDS=(
"free -m"
"i2ctransfer -f -y 1 w6@0x41 0x00 0xC0 0x55 0xAA 0x55 0xAA"
"i2ctransfer -f -y 1 w2@0x41 0x00 0xC0 r4"
"i2ctransfer -f -y 1 w6@0x41 0x00 0xC0 0xAA 0x55 0xAA 0x55"
"i2ctransfer -f -y 1 w2@0x41 0x00 0xC0 r4"
#"uptime"
#"cat /proc/loadavg"
#"cat /proc/meminfo"
#"df -h /tmp"
#"dmesg | tail -n 30"
)
# Seconds to wait after a full round (all COMMANDS executed) before the next one
INTERVAL_SEC=10
# Seconds to wait between two commands inside the same round (0 = no wait)
INTER_CMD_DELAY_SEC=0
# Per-command timeout in seconds; prevents a hung bmc-manager from freezing the
# whole loop. Set to 0 to disable.
CMD_TIMEOUT_SEC=30
# Stop after this many rounds. 0 = run until stopped manually.
MAX_ROUNDS=0
# Command runner. Final invocation is:
# "$RUNNER_BIN" "${RUNNER_ARGS[@]}" "<command string>"
# BMC side : RUNNER_BIN="bmc-manager" ; RUNNER_ARGS=(run)
# BMC by IP : RUNNER_BIN="bmc-manager" ; RUNNER_ARGS=(--ip 192.168.200.200 run)
# Host side : RUNNER_BIN="bash" ; RUNNER_ARGS=(-c)
RUNNER_BIN="bmc-manager"
RUNNER_ARGS=(run)
# stderr handling of the runner (bmc-manager banner lives here)
# 0 = log stderr only when the command fails (recommended, keeps log clean)
# 1 = always log stderr
LOG_STDERR=0
# Logging
LOG_DIR="" # empty = <script directory>/log
LOG_NAME="bmc_poll.log"
MAX_LOG_SIZE_MB=100 # rotate above this size; 0 = never rotate
MAX_LOG_KEEP=5 # keep .1 .. .N
# What `start` does with the log left behind by the previous run:
# new = begin with an empty log; previous content and rotated copies are
# discarded (default -- one run, one log)
# archive = rename the previous log to <log>.YYYYmmdd-HHMMSS, then begin empty
# append = keep appending to the existing log
START_LOG_MODE="new"
###############################################################################
# Internal
###############################################################################
SCRIPT_NAME="$(basename -- "${BASH_SOURCE[0]}")"
SCRIPT_PATH="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd)/${SCRIPT_NAME}"
[ -n "${LOG_DIR}" ] || LOG_DIR="$(dirname -- "${SCRIPT_PATH}")/log"
LOG_FILE="${LOG_DIR}/${LOG_NAME}"
PID_FILE="${LOG_DIR}/${LOG_NAME%.log}.pid"
LOG_TO_FILE=0
STOP=0
SLEEP_PID=""
ts() { date '+%Y-%m-%d %H:%M:%S'; }
die() { printf '[ERROR] %s\n' "$*" >&2; exit 1; }
on_signal() {
STOP=1
[ -n "${SLEEP_PID}" ] && kill "${SLEEP_PID}" 2>/dev/null
}
# sleep that can be aborted immediately by SIGTERM/SIGINT
interruptible_sleep() {
local sec="$1"
[ "${sec}" -gt 0 ] 2>/dev/null || return 0
sleep "${sec}" &
SLEEP_PID=$!
wait "${SLEEP_PID}" 2>/dev/null
SLEEP_PID=""
}
# echo the pid and return 0 when a live instance is found
is_running() {
local pid
[ -f "${PID_FILE}" ] || return 1
pid="$(cat "${PID_FILE}" 2>/dev/null)"
[[ "${pid}" =~ ^[0-9]+$ ]] || return 1
kill -0 "${pid}" 2>/dev/null || return 1
grep -qa "${SCRIPT_NAME}" "/proc/${pid}/cmdline" 2>/dev/null || return 1
printf '%s' "${pid}"
return 0
}
rotate_log_if_needed() {
local max_bytes size i
[ "${LOG_TO_FILE}" -eq 1 ] || return 0
[ "${MAX_LOG_SIZE_MB}" -gt 0 ] || return 0
[ -f "${LOG_FILE}" ] || return 0
max_bytes=$(( MAX_LOG_SIZE_MB * 1024 * 1024 ))
size="$(stat -c %s "${LOG_FILE}" 2>/dev/null || echo 0)"
[ "${size}" -ge "${max_bytes}" ] || return 0
for (( i = MAX_LOG_KEEP - 1; i >= 1; i-- )); do
[ -f "${LOG_FILE}.${i}" ] && mv -f "${LOG_FILE}.${i}" "${LOG_FILE}.$(( i + 1 ))"
done
mv -f "${LOG_FILE}" "${LOG_FILE}.1"
exec >>"${LOG_FILE}" 2>&1 # re-bind stdout/stderr to the new file
printf '[%s] INFO : log rotated at %s MB\n' "$(ts)" "${MAX_LOG_SIZE_MB}"
}
# Remove the rotated copies (.1 .. .N) only. Timestamped archives are kept.
remove_rotated_logs() {
local i
for (( i = 1; i <= MAX_LOG_KEEP + 1; i++ )); do
rm -f "${LOG_FILE}.${i}"
done
}
# Apply START_LOG_MODE just before a background run is launched.
prepare_log_on_start() {
local stamp
case "${START_LOG_MODE}" in
append)
return 0
;;
archive)
if [ -s "${LOG_FILE}" ]; then
stamp="$(date '+%Y%m%d-%H%M%S')"
mv -f "${LOG_FILE}" "${LOG_FILE}.${stamp}" \
|| die "cannot archive ${LOG_FILE}"
printf 'previous log archived: %s.%s\n' "${LOG_FILE}" "${stamp}"
fi
remove_rotated_logs
;;
new)
# Truncate rather than unlink: the inode is preserved, so a `tail -f`
# that is already attached keeps following the new run.
if [ -f "${LOG_FILE}" ]; then
: > "${LOG_FILE}" || die "cannot truncate ${LOG_FILE}"
fi
remove_rotated_logs
;;
*)
die "invalid START_LOG_MODE: ${START_LOG_MODE}"
;;
esac
}
run_one_command() {
local cmd="$1"
local out err rc t0 elapsed err_file
err_file="$(mktemp)"
t0=${SECONDS}
if [ "${CMD_TIMEOUT_SEC}" -gt 0 ]; then
out="$(timeout "${CMD_TIMEOUT_SEC}" \
"${RUNNER_BIN}" "${RUNNER_ARGS[@]}" "${cmd}" 2>"${err_file}")"
else
out="$("${RUNNER_BIN}" "${RUNNER_ARGS[@]}" "${cmd}" 2>"${err_file}")"
fi
rc=$?
elapsed=$(( SECONDS - t0 ))
err="$(cat "${err_file}" 2>/dev/null)"
rm -f "${err_file}"
printf '===== [%s] CMD: %s | rc=%d | %ds =====\n' "$(ts)" "${cmd}" "${rc}" "${elapsed}"
printf '%s\n' "${out}"
if [ "${rc}" -eq 124 ]; then
printf '[%s] WARN : timeout after %ds\n' "$(ts)" "${CMD_TIMEOUT_SEC}"
fi
if [ -n "${err}" ] && { [ "${LOG_STDERR}" -eq 1 ] || [ "${rc}" -ne 0 ]; }; then
printf -- '--- stderr ---\n%s\n' "${err}"
fi
printf '\n'
}
main_loop() {
local round=0 cmd
[ "${#COMMANDS[@]}" -gt 0 ] || die "COMMANDS is empty"
command -v "${RUNNER_BIN}" >/dev/null 2>&1 || die "runner not found: ${RUNNER_BIN}"
trap on_signal INT TERM
printf '#############################################################\n'
printf '[%s] START: pid=%d interval=%ds timeout=%ds cmds=%d\n' \
"$(ts)" "$$" "${INTERVAL_SEC}" "${CMD_TIMEOUT_SEC}" "${#COMMANDS[@]}"
printf '#############################################################\n'
while [ "${STOP}" -eq 0 ]; do
round=$(( round + 1 ))
printf -- '---------- ROUND %d @ %s ----------\n' "${round}" "$(ts)"
for cmd in "${COMMANDS[@]}"; do
[ "${STOP}" -eq 0 ] || break
run_one_command "${cmd}"
[ "${INTER_CMD_DELAY_SEC}" -gt 0 ] && interruptible_sleep "${INTER_CMD_DELAY_SEC}"
done
rotate_log_if_needed
if [ "${MAX_ROUNDS}" -gt 0 ] && [ "${round}" -ge "${MAX_ROUNDS}" ]; then
break
fi
# Fixed gap between rounds. For a fixed cadence instead, replace with:
# interruptible_sleep $(( INTERVAL_SEC - round_elapsed ))
[ "${STOP}" -eq 0 ] && interruptible_sleep "${INTERVAL_SEC}"
done
printf '[%s] STOP : pid=%d after %d round(s)\n' "$(ts)" "$$" "${round}"
}
###############################################################################
# Sub-commands
###############################################################################
do_start() {
local pid
pid="$(is_running)" && die "already running (pid=${pid})"
mkdir -p "${LOG_DIR}" || die "cannot create ${LOG_DIR}"
prepare_log_on_start
if command -v setsid >/dev/null 2>&1; then
setsid "${SCRIPT_PATH}" __daemon >/dev/null 2>&1 &
else
nohup "${SCRIPT_PATH}" __daemon >/dev/null 2>&1 &
disown 2>/dev/null
fi
sleep 1
pid="$(is_running)" || die "start failed, check ${LOG_FILE}"
printf 'started (pid=%s)\nlog: %s\n' "${pid}" "${LOG_FILE}"
}
do_daemon() {
mkdir -p "${LOG_DIR}" || die "cannot create ${LOG_DIR}"
LOG_TO_FILE=1
exec >>"${LOG_FILE}" 2>&1
printf '%d\n' "$$" > "${PID_FILE}"
trap 'rm -f "${PID_FILE}"' EXIT
main_loop
}
do_stop() {
local pid i
pid="$(is_running)" || { printf 'not running\n'; rm -f "${PID_FILE}"; return 0; }
kill -TERM -"${pid}" 2>/dev/null || kill -TERM "${pid}" 2>/dev/null
for (( i = 0; i < 20; i++ )); do
kill -0 "${pid}" 2>/dev/null || break
sleep 0.5
done
if kill -0 "${pid}" 2>/dev/null; then
printf 'SIGTERM ignored, sending SIGKILL\n'
kill -KILL -"${pid}" 2>/dev/null || kill -KILL "${pid}" 2>/dev/null
fi
rm -f "${PID_FILE}"
printf 'stopped (pid=%s)\n' "${pid}"
}
do_status() {
local pid
if pid="$(is_running)"; then
printf 'status : running (pid=%s)\n' "${pid}"
else
printf 'status : stopped\n'
fi
printf 'log : %s\n' "${LOG_FILE}"
[ -f "${LOG_FILE}" ] && printf 'size : %s\n' "$(du -h "${LOG_FILE}" | cut -f1)"
printf 'cmds : %d, interval=%ds\n' "${#COMMANDS[@]}" "${INTERVAL_SEC}"
}
usage() {
cat <<EOF
Usage: ${SCRIPT_NAME} {start|stop|status|fg|tail|clear}
start Run detached in the background; the previous log is discarded first
(see START_LOG_MODE)
stop Stop the background instance
status Show pid / log path / log size
fg Run in the foreground, output to the terminal (for a quick check)
tail tail -f the log file
clear Remove the log file and all rotated copies (must be stopped first)
Log file : ${LOG_FILE}
To add a command, edit the COMMANDS array at the top of this script.
EOF
}
###############################################################################
# Entry point
###############################################################################
case "${1:-}" in
start) do_start ;;
stop) do_stop ;;
status) do_status ;;
fg) LOG_TO_FILE=0; main_loop ;;
tail) tail -n 50 -f "${LOG_FILE}" ;;
clear)
is_running >/dev/null && die "still running, stop it first"
rm -f "${LOG_FILE}" "${LOG_FILE}".[0-9]*
printf 'log cleared\n'
;;
__daemon) do_daemon ;;
-h|--help|"") usage ;;
*) printf '[ERROR] unknown sub-command: %s\n\n' "$1" >&2; usage; exit 1 ;;
esac
@@ -0,0 +1,340 @@
#!/bin/bash
###############################################################################
# bmc_monitor_ddr.sh
#
# Version : V1.1.0
# Author : ETWen
# Date : 20260820
#
# Purpose : Periodically execute a list of commands on the BMC through
# `bmc-manager run "<cmd>"` from the SONiC host side, and append all
# output to a rotating log file. Designed to run detached in the
# background so no interactive SSH session to the BMC is required.
#
# Notes : - Everything runs on the HOST. Nothing is deployed to the BMC, so
# the BMC busybox toolchain (no base64 / no setsid ...) is a
# non-issue.
# - `bmc-manager run` prints the remote command output on stdout and
# its own banner on stderr; the two streams are captured separately
# so the log stays clean.
#
# Version History
# V1.0.0 20260820 Initial Version
# V1.1.0 20260820 `start` clears the previous log by default
# (START_LOG_MODE: new / archive / append)
###############################################################################
set -u
###############################################################################
# User Configurable Section -- normally this is the only part you need to edit
###############################################################################
# Commands executed on the BMC, one per line, in this order.
# Add a line to add a command; comment it out to disable it.
COMMANDS=(
"memtester 500M 1"
#"uptime"
#"cat /proc/loadavg"
#"cat /proc/meminfo"
#"df -h /tmp"
#"dmesg | tail -n 30"
)
# Seconds to wait after a full round (all COMMANDS executed) before the next one
INTERVAL_SEC=5
# Seconds to wait between two commands inside the same round (0 = no wait)
INTER_CMD_DELAY_SEC=0
# Per-command timeout in seconds; prevents a hung bmc-manager from freezing the
# whole loop. Set to 0 to disable.
CMD_TIMEOUT_SEC=3600
# Stop after this many rounds. 0 = run until stopped manually.
MAX_ROUNDS=0
# Command runner. Final invocation is:
# "$RUNNER_BIN" "${RUNNER_ARGS[@]}" "<command string>"
# BMC side : RUNNER_BIN="bmc-manager" ; RUNNER_ARGS=(run)
# BMC by IP : RUNNER_BIN="bmc-manager" ; RUNNER_ARGS=(--ip 192.168.200.200 run)
# Host side : RUNNER_BIN="bash" ; RUNNER_ARGS=(-c)
RUNNER_BIN="bmc-manager"
RUNNER_ARGS=(run)
# stderr handling of the runner (bmc-manager banner lives here)
# 0 = log stderr only when the command fails (recommended, keeps log clean)
# 1 = always log stderr
LOG_STDERR=0
# Logging
LOG_DIR="" # empty = <script directory>/log
LOG_NAME="bmc_ddr.log"
MAX_LOG_SIZE_MB=100 # rotate above this size; 0 = never rotate
MAX_LOG_KEEP=5 # keep .1 .. .N
# What `start` does with the log left behind by the previous run:
# new = begin with an empty log; previous content and rotated copies are
# discarded (default -- one run, one log)
# archive = rename the previous log to <log>.YYYYmmdd-HHMMSS, then begin empty
# append = keep appending to the existing log
START_LOG_MODE="new"
###############################################################################
# Internal
###############################################################################
SCRIPT_NAME="$(basename -- "${BASH_SOURCE[0]}")"
SCRIPT_PATH="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd)/${SCRIPT_NAME}"
[ -n "${LOG_DIR}" ] || LOG_DIR="$(dirname -- "${SCRIPT_PATH}")/log"
LOG_FILE="${LOG_DIR}/${LOG_NAME}"
PID_FILE="${LOG_DIR}/${LOG_NAME%.log}.pid"
LOG_TO_FILE=0
STOP=0
SLEEP_PID=""
ts() { date '+%Y-%m-%d %H:%M:%S'; }
die() { printf '[ERROR] %s\n' "$*" >&2; exit 1; }
on_signal() {
STOP=1
[ -n "${SLEEP_PID}" ] && kill "${SLEEP_PID}" 2>/dev/null
}
# sleep that can be aborted immediately by SIGTERM/SIGINT
interruptible_sleep() {
local sec="$1"
[ "${sec}" -gt 0 ] 2>/dev/null || return 0
sleep "${sec}" &
SLEEP_PID=$!
wait "${SLEEP_PID}" 2>/dev/null
SLEEP_PID=""
}
# echo the pid and return 0 when a live instance is found
is_running() {
local pid
[ -f "${PID_FILE}" ] || return 1
pid="$(cat "${PID_FILE}" 2>/dev/null)"
[[ "${pid}" =~ ^[0-9]+$ ]] || return 1
kill -0 "${pid}" 2>/dev/null || return 1
grep -qa "${SCRIPT_NAME}" "/proc/${pid}/cmdline" 2>/dev/null || return 1
printf '%s' "${pid}"
return 0
}
rotate_log_if_needed() {
local max_bytes size i
[ "${LOG_TO_FILE}" -eq 1 ] || return 0
[ "${MAX_LOG_SIZE_MB}" -gt 0 ] || return 0
[ -f "${LOG_FILE}" ] || return 0
max_bytes=$(( MAX_LOG_SIZE_MB * 1024 * 1024 ))
size="$(stat -c %s "${LOG_FILE}" 2>/dev/null || echo 0)"
[ "${size}" -ge "${max_bytes}" ] || return 0
for (( i = MAX_LOG_KEEP - 1; i >= 1; i-- )); do
[ -f "${LOG_FILE}.${i}" ] && mv -f "${LOG_FILE}.${i}" "${LOG_FILE}.$(( i + 1 ))"
done
mv -f "${LOG_FILE}" "${LOG_FILE}.1"
exec >>"${LOG_FILE}" 2>&1 # re-bind stdout/stderr to the new file
printf '[%s] INFO : log rotated at %s MB\n' "$(ts)" "${MAX_LOG_SIZE_MB}"
}
# Remove the rotated copies (.1 .. .N) only. Timestamped archives are kept.
remove_rotated_logs() {
local i
for (( i = 1; i <= MAX_LOG_KEEP + 1; i++ )); do
rm -f "${LOG_FILE}.${i}"
done
}
# Apply START_LOG_MODE just before a background run is launched.
prepare_log_on_start() {
local stamp
case "${START_LOG_MODE}" in
append)
return 0
;;
archive)
if [ -s "${LOG_FILE}" ]; then
stamp="$(date '+%Y%m%d-%H%M%S')"
mv -f "${LOG_FILE}" "${LOG_FILE}.${stamp}" \
|| die "cannot archive ${LOG_FILE}"
printf 'previous log archived: %s.%s\n' "${LOG_FILE}" "${stamp}"
fi
remove_rotated_logs
;;
new)
# Truncate rather than unlink: the inode is preserved, so a `tail -f`
# that is already attached keeps following the new run.
if [ -f "${LOG_FILE}" ]; then
: > "${LOG_FILE}" || die "cannot truncate ${LOG_FILE}"
fi
remove_rotated_logs
;;
*)
die "invalid START_LOG_MODE: ${START_LOG_MODE}"
;;
esac
}
run_one_command() {
local cmd="$1"
local out err rc t0 elapsed err_file
err_file="$(mktemp)"
t0=${SECONDS}
if [ "${CMD_TIMEOUT_SEC}" -gt 0 ]; then
out="$(timeout "${CMD_TIMEOUT_SEC}" \
"${RUNNER_BIN}" "${RUNNER_ARGS[@]}" "${cmd}" 2>"${err_file}")"
else
out="$("${RUNNER_BIN}" "${RUNNER_ARGS[@]}" "${cmd}" 2>"${err_file}")"
fi
rc=$?
elapsed=$(( SECONDS - t0 ))
err="$(cat "${err_file}" 2>/dev/null)"
rm -f "${err_file}"
printf '===== [%s] CMD: %s | rc=%d | %ds =====\n' "$(ts)" "${cmd}" "${rc}" "${elapsed}"
printf '%s\n' "${out}"
if [ "${rc}" -eq 124 ]; then
printf '[%s] WARN : timeout after %ds\n' "$(ts)" "${CMD_TIMEOUT_SEC}"
fi
if [ -n "${err}" ] && { [ "${LOG_STDERR}" -eq 1 ] || [ "${rc}" -ne 0 ]; }; then
printf -- '--- stderr ---\n%s\n' "${err}"
fi
printf '\n'
}
main_loop() {
local round=0 cmd
[ "${#COMMANDS[@]}" -gt 0 ] || die "COMMANDS is empty"
command -v "${RUNNER_BIN}" >/dev/null 2>&1 || die "runner not found: ${RUNNER_BIN}"
trap on_signal INT TERM
printf '#############################################################\n'
printf '[%s] START: pid=%d interval=%ds timeout=%ds cmds=%d\n' \
"$(ts)" "$$" "${INTERVAL_SEC}" "${CMD_TIMEOUT_SEC}" "${#COMMANDS[@]}"
printf '#############################################################\n'
while [ "${STOP}" -eq 0 ]; do
round=$(( round + 1 ))
printf -- '---------- ROUND %d @ %s ----------\n' "${round}" "$(ts)"
for cmd in "${COMMANDS[@]}"; do
[ "${STOP}" -eq 0 ] || break
run_one_command "${cmd}"
[ "${INTER_CMD_DELAY_SEC}" -gt 0 ] && interruptible_sleep "${INTER_CMD_DELAY_SEC}"
done
rotate_log_if_needed
if [ "${MAX_ROUNDS}" -gt 0 ] && [ "${round}" -ge "${MAX_ROUNDS}" ]; then
break
fi
# Fixed gap between rounds. For a fixed cadence instead, replace with:
# interruptible_sleep $(( INTERVAL_SEC - round_elapsed ))
[ "${STOP}" -eq 0 ] && interruptible_sleep "${INTERVAL_SEC}"
done
printf '[%s] STOP : pid=%d after %d round(s)\n' "$(ts)" "$$" "${round}"
}
###############################################################################
# Sub-commands
###############################################################################
do_start() {
local pid
pid="$(is_running)" && die "already running (pid=${pid})"
mkdir -p "${LOG_DIR}" || die "cannot create ${LOG_DIR}"
prepare_log_on_start
if command -v setsid >/dev/null 2>&1; then
setsid "${SCRIPT_PATH}" __daemon >/dev/null 2>&1 &
else
nohup "${SCRIPT_PATH}" __daemon >/dev/null 2>&1 &
disown 2>/dev/null
fi
sleep 1
pid="$(is_running)" || die "start failed, check ${LOG_FILE}"
printf 'started (pid=%s)\nlog: %s\n' "${pid}" "${LOG_FILE}"
}
do_daemon() {
mkdir -p "${LOG_DIR}" || die "cannot create ${LOG_DIR}"
LOG_TO_FILE=1
exec >>"${LOG_FILE}" 2>&1
printf '%d\n' "$$" > "${PID_FILE}"
trap 'rm -f "${PID_FILE}"' EXIT
main_loop
}
do_stop() {
local pid i
pid="$(is_running)" || { printf 'not running\n'; rm -f "${PID_FILE}"; return 0; }
kill -TERM -"${pid}" 2>/dev/null || kill -TERM "${pid}" 2>/dev/null
for (( i = 0; i < 20; i++ )); do
kill -0 "${pid}" 2>/dev/null || break
sleep 0.5
done
if kill -0 "${pid}" 2>/dev/null; then
printf 'SIGTERM ignored, sending SIGKILL\n'
kill -KILL -"${pid}" 2>/dev/null || kill -KILL "${pid}" 2>/dev/null
fi
rm -f "${PID_FILE}"
printf 'stopped (pid=%s)\n' "${pid}"
}
do_status() {
local pid
if pid="$(is_running)"; then
printf 'status : running (pid=%s)\n' "${pid}"
else
printf 'status : stopped\n'
fi
printf 'log : %s\n' "${LOG_FILE}"
[ -f "${LOG_FILE}" ] && printf 'size : %s\n' "$(du -h "${LOG_FILE}" | cut -f1)"
printf 'cmds : %d, interval=%ds\n' "${#COMMANDS[@]}" "${INTERVAL_SEC}"
}
usage() {
cat <<EOF
Usage: ${SCRIPT_NAME} {start|stop|status|fg|tail|clear}
start Run detached in the background; the previous log is discarded first
(see START_LOG_MODE)
stop Stop the background instance
status Show pid / log path / log size
fg Run in the foreground, output to the terminal (for a quick check)
tail tail -f the log file
clear Remove the log file and all rotated copies (must be stopped first)
Log file : ${LOG_FILE}
To add a command, edit the COMMANDS array at the top of this script.
EOF
}
###############################################################################
# Entry point
###############################################################################
case "${1:-}" in
start) do_start ;;
stop) do_stop ;;
status) do_status ;;
fg) LOG_TO_FILE=0; main_loop ;;
tail) tail -n 50 -f "${LOG_FILE}" ;;
clear)
is_running >/dev/null && die "still running, stop it first"
rm -f "${LOG_FILE}" "${LOG_FILE}".[0-9]*
printf 'log cleared\n'
;;
__daemon) do_daemon ;;
-h|--help|"") usage ;;
*) printf '[ERROR] unknown sub-command: %s\n\n' "$1" >&2; usage; exit 1 ;;
esac
@@ -0,0 +1,403 @@
#!/bin/bash
###############################################################################
# mgmt_ping_monitor.sh
#
# Version : V3.0.0
# Author : ETWen
# Date : 20260826
#
# Purpose : Ping from BOTH management NICs at the same time, continuously, and
# keep accumulating until stopped:
#
# 10G (eth0) -> TARGET_10G
# 1G (eth1) -> TARGET_1G
#
# `stop` renders a report into the log: the last TAIL_LINES entries
# per NIC, each one's ping statistics and verdict, then
# `ip -s link show` for both interfaces.
#
# Notes : - Both NICs are configured with iproute2 `replace`, which is
# idempotent, and both are left up. On a shared subnet the target's
# ARP can be answered by either NIC, so ARP_STRICT keeps each one
# to its own address.
# - ARP WARM-UP: the first packet after a neighbour entry expires is
# spent resolving ARP and is counted as loss. On the bench this
# produced three FAILs whose only missing packet was icmp_seq=1,
# every time, with zero NIC errors or drops. One discarded ping
# before the measured run removes that artefact, which is what lets
# MAX_LOSS_PCT stay at 0 and still mean something.
# - MANAGEMENT CONNECTIVITY IS IN USE while this runs. Drive it from
# the serial console.
#
# Version History
# V1.0.0 20260824 Initial Version (ifconfig, one NIC at a time)
# V2.0.0 20260824 iproute2; both NICs stay up; per-NIC target;
# ip -s link show + TX/RX delta per leg
# V3.0.0 20260826 Both NICs ping SIMULTANEOUSLY and continuously instead
# of alternating fixed bursts. Add the ARP warm-up.
# `stop` renders last-N per NIC + statistics + both
# ip -s link outputs into the log.
###############################################################################
set -u
###############################################################################
# User Configurable Section -- normally this is the only part you need to edit
###############################################################################
# --- 10G management port ---
IF_10G="eth0"
IP_10G="192.168.1.99"
TARGET_10G="192.168.1.30" # host this NIC pings
PLEN_10G=24 # prefix length, e.g. 24 = /24
GW_10G="" # default gateway; empty = do not touch routing
METRIC_10G=100 # lower metric wins for off-subnet traffic
# --- 1G management port ---
IF_1G="eth1"
IP_1G="192.168.1.101"
TARGET_1G="192.168.1.31"
PLEN_1G=24
GW_1G=""
METRIC_1G=200
# --- ping ---
PING_INTERVAL_SEC=1 # seconds between echo requests (ping -i)
MAX_LOSS_PCT=0 # loss above this marks the NIC FAIL
TAIL_LINES=20 # how many recent entries per NIC the report shows
# Extra ping flags. -D timestamps every line, -O prints a marker for a request
# that got no reply, so a drop is visible in the tail instead of just missing.
# Clear this if the platform's ping does not accept them.
PING_EXTRA_OPTS="-D -O"
# One discarded ping per NIC before the measured run, to resolve ARP. Without
# it the first packet of the run is lost to neighbour resolution and looks
# exactly like a link fault. 0 disables.
ARP_WARMUP=1
ARP_WARMUP_TIMEOUT=2 # seconds to wait for the warm-up reply
# Both NICs stay up. If they share a subnet the target's ARP can be answered by
# either one, so a reply may arrive on the NIC that did not send. This applies
# arp_ignore=1 / arp_announce=2 to both. RAM only -- reverts on reboot.
ARP_STRICT=1
# Prefix for the ip commands; empty because this already runs as root.
SUDO=""
# Seconds to wait after bringing a link up before pinging.
LINK_SETTLE_SEC=5
# Logging
LOG_DIR="" # empty = <script directory>/log
LOG_NAME="mgmt_ping.log"
# What `start` does with the log left behind by the previous run:
# new = discard it (default -- one run, one log)
# archive = rename it to <log>.YYYYmmdd-HHMMSS first
# append = keep it
START_LOG_MODE="new"
###############################################################################
# Internal
###############################################################################
SCRIPT_NAME="$(basename -- "${BASH_SOURCE[0]}")"
SCRIPT_DIR="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd)"
[ -n "${LOG_DIR}" ] || LOG_DIR="${SCRIPT_DIR}/log"
LOG_FILE="${LOG_DIR}/${LOG_NAME}"
BASE="${LOG_DIR}/${LOG_NAME%.log}"
PID_FILE="${BASE}.pid"
STATE_FILE="${BASE}.state"
RAW_10G="${BASE}_${IF_10G}.raw"
RAW_1G="${BASE}_${IF_1G}.raw"
ts() { date '+%Y-%m-%d %H:%M:%S'; }
die() { printf '[ERROR] %s\n' "$*" >&2; exit 1; }
# "<rx_packets> <tx_packets>" for an interface. ip -s link orders the columns
# "bytes packets errors ...", so packets is $2, not $1.
if_counters() {
${SUDO} ip -s link show "$1" 2>/dev/null | awk '
/RX:/ { getline; rx = $2 }
/TX:/ { getline; tx = $2 }
END { printf "%s %s", (rx == "" ? 0 : rx), (tx == "" ? 0 : tx) }'
}
# Bring the NIC up and (re)apply its address and default route.
setup_iface() {
local iface="$1" ipaddr="$2" plen="$3" gw="$4" metric="$5"
${SUDO} ip link set "${iface}" up 2>&1 || return 1
${SUDO} ip address replace "${ipaddr}/${plen}" dev "${iface}" 2>&1 || return 1
if [ -n "${gw}" ]; then
${SUDO} ip route replace default via "${gw}" dev "${iface}" \
metric "${metric}" 2>&1 || return 1
fi
return 0
}
apply_arp_strict() {
local i
[ "${ARP_STRICT}" -eq 1 ] || return 0
for i in "${IF_10G}" "${IF_1G}"; do
${SUDO} sysctl -qw "net.ipv4.conf.${i}.arp_ignore=1" 2>/dev/null
${SUDO} sysctl -qw "net.ipv4.conf.${i}.arp_announce=2" 2>/dev/null
done
}
# Resolve the neighbour so the measured run does not spend its first packet on
# ARP. Result is deliberately discarded.
arp_warmup() {
local iface="$1" target="$2"
[ "${ARP_WARMUP}" -eq 1 ] || return 0
ping -I "${iface}" -c 1 -W "${ARP_WARMUP_TIMEOUT}" "${target}" >/dev/null 2>&1
return 0
}
# echo "pid_10g pid_1g" and return 0 when both are alive
is_running() {
local p0 p1
[ -f "${PID_FILE}" ] || return 1
p0="$(sed -n '1p' "${PID_FILE}" 2>/dev/null)"
p1="$(sed -n '2p' "${PID_FILE}" 2>/dev/null)"
[[ "${p0}" =~ ^[0-9]+$ ]] || return 1
[[ "${p1}" =~ ^[0-9]+$ ]] || return 1
kill -0 "${p0}" 2>/dev/null || kill -0 "${p1}" 2>/dev/null || return 1
printf '%s %s' "${p0}" "${p1}"
return 0
}
###############################################################################
# Report
###############################################################################
# The recent entries for one NIC: replies and, thanks to -O, the requests that
# got none.
tail_entries() {
grep -aE 'bytes from|no answer|Unreachable|Time to live' "$1" 2>/dev/null \
| tail -n "${TAIL_LINES}"
}
# The trailing "--- x ping statistics ---" block.
stats_block() {
sed -n '/ping statistics ---/,$p' "$1" 2>/dev/null
}
# render_leg <label> <iface> <target> <raw> <rx0> <tx0>
render_leg() {
local label="$1" iface="$2" target="$3" raw="$4" rx0="$5" tx0="$6"
local st tx rx loss avg verdict c1 rx1 tx1 drx dtx
printf -- '----- %s : %s -> %s : last %s entries -----\n' \
"${label}" "${iface}" "${target}" "${TAIL_LINES}"
tail_entries "${raw}"
printf '\n'
st="$(stats_block "${raw}")"
printf '%s\n' "${st}"
tx="$(printf '%s' "${st}" | sed -n 's/^\([0-9]\+\) packets transmitted.*/\1/p' | tail -1)"
rx="$(printf '%s' "${st}" | sed -n 's/.*[, ]\([0-9]\+\) received.*/\1/p' | tail -1)"
loss="$(printf '%s' "${st}" | sed -n 's/.*[, ]\([0-9]\+\)% packet loss.*/\1/p' | tail -1)"
avg="$(printf '%s' "${st}" | sed -n 's|.*= [0-9.]*/\([0-9.]*\)/.*|\1|p' | tail -1)"
[ -n "${tx}" ] || tx=0
[ -n "${rx}" ] || rx=0
[ -n "${loss}" ] || loss=100
[ -n "${avg}" ] || avg="-"
c1="$(if_counters "${iface}")"; rx1="${c1% *}"; tx1="${c1#* }"
drx=$(( rx1 - rx0 )); dtx=$(( tx1 - tx0 ))
if [ "${loss}" -le "${MAX_LOSS_PCT}" ] && [ "${tx}" -gt 0 ]; then
verdict="PASS"
else
verdict="FAIL"
fi
# nic_tx/nic_rx are this interface's own counter delta over the whole run.
# A pass with nic_tx near zero means the traffic left on the other NIC.
printf '[%s] RESULT %-3s %-6s -> %-15s tx=%s rx=%s loss=%s%% rtt_avg=%sms nic_tx=%s nic_rx=%s %s\n\n' \
"$(ts)" "${label}" "${iface}" "${target}" "${tx}" "${rx}" "${loss}" "${avg}" \
"${dtx}" "${drx}" "${verdict}"
}
write_report() {
local rx0_10g tx0_10g rx0_1g tx0_1g started
started="$(sed -n '1p' "${STATE_FILE}" 2>/dev/null)"
rx0_10g="$(sed -n '2p' "${STATE_FILE}" 2>/dev/null)"; rx0_10g="${rx0_10g:-0}"
tx0_10g="$(sed -n '3p' "${STATE_FILE}" 2>/dev/null)"; tx0_10g="${tx0_10g:-0}"
rx0_1g="$(sed -n '4p' "${STATE_FILE}" 2>/dev/null)"; rx0_1g="${rx0_1g:-0}"
tx0_1g="$(sed -n '5p' "${STATE_FILE}" 2>/dev/null)"; tx0_1g="${tx0_1g:-0}"
{
printf '#############################################################\n'
printf '[%s] mgmt ping report\n' "$(ts)"
printf ' started : %s\n' "${started:-unknown}"
printf ' stopped : %s\n' "$(ts)"
printf ' 10G : %s %s/%s -> %s (metric %s)\n' \
"${IF_10G}" "${IP_10G}" "${PLEN_10G}" "${TARGET_10G}" "${METRIC_10G}"
printf ' 1G : %s %s/%s -> %s (metric %s)\n' \
"${IF_1G}" "${IP_1G}" "${PLEN_1G}" "${TARGET_1G}" "${METRIC_1G}"
printf ' ping : -i %s %s, max loss %s%%\n' \
"${PING_INTERVAL_SEC}" "${PING_EXTRA_OPTS}" "${MAX_LOSS_PCT}"
printf '#############################################################\n\n'
render_leg "10G" "${IF_10G}" "${TARGET_10G}" "${RAW_10G}" "${rx0_10g}" "${tx0_10g}"
render_leg "1G" "${IF_1G}" "${TARGET_1G}" "${RAW_1G}" "${rx0_1g}" "${tx0_1g}"
printf -- '--- ip -s link show %s ---\n' "${IF_10G}"
${SUDO} ip -s link show "${IF_10G}" 2>&1
printf '\n'
printf -- '--- ip -s link show %s ---\n' "${IF_1G}"
${SUDO} ip -s link show "${IF_1G}" 2>&1
} >> "${LOG_FILE}"
}
###############################################################################
# Sub-commands
###############################################################################
prepare_log_on_start() {
local stamp
case "${START_LOG_MODE}" in
append) return 0 ;;
archive)
if [ -s "${LOG_FILE}" ]; then
stamp="$(date '+%Y%m%d-%H%M%S')"
mv -f "${LOG_FILE}" "${LOG_FILE}.${stamp}" || die "cannot archive ${LOG_FILE}"
printf 'previous log archived: %s.%s\n' "${LOG_FILE}" "${stamp}"
fi
;;
new)
# Truncate rather than unlink, so a tail -f already attached keeps
# following the new run.
[ -f "${LOG_FILE}" ] && { : > "${LOG_FILE}" || die "cannot truncate ${LOG_FILE}"; }
;;
*) die "invalid START_LOG_MODE: ${START_LOG_MODE}" ;;
esac
}
do_start() {
local pids p0 p1 c
pids="$(is_running)" && die "already running (pids=${pids})"
command -v ip >/dev/null 2>&1 || die "ip (iproute2) not found"
command -v ping >/dev/null 2>&1 || die "ping not found"
mkdir -p "${LOG_DIR}" || die "cannot create ${LOG_DIR}"
prepare_log_on_start
: > "${RAW_10G}"
: > "${RAW_1G}"
setup_iface "${IF_10G}" "${IP_10G}" "${PLEN_10G}" "${GW_10G}" "${METRIC_10G}" \
|| die "cannot configure ${IF_10G}"
setup_iface "${IF_1G}" "${IP_1G}" "${PLEN_1G}" "${GW_1G}" "${METRIC_1G}" \
|| die "cannot configure ${IF_1G}"
apply_arp_strict
sleep "${LINK_SETTLE_SEC}"
# Spend the ARP resolution here, not on the first measured packet.
arp_warmup "${IF_10G}" "${TARGET_10G}"
arp_warmup "${IF_1G}" "${TARGET_1G}"
# Counter baseline, taken after the warm-up so its packets are excluded.
{ c="$(if_counters "${IF_10G}")"
printf '%s\n%s\n%s\n' "$(ts)" "${c% *}" "${c#* }"
c="$(if_counters "${IF_1G}")"
printf '%s\n%s\n' "${c% *}" "${c#* }"
} > "${STATE_FILE}"
# Both NICs ping at the same time and keep accumulating until stopped.
nohup ping -I "${IF_10G}" -i "${PING_INTERVAL_SEC}" ${PING_EXTRA_OPTS} \
"${TARGET_10G}" >> "${RAW_10G}" 2>&1 &
p0=$!
nohup ping -I "${IF_1G}" -i "${PING_INTERVAL_SEC}" ${PING_EXTRA_OPTS} \
"${TARGET_1G}" >> "${RAW_1G}" 2>&1 &
p1=$!
disown "${p0}" 2>/dev/null
disown "${p1}" 2>/dev/null
printf '%s\n%s\n' "${p0}" "${p1}" > "${PID_FILE}"
sleep 1
kill -0 "${p0}" 2>/dev/null || die "10G ping failed to start, see ${RAW_10G}"
kill -0 "${p1}" 2>/dev/null || die "1G ping failed to start, see ${RAW_1G}"
printf 'started (10G pid=%s, 1G pid=%s)\nlog: %s\n' "${p0}" "${p1}" "${LOG_FILE}"
}
do_stop() {
local pids p0 p1 i
pids="$(is_running)" || {
printf 'not running\n'
rm -f "${PID_FILE}"
return 0
}
p0="${pids% *}"; p1="${pids#* }"
# SIGINT, not SIGTERM: ping prints its statistics block on interrupt, and
# that block is what the report parses.
kill -INT "${p0}" 2>/dev/null
kill -INT "${p1}" 2>/dev/null
for (( i = 0; i < 20; i++ )); do
kill -0 "${p0}" 2>/dev/null || kill -0 "${p1}" 2>/dev/null || break
sleep 0.5
done
kill -KILL "${p0}" 2>/dev/null
kill -KILL "${p1}" 2>/dev/null
write_report
rm -f "${PID_FILE}"
printf 'stopped (10G pid=%s, 1G pid=%s)\nreport: %s\n' "${p0}" "${p1}" "${LOG_FILE}"
}
do_status() {
local pids
if pids="$(is_running)"; then
printf 'status : running (10G pid=%s, 1G pid=%s)\n' "${pids% *}" "${pids#* }"
else
printf 'status : stopped\n'
fi
printf 'log : %s\n' "${LOG_FILE}"
printf '10G : %s %s/%s -> %s\n' "${IF_10G}" "${IP_10G}" "${PLEN_10G}" "${TARGET_10G}"
printf '1G : %s %s/%s -> %s\n' "${IF_1G}" "${IP_1G}" "${PLEN_1G}" "${TARGET_1G}"
[ -f "${RAW_10G}" ] && printf '10G recv: %s\n' "$(grep -ac 'bytes from' "${RAW_10G}")"
[ -f "${RAW_1G}" ] && printf '1G recv: %s\n' "$(grep -ac 'bytes from' "${RAW_1G}")"
}
usage() {
cat <<EOF
Usage: ${SCRIPT_NAME} {start|stop|status|summary|clear}
start Configure both NICs, warm up ARP, then ping from BOTH at the same
time and keep accumulating. The previous log is discarded first.
stop Stop both pings and render the report into the log
status Show pids, configured NICs and replies received so far
summary Print just the RESULT lines from the log
clear Remove the log and the raw captures (must be stopped first)
Log file : ${LOG_FILE}
The report holds, per NIC, the last ${TAIL_LINES} entries and the ping
statistics, then "ip -s link show" for both interfaces.
Both NICs are used at once, so management connectivity is in play -- drive
this from the serial console. Edit the User Configurable Section at the top
to change interfaces, addresses, targets or the ping options.
EOF
}
###############################################################################
# Entry point
###############################################################################
case "${1:-}" in
start) do_start ;;
stop) do_stop ;;
status) do_status ;;
summary) grep -a 'RESULT' "${LOG_FILE}" 2>/dev/null || printf 'no results in %s\n' "${LOG_FILE}" ;;
clear)
is_running >/dev/null && die "still running, stop it first"
rm -f "${LOG_FILE}" "${LOG_FILE}".[0-9]* "${RAW_10G}" "${RAW_1G}" "${STATE_FILE}"
printf 'log cleared\n'
;;
-h|--help|"") usage ;;
*) printf '[ERROR] unknown sub-command: %s\n\n' "$1" >&2; usage; exit 1 ;;
esac
@@ -0,0 +1,431 @@
#!/bin/bash
###############################################################################
# port_prbs_monitor.sh
#
# Version : V1.0.0
# Author : ETWen
# Date : 20260826
#
# Purpose : Run a PRBS test on the two 100G uplinks AT THE SAME TIME and keep
# polling until stopped. Each port gets its own background worker:
#
# setup : sfputil lpmode off <port>
# phy diag <phy> prbs set <poly>
# phy diag <phy> prbsstat STArt Interval=<n>
# poll : phy diag <phy> prbs get <- PASS/FAIL comes from here
# phydiag <phy> prbsstat Ber <- recorded only
# stop : phydiag <phy> prbsstat STOp
# phydiag <phy> prbs clear
#
# Every command and its full output goes to that port's raw log.
# `stop` tears the test down and renders the report; `report`
# prints the PASS/FAIL tally per port.
#
# Notes : - A poll counts as PASS only when the output contains "PRBS OK!".
# bcmcmd exits 0 even when the BCM shell rejects a command, so the
# exit status cannot be used -- only the output can.
# - bcmcmd is always run with </dev/null. Without it, it inherits the
# polling loop's stdin and eats it, and the loop runs once.
#
# Version History
# V1.0.0 20260826 Initial Version
###############################################################################
set -u
###############################################################################
# User Configurable Section -- normally this is the only part you need to edit
###############################################################################
# --- port A ---
PORT_A_NAME="Ethernet513"
PORT_A_UNIT=0 # bcmcmd -n <unit>
PORT_A_PHY=268 # phy diag <phy>
# --- port B ---
PORT_B_NAME="Ethernet514"
PORT_B_UNIT=1
PORT_B_PHY=268
# Which ports `start` runs when no argument is given: both | a | b
# Override per run: start a / start b / start both
# Running one at a time is how you tell a genuine port fault from the DUT not
# coping with two PRBS streams at once.
PORTS="both"
# --- PRBS ---
PRBS_POLY="p=3" # passed to "prbs set"
PRBS_STAT_INTERVAL=5 # passed to "prbsstat STArt Interval="
POLL_INTERVAL_SEC=10 # seconds between polls
PASS_PATTERN="PRBS OK!" # a poll is PASS only if the output contains this
# Seconds to wait after the setup commands before the first poll, so the link
# has settled and the counters mean something.
SETTLE_SEC=10
# Stop after this many polls per port. 0 = run until stopped manually.
MAX_POLLS=0
# How many recent poll blocks the report shows per port.
TAIL_ENTRIES=20
# Prefix for privileged commands; empty because this already runs as root.
SUDO=""
BCMCMD="bcmcmd"
# Logging
LOG_DIR="" # empty = <script directory>/log
LOG_NAME="port_prbs.log"
# What `start` does with the log left behind by the previous run:
# new | archive | append
START_LOG_MODE="new"
###############################################################################
# Internal
###############################################################################
SCRIPT_NAME="$(basename -- "${BASH_SOURCE[0]}")"
SCRIPT_DIR="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd)"
SCRIPT_PATH="${SCRIPT_DIR}/${SCRIPT_NAME}"
[ -n "${LOG_DIR}" ] || LOG_DIR="${SCRIPT_DIR}/log"
LOG_FILE="${LOG_DIR}/${LOG_NAME}"
BASE="${LOG_DIR}/${LOG_NAME%.log}"
PID_FILE="${BASE}.pid"
STATE_FILE="${BASE}.state"
RAW_A="${BASE}_${PORT_A_NAME}.raw"
RAW_B="${BASE}_${PORT_B_NAME}.raw"
ts() { date '+%Y-%m-%d %H:%M:%S'; }
# port_tally <name> <raw> -> "<polls> <pass> <fail> <verdict>"
# A raw without a "PRBS START" line means the port was not run this session,
# which is SKIP -- reporting it as FAIL would make a deliberate single-port
# run look like half the hardware is broken.
port_tally() {
local name="$1" raw="$2" pass fail polls verdict
if [ ! -s "${raw}" ] || ! grep -aq "PRBS START ${name}" "${raw}" 2>/dev/null; then
printf '0 0 0 SKIP'; return 0
fi
pass=$(grep -ac "PRBS ${name} poll=[0-9]* PASS" "${raw}" 2>/dev/null); pass=${pass:-0}
fail=$(grep -ac "PRBS ${name} poll=[0-9]* FAIL" "${raw}" 2>/dev/null); fail=${fail:-0}
polls=$(( pass + fail ))
if [ "${polls}" -gt 0 ] && [ "${fail}" -eq 0 ]; then verdict=PASS; else verdict=FAIL; fi
printf '%s %s %s %s' "${polls}" "${pass}" "${fail}" "${verdict}"
}
die() { printf '[ERROR] %s\n' "$*" >&2; exit 1; }
# bcm <unit> <dsh command> -- output on stdout, never trusts the exit status.
# </dev/null so it cannot consume the caller's stdin.
bcm() {
local unit="$1" cmd="$2"
${SUDO} "${BCMCMD}" -n "${unit}" -c "dsh -c \"${cmd}\"" </dev/null 2>&1
}
# run_step <unit> <dsh command>
# The command and its full output are written to the LOG via stderr, because
# the worker has stderr pointed at the raw file. Only the output itself goes to
# stdout, so `out="$(run_step ...)"` captures the output without swallowing the
# log lines -- writing both to stdout would have put the whole record inside
# the variable and left the log with nothing but the poll verdicts.
run_step() {
local unit="$1" cmd="$2" out
out="$(bcm "${unit}" "${cmd}")"
{
printf '[%s] CMD : bcmcmd -n %s -c '\''dsh -c "%s"'\''\n' "$(ts)" "${unit}" "${cmd}"
printf '%s\n' "${out}"
} >&2
printf '%s' "${out}"
}
# Echo "<pid_a> <pid_b>" and return 0 while at least one worker is alive. A
# port that was not started this run is recorded as "-", so a single-port run
# is a first-class case rather than a half-broken two-port one.
is_running() {
local pa pb alive=0
[ -f "${PID_FILE}" ] || return 1
pa="$(sed -n '1p' "${PID_FILE}" 2>/dev/null)"; pa="${pa:--}"
pb="$(sed -n '2p' "${PID_FILE}" 2>/dev/null)"; pb="${pb:--}"
[[ "${pa}" =~ ^[0-9]+$ ]] && kill -0 "${pa}" 2>/dev/null && alive=1
[[ "${pb}" =~ ^[0-9]+$ ]] && kill -0 "${pb}" 2>/dev/null && alive=1
[ "${alive}" -eq 1 ] || return 1
printf '%s %s' "${pa}" "${pb}"
return 0
}
###############################################################################
# Worker -- one per port, launched by `start`
###############################################################################
WORKER_STOP=0
worker_on_signal() { WORKER_STOP=1; }
# __worker <name> <unit> <phy> <rawfile>
do_worker() {
local name="$1" unit="$2" phy="$3" raw="$4"
local n=0 pass=0 fail=0 out
exec >>"${raw}" 2>&1
trap worker_on_signal INT TERM
printf '#############################################################\n'
printf '[%s] PRBS START %s (unit %s, phy %s) poly=%s interval=%ss poll=%ss\n' \
"$(ts)" "${name}" "${unit}" "${phy}" "${PRBS_POLY}" \
"${PRBS_STAT_INTERVAL}" "${POLL_INTERVAL_SEC}"
printf '#############################################################\n'
# --- setup ---
printf '[%s] CMD : sfputil lpmode off %s\n' "$(ts)" "${name}"
${SUDO} sfputil lpmode off "${name}" </dev/null 2>&1
run_step "${unit}" "phy diag ${phy} prbs set ${PRBS_POLY}" >/dev/null
run_step "${unit}" "phy diag ${phy} prbsstat STArt Interval=${PRBS_STAT_INTERVAL}" >/dev/null
sleep "${SETTLE_SEC}"
# --- poll ---
while [ "${WORKER_STOP}" -eq 0 ]; do
n=$(( n + 1 ))
printf -- '---------- %s POLL %d @ %s ----------\n' "${name}" "${n}" "$(ts)"
out="$(run_step "${unit}" "phy diag ${phy} prbs get")"
# Ber is recorded for the log only; it does not decide the verdict.
run_step "${unit}" "phydiag ${phy} prbsstat Ber" >/dev/null
if printf '%s' "${out}" | grep -qF "${PASS_PATTERN}"; then
pass=$(( pass + 1 ))
printf '[%s] PRBS %s poll=%d PASS\n' "$(ts)" "${name}" "${n}"
else
fail=$(( fail + 1 ))
printf '[%s] PRBS %s poll=%d FAIL\n' "$(ts)" "${name}" "${n}"
fi
[ "${MAX_POLLS}" -gt 0 ] && [ "${n}" -ge "${MAX_POLLS}" ] && break
[ "${WORKER_STOP}" -eq 0 ] || break
sleep "${POLL_INTERVAL_SEC}"
done
# --- teardown ---
run_step "${unit}" "phydiag ${phy} prbsstat STOp" >/dev/null
run_step "${unit}" "phydiag ${phy} prbs clear" >/dev/null
printf '[%s] PRBS STOP %s polls=%d PASS=%d FAIL=%d\n' \
"$(ts)" "${name}" "${n}" "${pass}" "${fail}"
}
###############################################################################
# Report
###############################################################################
# render_port <name> <unit> <phy> <raw>
render_port() {
local name="$1" unit="$2" phy="$3" raw="$4"
local t polls pass fail verdict
t="$(port_tally "${name}" "${raw}")"
polls="$(printf '%s' "${t}" | awk '{print $1}')"
pass="$(printf '%s' "${t}" | awk '{print $2}')"
fail="$(printf '%s' "${t}" | awk '{print $3}')"
verdict="$(printf '%s' "${t}" | awk '{print $4}')"
if [ "${verdict}" = "SKIP" ]; then
printf -- '----- %s (unit %s, phy %s) : not run this session -----\n\n' \
"${name}" "${unit}" "${phy}"
printf '[%s] RESULT %-12s polls=%-4s PASS=%-4s FAIL=%-4s %s\n\n' \
"$(ts)" "${name}" "${polls}" "${pass}" "${fail}" "${verdict}"
return 0
fi
printf -- '----- %s (unit %s, phy %s) : last %s poll blocks -----\n' \
"${name}" "${unit}" "${phy}" "${TAIL_ENTRIES}"
grep -aE "^-{10} ${name} POLL|PRBS ${name} poll=|prbsstat Ber|^ *[0-9]+ *: " "${raw}" 2>/dev/null \
| tail -n $(( TAIL_ENTRIES * 3 ))
printf '\n'
printf '[%s] RESULT %-12s polls=%-4s PASS=%-4s FAIL=%-4s %s\n\n' \
"$(ts)" "${name}" "${polls}" "${pass}" "${fail}" "${verdict}"
}
write_report() {
local started
started="$(sed -n '1p' "${STATE_FILE}" 2>/dev/null)"
{
printf '#############################################################\n'
printf '[%s] 100G PRBS report\n' "$(ts)"
printf ' started : %s\n' "${started:-unknown}"
printf ' stopped : %s\n' "$(ts)"
printf ' %s : unit %s, phy %s\n' "${PORT_A_NAME}" "${PORT_A_UNIT}" "${PORT_A_PHY}"
printf ' %s : unit %s, phy %s\n' "${PORT_B_NAME}" "${PORT_B_UNIT}" "${PORT_B_PHY}"
printf ' poly=%s prbsstat Interval=%s poll every %ss\n' \
"${PRBS_POLY}" "${PRBS_STAT_INTERVAL}" "${POLL_INTERVAL_SEC}"
printf ' a poll is PASS only when the output contains "%s"\n' "${PASS_PATTERN}"
printf '#############################################################\n\n'
render_port "${PORT_A_NAME}" "${PORT_A_UNIT}" "${PORT_A_PHY}" "${RAW_A}"
render_port "${PORT_B_NAME}" "${PORT_B_UNIT}" "${PORT_B_PHY}" "${RAW_B}"
# Taken after prbsstat STOp + prbs clear, so this is the recovered
# state. During the test the same command would have reported DOWN.
printf -- '--- show interfaces status %s,%s (after prbsstat STOp + prbs clear) ---\n' \
"${PORT_A_NAME}" "${PORT_B_NAME}"
${SUDO} show interfaces status "${PORT_A_NAME},${PORT_B_NAME}" </dev/null 2>&1
} >> "${LOG_FILE}"
}
###############################################################################
# Sub-commands
###############################################################################
prepare_log_on_start() {
local stamp
case "${START_LOG_MODE}" in
append) return 0 ;;
archive)
if [ -s "${LOG_FILE}" ]; then
stamp="$(date '+%Y%m%d-%H%M%S')"
mv -f "${LOG_FILE}" "${LOG_FILE}.${stamp}" || die "cannot archive ${LOG_FILE}"
fi ;;
new)
[ -f "${LOG_FILE}" ] && { : > "${LOG_FILE}" || die "cannot truncate ${LOG_FILE}"; } ;;
*) die "invalid START_LOG_MODE: ${START_LOG_MODE}" ;;
esac
}
# do_start [both|a|b|<port name>]
do_start() {
local which="${1:-${PORTS}}" pids pa="-" pb="-" run_a=0 run_b=0
case "${which}" in
both|BOTH|all) run_a=1; run_b=1 ;;
a|A|"${PORT_A_NAME}") run_a=1 ;;
b|B|"${PORT_B_NAME}") run_b=1 ;;
*) die "unknown port selector '${which}' (use: both | a | b | ${PORT_A_NAME} | ${PORT_B_NAME})" ;;
esac
pids="$(is_running)" && die "already running (pids=${pids})"
command -v "${BCMCMD}" >/dev/null 2>&1 || die "${BCMCMD} not found"
mkdir -p "${LOG_DIR}" || die "cannot create ${LOG_DIR}"
prepare_log_on_start
# Only the selected ports are truncated. A raw with no "PRBS START" line is
# what the report uses to tell "not run this session" from "ran and failed".
[ "${run_a}" -eq 1 ] && : > "${RAW_A}"
[ "${run_b}" -eq 1 ] && : > "${RAW_B}"
printf '%s\n' "$(ts)" > "${STATE_FILE}"
# One worker per selected port; with both, they run at the same time.
# Invoked through bash so the file does not need the execute bit.
if [ "${run_a}" -eq 1 ]; then
setsid bash "${SCRIPT_PATH}" __worker \
"${PORT_A_NAME}" "${PORT_A_UNIT}" "${PORT_A_PHY}" "${RAW_A}" >/dev/null 2>&1 &
pa=$!
disown "${pa}" 2>/dev/null
fi
if [ "${run_b}" -eq 1 ]; then
setsid bash "${SCRIPT_PATH}" __worker \
"${PORT_B_NAME}" "${PORT_B_UNIT}" "${PORT_B_PHY}" "${RAW_B}" >/dev/null 2>&1 &
pb=$!
disown "${pb}" 2>/dev/null
fi
printf '%s\n%s\n' "${pa}" "${pb}" > "${PID_FILE}"
sleep 1
[ "${run_a}" -eq 1 ] && { kill -0 "${pa}" 2>/dev/null || die "${PORT_A_NAME} worker failed, see ${RAW_A}"; }
[ "${run_b}" -eq 1 ] && { kill -0 "${pb}" 2>/dev/null || die "${PORT_B_NAME} worker failed, see ${RAW_B}"; }
printf 'started (%s pid=%s, %s pid=%s)\nlog: %s\n' \
"${PORT_A_NAME}" "${pa}" "${PORT_B_NAME}" "${pb}" "${LOG_FILE}"
# Deliberately NOT printing "show interfaces status" here. With PRBS armed
# the link is out of normal operation and reports DOWN, which reads as a
# failure to anyone glancing at the console. The status is shown in the
# report instead, once PRBS has been cleared.
}
do_stop() {
local pids pa pb i
pids="$(is_running)" || { printf 'not running\n'; rm -f "${PID_FILE}"; return 0; }
pa="${pids% *}"; pb="${pids#* }"
# TERM lets each worker finish its poll and run prbsstat STOp / prbs clear.
[[ "${pa}" =~ ^[0-9]+$ ]] && kill -TERM "${pa}" 2>/dev/null
[[ "${pb}" =~ ^[0-9]+$ ]] && kill -TERM "${pb}" 2>/dev/null
for (( i = 0; i < 120; i++ )); do
kill -0 "${pa}" 2>/dev/null || kill -0 "${pb}" 2>/dev/null || break
sleep 0.5
done
if kill -0 "${pa}" 2>/dev/null || kill -0 "${pb}" 2>/dev/null; then
printf 'workers did not exit in time, sending SIGKILL (PRBS may be left running)\n' >&2
kill -KILL "${pa}" 2>/dev/null
kill -KILL "${pb}" 2>/dev/null
fi
write_report
rm -f "${PID_FILE}"
printf 'stopped\nreport: %s\n' "${LOG_FILE}"
}
do_report() {
local name raw pass fail polls verdict
printf '%-14s %-8s %-8s %-8s %s\n' PORT POLLS PASS FAIL RESULT
printf '%-14s %-8s %-8s %-8s %s\n' -------------- -------- -------- -------- ------
for spec in "${PORT_A_NAME}:${RAW_A}" "${PORT_B_NAME}:${RAW_B}"; do
name="${spec%%:*}"; raw="${spec#*:}"
set -- $(port_tally "${name}" "${raw}")
printf '%-14s %-8s %-8s %-8s %s\n' "${name}" "$1" "$2" "$3" "$4"
done
}
do_status() {
local pids
if pids="$(is_running)"; then
printf 'status : running (%s pid=%s, %s pid=%s)\n' \
"${PORT_A_NAME}" "${pids% *}" "${PORT_B_NAME}" "${pids#* }"
else
printf 'status : stopped\n'
fi
printf 'log : %s\n' "${LOG_FILE}"
do_report
}
usage() {
cat <<EOF
Usage: ${SCRIPT_NAME} {start [both|a|b]|stop|status|report|clear}
start Set up PRBS and poll in the background. With no argument it runs
\$PORTS (currently "${PORTS}"); "a" or "b" runs that port alone,
which is how you tell a genuine port fault from the DUT not coping
with two PRBS streams at once.
start both ports at the same time
start a ${PORT_A_NAME} only
start b ${PORT_B_NAME} only
Port status is NOT shown here: with PRBS armed the link reports
DOWN, which looks like a failure. See the report instead.
stop Stop polling, run prbsstat STOp + prbs clear on both ports, and
render the report into the log
report Print the PASS/FAIL tally per port
status Show worker pids plus the current tally
clear Remove the log and raw captures (must be stopped first)
Log file : ${LOG_FILE}
Raw : ${RAW_A}
${RAW_B}
A poll is PASS only when "phy diag <phy> prbs get" reports "${PASS_PATTERN}".
prbsstat Ber is captured alongside every poll but does not decide the verdict.
A port that was not run in this session reports SKIP, not FAIL.
EOF
}
###############################################################################
# Entry point
###############################################################################
case "${1:-}" in
start) shift; do_start "${1:-${PORTS}}" ;;
stop) do_stop ;;
status) do_status ;;
report) do_report ;;
clear)
is_running >/dev/null && die "still running, stop it first"
rm -f "${LOG_FILE}" "${LOG_FILE}".[0-9]* "${RAW_A}" "${RAW_B}" "${STATE_FILE}"
printf 'log cleared\n' ;;
__worker)
shift
do_worker "$1" "$2" "$3" "$4" ;;
-h|--help|"") usage ;;
*) printf '[ERROR] unknown sub-command: %s\n\n' "$1" >&2; usage; exit 1 ;;
esac
@@ -0,0 +1,203 @@
#!/bin/bash
###############################################################################
# usb_target.sh
#
# Version : V1.1.0
# Author : ETWen
# Date : 20260825
# Purpose : Auto-detect an inserted USB mass-storage device and report either
# its device node, its mount point, or both.
#
# stdout : requested value(s) only, for $(...) capture
# stderr : diagnostic messages
#
# Usage : usb_target.sh [-o dev|mnt|both|id] [-n] [-r seconds]
#
# -o dev print partition device node (e.g. /dev/sda1)
# -o mnt print mount point (e.g. /mnt/usb) [default]
# -o both print "<dev> <mnt>" on one line
# -o id print stable by-id path (/dev/disk/by-id/...)
# -n detect only, do not mount
# -r sec udev enumeration wait, default 15
#
# Exit : 0 success
# 1 no USB mass-storage found
# 2 multiple USB disks detected (refuse to guess)
# 3 no usable filesystem on the device
# 4 mount failed
# 5 mounted but not writable
#
# Version History
# V1.0.0 20260825 Initial Version
# V1.1.0 20260825 Add -o/-n/-r options; expose device node and by-id path
###############################################################################
set -u
MNT_BASE="/mnt/usb"
RETRY=15
OUTPUT="mnt"
DO_MOUNT=1
log() { echo "[usb] $*" >&2; } # diagnostics go to stderr; stdout stays clean
while getopts "o:nr:h" opt; do
case "$opt" in
o) OUTPUT="$OPTARG" ;;
n) DO_MOUNT=0 ;;
r) RETRY="$OPTARG" ;;
h) sed -n '3,30p' "$0" >&2; exit 0 ;;
*) log "invalid option"; exit 1 ;;
esac
done
shift $((OPTIND - 1))
case "$OUTPUT" in
dev|mnt|both|id) ;;
*) log "invalid -o value: $OUTPUT"; exit 1 ;;
esac
# "-o dev" / "-o id" alone does not require mounting.
[ "$OUTPUT" = "dev" ] && [ "$DO_MOUNT" = "1" ] && DO_MOUNT=0
[ "$OUTPUT" = "id" ] && [ "$DO_MOUNT" = "1" ] && DO_MOUNT=0
#------------------------------------------------------------------------------
# Identify the physical disk(s) backing rootfs / /host so they can be excluded.
# Needed because some platforms boot from a USB DOM, which also reports TRAN=usb.
#------------------------------------------------------------------------------
get_system_disk() {
local src
for mp in /host / ; do
src=$(findmnt -no SOURCE "$mp" 2>/dev/null) || continue
lsblk -no PKNAME "$src" 2>/dev/null | head -1
done | sort -u
}
#------------------------------------------------------------------------------
# List candidate USB disks (whole devices, not partitions).
#------------------------------------------------------------------------------
find_usb_disks() {
local sysdisks; sysdisks=$(get_system_disk)
lsblk -dn -o NAME,TYPE,TRAN,RM 2>/dev/null | while read -r name type tran rm; do
[ "$type" = "disk" ] || continue
case "$name" in loop*|ram*|dm-*|sr*|zram*) continue ;; esac
# Older util-linux may not expose the TRAN column; fall back to sysfs.
if [ "$tran" != "usb" ]; then
readlink -f "/sys/block/$name/device" 2>/dev/null | grep -q '/usb[0-9]' || continue
[ "$rm" = "1" ] || continue
fi
echo "$sysdisks" | grep -qx "$name" && { log "skip $name (system disk)"; continue; }
echo "/dev/$name"
done
}
#------------------------------------------------------------------------------
# Pick a mountable partition from a disk; fall back to the whole device for
# superfloppy layouts (filesystem written directly, no partition table).
#------------------------------------------------------------------------------
pick_partition() {
local disk="$1" p
p=$(lsblk -ln -o NAME,TYPE,FSTYPE "$disk" | \
awk '$2=="part" && $3!="" {print "/dev/"$1; exit}')
[ -n "$p" ] && { echo "$p"; return; }
[ -n "$(lsblk -dn -o FSTYPE "$disk")" ] && echo "$disk"
}
#------------------------------------------------------------------------------
# Resolve a device node to a stable /dev/disk/by-id path, if one exists.
#------------------------------------------------------------------------------
resolve_by_id() {
local dev="$1" real link
real=$(readlink -f "$dev")
for link in /dev/disk/by-id/*; do
[ -e "$link" ] || continue
case "$link" in *-part*|*) ;; esac
[ "$(readlink -f "$link")" = "$real" ] && { echo "$link"; return 0; }
done
return 1
}
#------------------------------------------------------------------------------
# Main
#------------------------------------------------------------------------------
disks=""
for i in $(seq 1 "$RETRY"); do
disks=$(find_usb_disks)
[ -n "$disks" ] && break
sleep 1
done
[ -z "$disks" ] && { log "no USB mass-storage found"; exit 1; }
n=$(echo "$disks" | wc -l)
if [ "$n" -gt 1 ]; then
log "multiple USB disks detected, refuse to guess:"
log "$disks"
exit 2
fi
disk="$disks"
part=$(pick_partition "$disk")
[ -z "$part" ] && { log "$disk has no usable filesystem"; exit 3; }
fstype=$(lsblk -no FSTYPE "$part")
log "found $part (disk=$disk fstype=$fstype)"
# --- device-node only: no mount needed ---------------------------------------
if [ "$OUTPUT" = "dev" ]; then
echo "$part"
exit 0
fi
if [ "$OUTPUT" = "id" ]; then
if byid=$(resolve_by_id "$part"); then
echo "$byid"
exit 0
fi
log "no by-id path for $part, falling back to device node"
echo "$part"
exit 0
fi
# --- mount point required ----------------------------------------------------
mp=$(lsblk -no MOUNTPOINT "$part" | head -1)
if [ -z "$mp" ] && [ "$DO_MOUNT" = "0" ]; then
log "$part is not mounted and -n was given"
exit 4
fi
if [ -z "$mp" ]; then
log "mounting $part -> $MNT_BASE"
mkdir -p "$MNT_BASE"
# exfat/ntfs are not always built into the platform kernel.
case "$fstype" in
exfat) grep -qw exfat /proc/filesystems || modprobe exfat 2>/dev/null ;;
ntfs) grep -qw ntfs3 /proc/filesystems || modprobe ntfs3 2>/dev/null ;;
esac
if ! mount -o rw,noatime "$part" "$MNT_BASE" 2>/dev/null; then
log "mount failed (fstype=$fstype)"
exit 4
fi
mp="$MNT_BASE"
# Verify it is actually writable (read-only media, dirty FAT, or full device).
if ! touch "$mp/.wtest" 2>/dev/null; then
log "mounted read-only or no space left"
umount "$mp"
exit 5
fi
rm -f "$mp/.wtest"
else
log "$part already mounted at $mp"
fi
if [ "$OUTPUT" = "both" ]; then
echo "$part $mp"
else
echo "$mp"
fi
exit 0
+13 -3
View File
@@ -1,12 +1,22 @@
;================================================================
; User Configuration
;================================================================
strTestcase="Margin"
project_name = "Blanton"
strTestcase="ENV" ;ENV,EMC,Margin...etc
EN_Margin = 1 ; 1: Enable Margin Test, 0: Disable Margin Test
EN_Margin = 0 ; 1: Enable Margin Test, 0: Disable Margin Test
EN_log = 1 ; 1: Enable log, 0: Disable log
;================================================================
; FAN_SPEED
;================================================================
FAN_SPEED = 30
;================================================================
; Switch Unit Population
;================================================================
SWB_UNIT0 = 1 ; 1 = this DUT has switch unit 0, wait for it ; 0 = skip
SWB_UNIT1 = 1 ; 1 = this DUT has switch unit 1, wait for it ; 0 = skip
;================================================================
; Prompt Definitions
;================================================================
@@ -16,3 +16,8 @@ wait prompt_sonic_root
sendln "show platform leak status"
wait prompt_sonic_root
sendln "show platform leak channels"
wait prompt_sonic_root
sendln "tpm-dut-test fru"
wait prompt_sonic_root
sendln "tpm-dut-test tpm-read"
+8 -5
View File
@@ -1,8 +1,11 @@
; All Event
wait prompt_sonic_root
sendln "date"
sendln "dmesg -T | grep -Ei 'error|fail(ed|ure)?|fault|critical|panic|timeout'"
; i2c event
wait prompt_sonic_root
sendln "dmesg | grep -i error"
sendln "dmesg -T | grep -Ei 'error|fail(ed|ure)?|fault|critical|panic|timeout' | grep -i i2c"
; Clear Event
wait prompt_sonic_root
sendln "dmesg | grep -i fail"
wait prompt_sonic_root
sendln "dmesg | grep -i warning"
sendln "dmesg -C"
+105
View File
@@ -0,0 +1,105 @@
; =============================================================================
; File : utils/wait_init.ttl
; Version : V3.0.0
; Date : 2026-08-21
; Author : ETWen
; =============================================================================
; Wait until every switch unit ENABLED below reports at least WT_MIN ports up:
; bcmcmd -n <u> -c ps | grep -w up | wc -l
;
; The DUT is not always fully populated. Which units exist is declared ONCE in
; config.ttl, so the traffic blocks in Script A/B/C and this wait agree:
; both units -> SWB_UNIT0 = 1 , SWB_UNIT1 = 1
; unit 0 only -> SWB_UNIT0 = 1 , SWB_UNIT1 = 0
; unit 1 only -> SWB_UNIT0 = 0 , SWB_UNIT1 = 1
; no unit -> SWB_UNIT0 = 0 , SWB_UNIT1 = 0 (bypass, returns immediately)
;
; Enter : prompt of the previous command is NOT consumed.
; Exit : a command has been sent, prompt NOT consumed (caller does the wait).
;
; Why the "PORTS0=" marker: TTL can only test for a string, so reading a count
; means capturing it. Wrapping the number in an echo gives waitregex a unique
; anchor, and the ECHO of the command cannot false-match -- it reads
; "PORTS0=$(bcmcmd ..." and the pattern requires a digit right after the "=".
;
; Version History:
; V1.0.0 2026-08-21 Initial Version (waited on "show platform temperature"
; until it stopped reporting "Thermal Not detected")
; V2.0.0 2026-08-21 Wait on the bcmcmd port-up count of BOTH switch units
; instead of the thermal sensors; proceed only when both
; are greater than WT_MIN (216).
; V3.0.0 2026-08-21 Add WT_UNIT0 / WT_UNIT1 so a partly populated DUT can
; be declared: wait on the enabled units only, and bypass
; entirely when neither is set.
; V3.1.0 2026-08-21 Take the population from SWB_UNIT0 / SWB_UNIT1 in
; config.ttl instead of local copies, so this wait and the
; traffic blocks cannot drift apart.
; V3.1.1 2026-08-25 WT_INTERVAL 10 -> 60. Fix the WT_MIN comments: the test
; has been `< WT_MIN` (at least) since the threshold was
; corrected, but the text still said "more than", which is
; the off-by-one that once hung Script A.
; =============================================================================
; ---- which units to wait for: SWB_UNIT0 / SWB_UNIT1, set in config.ttl ------
WT_MIN = 216 ; an enabled unit must report AT LEAST this many ports up
; (216 = 108 loopback pairs x 2, i.e. every cabled port)
WT_INTERVAL = 60 ; seconds between polls
; -----------------------------------------------------------------------------
timeout = 60 ; per-wait cap; bcmcmd ps is not instant
wait prompt_sonic_root
; nothing declared -> bypass
if SWB_UNIT0 = 0 then
if SWB_UNIT1 = 0 then
goto NO_UNITS
endif
endif
:WAIT_PORTS
wt_ok = 1
if SWB_UNIT0 = 1 then
wt_up0 = 0
sendln "echo PORTS0=$(bcmcmd -n 0 -c ps | grep -w up | wc -l)"
waitregex "PORTS0=[ ]*([0-9]+)"
if result = 1 then
str2int wt_up0 groupmatchstr1
endif
wait prompt_sonic_root
if wt_up0 < WT_MIN then
wt_ok = 0
endif
endif
if SWB_UNIT1 = 1 then
wt_up1 = 0
sendln "echo PORTS1=$(bcmcmd -n 1 -c ps | grep -w up | wc -l)"
waitregex "PORTS1=[ ]*([0-9]+)"
if result = 1 then
str2int wt_up1 groupmatchstr1
endif
wait prompt_sonic_root
if wt_up1 < WT_MIN then
wt_ok = 0
endif
endif
if wt_ok = 1 then
goto PORTS_OK
endif
pause WT_INTERVAL
goto WAIT_PORTS
; ---- exits ------------------------------------------------------------------
; Both leave exactly one prompt unconsumed, so the caller's next
; "wait prompt_sonic_root" has something to match.
:NO_UNITS
sendln "echo wait_init:no-switch-unit-declared,bypassing-port-wait"
goto DONE
:PORTS_OK
sendln ""
:DONE
timeout = 0
+63
View File
@@ -0,0 +1,63 @@
echo PORTS0=$(bcmcmd -n 0 -c ps | grep -w up | wc -l)
echo PORTS0=$(bcmcmd -n 1 -c ps | grep -w up | wc -l)
show interfaces status Ethernet513,Ethernet514
//100G Port 513
sudo sfputil lpmode off Ethernet513
bcmcmd -n 0 -c 'dsh -c "phy diag 268 prbs set p=3"'
bcmcmd -n 0 -c 'dsh -c "phy diag 268 prbsstat STArt Interval=5"'
bcmcmd -n 0 -c 'dsh -c "phy diag 268 prbs get"'
bcmcmd -n 0 -c 'dsh -c "phydiag 268 prbsstat Ber"'
bcmcmd -n 0 -c 'dsh -c "phydiag 268 prbsstat STOp"'
bcmcmd -n 0 -c 'dsh -c "phydiag 268 prbs clear"'
//100G Port 514
sudo sfputil lpmode off Ethernet514
bcmcmd -n 1 -c 'dsh -c "phy diag 268 prbs set p=3"'
bcmcmd -n 1 -c 'dsh -c "phy diag 268 prbsstat STArt Interval=5"'
bcmcmd -n 1 -c 'dsh -c "phy diag 268 prbs get"'
bcmcmd -n 1 -c 'dsh -c "phydiag 268 prbsstat Ber"'
bcmcmd -n 1 -c 'dsh -c "phydiag 268 prbsstat STOp"'
bcmcmd -n 1 -c 'dsh -c "phydiag 268 prbs clear"'
//100G Port 513
sudo sfputil lpmode off Ethernet513
sudo sfputil lpmode off Ethernet514
bcmcmd -n 0 -c 'dsh -c "phy diag 268 prbs set p=3"'
sleep 1
bcmcmd -n 1 -c 'dsh -c "phy diag 268 prbs set p=3"'
bcmcmd -n 0 -c 'dsh -c "phy diag 268 prbsstat STArt Interval=5"'
sleep 1
bcmcmd -n 1 -c 'dsh -c "phy diag 268 prbsstat STArt Interval=5"'
bcmcmd -n 0 -c 'dsh -c "phy diag 268 prbs get"'
sleep 1
bcmcmd -n 1 -c 'dsh -c "phy diag 268 prbs get"'
bcmcmd -n 0 -c 'dsh -c "phydiag 268 prbsstat Ber"'
sleep 1
bcmcmd -n 1 -c 'dsh -c "phydiag 268 prbsstat Ber"'
bcmcmd -n 0 -c 'dsh -c "phydiag 268 prbsstat STOp"'
sleep 1
bcmcmd -n 1 -c 'dsh -c "phydiag 268 prbsstat STOp"'
bcmcmd -n 0 -c 'dsh -c "phydiag 268 prbs clear"'
sleep 1
bcmcmd -n 1 -c 'dsh -c "phydiag 268 prbs clear"'
+32
View File
@@ -0,0 +1,32 @@
bcmcmd -n 0 -c '*:port all lb=mac'
bcmcmd -n 0 -c '*:l2 learn off'
bcmcmd -n 0 -c '*:dsh -c "test mode nr=yes"'
bcmcmd -n 0 -c '*:dsh -c "tr 518 Scenario=6 Profile=2 Count=260 PktSize=324 PortList=1-259,270-565 rm=ext CheckLinkTime=20 CheckDataTime=200 testphase=2"'
bcmcmd -n 1 -c "*:port all lb=mac"
bcmcmd -n 1 -c "*:l2 learn off"
bcmcmd -n 1 -c "*:dsh -c 'test mode nr=yes'"
bcmcmd -n 1 -c "*:dsh -c 'tr 518 Scenario=6 Profile=2 Count=260 PktSize=324 PortList=1-259,270-565 rm=ext CheckLinkTime=20 CheckDataTime=200 testphase=2'"
bcmcmd -n 0 -c "*:port all lb=mac"
sleep 1
bcmcmd -n 1 -c "*:port all lb=mac"
sleep 1
bcmcmd -n 0 -c "*:l2 learn off"
sleep 1
bcmcmd -n 1 -c "*:l2 learn off"
sleep 1
bcmcmd -n 0 -c "*:dsh -c 'test mode nr=yes'"
sleep 1
bcmcmd -n 1 -c "*:dsh -c 'test mode nr=yes'"
sleep 1
bcmcmd -n 0 -c "*:dsh -c 'tr 518 Scenario=6 Profile=2 Count=260 PktSize=324 PortList=1-259,270-565 rm=ext CheckLinkTime=20 CheckDataTime=200 testphase=2'"
sleep 1
bcmcmd -n 1 -c "*:dsh -c 'tr 518 Scenario=6 Profile=2 Count=260 PktSize=324 PortList=1-259,270-565 rm=ext CheckLinkTime=20 CheckDataTime=200 testphase=2'"
sleep 1
root@(none):/usr/local/bin# swutil *:l2 learn off
root@(none):/usr/local/bin# swutil *:dsh -c \"test mode nr=yes\"
root@(none):/usr/local/bin# swutil *:dsh -c \"tr 518 Scenario=6 Profile=2 Count=260 PktSize=324 PortList=1-259,270-565 rm=ext CheckLinkTime=20 CheckDataTime=200 testphase=2"
+7
View File
@@ -0,0 +1,7 @@
echo 18-0020 > /sys/bus/i2c/drivers/max31790/unbind
sleep .5
echo 18-0020 > /sys/bus/i2c/drivers/max31790/bind
echo 25-0020 > /sys/bus/i2c/drivers/max31790/unbind
sleep .5
echo 25-0020 > /sys/bus/i2c/drivers/max31790/bind
fan-speed-control.sh 30
+59
View File
@@ -0,0 +1,59 @@
#!/bin/bash
# =============================================================================
# gen_channel_map.sh - Build docs/LTC2980_channel_map.csv from settings/*.conf
# Version : V1.0.0
# Date : 2026-08-27
# Author : ETWen
# =============================================================================
# Runs on the DEV machine, not the DUT. Reads every LTC2980 margin settings
# file and flattens CH<n>_VNOM / CH<n>_NET into one CSV:
#
# Board,CONN,Ch,NetName,Vnom
# CB,CONN13,CH0,V5P0_ALW,5.0
#
# NC channels keep their literal NC / "-" so the page numbering still lines up
# (ch/8 -> CHIPS[] index, ch%8 -> LTC2977 PAGE). Do not filter them out.
#
# Usage:
# tools/gen_channel_map.sh # write docs/LTC2980_channel_map.csv
# tools/gen_channel_map.sh - # write to stdout
# =============================================================================
set -eu
REPO_ROOT="$(cd "$(dirname "$0")/.." && pwd)"
SET_DIR="${REPO_ROOT}/src/Script_ABC_Blanton/Blanton_Script/LTC2980_Margin_Script/settings"
OUT="${1:-${REPO_ROOT}/docs/LTC2980_channel_map.csv}"
[ -d "${SET_DIR}" ] || { echo "settings dir not found: ${SET_DIR}" >&2; exit 1; }
# 3-argument match(str, regex, array) is a gawk extension. Under mawk it is a
# syntax error at best and an empty CSV at worst - fail loudly instead.
awk 'BEGIN { if (match("a", /a/, m) != 1) exit 1 }' 2>/dev/null \
|| { echo "need gawk (3-arg match); this awk is: $(awk --version 2>&1 | head -1)" >&2; exit 1; }
gen() {
printf 'Board,CONN,Ch,NetName,Vnom\n'
# CB first, then SWB0/SWB1 CONN13..CONN16 - matches the physical walk order.
for f in "${SET_DIR}"/Blanton_CB_CONN*.conf \
"${SET_DIR}"/Blanton_SWB0_CONN*.conf \
"${SET_DIR}"/Blanton_SWB1_CONN*.conf ; do
[ -f "${f}" ] || continue
base="$(basename "${f}" .conf)" # Blanton_SWB0_CONN13
board="$(echo "${base}" | cut -d_ -f2)" # SWB0
conn="$(echo "${base}" | cut -d_ -f3)" # CONN13
# .conf files are LF per .gitattributes, but strip CR anyway - a stray
# CR would land inside the net name and travel into the CSV unseen.
sed 's/\r$//' "${f}" | awk -v b="${board}" -v c="${conn}" '
match($0, /^CH([0-9]+)_VNOM="([^"]*)"[^"]*CH[0-9]+_NET="([^"]*)"/, m) {
printf "%s,%s,CH%s,%s,%s\n", b, c, m[1], m[3], m[2]
}'
done
}
if [ "${OUT}" = "-" ]; then
gen
else
gen > "${OUT}"
echo "wrote ${OUT} ($(($(wc -l < "${OUT}") - 1)) channels)"
fi