fix(stress): Keep the SSD and USB jobs alive for the whole soak

qfx5252-stress-usb was started with --runtime 60. It finished a minute
into an overnight run and the USB path sat idle for the rest of the
night - nothing failed, nothing said so, and the log looked complete.
SSD had the same shape at --runtime 3600. Both are now 86400, and the
BMC DDR memtester drops its loop count so it runs until stopped.

86400 is 24h, not forever. Past that SSD and USB die quietly and the
only clue is the shorter bgctl list Script B already prints each round.
Said so in the version history, the release notes and ARCHITECTURE.

Adds blanton_multiphase_margin.sh: the multiphase controllers on both
switch boards over the native i2c buses (SWB0 on i2c-13/15, SWB1 on
-14/-16), 19 rails each. high3/low3/comboA/comboB set voltages and print
one line per rail; save commits them and prints Multiphase Save All,
because that list has no per-rail mapping to report against.

Two hazards it carries: save is STORE_USER_ALL and permanent, and these
are NOT the LTC2980s margin.sh drives - the two tools reach the same
rails from opposite sides with no interlock between them.

The emitted commands were diffed against tools/Multiphase_*.txt: argv
identical for all five. The rail-to-netname mapping was cross-checked
against the bus numbers rather than trusted by line position - blocks
1..19 are all bus 13/15 and 20..38 all 14/16, matching the SWB0/SWB1
split of Multiphase_Netname.

blanton_traffic_linespeed v0.7.0 appends PatternRandom=yes to every tx
including ce0, and the defaults become tx 200 length=324. A fixed
pattern can sit on a benign bit sequence for a whole run; a changing one
is what exposes a weak lane. -r no forces fixed, -r "" omits the
argument for an SDK that will not take it. Bytes per burst went 51200 ->
64800, so absolute counter totals do not carry across this build - the
pair cross-check that decides PASS/FAIL does.

Script A -> V1.1.5.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014RetWKFZFG1ZHcitQAyhwM
This commit is contained in:
2026-09-01 11:04:45 +08:00
co-authored by Claude Opus 5
parent db44f72352
commit 215fed6e95
13 changed files with 2061 additions and 20 deletions
+18 -2
View File
@@ -9,7 +9,7 @@ DUT 端的取數能力由一組 **bash 工具庫** 提供(CB FPGA `pcimem` →
完整架構見 [ARCHITECTURE.md](ARCHITECTURE.md)。
**目前狀態(2026-08-31**Script A 為 **V1.1.4**(A 集中記錄所有 TTL 的變更)。涵蓋
**目前狀態(2026-09-01**Script A 為 **V1.1.5**(A 集中記錄所有 TTL 的變更)。涵蓋
PCIe AER / DDR EDAC / PMON+TPM / 電壓 margin9 顆 LTC2980/ SWB loopback 線速流量 /
10G+1G 管理網路 ping / BMC DDR 與 USB / NVMe 健康,壓力層走平台的 `bgctl`
switch unit 數量由 `config.ttl``SWB_UNIT0` / `SWB_UNIT1` 宣告,風扇轉速由 `FAN_SPEED` 指定
@@ -26,7 +26,7 @@ NFC tag 讀取由 `nfc_polling.sh` 背景輪詢(A 做基線、B 起、C 停並
另有 **`4_Blanton_Script_thermal_safety.ttl`**(熱 / 安規,**獨立跑不接 A→B→C**,滿載後每分鐘
取 PMON + 兩片 SWB 的軌功耗;**沒有收尾段**,中斷後要手動 `blanton_tr518.sh stop`)。
流程圖:`docs/Blanton_Test_Flow.drawio`
最後一次發布是 **V1.1.3**;SWB 的 I2C 通道與位址已於 2026-08-19 上機驗證。
最後一次發布是 **V1.1.4**;SWB 的 I2C 通道與位址已於 2026-08-19 上機驗證。
margin 的 `VNOM` 已於 2026-08-30 改為 Blanton 實測中位數(144 條軌中 86 條上調 0.8~2.5%)。
交付用 `./publish/publish.sh` 打包。
@@ -117,6 +117,16 @@ bash ~/Blanton_Script/port_prbs_monitor.sh report # PASS/FAIL 統計(沒
bash ~/Blanton_Script/port_prbs_monitor.sh stop # 收尾會跑 prbsstat STOp + prbs clear
# ⚠️ PRBS 打起來時 link 會顯示 down,這是正常的 —— 所以 start 刻意不印 port status
# ── 多相 VRM margin(原生 i2c,38 條軌)──────────────
./Blanton_Script/blanton_multiphase_margin.sh high3 # 全部 +3%
./Blanton_Script/blanton_multiphase_margin.sh low3 # 全部 -3%
./Blanton_Script/blanton_multiphase_margin.sh comboA # 混合,組合 A
./Blanton_Script/blanton_multiphase_margin.sh comboB # 混合,組合 B
./Blanton_Script/blanton_multiphase_margin.sh save # ⚠️ 永久寫 NVM
# -u 0 / -u 1 單邊(SWB0=i2c-13/15SWB1=i2c-14/16);-n 只印指令
# 逐條印 SWB0_0V9_ANLG0 - 0.930V Configure Donesave 只印 Multiphase Save All
# ⚠️ 這不是 margin.sh 管的 LTC2980,兩支摸同一批軌,不要同時用
# ── 全機 SerDes BERPRBS,期間 link 會顯示 DOWN)────
./Blanton_Script/blanton_ber.sh init # prbs set p=3 + prbs get(檢查每條 lane 鎖定)
./Blanton_Script/blanton_ber.sh start # prbsstat STArt Interval=30
@@ -222,6 +232,12 @@ Tera Term 連 COM port (115200-8-N-1)
- ⚠️ **ICB/PDB 一律走 VI2C`blanton_icb_vi2c.sh`),不要走 VSPI**VSPI ~18 ms/reg 會讓 I2C
byte 相間隔拉到 ~50 ms,踩中 SMBus 2535 ms timeout,裝置直接 NACK。實測全掃 9.4 s vs 34.0 s。
- ⚠️ **I2C 讀一律 Repeated START**(不是 STOP + START),否則 PMBus/SMBus 裝置(VRM、EFUSE)不理。
- ⚠️ **`--runtime 86400` 是 24 小時上限**:超過一天的 soak,SSD / USB 壓力會安靜結束,
剩下的時間那兩項閒置。`bgctl list` 的數量變少是唯一的線索,**不會報警**
- ⚠️ **多相 VRM 與 LTC2980 是同一批軌的兩條路**`blanton_multiphase_margin.sh`(原生 i2c
`margin.sh`LTC2980 PMBus)都假設自己是唯一寫入者,**沒有互斥檢查**,只靠人記得
- ⚠️ **traffic 的計數器基準在 v0.7.0 變了**`tx 100 length=512``tx 200 length=324`
每 burst 的位元組數 +27%。PASS/FAIL 是成對交叉比對不受影響,但**絕對值不能跟舊 log 比**
- ⚠️ **Script 4thermal/safety)沒有收尾**`while 1` 之後就是檔尾。中斷後 DUT 停在
`port cd lb=mac` / `l2 learn off` / `test mode nr=yes``mlucas-avx2` 也還在背景。
下一個測試會在被污染的狀態上開始,而且 **`show interfaces status` 看不出來**。