Compare commits

..
3 Commits
Author SHA1 Message Date
etwenandClaude Opus 5.5 4db9b45ce7 docs: Record V1.2.4 in Script A's history and add the release note
Script A: header to V1.2.4 / 2026-10-01; entries for tray_id_stress.sh,
the Script A and Script B hooks, and 6_Blanton_Script_BER_Cycle.ttl,
with notes on the exec bit, the sleep-vs-Interval rule, the hard-coded
BDF, the hostname, the SONiC-only fan setting and the placeholder
password.

CLAUDE.md: status and last release to V1.2.4; the Tray ID stress and the
BER cycle in the overview; a commands block for tray_id_stress.sh; a
gotcha that prbsstat Ber must be read after the first interval is
published, and that Script A / C's own BER (Interval=30, sleep 30) sits
on that edge and has not been verified.

ARCHITECTURE.md: Script A ref and the tag checkbox to V1.2.4; Script 5's
current file name and the new macro 6 in the tree; tray_id_stress.sh in
the tree.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UniUXkxgCG2Ty74advL8Pd
2026-10-01 13:32:14 +08:00
etwenandClaude Opus 5.5 7937ade30e feat(ttl): Add the Diag / SONiC BER cycle macro
6_Blanton_Script_BER_Cycle.ttl is a standalone macro, not part of A/B/C.
Each cycle measures PRBS BER on unit 0 in the Diag OS, cold-boots into
SONiC by writing the CB FPGA, measures again, and cold-boots back; while
1, no end. The boot target is chosen in the BIOS Boot Manager ("USB
Device" = Diag, "Hard Drive" = SONiC) and confirmed by the boot
signature. The log goes to Logs\Blanton_BER_Cycle_<ts>.log.

prbsstat runs with Interval=30 and the BER is read after sleep 40. The
sleep has to be longer than the interval: checked against the error
counts 'prbs get' reports in the same cycle, sleep 10 read Diag ~60x and
SONiC ~40x low, and sleep 30 still read Diag ~60x low because its Ber is
sent 30.15 s after STArt, before the first interval is published.
Interval=30 + sleep 40 matched on both sides over 73 cycles.

Known limits: the SONiC -> Diag boot writes a hard-coded BDF
(0000:05:00.0); the prompts assume the hostname 'sonic'; the fan is set
to 30 % on the SONiC side only; the password is the placeholder.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UniUXkxgCG2Ty74advL8Pd
2026-10-01 13:32:14 +08:00
etwenandClaude Opus 5.5 ebb2260798 feat(i2c): Add a Tray ID I2C stress and run it in Script A and B
tray_id_stress.sh writes 0xA5 and 0x5A to register 0x02 of device 0x70
on buses 34, 35, 38 and 39, reading each back and comparing. The two
patterns put every bit in both states. It prints one PASS / FAIL / ERROR
line per step and a per-bus Summary with a single RESULT line; exit 0 is
all PASS, 1 is any FAIL or I2C error, 2 is bad arguments. FAIL (wrong
value read) and ERROR (i2cset / i2cget failed) are counted apart.

The original value is read first and written back at the end, verified
by a read; -R skips that, -n N repeats the sequence. Buses, address,
register and patterns are editable at the top of the file.

Script A runs one pass after show_dmesg as the baseline. Script B runs
one pass in every soak round, before the sleep 60.

The script is called by path and needs the exec bit on the DUT.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UniUXkxgCG2Ty74advL8Pd
2026-10-01 13:32:14 +08:00
7 changed files with 610 additions and 7 deletions
+5 -3
View File
@@ -161,7 +161,8 @@ Blanton_TTL_Script/
└── Script_ABC_Blanton/ # ← Tera Term 的工作目錄(整包給測試員)
│
├── 4_Blanton_Script_thermal_safety.ttl # 熱 / 安規:獨立跑,不接 A→B→C
├── 5_Blanton_Script_Reboot_Cycle.ttl # 重開機循環:獨立跑,while 1 無止境(未上機驗證)
├── 5_Blanton_Script_Boot_Cycle.ttl # 開機循環(warm / cold,EN_Cold_Warm_boot):獨立跑,while 1 無止境
├── 6_Blanton_Script_BER_Cycle.ttl # Diag ↔ SONiC 冷開機循環,兩邊各量一次 unit 0 BER:獨立跑,while 1
├── config.ttl # 全域設定:testcase 名、EN_Margin / EN_TX_FIR / EN_log / EN_timestamp、
│ # 風扇與 SWB 宣告、SSD/USB/BMC_USB 壓力秒數、三個 prompt 字串
├── hammer/ # 🚫 gitignored — mlucas ELF binary(~99 MB),DUT image 自帶,
@@ -219,6 +220,7 @@ Blanton_TTL_Script/
├── ddr5_temp.sh # DDR5 DIMM 溫度(SPD5118 @ i2c-5 0x50/0x51,reg 0x31)
├── hsc_temp_readback.sh # LTC4287 HSC 溫度(i2c-28 0x40/0x45,內/外部感溫)
├── avt_pmpower.sh # AMD CPU 封裝功耗(AVTCMD pmm get_pmdata;AVT 是 NDA,不隨包)
├── tray_id_stress.sh # Tray ID I2C stress(bus 34/35/38/39 0x70 reg 0x02,寫 0xA5/0x5A 讀回)
├── usb_target.sh # 偵測 USB 裝置節點/掛載點(排除系統碟)
├── bmc_monitor.sh # BMC 監控:free -m + I2C 寫入/讀回圖樣測試
├── bmc_monitor_ddr.sh # ⚠️ 已停用:V1.1.9 起 B/C 內的註解區塊已刪除,DDR 壓力走 bgctl + bmc-manager
@@ -381,7 +383,7 @@ PROFILE_CHANNELS=( "0:high:8" "1:low:8" ... ) # "channel:operation:change_perc
| 模組 | 進入點 | 功能 |
|------|--------|------|
| **Script A(Pre-test)** | `1_Blanton_Script_A.ttl` V1.2.3 | root 登入 → `date -s` 對時 → 清 job log → **收掉前一輪殘留**(`hw-test-session finish` → `bgctl reset --yes` → `bgctl stop --all` → `bgctl list` 確認清空)→ `hw-test-session start` → **設定風扇轉速** → **等資料面就緒**(`wait_init.ttl`)→ DUT 清單(boot/version/fwutil/syseeprom/ssdhealth/TPM/NVMe)→ BMC enroll + version/status → source bash 工具 → 清/讀 7 組 PCIe AER + `ras-mc-ctl --summary` → `lspci` → margin → PMON → dmesg → **10G/1G 管理網路 ping** → **NFC 基線**(start → 30s → stop → report)→ 100G uplink 設定與狀態 → **BER 基線掃描**(依 `SWB_UNIT`,V1.2.1 新增)→ **一輪流量基線** → **SerDes DSC 基線**(`EN_TX_FIR`)→ `show uptime` |
| **Script A(Pre-test)** | `1_Blanton_Script_A.ttl` V1.2.4 | root 登入 → `date -s` 對時 → 清 job log → **收掉前一輪殘留**(`hw-test-session finish` → `bgctl reset --yes` → `bgctl stop --all` → `bgctl list` 確認清空)→ `hw-test-session start` → **設定風扇轉速** → **等資料面就緒**(`wait_init.ttl`)→ DUT 清單(boot/version/fwutil/syseeprom/ssdhealth/TPM/NVMe)→ BMC enroll + version/status → source bash 工具 → 清/讀 7 組 PCIe AER + `ras-mc-ctl --summary` → `lspci` → margin → PMON → dmesg → **10G/1G 管理網路 ping** → **NFC 基線**(start → 30s → stop → report)→ 100G uplink 設定與狀態 → **BER 基線掃描**(依 `SWB_UNIT`,V1.2.1 新增)→ **一輪流量基線** → **SerDes DSC 基線**(`EN_TX_FIR`)→ `show uptime` |
| **Script B(Stress)** | `2_Blanton_Script_B.ttl` | **V1.1.8 起 `mlucas-avx2 -cpu 0:15` 由 `systemd-run --unit=mlucas-amm`(`Type=exec`、`Restart=always`)啟動,不再是 shell 的 `&` 背景 job** + `bgctl run` 的 `memtester` / SSD / USB 壓力(v1.1.7 起 SSD/USB 的 `--runtime`/`--passes` 讀 `config.ttl` 的 `SSD_RUNTIME`/`SSD_COUNT`、`USB_RUNTIME`/`USB_COUNT`;BMC memtester 不帶次數 = 跑到被停) + BMC DDR(`bmc-manager run memtester`)→ BMC USB net test(探測 cdc_ncm 介面,`systemd-run` 掛 `BMC_USB_RUNTIME` 秒的 ping,V1.1.8 前寫死 4 小時)→ 啟動 mgmt ping 監控 → **啟動 NFC 輪詢** → `jobs` / `systemctl is-active mlucas-amm` / `bgctl list` → traffic(依 `SWB_UNIT`)→ `while 1` 每輪 PMON + `bgctl list`/`jobs`/`systemctl is-active mlucas-amm` + `sleep 60`(**V1.1.4 起 margin 不在迴圈內**;V1.2.0 起等待走 DUT 端 `sleep` 而非 macro 端 `pause`)|
| **Script C(Post-test)** | `3_Blanton_Script_C.ttl` | `kill $(jobs -p)` + **`systemctl stop mlucas-amm`**(systemd unit 不是 shell job,`jobs -p` 抓不到)+ `bgctl stop --all` / `reset` + 停 BMC USB unit、mgmt ping 與 **NFC 輪詢** → PMON + dmesg → 7 組 AER + `ras-mc-ctl` → BMC USB `journalctl` **寫進 `log/bmc_usb_net.log`**(不再印在 console) → `ip -s link show eth0/eth1` → `cat` mlucas log(並 `cp` 進 job 目錄一起帶走)→ **NFC `report`** → traffic `stop` + `report` → **SerDes DSC 收尾**(`EN_TX_FIR`)→ **BER 掃描**(依 `SWB_UNIT`)→ NVMe 健康 → `show reboot-cause` → **margin 掃描**(`EN_Margin`,V1.1.4 由 Script B 移來)→ `uptime` → **job log 複製到 USB(帶時戳)** → `exit` + wait『Script done』→ `hw-test-session finish` |
| **Script 4(Thermal / Safety)** | `4_Blanton_Script_thermal_safety.ttl` | **獨立腳本,不接 A→B→C。** Script A 式的環境建立(登入 / session / 風扇 / `wait_init` / DUT 清單 / BMC / PCIe 清除 / `pcie_bus` / PMON / dmesg)→ **滿載**(`mlucas-avx2 -cpu 0:15`,V1.1.8 起與 Script B 同樣掛成 `mlucas-amm` systemd unit + `blanton_tr518.sh start`)→ `while 1`:PMON + `TH6_SWB0/1_power_readback.sh` + `avt_pmpower.sh`(CPU 封裝功耗,V1.2.3)+ `bgctl list` / `jobs` / `systemctl is-active mlucas-amm`,`pause 60`。**沒有收尾段** —— 中斷後 DUT 仍在 `port cd lb=mac` / `l2 learn off` / `test mode nr=yes`,且 CPU 仍滿載,要手動 `blanton_tr518.sh stop` + `systemctl stop mlucas-amm` |
@@ -812,7 +814,7 @@ margin status、全部 VRM 軌的 Vin/Vout/Iout/Temp、CB + ICB 溫感讀值,
- [ ] `CHANGELOG.md`:延續三支腳本頂端的 Version History,集中成一份
- [ ] 三支腳本的 header 版號與 `CHANGELOG.md` 對齊,`config.ttl` 加 `script_version` 並寫進 log 開頭
- [ ] `docs/TTL_Script_Blanton_Status_*.xlsx` 更新為全 OK,或把剩餘項目移進 Future Extensions
- [x] 打 git tag,push `nas` + `gitea`(已至 `V1.2.3`;`publish/` 依 Script A 檔頭版號打包)
- [x] 打 git tag,push `nas` + `gitea`(已至 `V1.2.4`;`publish/` 依 Script A 檔頭版號打包)
**驗收條件:** 從 zip 解壓到一台乾淨的測試 NB,照 README 操作,不看原始碼就能完成一輪 A→B→C
並產出 summary;`publish/` 內容與 git tag 一致。
+16 -2
View File
@@ -9,7 +9,7 @@ DUT 端的取數能力由一組 **bash 工具庫** 提供(CB FPGA `pcimem` →
完整架構見 [ARCHITECTURE.md](ARCHITECTURE.md)。
**目前狀態(2026-09-23)**:Script A 為 **V1.2.3**(A 集中記錄所有 TTL 的變更)。涵蓋
**目前狀態(2026-10-01)**:Script A 為 **V1.2.4**(A 集中記錄所有 TTL 的變更)。涵蓋
PCIe AER / DDR EDAC / PMON+TPM / 電壓 margin(9 顆 LTC2980)/ SWB loopback 線速流量 /
10G+1G 管理網路 ping / BMC DDR 與 USB / NVMe 健康,壓力層走平台的 `bgctl`。
switch unit 數量由 `config.ttl` 的 `SWB_UNIT0` / `SWB_UNIT1` 宣告,風扇轉速由 `FAN_SPEED` 指定
@@ -36,11 +36,14 @@ NFC tag 讀取由 `nfc_polling.sh` 背景輪詢(A 做基線、B 起、C 停並
取 PMON + 兩片 SWB 的軌功耗 + **AMD CPU 封裝功耗**(`avt_pmpower.sh`,V1.2.3);**沒有收尾段**,中斷後要手動 `blanton_tr518.sh stop`)。
**V1.2.3 起 PMON 快照(`utils/setup_pmon.ttl`)多了 DDR5 DIMM 溫度(`ddr5_temp.sh`)與 LTC4287 HSC 溫度
(`hsc_temp_readback.sh`)**,A/B/C/4/5 全部吃到;HSC 那支每次約 +4 秒。
**V1.2.4 新增 Tray ID I2C stress(`tray_id_stress.sh`)**:Script A 跑一次當基線、Script B 每輪 soak 跑一次。
另新增 **`6_Blanton_Script_BER_Cycle.ttl`**(Diag ↔ SONiC 冷開機循環量 BER,**獨立跑不接 A→B→C**),
操作與分析見 `tmp/BER_Test_Cycle/runbook.md`(tmp 不進 git)。
以及 **`5_Blanton_Script_Boot_Cycle.ttl`**(重開機循環,**獨立跑不接 A→B→C**:登入 → 風扇 →
Check Pt1~Pt13 → `reboot` → `wait "Booting in blind mode"`,`cnt_cycle` 每輪遞增,`while 1` 無止境,
要停就關 Tera Term 的 macro)。**尚未上機驗證**。
流程圖:`docs/Blanton_Test_Flow.drawio`。
最後一次發布是 **V1.2.3**;SWB 的 I2C 通道與位址已於 2026-08-19 上機驗證。
最後一次發布是 **V1.2.4**;SWB 的 I2C 通道與位址已於 2026-08-19 上機驗證。
margin 的 `VNOM` 已於 2026-08-30 改為 Blanton 實測中位數(144 條軌中 86 條上調 0.8~2.5%)。
交付用 `./publish/publish.sh` 打包。
@@ -140,6 +143,12 @@ bash ~/Blanton_Script/port_prbs_monitor.sh stop # 收尾會跑 prbsstat
# ⚠️ AVT 是 NDA,不在交付包裡:DUT 上要先解壓到 ~/AVT_Linux_NDA_2.8.37/,沒有就每輪 exit 2
# ⚠️ 數值是 SMU telemetry 不是量測,要準就對照 VR 的 PMBus READ_POUT
# ── Tray ID I2C stress(V1.2.4,Script A / B 已接)────
~/Blanton_Script/tray_id_stress.sh # bus 34/35/38/39,0x70 reg 0x02:寫 0xA5/0x5A 讀回比對 → Summary
~/Blanton_Script/tray_id_stress.sh -n 100 # 重複 100 輪
~/Blanton_Script/tray_id_stress.sh -R # 跑完不還原(預設會寫回原值並讀回確認)
# exit 0 = 全 PASS、1 = 有 FAIL(讀回不符)或 ERROR(i2cset/i2cget 本身失敗)、2 = 參數錯
# ── PDB / CPB power brick 讀值 ──────────────────────
./Blanton_Script/pwr_brick.sh show # Vin/Iin/Vout/Iout/Temp × 5 顆
./Blanton_Script/pwr_brick.sh show -r # 多印 raw word
@@ -357,6 +366,11 @@ Tera Term 連 COM port (115200-8-N-1)
- ⚠️ **`stress_hhmd.py` / `stress_pcie.py` 已於 2026-08-17 由 KC 移除**(Status 表 ScriptB #5/#7)。
Script B 現在是 **8 個背景 job**(1 mlucas + 5 mem + ssd + usb),不是 10 個。
I2C/PCIe 的持續讀取壓力目前沒有替代品。
- ⚠️ **`prbsstat Ber` 要等第一個 interval 結算之後才讀:sleep 必須「大於」Interval**(2026-09-30 驗證)。
`6_Blanton_Script_BER_Cycle.ttl` 用 `Interval=30` + `sleep 40`。拿同一輪 `prbs get` 的錯誤數交叉比對:sleep 10 時 Diag 低 ~60×、
SONiC 低 ~40×;sleep 30(= Interval)時 Diag 仍低 ~60×(Ber 在 STArt 後 30.15 s 送出,剛好在結算前)。
⚠️ **Script A / C 的 BER 也是 `blanton_ber.sh start`(Interval=30)→ TTL `sleep 30` → `report`**,跑在 SONiC 上 —— 正是「sleep = Interval」的邊界情況,
能不能趕上第一個 interval 結算**未驗證**。建議改成 `sleep 40`(或 `BER_INTERVAL` 調小),並用 `validate_prbsget.py` 的方法驗證一次
- ⚠️ **`pause` 是停 macro,`sleep` 是停 DUT**(V1.2.0 起 A/B/C 一律用後者):`pause N` 期間 macro 不理會
DUT,shell 是閒著的,等於用猜的估上一個指令跑多久;`sendln "sleep N"` + `wait prompt_sonic_root`
把等待放到 DUT 上,prompt 回來才繼續,兩邊不會走散,而且這一段會帶時戳進 log。
+54
View File
@@ -0,0 +1,54 @@
# v1.2.4 — Tray ID I2C stress in Script A and B, and a Diag / SONiC BER cycle that reads the BER correctly
## ✨ New features
**`tray_id_stress.sh` — Tray ID I2C write / read-back stress**
* Buses `34` / `35` / `38` / `39`, device `0x70`, register `0x02`. Each bus gets `0xA5` written and read back, then `0x5A` written and read back — the two patterns put every bit in both states.
* One line per step (`PASS` / `FAIL` / `ERROR`), then a per-bus **Summary** table and a single `Tray ID stress RESULT: PASS|FAIL` line. Exit `0` = all PASS, `1` = any FAIL or I2C error, `2` = bad arguments.
* `FAIL` (the read worked but returned the wrong value) and `ERROR` (`i2cset` / `i2cget` itself failed) are counted separately, because they point at different faults.
* The original register value is read first and **written back at the end**, verified by a read. `-R` skips that; `-n N` repeats the whole sequence N times. Buses, address, register and patterns are editable at the top of the file.
**Script A and Script B run it**
* **Script A** runs one pass after `show_dmesg` as the baseline.
* **Script B** runs one pass in every soak round, before `sleep 60` — 12 writes per round, registers restored each time.
**`6_Blanton_Script_BER_Cycle.ttl` — Diag ↔ SONiC BER cycle (standalone)**
* Each cycle measures PRBS BER on unit 0 in the Diag OS, cold-boots into SONiC, measures again, and cold-boots back to Diag. `while 1`, no end. The boot target is picked in the BIOS Boot Manager (`USB Device` = Diag, `Hard Drive` = SONiC).
* Log: `Logs\Blanton_BER_Cycle_<ts>.log`, timestamps always on.
## 🐛 Bug fixes
**`prbsstat Ber` was read before the first interval had been published — BER read 40–60× low**
* The BER cycle originally read `prbsstat Ber` 10 s after `prbsstat STArt Interval=30`. Checked against the error counts that `prbs get` reports in the same cycle, that read **Diag ~60× low and SONiC ~40× low**.
* Moving to `sleep 30` (equal to the interval) fixed SONiC but **not Diag**: its `Ber` goes out 30.15 s after `STArt`, just before the first interval is published. SONiC's `bcmcmd` adds ~0.3 s and just makes it.
* The macro now uses **`Interval=30` + `sleep 40`**. Over 73 cycles both sides agree with the `prbs get` estimate to within 0.1 decade. The rule: **the sleep must be longer than the interval.**
## 📦 Downloads
| File | Contents |
|---|---|
| `Script_ABC_Blanton_V1.2.4.zip` | The full Tera Term working directory, including `Blanton_Script/` to copy onto the DUT |
```bash
chmod +x ~/Blanton_Script/*.sh
grep -rlU $'\r' ~/Blanton_Script # expect no output
```
## ⚠️ Before you run
* **`chmod +x` after deploying.** Script A and B call `tray_id_stress.sh` by path; without the exec bit every call is `Permission denied` and the round carries on without the check.
* **Any BER log taken before 2026-09-30 14:13 reads 40–60× low** (BER cycle with `sleep ≤ Interval`). Relative Diag-vs-SONiC trends are usable; absolute values are not.
* **Script A / C's BER is the same edge case**: `blanton_ber.sh start` (Interval=30) → `sleep 30` → `report`, on SONiC. Whether it catches the first interval has **not been verified**. Lengthening that sleep to 40 is the safe change.
* **The BER cycle's SONiC → Diag boot writes a hard-coded BDF** (`pcimem /sys/bus/pci/devices/0000:05:00.0/resource0 0x480`). Check the CB FPGA BDF on every unit before running it.
* **The BER cycle expects the hostname `sonic`.** A unit with another hostname (e.g. `Margin02`) needs `prompt_sonic` / `prompt_sonic_root` edited, or the macro waits forever.
* **The BER cycle sets the fan to 30 % on the SONiC side only** — SONiC runs ~11 °C hotter than Diag, a variable in any Diag-vs-SONiC comparison.
* **Its SONiC password is the `YourPaSsWoRd` placeholder**, and it does not read `config.ttl` or `secret/`.
* **Script 4 has no teardown**: finish it with `blanton_tr518.sh stop` **and** `systemctl stop mlucas-amm`.
* **`FAN_SPEED` is 100.**
## 🔗 Links
* [ARCHITECTURE.md](https://etgit.et-wen.com/etwen/Blanton_TTL_Script/src/branch/main/ARCHITECTURE.md) — test flow, data models, constraints
* [CLAUDE.md](https://etgit.et-wen.com/etwen/Blanton_TTL_Script/src/branch/main/CLAUDE.md) — commands and the bench gotchas
**Full changelog:** [V1.2.3...V1.2.4](https://etgit.et-wen.com/etwen/Blanton_TTL_Script/compare/V1.2.3...V1.2.4)
+60 -2
View File
@@ -1,7 +1,7 @@
; =============================================================================
; Script A for Blanton
; Version : V1.2.3
; Date : 2026-09-23
; Version : V1.2.4
; Date : 2026-10-01
; Author : ETWen
; =============================================================================
; Version History:
@@ -740,6 +740,60 @@
; !! without -q the full PM table is printed every round, and
; Script 4 loops once a minute - the log grows by one full
; table per minute
; V1.2.4 2026-10-01 Blanton_Script/tray_id_stress.sh Created v1.0.0 - Tray ID I2C write / read-back stress
; Buses 34 / 35 / 38 / 39, device 0x70, register 0x02. Each
; bus: write 0xA5, read back, compare; write 0x5A, read back,
; compare - the two patterns put every bit in both states
; One line per step (PASS / FAIL / ERROR), then a per-bus
; Summary table and 'Tray ID stress RESULT: PASS|FAIL'
; Exit 0 = all PASS, 1 = any FAIL or I2C error, 2 = usage
; -n N repeats the sequence N times; buses / addr / reg /
; patterns are editable at the top of the file
; The original value is read first and written back at the
; end (verified by a read). -R leaves the register at 0x5A
; !! FAIL = the read worked but returned the wrong value;
; ERROR = i2cset / i2cget itself failed. They are counted
; apart because they point at different faults
; !! a bus whose original value cannot be read is NOT
; restored (there is nothing to restore to) - the line
; says so
; !! called by path, so it needs the exec bit on the DUT:
; chmod +x ~/Blanton_Script/*.sh
; ScriptA Add Tray ID stress after show_dmesg (one pass, baseline)
; ScriptB Add Tray ID stress to every soak round, before sleep 60
; Each round now writes 0xA5 / 0x5A to four Tray ID
; registers and restores them - 12 writes per round
; 6_Blanton_Script_BER_Cycle.ttl Added - Diag / SONiC BER cycle, a STANDALONE macro (not A/B/C)
; Each cycle: PRBS BER on unit 0 in the Diag OS, cold boot
; into SONiC, the same PRBS BER again, cold boot back to
; Diag; while 1, no end. Boot selection drives the BIOS
; Boot Manager ('USB Device' = Diag, 'Hard Drive' = SONiC)
; Log: Logs\Blanton_BER_Cycle_<ts>.log, timestamps always on
; Report tools (not shipped): tmp/BER_Test_Cycle/
; parse_multi.py, gen_multi_html.py, validate_prbsget.py
; prbsstat STArt Interval=30, then sleep 40, then Ber
; !! the sleep MUST be longer than the Interval. Checked
; against the error counts that 'prbs get' reports in the
; same cycle: with sleep 10 Diag read ~60x low and SONiC
; ~40x low; with sleep 30 (= Interval) Diag was still
; ~60x low - its Ber is sent 30.15 s after STArt, just
; before the first interval is published. Interval=30 +
; sleep 40 matched on both sides over 73 cycles. Every
; BER log taken before 2026-09-30 14:13 is low by 40-60x
; !! SONiC -> Diag cold boot writes pcimem
; /sys/bus/pci/devices/0000:05:00.0/resource0 0x480 with a
; hard-coded BDF (the Diag side uses fpga_switch.sh, which
; finds the FPGA itself). Check the BDF on every unit
; !! the SONiC password is the YourPaSsWoRd placeholder and
; the macro does not include config.ttl or secret/
; !! prompt_sonic_root is 'root@sonic:~#'. A unit whose
; hostname is not 'sonic' (M02 = Margin02) needs the
; prompts edited or the macro waits forever
; !! fan-speed-control.sh 30 is set on the SONiC side only;
; SONiC runs ~11 C hotter than Diag, a variable in any
; Diag vs SONiC comparison
; !! measures unit 0 only, port 268 not in the list, yet
; wait_init still waits for BOTH units (SWB_UNIT0/1 = 1)
; =============================================================================
include "config.ttl"
@@ -897,6 +951,10 @@ endif
include "utils/setup_pmon.ttl"
include "utils/show_dmesg.ttl"
; ========== Tray ID Stress Test ==========
wait prompt_sonic_root
sendln "~/Blanton_Script/tray_id_stress.sh"
;include "utils/xxx.ttl"
;wait prompt_sonic_root
;flushrecv
@@ -186,6 +186,11 @@ while 1
sendln "jobs"
wait prompt_sonic_root
sendln "systemctl is-active mlucas-amm"
; ========== Tray ID Stress Test ==========
wait prompt_sonic_root
sendln "~/Blanton_Script/tray_id_stress.sh"
wait prompt_sonic_root
sendln "sleep 60"
endwhile
@@ -0,0 +1,315 @@
;================================================================
; Prompt Definitions
;================================================================
prompt_login = "sonic login:"
prompt_sonic = "admin@sonic:~$"
prompt_sonic_root = "root@sonic:~#"
prompt_diag = "root@(none):~#"
; ============================================================
; [1] Config
; ============================================================
SWB_UNIT0 = 1 ; 1 = this DUT has switch unit 0, wait for it ; 0 = skip
SWB_UNIT1 = 1 ; 1 = this DUT has switch unit 1, wait for it ; 0 = skip
bb_post_hotkey = 'Post Code>>:0226' ; 看到此 POST code 後開始送 ESC
bb_esc_ack = 'Esc is pressed' ; BIOS 收到 ESC 的回應訊息
bb_fp_continue = 'continue to booting' ; Front Page:Continue 反白時的說明欄(同一 row 內的字串)
bb_fp_bootmgr = 'you to the Boot Manager' ; Front Page:Boot Manager 反白時的說明欄
bb_bm_header = 'EFI Boot Devices' ; Boot Manager 清單標題
bb_bm_footer = 'ESC to exit' ; Boot Manager 頁面最後一段文字(畫完判定)
bb_key_esc = #27
bb_key_down = #27'[B'
bb_key_enter = #13
;diag_tok = 'KINGSHARKS' ; EFI USB Device (KINGSHARKS-ANTU04)
;sonic_tok = 'J983670007' ; EFI Hard Drive (J983670007-TS2TMTE712P-VS2)
diag_tok = 'USB Device' ; EFI USB Device (KINGSHARKS-ANTU04)
sonic_tok = 'Hard Drive' ; EFI Hard Drive (J983670007-TS2TMTE712P-VS2)
diag_sig = 'ast1813b_diag' ; EFI Shell 執行 startup.nsh 載入的 diag kernel
sonic_sig = 'SONiC-OS' ; SONiC GRUB entry 名稱(請依實際 GRUB 畫面確認)
gettime str_starttime "%Y%m%d-%H%M%S"
getdir mdir
sprintf2 filename '%s\Logs\%s_%s_%s.log' mdir "Blanton" "BER_Cycle" str_starttime
; logopen <file> <binary> <append> <PlainText> <timestamp>
; binary=0 text mode, so PlainText / timestamp are honoured
; append=1 keep earlier cycles, do not overwrite
; PlainText=1 strip the ANSI colour codes SONiC emits
logopen filename 0 1 1 1
test_cycle = 0
while 1
test_cycle = test_cycle + 1
flushrecv
logwrite ''#13#10
sprintf2 log_cycle_msg 'BER Diag/SONiC Cycle Test : %d'#13#10 test_cycle
logwrite '#####################################'#13#10
logwrite log_cycle_msg
logwrite '#####################################'#13#10
settitle log_cycle_msg
; ============================================================
; [2] Call : Script where switching is needed
; ============================================================
; ====== Diag Test ======
wait prompt_diag
sendln 'swutil 0:dsh -c "phydiag 1-24,40-96,112-168,184-240,256-259,270-308,310-327,342-380,382-399,414-452,454-471,486-524,526-543,558-565 prbs set p=3"'
wait prompt_diag
sendln 'swutil 0:dsh -c "phydiag 1-24,40-96,112-168,184-240,256-259,270-308,310-327,342-380,382-399,414-452,454-471,486-524,526-543,558-565 prbs get"'
wait prompt_diag
sendln 'swutil 0:dsh -c "phydiag 1-24,40-96,112-168,184-240,256-259,270-308,310-327,342-380,382-399,414-452,454-471,486-524,526-543,558-565 prbs get"'
wait prompt_diag
sendln 'swutil 0:dsh -c "phydiag 1-24,40-96,112-168,184-240,256-259,270-308,310-327,342-380,382-399,414-452,454-471,486-524,526-543,558-565 prbsstat STArt Interval=30"'
wait prompt_diag
sendln "sleep 40"
wait prompt_diag
sendln 'swutil 0:dsh -c "phydiag 1-24,40-96,112-168,184-240,256-259,270-308,310-327,342-380,382-399,414-452,454-471,486-524,526-543,558-565 prbsstat Ber"'
wait prompt_diag
sendln 'swutil 0:dsh -c "phydiag 1-24,40-96,112-168,184-240,256-259,270-308,310-327,342-380,382-399,414-452,454-471,486-524,526-543,558-565 prbsstat STOp"'
wait prompt_diag
sendln 'swutil 0:dsh -c "phydiag 1-24,40-96,112-168,184-240,256-259,270-308,310-327,342-380,382-399,414-452,454-471,486-524,526-543,558-565 prbs clear"'
wait prompt_diag
sendln 'swutil 0:dsh -c "phydiag 256-259 dsc"'
wait prompt_diag
sendln 'swutil 0:show temp'
wait prompt_diag
sendln "swutil *:extphy core_diag all"
wait prompt_diag
; ---- Diag -> SONiC ----
sendln 'fpga_switch.sh cb 0 0x480 0xD0000'
bb_target_tok = sonic_tok
bb_sig_target = sonic_sig
bb_sig_other = diag_sig
call bb_boot_to
if bb_ok = 0 then
sprintf2 bb_msg 'Boot to SONiC failed: %s' bb_err
messagebox bb_msg 'bios_boot'
end
endif
; ========== SONiC Test ==========
wait prompt_login
sendln "admin"
wait "Password:"
sendln "YourPaSsWoRd"
wait prompt_sonic
sendln "sudo -i"
wait prompt_sonic_root
sendln "sleep 10"
; ========== FAN SPEED ==========
wait prompt_sonic_root
sendln "echo 18-0020 > /sys/bus/i2c/drivers/max31790/unbind"
wait prompt_sonic_root
sendln "sleep .5"
wait prompt_sonic_root
sendln "echo 18-0020 > /sys/bus/i2c/drivers/max31790/bind"
wait prompt_sonic_root
sendln "echo 25-0020 > /sys/bus/i2c/drivers/max31790/unbind"
wait prompt_sonic_root
sendln "sleep .5"
wait prompt_sonic_root
sendln "echo 25-0020 > /sys/bus/i2c/drivers/max31790/bind"
wait prompt_sonic_root
sprintf2 fan_ctrl_cmd 'fan-speed-control.sh %d' 30
sendln fan_ctrl_cmd
wait "Set all configured fan channels to"
sendln "y"
; ========== Wait DUT Init ==========
include "utils/wait_init.ttl"
wait prompt_sonic_root
sendln 'bcmcmd -n 0 -c " dsh -c ' 39 'phydiag 1-24,40-96,112-168,184-240,256-259,270-308,310-327,342-380,382-399,414-452,454-471,486-524,526-543,558-565 prbs set p=3' 39 ' "'
wait prompt_sonic_root
sendln 'bcmcmd -n 0 -c " dsh -c ' 39 'phydiag 1-24,40-96,112-168,184-240,256-259,270-308,310-327,342-380,382-399,414-452,454-471,486-524,526-543,558-565 prbs get' 39 ' "'
wait prompt_sonic_root
sendln 'bcmcmd -n 0 -c " dsh -c ' 39 'phydiag 1-24,40-96,112-168,184-240,256-259,270-308,310-327,342-380,382-399,414-452,454-471,486-524,526-543,558-565 prbs get' 39 ' "'
wait prompt_sonic_root
sendln 'bcmcmd -n 0 -c " dsh -c ' 39 'phydiag 1-24,40-96,112-168,184-240,256-259,270-308,310-327,342-380,382-399,414-452,454-471,486-524,526-543,558-565 prbsstat STArt Interval=30' 39 ' "'
wait prompt_sonic_root
sendln "sleep 40"
wait prompt_sonic_root
sendln 'bcmcmd -n 0 -c " dsh -c ' 39 'phydiag 1-24,40-96,112-168,184-240,256-259,270-308,310-327,342-380,382-399,414-452,454-471,486-524,526-543,558-565 prbsstat Ber' 39 ' "'
wait prompt_sonic_root
sendln 'bcmcmd -n 0 -c " dsh -c ' 39 'phydiag 1-24,40-96,112-168,184-240,256-259,270-308,310-327,342-380,382-399,414-452,454-471,486-524,526-543,558-565 prbsstat STOp' 39 ' "'
wait prompt_sonic_root
sendln 'bcmcmd -n 0 -c " dsh -c ' 39 'phydiag 1-24,40-96,112-168,184-240,256-259,270-308,310-327,342-380,382-399,414-452,454-471,486-524,526-543,558-565 prbs clear' 39 ' "'
wait prompt_sonic_root
sendln 'bcmcmd -n 0 -c "dsh -c ' 39 'phydiag 256-259 dsc' 39 '"'
wait prompt_sonic_root
sendln 'show plat temperature'
; ---- SONiC -> Diag ----
wait prompt_sonic_root
sendln 'pcimem /sys/bus/pci/devices/0000:05:00.0/resource0 0x480 w 0xD0000'
bb_target_tok = diag_tok
bb_sig_target = diag_sig
bb_sig_other = sonic_sig
call bb_boot_to
if bb_ok = 0 then
sprintf2 bb_msg 'Boot to Diag failed: %s' bb_err
messagebox bb_msg 'bios_boot'
end
endif
endwhile
logclose
end
; ============================================================
; [3] Subroutine
; ============================================================
:bb_boot_to
bb_saved_to = timeout
call bb_run
timeout = bb_saved_to
mtimeout = 0
return
:bb_run
bb_ok = 0
bb_err = ''
call bb_enter_bios
if bb_ok = 0 return
call bb_goto_bootmgr
if bb_ok = 0 return
call bb_find_idx
if bb_ok = 0 return
call bb_select
call bb_verify
return
; ------------------------------------------------------------
; 等 POST 0226,之後連送 ESC 直到 BIOS 回應 'Esc is pressed'
; ------------------------------------------------------------
:bb_enter_bios
bb_ok = 0
flushrecv ; 清掉 reboot 前的舊資料,避免誤比對
timeout = 300
mtimeout = 0
wait bb_post_hotkey
if result = 0 then
bb_err = 'timeout waiting POST code 0226'
return
endif
timeout = 0
mtimeout = 300
for bb_i 1 200 ; 300 ms x 200 = 最多約 60 秒
send bb_key_esc
wait bb_esc_ack
if result > 0 then
bb_ok = 1
break ; 收到 ack 立即停送,避免多餘 ESC 把 Front Page 退掉
endif
next
mtimeout = 0
if bb_ok = 0 bb_err = 'no ESC ack from BIOS'
return
; ------------------------------------------------------------
; Front Page:預設反白在 Continue,按一次 Down 到 Boot Manager
; 以右側說明欄文字確認反白位置,再按 Enter
; ------------------------------------------------------------
:bb_goto_bootmgr
bb_ok = 0
timeout = 30
wait bb_fp_continue
if result = 0 then
bb_err = 'Front Page not shown'
return
endif
mpause 500
send bb_key_down
timeout = 5
wait bb_fp_bootmgr
if result = 0 then
bb_err = 'cannot highlight Boot Manager'
return
endif
mpause 300
send bb_key_enter
timeout = 15
wait bb_bm_header
if result = 0 then
bb_err = 'Boot Manager page not shown'
return
endif
bb_ok = 1
return
; ------------------------------------------------------------
; 解析 Boot Manager 清單,計算目標 entry 的 index(0-based)
; 每個 entry 以 'EFI '(含 UEFI)或 'SVOS' 開頭計數,命中目標字串即停止
; ------------------------------------------------------------
:bb_find_idx
bb_ok = 0
bb_idx = -1
bb_cnt = 0
timeout = 0
mtimeout = 3000
for bb_i 1 20
wait 'EFI ' 'SVOS' bb_target_tok
if result = 0 break ; 清單畫完仍未命中
if result = 3 then
bb_idx = bb_cnt - 1 ; 目標字串前的 'EFI ' 屬於目標 entry 本身
break
endif
bb_cnt = bb_cnt + 1
next
mtimeout = 0
if bb_idx < 0 then
bb_err = 'target entry not found in Boot Manager'
return
endif
timeout = 5
wait bb_bm_footer ; 等整頁畫完才開始按鍵
bb_ok = 1
return
; ------------------------------------------------------------
; 預設反白在第一個 entry,往下移 bb_idx 格後按 Enter
; ------------------------------------------------------------
:bb_select
mpause 300
if bb_idx > 0 then ; TTL 的 for 1 0 會倒數執行,必須先擋掉
for bb_i 1 bb_idx
send bb_key_down
timeout = 3
wait bb_bm_footer ; BIOS 每按一次會整頁重畫,畫完再按下一次
mpause 200
next
endif
send bb_key_enter
; 之後不可再送任何按鍵:Diag 走 EFI Shell,
; 任何按鍵都會被當成「跳過 / 中斷 startup.nsh」
return
; ------------------------------------------------------------
; 同時等目標與另一個 image 的特徵字串
; ------------------------------------------------------------
:bb_verify
bb_ok = 0
timeout = 300
wait bb_sig_target bb_sig_other
if result = 1 then
bb_ok = 1
elseif result = 2 then
bb_err = 'booted into wrong image'
else
bb_err = 'timeout waiting boot signature'
endif
return
@@ -0,0 +1,155 @@
#!/bin/bash
#==============================================================================
# Author : ETWen
# Date : 20261001
# Purpose : Tray ID I2C stress - write a pattern to the Tray ID register on each
# bus, read it back and compare. Patterns 0xA5 and 0x5A exercise every
# bit in both states (1010_0101 / 0101_1010).
# Usage : ./tray_id_stress.sh [-n loops] [-R] [-h]
# -n N repeat the whole sequence N times (default 1)
# -R do NOT restore the original register value at the end
# -h help
# Buses / address / register / patterns are set in the block below.
#
# Exit : 0 = all PASS, 1 = any FAIL or I2C error, 2 = usage
#
# Version History:
# V1.0.0 20261001 Initial Version
#==============================================================================
#------------------------------------------------------------------------------
# Settings - edit here
#------------------------------------------------------------------------------
BUSES=(34 35 38 39) # i2c buses, one Tray ID device each
ADDR=0x70 # device address
REG=0x02 # Tray ID register
PATTERNS=(0xa5 0x5a) # written in this order, each read back and compared
RESTORE=1 # 1 = write the original value back when done
LOOPS=1
usage() {
sed -n '/^# Usage/,/^# Exit/p' "$0" | sed 's/^# \{0,1\}//'
}
while getopts ":n:Rh" opt; do
case "$opt" in
n) LOOPS="$OPTARG" ;;
R) RESTORE=0 ;;
h) usage; exit 0 ;;
:) echo "ERROR: -$OPTARG needs a value" >&2; usage >&2; exit 2 ;;
\?) echo "ERROR: unknown option -$OPTARG" >&2; usage >&2; exit 2 ;;
esac
done
if ! [[ "$LOOPS" =~ ^[1-9][0-9]*$ ]]; then
echo "ERROR: -n must be a positive integer (got '$LOOPS')" >&2; exit 2
fi
for t in i2cset i2cget; do
command -v "$t" >/dev/null 2>&1 || { echo "ERROR: $t not found" >&2; exit 2; }
done
# Counters per bus
declare -A N_PASS N_FAIL N_ERR ORIG
for b in "${BUSES[@]}"; do N_PASS[$b]=0; N_FAIL[$b]=0; N_ERR[$b]=0; ORIG[$b]=""; done
# Read one byte; prints the value (0xNN) on success, returns 1 on failure.
rd() {
local out
out=$(i2cget -f -y "$1" "$ADDR" "$REG" 2>&1)
if [[ $? -ne 0 || ! "$out" =~ ^0x[0-9a-fA-F]{1,2}$ ]]; then
echo "$out"; return 1
fi
echo "$out"
}
TS() { date '+%Y-%m-%d %H:%M:%S'; }
echo "=========================================================="
echo " Tray ID I2C stress $(TS)"
echo " buses: ${BUSES[*]} addr $ADDR reg $REG"
echo " patterns: ${PATTERNS[*]} loops: $LOOPS restore: $RESTORE"
echo "=========================================================="
#------------------------------------------------------------------------------
# Original values (so they can be put back)
#------------------------------------------------------------------------------
for b in "${BUSES[@]}"; do
if v=$(rd "$b"); then
ORIG[$b]="$v"
printf "[bus %-2s %s reg %s] original value: %s\n" "$b" "$ADDR" "$REG" "$v"
else
printf "[bus %-2s %s reg %s] original value: READ ERROR (%s)\n" "$b" "$ADDR" "$REG" "$v"
fi
done
echo "----------------------------------------------------------"
#------------------------------------------------------------------------------
# Write / read-back
#------------------------------------------------------------------------------
for (( loop = 1; loop <= LOOPS; loop++ )); do
(( LOOPS > 1 )) && echo "--- loop $loop / $LOOPS ---"
for b in "${BUSES[@]}"; do
for p in "${PATTERNS[@]}"; do
tag=$(printf "[bus %-2s %s reg %s]" "$b" "$ADDR" "$REG")
if ! werr=$(i2cset -f -y "$b" "$ADDR" "$REG" "$p" 2>&1); then
N_ERR[$b]=$(( N_ERR[$b] + 1 ))
echo "$tag write $p -> WRITE ERROR (${werr:-no message}) ERROR"
continue
fi
if ! got=$(rd "$b"); then
N_ERR[$b]=$(( N_ERR[$b] + 1 ))
echo "$tag write $p -> READ ERROR (${got:-no message}) ERROR"
continue
fi
if (( got == p )); then
N_PASS[$b]=$(( N_PASS[$b] + 1 ))
echo "$tag write $p -> read $got PASS"
else
N_FAIL[$b]=$(( N_FAIL[$b] + 1 ))
echo "$tag write $p -> read $got FAIL"
fi
done
done
done
#------------------------------------------------------------------------------
# Restore
#------------------------------------------------------------------------------
if (( RESTORE )); then
echo "----------------------------------------------------------"
for b in "${BUSES[@]}"; do
o=${ORIG[$b]}; v=""
if [[ -z "$o" ]]; then
printf "[bus %-2s] restore skipped: original value was not readable\n" "$b"
continue
fi
if i2cset -f -y "$b" "$ADDR" "$REG" "$o" >/dev/null 2>&1 && v=$(rd "$b") && (( v == o )); then
printf "[bus %-2s] restored %s\n" "$b" "$o"
else
printf "[bus %-2s] RESTORE FAILED: wanted %s, read %s\n" "$b" "$o" "${v:-?}"
N_ERR[$b]=$(( N_ERR[$b] + 1 ))
fi
done
fi
#------------------------------------------------------------------------------
# Summary
#------------------------------------------------------------------------------
per_bus=$(( LOOPS * ${#PATTERNS[@]} ))
tp=0; tf=0; te=0
echo
echo "=========================== Summary =========================="
printf "%-6s %-6s %-6s %8s %6s %6s %6s %s\n" "BUS" "ADDR" "REG" "CHECKS" "PASS" "FAIL" "ERROR" "RESULT"
for b in "${BUSES[@]}"; do
p=${N_PASS[$b]}; f=${N_FAIL[$b]}; e=${N_ERR[$b]}
tp=$(( tp + p )); tf=$(( tf + f )); te=$(( te + e ))
r=PASS; (( f > 0 || e > 0 || p != per_bus )) && r=FAIL
printf "%-6s %-6s %-6s %8s %6s %6s %6s %s\n" "$b" "$ADDR" "$REG" "$per_bus" "$p" "$f" "$e" "$r"
done
total=$(( per_bus * ${#BUSES[@]} ))
overall=PASS; (( tf > 0 || te > 0 || tp != total )) && overall=FAIL
printf "%-6s %-6s %-6s %8s %6s %6s %6s %s\n" "TOTAL" "" "" "$total" "$tp" "$tf" "$te" "$overall"
echo "=============================================================="
echo "Tray ID stress RESULT: $overall ($(TS))"
[[ "$overall" == PASS ]] && exit 0 || exit 1