Compare commits
3
Commits
V1.1.7
...
4648f936c7
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
4648f936c7 | ||
|
|
9cdd651d79 | ||
|
|
4f2f9ed090 |
@@ -31,3 +31,6 @@ publish/*
|
|||||||
|
|
||||||
# 尚未上機驗證的腳本暫存區(不入 repo,測過再搬進 src/)
|
# 尚未上機驗證的腳本暫存區(不入 repo,測過再搬進 src/)
|
||||||
tmp/
|
tmp/
|
||||||
|
|
||||||
|
# mlucas 的 ELF binary(~99 MB),DUT image 已內含,不入 repo
|
||||||
|
src/Script_ABC_Blanton/hammer/
|
||||||
|
|||||||
+36
-6
@@ -112,7 +112,7 @@ Blanton_TTL_Script/
|
|||||||
│
|
│
|
||||||
├── CLAUDE.md # 專案記憶 & 給 Claude 的指令
|
├── CLAUDE.md # 專案記憶 & 給 Claude 的指令
|
||||||
├── ARCHITECTURE.md # 本架構文件
|
├── ARCHITECTURE.md # 本架構文件
|
||||||
├── .gitignore # secret/ + For_AI/ + tmp/ + logs + publish 產出
|
├── .gitignore # secret/ + For_AI/ + tmp/ + hammer/ + logs + publish 產出
|
||||||
├── .gitattributes # 換行符政策:.sh/.conf 釘 LF、.ttl 保持 CRLF
|
├── .gitattributes # 換行符政策:.sh/.conf 釘 LF、.ttl 保持 CRLF
|
||||||
│
|
│
|
||||||
├── docs/ # 規格與狀態文件(客戶提供 / 專案產出)
|
├── docs/ # 規格與狀態文件(客戶提供 / 專案產出)
|
||||||
@@ -161,7 +161,10 @@ Blanton_TTL_Script/
|
|||||||
│
|
│
|
||||||
├── 4_Blanton_Script_thermal_safety.ttl # 熱 / 安規:獨立跑,不接 A→B→C
|
├── 4_Blanton_Script_thermal_safety.ttl # 熱 / 安規:獨立跑,不接 A→B→C
|
||||||
├── 5_Blanton_Script_Reboot_Cycle.ttl # 重開機循環:獨立跑,while 1 無止境(未上機驗證)
|
├── 5_Blanton_Script_Reboot_Cycle.ttl # 重開機循環:獨立跑,while 1 無止境(未上機驗證)
|
||||||
├── config.ttl # 全域設定:testcase 名、EN_Margin / EN_log、三個 prompt 字串
|
├── config.ttl # 全域設定:testcase 名、EN_Margin / EN_log / EN_timestamp、
|
||||||
|
│ # 風扇與 SWB 宣告、SSD/USB/BMC_USB 壓力秒數、三個 prompt 字串
|
||||||
|
├── hammer/ # 🚫 gitignored — mlucas ELF binary(~99 MB),DUT image 自帶,
|
||||||
|
│ # publish.sh 也排除;Script B / Script 4 只負責 chmod +x 後執行
|
||||||
├── 1_Blanton_Script_A.ttl # Pre-test:登入 → 對時 → source 工具 → 清 AER → 抓基線
|
├── 1_Blanton_Script_A.ttl # Pre-test:登入 → 對時 → source 工具 → 清 AER → 抓基線
|
||||||
├── 2_Blanton_Script_B.ttl # Stress:開壓力程序 → while 1 每輪抓 fan/temp/margin
|
├── 2_Blanton_Script_B.ttl # Stress:開壓力程序 → while 1 每輪抓 fan/temp/margin
|
||||||
├── 3_Blanton_Script_C.ttl # Post-test:kill → 再抓一次 → 比對 stress log
|
├── 3_Blanton_Script_C.ttl # Post-test:kill → 再抓一次 → 比對 stress log
|
||||||
@@ -367,9 +370,9 @@ PROFILE_CHANNELS=( "0:high:8" "1:low:8" ... ) # "channel:operation:change_perc
|
|||||||
|
|
||||||
| 模組 | 進入點 | 功能 |
|
| 模組 | 進入點 | 功能 |
|
||||||
|------|--------|------|
|
|------|--------|------|
|
||||||
| **Script A(Pre-test)** | `1_Blanton_Script_A.ttl` V1.1.7 | root 登入 → `date -s` 對時 → 清 job log → **收掉前一輪殘留**(`hw-test-session finish` → `bgctl reset --yes` → `bgctl stop --all` → `bgctl list` 確認清空)→ `hw-test-session start` → **設定風扇轉速** → **等資料面就緒**(`wait_init.ttl`)→ DUT 清單(boot/version/fwutil/syseeprom/ssdhealth/TPM/NVMe)→ BMC enroll + version/status → source bash 工具 → 清/讀 7 組 PCIe AER + `ras-mc-ctl --summary` → `lspci` → margin → PMON → dmesg → **10G/1G 管理網路 ping** → **NFC 基線**(start → 30s → stop → report)→ 100G uplink 設定與狀態 → **一輪流量基線** → `show uptime` |
|
| **Script A(Pre-test)** | `1_Blanton_Script_A.ttl` V1.1.8 | root 登入 → `date -s` 對時 → 清 job log → **收掉前一輪殘留**(`hw-test-session finish` → `bgctl reset --yes` → `bgctl stop --all` → `bgctl list` 確認清空)→ `hw-test-session start` → **設定風扇轉速** → **等資料面就緒**(`wait_init.ttl`)→ DUT 清單(boot/version/fwutil/syseeprom/ssdhealth/TPM/NVMe)→ BMC enroll + version/status → source bash 工具 → 清/讀 7 組 PCIe AER + `ras-mc-ctl --summary` → `lspci` → margin → PMON → dmesg → **10G/1G 管理網路 ping** → **NFC 基線**(start → 30s → stop → report)→ 100G uplink 設定與狀態 → **一輪流量基線** → `show uptime` |
|
||||||
| **Script B(Stress)** | `2_Blanton_Script_B.ttl` | `mlucas-avx2 -cpu 0:15` + `bgctl run` 的 `memtester` / SSD / USB 壓力(v1.1.7 起 SSD/USB 的 `--runtime`/`--passes` 讀 `config.ttl` 的 `SSD_RUNTIME`/`SSD_COUNT`、`USB_RUNTIME`/`USB_COUNT`;BMC memtester 不帶次數 = 跑到被停) + BMC DDR(`bmc-manager run memtester`)→ BMC USB net test(探測 cdc_ncm 介面,`systemd-run` 掛 4 小時 ping)→ 啟動 mgmt ping 監控 → **啟動 NFC 輪詢** → `jobs` / `bgctl list` → traffic(依 `SWB_UNIT`)→ `while 1` 每輪 PMON + `bgctl list`/`jobs` + `pause 60`(**V1.1.4 起 margin 不在迴圈內**)|
|
| **Script B(Stress)** | `2_Blanton_Script_B.ttl` | **V1.1.8 起 `mlucas-avx2 -cpu 0:15` 由 `systemd-run --unit=mlucas-amm`(`Type=exec`、`Restart=always`)啟動,不再是 shell 的 `&` 背景 job** + `bgctl run` 的 `memtester` / SSD / USB 壓力(v1.1.7 起 SSD/USB 的 `--runtime`/`--passes` 讀 `config.ttl` 的 `SSD_RUNTIME`/`SSD_COUNT`、`USB_RUNTIME`/`USB_COUNT`;BMC memtester 不帶次數 = 跑到被停) + BMC DDR(`bmc-manager run memtester`)→ BMC USB net test(探測 cdc_ncm 介面,`systemd-run` 掛 `BMC_USB_RUNTIME` 秒的 ping,V1.1.8 前寫死 4 小時)→ 啟動 mgmt ping 監控 → **啟動 NFC 輪詢** → `jobs` / `systemctl is-active mlucas-amm` / `bgctl list` → traffic(依 `SWB_UNIT`)→ `while 1` 每輪 PMON + `bgctl list`/`jobs`/`systemctl is-active mlucas-amm` + `pause 60`(**V1.1.4 起 margin 不在迴圈內**)|
|
||||||
| **Script C(Post-test)** | `3_Blanton_Script_C.ttl` | `kill $(jobs -p)` + `bgctl stop --all` / `reset` + 停 BMC USB unit、mgmt ping 與 **NFC 輪詢** → PMON + dmesg → 7 組 AER + `ras-mc-ctl` → BMC USB `journalctl` **寫進 `log/bmc_usb_net.log`**(不再印在 console) → `ip -s link show eth0/eth1` → `cat` mlucas log → **NFC `report`** → traffic `stop` + `report` → **BER 掃描**(依 `SWB_UNIT`)→ NVMe 健康 → `show reboot-cause` → **margin 掃描**(`EN_Margin`,V1.1.4 由 Script B 移來)→ `uptime` → **job log 複製到 USB(帶時戳)** → `exit` + wait『Script done』→ `hw-test-session finish` |
|
| **Script C(Post-test)** | `3_Blanton_Script_C.ttl` | `kill $(jobs -p)` + **`systemctl stop mlucas-amm`**(systemd unit 不是 shell job,`jobs -p` 抓不到)+ `bgctl stop --all` / `reset` + 停 BMC USB unit、mgmt ping 與 **NFC 輪詢** → PMON + dmesg → 7 組 AER + `ras-mc-ctl` → BMC USB `journalctl` **寫進 `log/bmc_usb_net.log`**(不再印在 console) → `ip -s link show eth0/eth1` → `cat` mlucas log(並 `cp` 進 job 目錄一起帶走)→ **NFC `report`** → traffic `stop` + `report` → **BER 掃描**(依 `SWB_UNIT`)→ NVMe 健康 → `show reboot-cause` → **margin 掃描**(`EN_Margin`,V1.1.4 由 Script B 移來)→ `uptime` → **job log 複製到 USB(帶時戳)** → `exit` + wait『Script done』→ `hw-test-session finish` |
|
||||||
| **Script 4(Thermal / Safety)** | `4_Blanton_Script_thermal_safety.ttl` | **獨立腳本,不接 A→B→C。** Script A 式的環境建立(登入 / session / 風扇 / `wait_init` / DUT 清單 / BMC / PCIe 清除 / `pcie_bus` / PMON / dmesg)→ **滿載**(`mlucas-avx2 -cpu 0:15` 背景 + `blanton_tr518.sh start`)→ `while 1`:PMON + `TH6_SWB0/1_power_readback.sh` + `bgctl list` / `jobs`,`pause 60`。**沒有收尾段** —— 中斷後 DUT 仍在 `port cd lb=mac` / `l2 learn off` / `test mode nr=yes`,要手動 `blanton_tr518.sh stop` |
|
| **Script 4(Thermal / Safety)** | `4_Blanton_Script_thermal_safety.ttl` | **獨立腳本,不接 A→B→C。** Script A 式的環境建立(登入 / session / 風扇 / `wait_init` / DUT 清單 / BMC / PCIe 清除 / `pcie_bus` / PMON / dmesg)→ **滿載**(`mlucas-avx2 -cpu 0:15` 背景 + `blanton_tr518.sh start`)→ `while 1`:PMON + `TH6_SWB0/1_power_readback.sh` + `bgctl list` / `jobs`,`pause 60`。**沒有收尾段** —— 中斷後 DUT 仍在 `port cd lb=mac` / `l2 learn off` / `test mode nr=yes`,要手動 `blanton_tr518.sh stop` |
|
||||||
| **Script 5(Reboot Cycle)** | `5_Blanton_Script_Reboot_Cycle.ttl` | **獨立腳本,不接 A→B→C。** `while 1`:登入 → `sudo -i` → 對時 → 風扇 → **Check Pt1~Pt13**(DUT/BMC info、mgmt ping、NFC、SSD、USB、BMC USB、port status + 15s traffic、BER、NVMe、platform data、dmesg i2c、PCIe error counters)→ `show uptime` → `reboot` → `wait "Booting in blind mode"`。`cnt_cycle` 每輪 +1 印在標頭。**沒有結束條件** —— 要停就關掉 Tera Term 的 macro。**尚未上機驗證** |
|
| **Script 5(Reboot Cycle)** | `5_Blanton_Script_Reboot_Cycle.ttl` | **獨立腳本,不接 A→B→C。** `while 1`:登入 → `sudo -i` → 對時 → 風扇 → **Check Pt1~Pt13**(DUT/BMC info、mgmt ping、NFC、SSD、USB、BMC USB、port status + 15s traffic、BER、NVMe、platform data、dmesg i2c、PCIe error counters)→ `show uptime` → `reboot` → `wait "Booting in blind mode"`。`cnt_cycle` 每輪 +1 印在標頭。**沒有結束條件** —— 要停就關掉 Tera Term 的 macro。**尚未上機驗證** |
|
||||||
| **SWB loopback 流量** | `blanton_traffic_linespeed <sub> [-u 0\|1\|all]` | `init`(VLAN 30..137,成對 `cdN`/`cdN+32`)→ `clear` → `start`(`tx 200 length=324 PatternRandom=yes`,v0.7.0 起)→ `stop` → `report`(per-pair TX/RX 交叉比對 PASS/FAIL);另有 `ps`(link/speed)與 `run`(一條龍)。**V0.6.0 起含 100G uplink `ce0`**(VLAN 138):它沒有 loopback 對象,靠 **ingress mirror 打回自己**維持循環,`report` 最後一列就是它,底下再接一份跨 unit summary |
|
| **SWB loopback 流量** | `blanton_traffic_linespeed <sub> [-u 0\|1\|all]` | `init`(VLAN 30..137,成對 `cdN`/`cdN+32`)→ `clear` → `start`(`tx 200 length=324 PatternRandom=yes`,v0.7.0 起)→ `stop` → `report`(per-pair TX/RX 交叉比對 PASS/FAIL);另有 `ps`(link/speed)與 `run`(一條龍)。**V0.6.0 起含 100G uplink `ce0`**(VLAN 138):它沒有 loopback 對象,靠 **ingress mirror 打回自己**維持循環,`report` 最後一列就是它,底下再接一份跨 unit summary |
|
||||||
@@ -479,6 +482,8 @@ pwr_data
|
|||||||
|
|
||||||
1. **A → B → C 必須依序跑,且 B 之後才能跑 C。** Script B 的背景 job 只存在於那個 shell session,
|
1. **A → B → C 必須依序跑,且 B 之後才能跑 C。** Script B 的背景 job 只存在於那個 shell session,
|
||||||
Script C 的 `kill $(jobs -p)` 依賴同一個 session;中途斷線就得手動收程序。
|
Script C 的 `kill $(jobs -p)` 依賴同一個 session;中途斷線就得手動收程序。
|
||||||
|
V1.1.8 起 `mlucas-amm` 例外 —— 它是 systemd transient unit,**跨 session 存活**,
|
||||||
|
所以 Script C 必須另外 `systemctl stop mlucas-amm`(見第 17 條)。
|
||||||
2. **A / B / C 共用同一個 log 檔**:只有 Script A `logopen`,B / C 用 `logwrite` 續寫分隔線。
|
2. **A / B / C 共用同一個 log 檔**:只有 Script A `logopen`,B / C 用 `logwrite` 續寫分隔線。
|
||||||
跑 B/C 前若關掉了 Tera Term,log 就斷了。
|
跑 B/C 前若關掉了 Tera Term,log 就斷了。
|
||||||
3. **PCIe AER 的判定是「差值」不是「絕對值」**:Script A 抓基線、Script C 抓結果,
|
3. **PCIe AER 的判定是「差值」不是「絕對值」**:Script A 抓基線、Script C 抓結果,
|
||||||
@@ -535,6 +540,19 @@ pwr_data
|
|||||||
**ping 成功但 `nic_tx` 接近 0 就代表封包從另一張卡出去了**。兩張卡在不同網段時可設 0。
|
**ping 成功但 `nic_tx` 接近 0 就代表封包從另一張卡出去了**。兩張卡在不同網段時可設 0。
|
||||||
16. **客戶 NDA**:`docs/` 內含客戶 FPGA 規格與 schematic 衍生資訊,**本 repo 只推 NAS + Gitea,
|
16. **客戶 NDA**:`docs/` 內含客戶 FPGA 規格與 schematic 衍生資訊,**本 repo 只推 NAS + Gitea,
|
||||||
絕不推 GitHub**。
|
絕不推 GitHub**。
|
||||||
|
17. **改用 `systemd-run` 起壓力工作,就要自己補停的那一行。** `utils/kill_all_process.ttl` 的內容
|
||||||
|
只有 `kill $(jobs -p)`,而 systemd unit 不是 shell job,它抓不到。V1.1.8 把 `mlucas-avx2`
|
||||||
|
搬到 `mlucas-amm` unit(好處:掉線不會停、`jobs` 看不見的問題消失、`Restart=always` 會自動拉回),
|
||||||
|
代價是 Script C 少了那行 `systemctl stop` 就會讓 CPU 滿載燒進收工量測與下一輪測試 ——
|
||||||
|
而 `show interfaces status` 之類的檢查完全看不出來。目前 `mlucas-amm` 與
|
||||||
|
`qfx5252-bmc-usb-net-test` 兩個 unit 都已在 Script C 收掉;
|
||||||
|
⚠️ **`4_Blanton_Script_thermal_safety.ttl` 的 `mlucas-avx2` 仍是 `&` 背景 job**,
|
||||||
|
兩支腳本目前用不同機制,改動時不要互相套用。
|
||||||
|
18. **`config.ttl` 少一個變數會讓 macro 當場中止**,不是當成 0。V1.1.8 新增 `EN_timestamp`
|
||||||
|
(Script A `logopen` 的第 5 個參數)與 `BMC_USB_RUNTIME`(Script B 的 ping unit)。
|
||||||
|
只更新 `.ttl` 卻沿用舊的 `config.ttl`,Script A 會停在 `logopen` 那一行。
|
||||||
|
另外 **時間戳的「格式」不在 macro 裡** —— 它在 Tera Term 的
|
||||||
|
Setup → Additional settings → Log,存在那台電腦的 `TERATERM.INI`,會跟著 PC 而不是跟著腳本走。
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
@@ -768,7 +786,7 @@ margin status、全部 VRM 軌的 Vin/Vout/Iout/Temp、CB + ICB 溫感讀值,
|
|||||||
- [ ] `CHANGELOG.md`:延續三支腳本頂端的 Version History,集中成一份
|
- [ ] `CHANGELOG.md`:延續三支腳本頂端的 Version History,集中成一份
|
||||||
- [ ] 三支腳本的 header 版號與 `CHANGELOG.md` 對齊,`config.ttl` 加 `script_version` 並寫進 log 開頭
|
- [ ] 三支腳本的 header 版號與 `CHANGELOG.md` 對齊,`config.ttl` 加 `script_version` 並寫進 log 開頭
|
||||||
- [ ] `docs/TTL_Script_Blanton_Status_*.xlsx` 更新為全 OK,或把剩餘項目移進 Future Extensions
|
- [ ] `docs/TTL_Script_Blanton_Status_*.xlsx` 更新為全 OK,或把剩餘項目移進 Future Extensions
|
||||||
- [x] 打 git tag,push `nas` + `gitea`(已至 `V1.1.7`;`publish/` 依 Script A 檔頭版號打包)
|
- [x] 打 git tag,push `nas` + `gitea`(已至 `V1.1.8`;`publish/` 依 Script A 檔頭版號打包)
|
||||||
|
|
||||||
**驗收條件:** 從 zip 解壓到一台乾淨的測試 NB,照 README 操作,不看原始碼就能完成一輪 A→B→C
|
**驗收條件:** 從 zip 解壓到一台乾淨的測試 NB,照 README 操作,不看原始碼就能完成一輪 A→B→C
|
||||||
並產出 summary;`publish/` 內容與 git tag 一致。
|
並產出 summary;`publish/` 內容與 git tag 一致。
|
||||||
@@ -817,6 +835,16 @@ margin status、全部 VRM 軌的 Vin/Vout/Iout/Temp、CB + ICB 溫感讀值,
|
|||||||
- **`tmp/blanton_reboottest.sh`(bash 版重開循環)也未上機驗證**:不用 Tera Term、由 systemd 在
|
- **`tmp/blanton_reboottest.sh`(bash 版重開循環)也未上機驗證**:不用 Tera Term、由 systemd 在
|
||||||
每次開機跑一輪並自動 `reboot`,登入下 `stop` 才停。整個 `tmp/` 是 gitignored,**不會進交付包** ——
|
每次開機跑一輪並自動 `reboot`,登入下 `stop` 才停。整個 `tmp/` 是 gitignored,**不會進交付包** ——
|
||||||
驗證過再搬進 `Blanton_Script/`。未驗證的腳本混進 `Blanton_Script/` 之後,測試員分不出哪支能信。
|
驗證過再搬進 `Blanton_Script/`。未驗證的腳本混進 `Blanton_Script/` 之後,測試員分不出哪支能信。
|
||||||
|
- **交付包不含 `hammer/`,但 Script B 與 Script 4 都要用它**:V1.1.8 起 `hammer/` 是 gitignored
|
||||||
|
且被 `publish.sh` 排除(~99 MB 的 mlucas binary,進 git 就永久留在 history)。這是刻意的 ——
|
||||||
|
binary 由 DUT image 帶 —— 但代表**測試員解開交付包後 `~/hammer` 必須已經在 DUT 上**,
|
||||||
|
否則 Script B 會在 `chmod +x ~/hammer/tools/amd/mlucas-avx2` 那一行失敗,而且 macro 不會停,
|
||||||
|
接著的 `systemd-run` 只是起一個立刻失敗又被 `Restart=always` 反覆重拉的 unit。
|
||||||
|
上機前先 `ls ~/hammer/tools/amd/mlucas-avx2` 確認一次。
|
||||||
|
- **`mlucas-amm` 的 `~` 依賴 systemd 的環境**:unit 是
|
||||||
|
`/bin/bash -c '~/hammer/... >> ~/hammer/mlucas_amm_log'`,system service 沒有 `User=` 時 systemd
|
||||||
|
**不會設 `HOME`**,bash 才退回 passwd 的家目錄(root → `/root`)解出同一條路徑。目前會通,
|
||||||
|
但那是剛好對上而不是保證;改成絕對路徑 `/root/hammer/...` 就不用賭。**尚未上機驗證。**
|
||||||
- **Script C 的 USB 掛載走 `mount` 而非 `usb_target.sh -o mnt`**:目前是 `-o dev` 取節點再自己
|
- **Script C 的 USB 掛載走 `mount` 而非 `usb_target.sh -o mnt`**:目前是 `-o dev` 取節點再自己
|
||||||
`mount <dev> /mnt/usb`,繞過了工具內建的 `mkdir -p`、exfat/ntfs `modprobe` 與可寫性驗證。
|
`mount <dev> /mnt/usb`,繞過了工具內建的 `mkdir -p`、exfat/ntfs `modprobe` 與可寫性驗證。
|
||||||
掛載點不存在、檔案系統模組沒載、或媒體唯讀時只會失敗一行,job log 就沒帶出來。
|
掛載點不存在、檔案系統模組沒載、或媒體唯讀時只會失敗一行,job log 就沒帶出來。
|
||||||
@@ -833,6 +861,8 @@ margin status、全部 VRM 軌的 Vin/Vout/Iout/Temp、CB + ICB 溫感讀值,
|
|||||||
(`SSD_RUNTIME`/`SSD_COUNT`、`USB_RUNTIME`/`USB_COUNT`,COUNT 直接對應 `--passes`)。
|
(`SSD_RUNTIME`/`SSD_COUNT`、`USB_RUNTIME`/`USB_COUNT`,COUNT 直接對應 `--passes`)。
|
||||||
`qfx5252-stress-*` 跑滿 RUNTIME 秒就結束,**`--passes` 不會把它延長**,所以
|
`qfx5252-stress-*` 跑滿 RUNTIME 秒就結束,**`--passes` 不會把它延長**,所以
|
||||||
`RUNTIME=600, COUNT=720` 是 10 分鐘的壓力而不是 120 小時。要跑滿一天就把 RUNTIME 填 `86400`。
|
`RUNTIME=600, COUNT=720` 是 10 分鐘的壓力而不是 120 小時。要跑滿一天就把 RUNTIME 填 `86400`。
|
||||||
|
**V1.1.8 起出貨預設是 `RUNTIME=3600`(1 小時)**,`BMC_USB_RUNTIME` 則是 `172800`(2 天)——
|
||||||
|
三個壓力源的長度目前刻意不一致,開跑前要照當次 soak 的時間各自對一次。
|
||||||
壓力提早結束時 Script B 每輪的 `bgctl list` 數量會變少,但**沒有任何東西會主動報警**;
|
壓力提早結束時 Script B 每輪的 `bgctl list` 數量會變少,但**沒有任何東西會主動報警**;
|
||||||
要比 24 小時更久則得加一段重啟 job 的邏輯。
|
要比 24 小時更久則得加一段重啟 job 的邏輯。
|
||||||
- **BMC USB 的 service log 只剩隨身碟一份**:Script C 把 `journalctl` 導進
|
- **BMC USB 的 service log 只剩隨身碟一份**:Script C 把 `journalctl` 導進
|
||||||
|
|||||||
@@ -9,14 +9,17 @@ DUT 端的取數能力由一組 **bash 工具庫** 提供(CB FPGA `pcimem` →
|
|||||||
|
|
||||||
完整架構見 [ARCHITECTURE.md](ARCHITECTURE.md)。
|
完整架構見 [ARCHITECTURE.md](ARCHITECTURE.md)。
|
||||||
|
|
||||||
**目前狀態(2026-09-04)**:Script A 為 **V1.1.7**(A 集中記錄所有 TTL 的變更)。涵蓋
|
**目前狀態(2026-09-06)**:Script A 為 **V1.1.8**(A 集中記錄所有 TTL 的變更)。涵蓋
|
||||||
PCIe AER / DDR EDAC / PMON+TPM / 電壓 margin(9 顆 LTC2980)/ SWB loopback 線速流量 /
|
PCIe AER / DDR EDAC / PMON+TPM / 電壓 margin(9 顆 LTC2980)/ SWB loopback 線速流量 /
|
||||||
10G+1G 管理網路 ping / BMC DDR 與 USB / NVMe 健康,壓力層走平台的 `bgctl`。
|
10G+1G 管理網路 ping / BMC DDR 與 USB / NVMe 健康,壓力層走平台的 `bgctl`。
|
||||||
switch unit 數量由 `config.ttl` 的 `SWB_UNIT0` / `SWB_UNIT1` 宣告,風扇轉速由 `FAN_SPEED` 指定
|
switch unit 數量由 `config.ttl` 的 `SWB_UNIT0` / `SWB_UNIT1` 宣告,風扇轉速由 `FAN_SPEED` 指定
|
||||||
(預設 100 = 全速;soak 是熱測試,散熱不該是變因)。
|
(預設 100 = 全速;soak 是熱測試,散熱不該是變因)。
|
||||||
測試 job log 收在 DUT 的 `/host/hw-eval/jobs/`,Script C 結束時掛好 USB 再帶時戳複製到 `/mnt/usb/`。
|
測試 job log 收在 DUT 的 `/host/hw-eval/jobs/`,Script C 結束時掛好 USB 再帶時戳複製到 `/mnt/usb/`。
|
||||||
100G PRBS(`port_prbs_monitor.sh`)已隨包交付,但 **A/B/C 的呼叫仍註解掉**,還在 bring-up。
|
100G PRBS(`port_prbs_monitor.sh`)已隨包交付,但 **A/B/C 的呼叫仍註解掉**,還在 bring-up。
|
||||||
Script B 的 soak 迴圈每輪印 `bgctl list` + `jobs`,並 `pause 60`(V1.0.10 之前完全沒有 pause)。
|
Script B 的 soak 迴圈每輪印 `bgctl list` + `jobs` + `systemctl is-active mlucas-amm`,並 `pause 60`
|
||||||
|
(V1.0.10 之前完全沒有 pause)。V1.1.8 起 AMD CPU 壓力(`mlucas-avx2`)由 `systemd-run` 掛成
|
||||||
|
`mlucas-amm` transient unit(`Restart=always`),不再是 shell 的 `&` 背景 job —— 掉線也不會停,
|
||||||
|
Script C 用 `systemctl stop mlucas-amm` 收掉並把 log 複製到隨身碟。
|
||||||
V1.1.0 新增全機 SerDes BER 掃描(`blanton_ber.sh`,**Script C 已接**,依 `SWB_UNIT` 分支)、
|
V1.1.0 新增全機 SerDes BER 掃描(`blanton_ber.sh`,**Script C 已接**,依 `SWB_UNIT` 分支)、
|
||||||
TR518 負載(`blanton_tr518.sh`,**TTL 尚未接**)、SWB 軌功耗(`TH6_SWB{0,1}_power_readback.sh`),
|
TR518 負載(`blanton_tr518.sh`,**TTL 尚未接**)、SWB 軌功耗(`TH6_SWB{0,1}_power_readback.sh`),
|
||||||
線速流量也納入 100G `ce0`(VLAN 138,靠 ingress mirror 維持循環)。
|
線速流量也納入 100G `ce0`(VLAN 138,靠 ingress mirror 維持循環)。
|
||||||
@@ -255,7 +258,17 @@ Tera Term 連 COM port (115200-8-N-1)
|
|||||||
`SSD_RUNTIME`/`SSD_COUNT`、`USB_RUNTIME`/`USB_COUNT` 指定(COUNT 就是 `--passes`)。
|
`SSD_RUNTIME`/`SSD_COUNT`、`USB_RUNTIME`/`USB_COUNT` 指定(COUNT 就是 `--passes`)。
|
||||||
`qfx5252-stress-*` 跑滿 RUNTIME 秒就停,**`--passes` 不會把它延長** —— `RUNTIME=600` 就是
|
`qfx5252-stress-*` 跑滿 RUNTIME 秒就停,**`--passes` 不會把它延長** —— `RUNTIME=600` 就是
|
||||||
10 分鐘,不是 600×COUNT。要 24 小時就填 `86400`。壓力提早結束時 `bgctl list` 數量變少是
|
10 分鐘,不是 600×COUNT。要 24 小時就填 `86400`。壓力提早結束時 `bgctl list` 數量變少是
|
||||||
唯一線索,**不會報警**
|
唯一線索,**不會報警**。V1.1.8 起出貨預設改為 `RUNTIME=3600`(1 小時),2 天 soak 要填 `172800`
|
||||||
|
- ⚠️ **`utils/kill_all_process.ttl` 只有 `kill $(jobs -p)`,抓不到 systemd unit**:把壓力工作改成
|
||||||
|
`systemd-run` 之後,Script C 一定要補對應的 `systemctl stop <unit>`,否則它會活過 Script C 燒進
|
||||||
|
下一輪,**收工的溫度/功耗全部是在滿載下量的**。V1.1.8 已為 `mlucas-amm` 補上;
|
||||||
|
**Script 4 的 `mlucas-avx2` 仍是 `&` 背景 job**,兩支目前用不同機制,改動時別互相套用
|
||||||
|
- ⚠️ **`config.ttl` 少一個變數會讓 macro 當場中止,不是當成 0**:`EN_timestamp`(V1.1.8 新增,
|
||||||
|
餵 Script A `logopen` 的第 5 個參數)、`BMC_USB_RUNTIME` 都是必要項。只更新 `.ttl` 卻沿用舊的
|
||||||
|
`config.ttl`,Script A 會停在 `logopen` 那一行
|
||||||
|
- ⚠️ **log 時間戳的「格式」不在腳本裡**:`EN_timestamp` 只決定開不開,格式在 Tera Term 的
|
||||||
|
Setup → Additional settings → Log,存在**那台電腦的 `TERATERM.INI`**。同一份 macro 在兩台機器上
|
||||||
|
可以產生不同格式的 log,跨機比對前先確認
|
||||||
- ⚠️ **多相 VRM 與 LTC2980 是同一批軌的兩條路**:`blanton_multiphase_margin.sh`(原生 i2c)
|
- ⚠️ **多相 VRM 與 LTC2980 是同一批軌的兩條路**:`blanton_multiphase_margin.sh`(原生 i2c)
|
||||||
和 `margin.sh`(LTC2980 PMBus)都假設自己是唯一寫入者,**沒有互斥檢查**,只靠人記得
|
和 `margin.sh`(LTC2980 PMBus)都假設自己是唯一寫入者,**沒有互斥檢查**,只靠人記得
|
||||||
- ⚠️ **traffic 的計數器基準在 v0.7.0 變了**:`tx 100 length=512` → `tx 200 length=324`,
|
- ⚠️ **traffic 的計數器基準在 v0.7.0 變了**:`tx 100 length=512` → `tx 200 length=324`,
|
||||||
@@ -355,5 +368,9 @@ Tera Term 連 COM port (115200-8-N-1)
|
|||||||
- `tmp/` — 🚫 gitignored:**還沒上機驗證**的腳本暫存區(目前 `blanton_reboottest.sh`)。
|
- `tmp/` — 🚫 gitignored:**還沒上機驗證**的腳本暫存區(目前 `blanton_reboottest.sh`)。
|
||||||
上機測過才搬進 `src/Script_ABC_Blanton/Blanton_Script/` 並進 git —— 未驗證的東西不該混在
|
上機測過才搬進 `src/Script_ABC_Blanton/Blanton_Script/` 並進 git —— 未驗證的東西不該混在
|
||||||
交付包裡,publish 時測試員分不出哪支能信
|
交付包裡,publish 時測試員分不出哪支能信
|
||||||
|
- `src/Script_ABC_Blanton/hammer/` — 🚫 gitignored:mlucas 的 ELF binary(~99 MB,avx512 / avx2 /
|
||||||
|
avx / sse2 / generic-c)。Script B 與 Script 4 會 `chmod +x` 後執行,但**它由 DUT image 帶**,
|
||||||
|
不從這個 repo 發 —— binary 一旦進 git 就永久留在 history,每個 clone 都要付這個代價。
|
||||||
|
`publish.sh` 也把它排除,所以打包結果不會因為誰的工作目錄有沒有 binary 而變成 100 MB
|
||||||
- `secret/` — 🚫 gitignored(除 `README.md` 與 `*.example`):DUT 帳密、per-unit BDF 覆寫、COM 設定
|
- `secret/` — 🚫 gitignored(除 `README.md` 與 `*.example`):DUT 帳密、per-unit BDF 覆寫、COM 設定
|
||||||
- `For_AI/` — 🚫 gitignored:AI 協作素材(波形截圖、草稿筆記)
|
- `For_AI/` — 🚫 gitignored:AI 協作素材(波形截圖、草稿筆記)
|
||||||
|
|||||||
@@ -0,0 +1,71 @@
|
|||||||
|
# v1.1.8 — The CPU stress no longer dies with your terminal
|
||||||
|
|
||||||
|
## ✨ New features
|
||||||
|
|
||||||
|
**`mlucas` now runs as a systemd unit, so it survives a lost session**
|
||||||
|
* The AMD CPU load is started as `systemd-run --unit=mlucas-amm --property=Type=exec --property=Restart=always --property=RestartSec=2`, instead of being backgrounded with `&` in the login shell.
|
||||||
|
* As a shell job it belonged to that one terminal: close Tera Term, drop the console, reconnect on a different session, and the load was gone — while `jobs` from the new session showed nothing to tell you. A multi-day soak could spend most of its length with an idle CPU and produce thermal numbers that looked fine because nothing was loading the part.
|
||||||
|
* `Restart=always` brings it back if it exits on its own, and the log is appended (`>>`) inside the unit while being truncated (`>`) once before the unit starts, so a restart adds to the run rather than erasing it.
|
||||||
|
* Script B now prints `systemctl is-active mlucas-amm` beside `jobs`, both in the one-shot check and in every pass of the monitor loop. One word — `active` — tells you the CPU is still loaded.
|
||||||
|
|
||||||
|
> The old `&` line is kept commented out directly above the new one, to fall back to if `systemd-run` is not available on a build.
|
||||||
|
|
||||||
|
**Script C stops the unit and brings its log home**
|
||||||
|
* `systemctl stop mlucas-amm` was added next to the existing `kill $(jobs -p)`, and `mlucas_amm_log` is copied into `/host/hw-eval/jobs/` with the other stress logs.
|
||||||
|
* `kill $(jobs -p)` cannot reach a systemd unit, so without this the load would have run straight through the post-test snapshot and into the next test — every temperature and power reading taken at full load, with nothing in `show interfaces status` to reveal it.
|
||||||
|
|
||||||
|
**`EN_timestamp` — turn the per-line log timestamp on or off from `config.ttl`**
|
||||||
|
* Script A's `logopen` takes its 5th argument from the new flag: `logopen filename 0 1 1 EN_timestamp`. The argument order (`<file> <binary> <append> <PlainText> <timestamp>`) is now spelled out above the call.
|
||||||
|
|
||||||
|
**`BMC_USB_RUNTIME` — the BMC USB ping duration is a knob, not a constant**
|
||||||
|
* It was hard-coded to `-w 14400` (4 hours), which no longer matched the soak lengths the SSD and USB knobs were being set to. The shipped default is `172800` (2 days).
|
||||||
|
|
||||||
|
**The shipped stress defaults are now one hour**
|
||||||
|
* `SSD_RUNTIME` and `USB_RUNTIME` go from 600 to `3600`. v1.1.7 shipped a 10 minute stress by default.
|
||||||
|
|
||||||
|
## 🐛 Bug fixes
|
||||||
|
|
||||||
|
**`publish.sh` produced a different bundle depending on whose machine it ran on**
|
||||||
|
* `rsync` copied whatever sat in `src/Script_ABC_Blanton/`, so a developer with the `hammer/` mlucas binaries in their working tree shipped a ~100 MB zip while a fresh clone shipped ~200 KB. `hammer/` is now excluded from the bundle and ignored by git.
|
||||||
|
* `publish.sh -h` also printed the first lines of code along with the usage text, and would have truncated the usage block as the header grew. It now reads the header up to the first line of code.
|
||||||
|
|
||||||
|
## 📦 Downloads
|
||||||
|
|
||||||
|
| File | Contents |
|
||||||
|
|---|---|
|
||||||
|
| `Script_ABC_Blanton_V1.1.8.zip` | The full Tera Term working directory, including `Blanton_Script/` to copy onto the DUT |
|
||||||
|
|
||||||
|
**After copying to the DUT:**
|
||||||
|
|
||||||
|
```bash
|
||||||
|
chmod +x ~/Blanton_Script/*.sh # see below
|
||||||
|
grep -rlU $'\r' ~/Blanton_Script # expect no output
|
||||||
|
ls ~/hammer/tools/amd/mlucas-avx2 # must already be there - see below
|
||||||
|
```
|
||||||
|
|
||||||
|
`blanton_ber.sh`, `nfc_polling.sh`, `blanton_tr518.sh`, `blanton_multiphase_margin.sh`, `pwr_brick.sh` and the two `TH6_SWB*_power_readback.sh` are invoked through `./` or `~/`, so they need the execute bit along with `bmc_monitor.sh` and `usb_target.sh` — neither git nor a Windows/USB copy carries it reliably.
|
||||||
|
|
||||||
|
## ⚠️ Before you run
|
||||||
|
|
||||||
|
* **Replace `config.ttl` along with the macros.** `EN_timestamp` and `BMC_USB_RUNTIME` are new, and Tera Term aborts the macro on an undefined variable rather than treating it as 0 — keeping an old `config.ttl` stops Script A at the `logopen` line.
|
||||||
|
* **`hammer/` is not in the bundle.** The ~99 MB of mlucas binaries comes with the DUT image, not from this repo. If `~/hammer/tools/amd/mlucas-avx2` is missing, Script B does not stop — it starts a unit that fails and is restarted every 2 seconds. Check for the file before you start.
|
||||||
|
* **The timestamp format is not in the macro.** `EN_timestamp` only decides whether there is one; the format lives in Tera Term's Setup → Additional settings → Log, stored in that PC's `TERATERM.INI`. Two benches running the same macro can produce differently stamped logs.
|
||||||
|
* **The three stress durations are deliberately different**: SSD and USB at 1 hour, the BMC USB ping at 2 days. Set each to match the soak you are actually running.
|
||||||
|
* **`--runtime` is the job's overall limit, not a per-pass one.** `qfx5252-stress-*` stops after `RUNTIME` seconds whatever `--passes` says. For a 24 hour soak set `RUNTIME` to `86400`.
|
||||||
|
* **`mlucas-amm` has not been run on hardware yet.** The unit resolves `~` through bash's fallback to the passwd entry, because systemd does not set `HOME` for a system service without `User=`. It should land on `/root/hammer`, the same path the interactive shell uses — confirm it on the bench, or change both paths to an absolute `/root/hammer/...`.
|
||||||
|
* **Script 4 still backgrounds `mlucas-avx2` with `&`.** The two macros use different mechanisms now; do not copy one's teardown into the other.
|
||||||
|
* **Script 5 (reboot cycle) is still unverified on hardware**, still has no exit condition, and `wait "Booting in blind mode"` still has no `timeout` guard.
|
||||||
|
* **`blanton_multiphase_margin.sh save` is permanent**, and must not be mixed with `margin.sh` — same rails, two access paths, no interlock.
|
||||||
|
* **`READ_VIN` and `READ_IIN` are still not in the ADPM12200 coefficient table.**
|
||||||
|
* **The BMC USB service log only exists on the USB stick.**
|
||||||
|
* **These three cannot share the switch**: `blanton_ber.sh`, `blanton_tr518.sh`, `blanton_traffic_linespeed`.
|
||||||
|
* **`FAN_SPEED` is 100.**
|
||||||
|
* **Still unverified from v1.1.0**: the `ce0` ingress mirror, the 120 s settle in `blanton_tr518.sh`, and `test mode nr=no` on stop.
|
||||||
|
|
||||||
|
## 🔗 Links
|
||||||
|
|
||||||
|
* [ARCHITECTURE.md](https://etgit.et-wen.com/etwen/Blanton_TTL_Script/src/branch/main/ARCHITECTURE.md) — test flow, data models, constraints
|
||||||
|
* [CLAUDE.md](https://etgit.et-wen.com/etwen/Blanton_TTL_Script/src/branch/main/CLAUDE.md) — commands and the bench gotchas
|
||||||
|
* [Blanton_Test_Flow.drawio](https://etgit.et-wen.com/etwen/Blanton_TTL_Script/src/branch/main/docs/Blanton_Test_Flow.drawio) — the scripts side by side
|
||||||
|
|
||||||
|
**Full changelog:** [V1.1.7...V1.1.8](https://etgit.et-wen.com/etwen/Blanton_TTL_Script/compare/V1.1.7...V1.1.8)
|
||||||
+14
-2
@@ -7,6 +7,9 @@
|
|||||||
# publish/Script_ABC_Blanton_<Version>, taking <Version>
|
# publish/Script_ABC_Blanton_<Version>, taking <Version>
|
||||||
# from the "; Version : Vx.y.z" header of
|
# from the "; Version : Vx.y.z" header of
|
||||||
# 1_Blanton_Script_A.ttl.
|
# 1_Blanton_Script_A.ttl.
|
||||||
|
# V1.0.1 2026-09-06 Exclude hammer/ (the mlucas binaries) from the bundle,
|
||||||
|
# so the output no longer depends on whether the developer
|
||||||
|
# has them in their working tree.
|
||||||
# =============================================================================
|
# =============================================================================
|
||||||
#
|
#
|
||||||
# The version is READ FROM Script A rather than passed in, so the folder name
|
# The version is READ FROM Script A rather than passed in, so the folder name
|
||||||
@@ -33,7 +36,9 @@ info() { printf '[\033[34mINFO\033[0m] %s\n' "$*"; }
|
|||||||
ok() { printf '[\033[32m OK \033[0m] %s\n' "$*"; }
|
ok() { printf '[\033[32m OK \033[0m] %s\n' "$*"; }
|
||||||
die() { printf '[\033[31mERR \033[0m] %s\n' "$*" >&2; exit 1; }
|
die() { printf '[\033[31mERR \033[0m] %s\n' "$*" >&2; exit 1; }
|
||||||
|
|
||||||
usage() { sed -n '2,26p' "${BASH_SOURCE[0]}" | sed 's/^# \{0,1\}//'; exit 0; }
|
# Print the header block above, up to (but not including) the first line of
|
||||||
|
# code, so adding a Version History entry never truncates the usage text.
|
||||||
|
usage() { sed -n '2,/^set -u/p' "${BASH_SOURCE[0]}" | sed '$d' | sed 's/^# \{0,1\}//'; exit 0; }
|
||||||
|
|
||||||
while [ $# -gt 0 ]; do
|
while [ $# -gt 0 ]; do
|
||||||
case "$1" in
|
case "$1" in
|
||||||
@@ -86,12 +91,19 @@ fi
|
|||||||
# Captured console logs are per-run and often carry DUT serials, so they are
|
# Captured console logs are per-run and often carry DUT serials, so they are
|
||||||
# never shipped; the empty logs/ directory itself is kept because Script A's
|
# never shipped; the empty logs/ directory itself is kept because Script A's
|
||||||
# logopen writes into it.
|
# logopen writes into it.
|
||||||
|
#
|
||||||
|
# hammer/ is excluded as well: it is ~99 MB of mlucas ELF binaries that the DUT
|
||||||
|
# image already carries, and it is gitignored. Without this exclude the bundle
|
||||||
|
# would depend on whether the developer happened to have the binaries sitting in
|
||||||
|
# src/ - 100 MB from one machine, 200 KB from a fresh clone.
|
||||||
mkdir -p -- "${DEST}" || die "cannot create ${DEST}"
|
mkdir -p -- "${DEST}" || die "cannot create ${DEST}"
|
||||||
if command -v rsync >/dev/null 2>&1; then
|
if command -v rsync >/dev/null 2>&1; then
|
||||||
rsync -a --exclude='*.log' "${SRC_DIR}/" "${DEST}/" || die "rsync failed"
|
rsync -a --exclude='*.log' --exclude='hammer/' "${SRC_DIR}/" "${DEST}/" \
|
||||||
|
|| die "rsync failed"
|
||||||
else
|
else
|
||||||
cp -a -- "${SRC_DIR}/." "${DEST}/" || die "cp failed"
|
cp -a -- "${SRC_DIR}/." "${DEST}/" || die "cp failed"
|
||||||
find "${DEST}" -type f -name '*.log' -delete
|
find "${DEST}" -type f -name '*.log' -delete
|
||||||
|
rm -rf -- "${DEST}/hammer"
|
||||||
fi
|
fi
|
||||||
|
|
||||||
# Tera Term's logopen target, in case the source tree never had one
|
# Tera Term's logopen target, in case the source tree never had one
|
||||||
|
|||||||
@@ -1,7 +1,7 @@
|
|||||||
; =============================================================================
|
; =============================================================================
|
||||||
; Script A for Blanton
|
; Script A for Blanton
|
||||||
; Version : V1.1.7
|
; Version : V1.1.8
|
||||||
; Date : 2026-09-04
|
; Date : 2026-09-06
|
||||||
; Author : ETWen
|
; Author : ETWen
|
||||||
; =============================================================================
|
; =============================================================================
|
||||||
; Version History:
|
; Version History:
|
||||||
@@ -341,6 +341,73 @@
|
|||||||
; sprintf2 cmd_ssd ... SSD_RUNTIME SSD_COUNT, and the same
|
; sprintf2 cmd_ssd ... SSD_RUNTIME SSD_COUNT, and the same
|
||||||
; for USB; the old hard-coded lines are left commented out
|
; for USB; the old hard-coded lines are left commented out
|
||||||
; next to them
|
; next to them
|
||||||
|
; V1.1.8 2026-09-06
|
||||||
|
; ScriptA Drive logopen's timestamp flag from EN_timestamp
|
||||||
|
; logopen filename 0 1 1 EN_timestamp, with the argument
|
||||||
|
; order spelled out above the call: <file> <binary>
|
||||||
|
; <append> <PlainText> <timestamp>
|
||||||
|
; binary must stay 0. In binary mode Tera Term writes the
|
||||||
|
; raw stream and both PlainText and the timestamp are
|
||||||
|
; ignored
|
||||||
|
; !! EN_timestamp has to exist in config.ttl. An undefined
|
||||||
|
; variable aborts the macro at logopen, it is not read
|
||||||
|
; as 0
|
||||||
|
; !! the timestamp FORMAT lives in TERATERM.INI (Setup >
|
||||||
|
; Additional settings > Log), so it follows the PC and
|
||||||
|
; not this script. Two testers can produce differently
|
||||||
|
; stamped logs from one macro
|
||||||
|
; ScriptB Run mlucas as a systemd unit, not a shell background job
|
||||||
|
; systemd-run --unit=mlucas-amm --property=Type=exec
|
||||||
|
; --property=Restart=always --property=RestartSec=2
|
||||||
|
; The '&' job belonged to the interactive shell: it died
|
||||||
|
; with the session, and 'jobs' could not see it from a
|
||||||
|
; reconnected one, so a soak that lost its terminal quietly
|
||||||
|
; stopped loading the CPU
|
||||||
|
; The log is truncated with '>' before the unit starts and
|
||||||
|
; appended with '>>' inside it, so a restart adds to the
|
||||||
|
; run instead of erasing it
|
||||||
|
; 'systemctl is-active mlucas-amm' now sits beside 'jobs',
|
||||||
|
; both in the one-shot check and in the monitor loop at the
|
||||||
|
; end
|
||||||
|
; !! the old '&' line is kept commented out directly above
|
||||||
|
; it, to fall back to if systemd-run is unavailable
|
||||||
|
; ScriptB Take the BMC USB ping duration from BMC_USB_RUNTIME
|
||||||
|
; sprintf2 cmd '... -w %d ...' ncm_if BMC_USB_RUNTIME. It
|
||||||
|
; was hard-coded to 14400 (4 h), which no longer matched
|
||||||
|
; the soak the SSD and USB knobs were set for
|
||||||
|
; ScriptC Stop mlucas-amm and copy its log to the USB stick
|
||||||
|
; utils/kill_all_process.ttl is only "kill $(jobs -p)", and
|
||||||
|
; a systemd unit is not a shell job
|
||||||
|
; ScriptB's change on its own would have left mlucas
|
||||||
|
; loading all 16 cores through ScriptC and into the next
|
||||||
|
; run, so every temperature and power reading after it
|
||||||
|
; would have been taken under load
|
||||||
|
; cp ~/hammer/tools/amd/mlucas_amm_log /host/hw-eval/jobs/
|
||||||
|
; joins the other stress logs
|
||||||
|
; !! ScriptB and ScriptC change together. Whoever moves a
|
||||||
|
; stress job to systemd owns the matching stop in
|
||||||
|
; ScriptC
|
||||||
|
; config.ttl Add EN_timestamp and BMC_USB_RUNTIME, raise stress runtimes
|
||||||
|
; EN_timestamp = 1 feeds ScriptA's logopen; BMC_USB_RUNTIME
|
||||||
|
; = 172800 feeds ScriptB's ping unit
|
||||||
|
; SSD_RUNTIME and USB_RUNTIME 600 -> 3600, so the shipped
|
||||||
|
; default is a one hour stress rather than the ten minutes
|
||||||
|
; V1.1.7 went out with
|
||||||
|
; !! --runtime is still the job's OVERALL limit. RUNTIME
|
||||||
|
; 3600 = one hour of stress; COUNT does not multiply it.
|
||||||
|
; For a 2 day soak set RUNTIME 172800
|
||||||
|
; .gitignore Ignore src/Script_ABC_Blanton/hammer/
|
||||||
|
; ~99 MB of unstripped mlucas ELF binaries (avx512 / avx2 /
|
||||||
|
; avx / sse2 / generic-c) that ScriptB and the thermal
|
||||||
|
; macro chmod and run on the DUT
|
||||||
|
; !! they are deliberately NOT in git. A binary tree that
|
||||||
|
; large is permanent once committed and every clone pays
|
||||||
|
; for it forever; the DUT image already carries hammer/
|
||||||
|
; publish/publish.sh Exclude hammer/ from the delivery bundle
|
||||||
|
; rsync copies whatever sits in src/, so a developer who
|
||||||
|
; happened to have the binaries locally would have shipped
|
||||||
|
; a 100 MB zip while a clone shipped 200 KB. The bundle is
|
||||||
|
; now the same either way
|
||||||
; =============================================================================
|
; =============================================================================
|
||||||
include "config.ttl"
|
include "config.ttl"
|
||||||
|
|
||||||
@@ -353,7 +420,11 @@ getdir mdir
|
|||||||
|
|
||||||
if EN_log = 1 then
|
if EN_log = 1 then
|
||||||
sprintf2 filename '%s\Logs\%s_%s_%s.log' mdir project_name strTestcase str_starttime
|
sprintf2 filename '%s\Logs\%s_%s_%s.log' mdir project_name strTestcase str_starttime
|
||||||
logopen filename 0 1 1 1
|
; logopen <file> <binary> <append> <PlainText> <timestamp>
|
||||||
|
; binary=0 text mode, so PlainText / timestamp are honoured
|
||||||
|
; append=1 keep earlier cycles, do not overwrite
|
||||||
|
; PlainText=1 strip the ANSI colour codes SONiC emits
|
||||||
|
logopen filename 0 1 1 EN_timestamp
|
||||||
endif
|
endif
|
||||||
|
|
||||||
; ========== Root Login ==========
|
; ========== Root Login ==========
|
||||||
|
|||||||
@@ -15,10 +15,15 @@ pause 1
|
|||||||
;sendln "command"
|
;sendln "command"
|
||||||
|
|
||||||
; ========== AMD CPU STRESS TEST ==========
|
; ========== AMD CPU STRESS TEST ==========
|
||||||
|
; Clear the log file first
|
||||||
|
wait prompt_sonic_root
|
||||||
|
sendln "> ~/hammer/tools/amd/mlucas_amm_log"
|
||||||
wait prompt_sonic_root
|
wait prompt_sonic_root
|
||||||
sendln 'chmod +x ~/hammer/tools/amd/mlucas-avx2'
|
sendln 'chmod +x ~/hammer/tools/amd/mlucas-avx2'
|
||||||
wait prompt_sonic_root
|
wait prompt_sonic_root
|
||||||
sendln "~/hammer/tools/amd/mlucas-avx2 -s l -cpu 0:15 > ~/hammer/tools/amd/mlucas_amm_log 2>&1 &"
|
;sendln "~/hammer/tools/amd/mlucas-avx2 -s l -cpu 0:15 > ~/hammer/tools/amd/mlucas_amm_log 2>&1 &"
|
||||||
|
sendln "systemd-run --unit=mlucas-amm --property=Type=exec --property=Restart=always --property=RestartSec=2 /bin/bash -c '~/hammer/tools/amd/mlucas-avx2 -s l -cpu 0:15 >> ~/hammer/tools/amd/mlucas_amm_log 2>&1'"
|
||||||
|
|
||||||
|
|
||||||
; ========== BMC DDR Stress Test ==========
|
; ========== BMC DDR Stress Test ==========
|
||||||
;wait prompt_sonic_root
|
;wait prompt_sonic_root
|
||||||
@@ -65,6 +70,8 @@ sendln cmd
|
|||||||
wait prompt_sonic_root
|
wait prompt_sonic_root
|
||||||
sendln "jobs"
|
sendln "jobs"
|
||||||
wait prompt_sonic_root
|
wait prompt_sonic_root
|
||||||
|
sendln "systemctl is-active mlucas-amm"
|
||||||
|
wait prompt_sonic_root
|
||||||
sendln "bgctl list"
|
sendln "bgctl list"
|
||||||
|
|
||||||
; ========== BMC USB Test ==========
|
; ========== BMC USB Test ==========
|
||||||
@@ -89,7 +96,7 @@ wait prompt_sonic_root
|
|||||||
sendln 'systemctl stop qfx5252-bmc-usb-net-test.service 2>/dev/null || true'
|
sendln 'systemctl stop qfx5252-bmc-usb-net-test.service 2>/dev/null || true'
|
||||||
wait prompt_sonic_root
|
wait prompt_sonic_root
|
||||||
sendln 'systemctl reset-failed qfx5252-bmc-usb-net-test.service 2>/dev/null || true'
|
sendln 'systemctl reset-failed qfx5252-bmc-usb-net-test.service 2>/dev/null || true'
|
||||||
sprintf2 cmd 'systemd-run --unit=qfx5252-bmc-usb-net-test --property=Type=exec /usr/bin/ping -I %s -i 1 -W 2 -w 14400 192.168.200.200' ncm_if
|
sprintf2 cmd 'systemd-run --unit=qfx5252-bmc-usb-net-test --property=Type=exec /usr/bin/ping -I %s -i 1 -W 2 -w %d 192.168.200.200' ncm_if BMC_USB_RUNTIME
|
||||||
wait prompt_sonic_root
|
wait prompt_sonic_root
|
||||||
sendln cmd
|
sendln cmd
|
||||||
|
|
||||||
@@ -179,5 +186,7 @@ while 1
|
|||||||
sendln "bgctl list"
|
sendln "bgctl list"
|
||||||
wait prompt_sonic_root
|
wait prompt_sonic_root
|
||||||
sendln "jobs"
|
sendln "jobs"
|
||||||
|
wait prompt_sonic_root
|
||||||
|
sendln "systemctl is-active mlucas-amm"
|
||||||
pause 60
|
pause 60
|
||||||
endwhile
|
endwhile
|
||||||
|
|||||||
@@ -12,6 +12,11 @@ pause 1
|
|||||||
|
|
||||||
; ========== Kill Process ==========
|
; ========== Kill Process ==========
|
||||||
include "utils/kill_all_process.ttl"
|
include "utils/kill_all_process.ttl"
|
||||||
|
wait prompt_sonic_root
|
||||||
|
sendln "systemctl is-active mlucas-amm"
|
||||||
|
wait prompt_sonic_root
|
||||||
|
sendln "systemctl stop mlucas-amm"
|
||||||
|
|
||||||
; bgctrl all stop
|
; bgctrl all stop
|
||||||
wait prompt_sonic_root
|
wait prompt_sonic_root
|
||||||
sendln "bgctl stop --all"
|
sendln "bgctl stop --all"
|
||||||
@@ -211,6 +216,8 @@ sendln cmd
|
|||||||
|
|
||||||
; ========== Copy Stress Log to USB ==========
|
; ========== Copy Stress Log to USB ==========
|
||||||
wait prompt_sonic_root
|
wait prompt_sonic_root
|
||||||
|
sendln "cp ~/hammer/tools/amd/mlucas_amm_log /host/hw-eval/jobs/"
|
||||||
|
wait prompt_sonic_root
|
||||||
sendln "cp ./Blanton_Script/log/bmc_usb_net.log /host/hw-eval/jobs/"
|
sendln "cp ./Blanton_Script/log/bmc_usb_net.log /host/hw-eval/jobs/"
|
||||||
wait prompt_sonic_root
|
wait prompt_sonic_root
|
||||||
sendln "cp ./Blanton_Script/log/bmc_poll.log /host/hw-eval/jobs/"
|
sendln "cp ./Blanton_Script/log/bmc_poll.log /host/hw-eval/jobs/"
|
||||||
@@ -220,7 +227,6 @@ wait prompt_sonic_root
|
|||||||
sendln "cp ./Blanton_Script/log/nfc_poll.log /host/hw-eval/jobs/"
|
sendln "cp ./Blanton_Script/log/nfc_poll.log /host/hw-eval/jobs/"
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
wait prompt_sonic_root
|
wait prompt_sonic_root
|
||||||
getdate ts_date "%Y%m%d"
|
getdate ts_date "%Y%m%d"
|
||||||
gettime ts_time "%H%M"
|
gettime ts_time "%H%M"
|
||||||
|
|||||||
@@ -6,6 +6,7 @@ strTestcase="ENV" ;ENV,EMC,Margin...etc
|
|||||||
|
|
||||||
EN_Margin = 0 ; 1: Enable Margin Test, 0: Disable Margin Test
|
EN_Margin = 0 ; 1: Enable Margin Test, 0: Disable Margin Test
|
||||||
EN_log = 1 ; 1: Enable log, 0: Disable log
|
EN_log = 1 ; 1: Enable log, 0: Disable log
|
||||||
|
EN_timestamp = 1 ; 1: Enable timestamp in log, 0: Disable timestamp in log
|
||||||
|
|
||||||
;================================================================
|
;================================================================
|
||||||
; FAN_SPEED
|
; FAN_SPEED
|
||||||
@@ -20,10 +21,11 @@ SWB_UNIT1 = 1 ; 1 = this DUT has switch unit 1, wait for it ; 0 = skip
|
|||||||
;================================================================
|
;================================================================
|
||||||
; Stress_Test
|
; Stress_Test
|
||||||
;================================================================
|
;================================================================
|
||||||
SSD_RUNTIME = 600 ; Unit seconds, 600 = 10 minutes
|
SSD_RUNTIME = 3600 ; Unit seconds, 3600 = 1 hour
|
||||||
SSD_COUNT = 720 ; Total time = SSD_RUNTIME * SSD_COUNT
|
SSD_COUNT = 240 ; Total time = SSD_RUNTIME * SSD_COUNT
|
||||||
USB_RUNTIME = 600 ; Unit seconds, 600 = 10 minutes
|
USB_RUNTIME = 3600 ; Unit seconds, 3600 = 1 hour
|
||||||
USB_COUNT = 720 ; Total time = USB_RUNTIME * USB_COUNT
|
USB_COUNT = 240 ; Total time = USB_RUNTIME * USB_COUNT
|
||||||
|
BMC_USB_RUNTIME = 172800 ; Unit seconds, 172800 = 2 days
|
||||||
|
|
||||||
;================================================================
|
;================================================================
|
||||||
; Prompt Definitions
|
; Prompt Definitions
|
||||||
|
|||||||
Reference in New Issue
Block a user