6 Commits
Author SHA1 Message Date
etwenandClaude Opus 5 5e1e4a6a8f docs(release): Add v1.0.8 notes; wire mgmt ping status into B and C
Script A: the baseline ping window is 30 s rather than 45, and the log
is cleared afterwards so Script B starts from nothing.

Script B: `status` after `start`, and Script C: `status` before `stop`
plus a `cat` of the report. The status line shows both pids and the
replies received per NIC, so a dead link is visible while the soak is
still running instead of only at the end.

docs/release-notes/v1.0.8.md written in the house format.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014RetWKFZFG1ZHcitQAyhwM
2026-08-26 10:13:34 +08:00
etwenandClaude Opus 5 a31c186142 feat(mgmt): Ping both NICs at once, and stop counting ARP as loss
The bench run reported FAIL=3 with no NIC errors and no drops. All three
failures were on eth1 and all three were missing exactly icmp_seq=1,
with the other nine replies present and sub-millisecond. That is
neighbour resolution: once the ARP entry for the target expires, the
first echo request is spent resolving it and ping counts it as loss.
Rounds sat about 24 s apart, right on the edge of the default reachable
time, so it happened on some rounds and not others.

mgmt_ping_monitor.sh V3.0.0:

- One discarded ping per NIC before the measured run resolves ARP, so
  MAX_LOSS_PCT can stay at 0 and a FAIL means a real drop. Raising the
  threshold instead would have hidden genuine single-packet loss, and a
  longer settle would not help -- nothing triggers ARP until the ping.
- Both NICs now ping at the same time and keep accumulating until
  stopped, rather than alternating fixed bursts. Each writes its own raw
  capture.
- `stop` sends SIGINT rather than SIGTERM, because ping prints its
  statistics block on interrupt and that block is what the report
  parses, then renders into the log: the last TAIL_LINES entries per
  NIC, each one's statistics and verdict, and `ip -s link show` for both
  interfaces.
- `-D -O` are on by default so every line carries a timestamp and a
  request that got no reply is visible in the tail instead of merely
  absent. Clear PING_EXTRA_OPTS if the platform's ping rejects them.

Since the script no longer re-execs itself to daemonise, `bash
mgmt_ping_monitor.sh start` works without the execute bit.

.gitignore: the new *.raw / *.pid / *.state runtime files were outside
the *.log rule and would have been committed.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014RetWKFZFG1ZHcitQAyhwM
2026-08-26 09:53:20 +08:00
etwenandClaude Opus 5 1b8d5a8e91 feat(usb): Detect the USB target instead of assuming /dev/sda1
New Blanton_Script/usb_target.sh reports a USB mass-storage device's
node, mount point or by-id path. It excludes whatever disk backs / and
/host -- this platform can boot from a USB DOM, which also reports
TRAN=usb -- refuses to guess when several USB disks are present, falls
back to the whole device for superfloppy media, and when mounting,
checks the result is actually writable rather than trusting that mount
succeeded.

Script B asks it for the device node and builds the stress command from
the answer, so a stick that enumerates as sdb no longer sends the write
test at whatever /dev/sda1 happens to be.

The device is carried back through an echo marker, split in the shell
literal as "<<D""EV=..." so the command echo cannot satisfy the pattern
the macro waits for -- the same false-match guard used for PORTS0= and
NCMIF=, arranged differently.

Script B: DDR memtester drops its 100-pass limit and runs until stopped,
matching the other soak loads.

Fix: Script C ran `bgctl reset --yes` while killing processes, which is
before the job logs are copied to USB. Moved to after the archive and
after hw-test-session finish, so the run's own logs are collected before
anything clears them.

Script A -> V1.0.8 history updated.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014RetWKFZFG1ZHcitQAyhwM
2026-08-25 17:03:36 +08:00
etwenandClaude Opus 5 cac37583e7 feat(ttl): Include mgmt_ping.log in the USB archive
Script C now copies log/mgmt_ping.log into
/host/hw-eval/current/jobs/ alongside bmc_poll.log, so the management
link results leave the DUT with the run instead of staying behind. Both
monitors' logs are now covered.

Script A history: the two copies are one action in one block, so they
share a single V1.0.8 line rather than reading as separate changes.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014RetWKFZFG1ZHcitQAyhwM
2026-08-25 15:57:23 +08:00
etwenandClaude Opus 5 802ee4d467 feat(ttl): Fold bmc_poll.log into the USB archive
Script C copies log/bmc_poll.log into /host/hw-eval/current/jobs/ before
that directory is archived to USB, so the BMC sampling and I2C
read-back results leave the DUT with the run they belong to rather than
only existing in the Tera Term capture.

The wait moves ahead of the getdate/sprintf2 block; those build strings
without touching the terminal, so the prompt is still consumed exactly
once before the copy is sent.

Script A history updated under the existing V1.0.8 block -- that version
is not tagged yet, so this is the same batch.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014RetWKFZFG1ZHcitQAyhwM
2026-08-25 15:55:05 +08:00
etwenandClaude Opus 5 166b53983f feat(bmc): Re-enable bmc_monitor with an I2C write/read-back test
bmc_monitor.sh comes back into the run with a second job beyond the
`free -m` sampling: an I2C pattern test against bus 1, address 0x41,
offset 0x00C0 -- write 55AA55AA and read it back, then write AA55AA55
and read that back. Two complementary patterns catch a stuck bit either
way round, and repeating it through the soak turns an intermittent I2C
fault into something the log shows rather than something the tester has
to catch live.

Script B starts it alongside the other stress; Script C stops it and
cats log/bmc_poll.log.

bmc_monitor_ddr.sh stays disabled -- BMC DDR stress runs through
bgctl + bmc-manager.

Docs: the "disabled" notes I added last time now applied to only half
the pair and read as wrong for bmc_monitor.sh. Split so the monitor is
documented as live with its I2C test, and only the _ddr variant is
marked disabled.

Script A -> V1.0.8.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014RetWKFZFG1ZHcitQAyhwM
2026-08-25 15:50:56 +08:00
10 changed files with 610 additions and 343 deletions
+5
View File
@@ -12,6 +12,11 @@ For_AI/
!**/logs/.gitkeep !**/logs/.gitkeep
!**/Logs/.gitkeep !**/Logs/.gitkeep
# monitor 腳本的執行期產物(原始擷取、pid、狀態)
*.raw
*.pid
*.state
# 打包輸出:產出不進 git,打包工具本身要進 # 打包輸出:產出不進 git,打包工具本身要進
publish/* publish/*
!publish/.gitkeep !publish/.gitkeep
+8 -6
View File
@@ -178,8 +178,8 @@ Blanton_TTL_Script/
│ # init/clear/show/start/stop/ps/report/run │ # init/clear/show/start/stop/ps/report/run
├── mgmt_ping_monitor.sh # 10G/1G 管理網路 ping 監控(iproute2 ├── mgmt_ping_monitor.sh # 10G/1G 管理網路 ping 監控(iproute2
│ # eth0→TARGET_10G、eth1→TARGET_1G,兩張卡都保持 up │ # eth0→TARGET_10G、eth1→TARGET_1G,兩張卡都保持 up
├── bmc_monitor.sh # ⚠️ 已停用:B/C 內呼叫處已註解,改走 bgctl + bmc-manager ├── bmc_monitor.sh # BMC 監控:free -m + I2C 寫入/讀回圖樣測試
├── bmc_monitor_ddr.sh # ⚠️ 已停用,同上 ├── bmc_monitor_ddr.sh # ⚠️ 已停用:B/C 內呼叫處已註解,DDR 壓力改走 bgctl + bmc-manager
│ # ↑ 三支都要 chmod +x 才能跑(見 Key Constraints │ # ↑ 三支都要 chmod +x 才能跑(見 Key Constraints
└── LTC2980_Margin_Script/ # 電壓 margin 子專案(另有獨立 repo └── LTC2980_Margin_Script/ # 電壓 margin 子專案(另有獨立 repo
├── margin.sh # margin_init/status/set/apply_profile/save ├── margin.sh # margin_init/status/set/apply_profile/save
@@ -312,7 +312,8 @@ PROFILE_CHANNELS=( "0:high:8" "1:low:8" ... ) # "channel:operation:change_perc
| **電源資料** | `pwr_data` / `pwr_data_pdb` / `pwr_pdb_id` | SWB VRM 全軌 Vin/Vout/Iout/Temp 對齊表;MP29816 解析度**執行期從 `MFR_VOUT_SCALE_LOOP` 讀**,不寫死 | | **電源資料** | `pwr_data` / `pwr_data_pdb` / `pwr_pdb_id` | SWB VRM 全軌 Vin/Vout/Iout/Temp 對齊表;MP29816 解析度**執行期從 `MFR_VOUT_SCALE_LOOP` 讀**,不寫死 |
| **電壓 Margin** | `margin_init/status/set/apply_profile/save` + `*_all.sh` | 9 顆 LTC2980CB CONN13、SWB0/1 CONN13~16)批次 high/low margin 與狀態掃描。v2.6.0 起 SWB 路徑真正接上共用 cb_i2c 後端,並有 `margin_swb_info/probe/scan/reset/sem` 除錯包裝 | | **電壓 Margin** | `margin_init/status/set/apply_profile/save` + `*_all.sh` | 9 顆 LTC2980CB CONN13、SWB0/1 CONN13~16)批次 high/low margin 與狀態掃描。v2.6.0 起 SWB 路徑真正接上共用 cb_i2c 後端,並有 `margin_swb_info/probe/scan/reset/sem` 除錯包裝 |
| **管理網路 ping** | `mgmt_ping_monitor.sh {start\|stop\|status\|fg\|tail\|clear\|summary}` | 一輪 = 10G leg + 1G leg,各自 `ip address/route replace``ping -I <if>` 打自己的對端;兩張卡都保持 up。每 leg 記 `ip -s link show` 與該卡的 TX/RX packet delta,另出一行可 grep 的 `RESULT` | | **管理網路 ping** | `mgmt_ping_monitor.sh {start\|stop\|status\|fg\|tail\|clear\|summary}` | 一輪 = 10G leg + 1G leg,各自 `ip address/route replace``ping -I <if>` 打自己的對端;兩張卡都保持 up。每 leg 記 `ip -s link show` 與該卡的 TX/RX packet delta,另出一行可 grep 的 `RESULT` |
| **BMC 監控 / 壓力** | `bmc_monitor.sh` / `bmc_monitor_ddr.sh` | ⚠️ **已停用** —— B/C 內呼叫處已註解,改由 `bgctl` + `bmc-manager run memtester` 取代。檔案保留未刪 | | **BMC 監控** | `bmc_monitor.sh {start\|stop\|status\|fg\|tail\|clear}` | 由 host 端 `bmc-manager run` 週期取樣 `free -m`,另加一組 **I2C 寫入/讀回圖樣測試**bus 1、addr 0x41、offset 0x00C0,寫 `55AA55AA` 讀回、再寫 `AA55AA55` 讀回)。B 啟動、C 停止並 `cat log/bmc_poll.log` |
| **BMC DDR 壓力** | `bmc_monitor_ddr.sh` | ⚠️ **已停用** —— B/C 內呼叫處已註解,改由 `bgctl run bmc-manager run memtester` 取代。檔案保留未刪 |
| **打包交付** | `publish/publish.sh [-v Ver] [-f] [-z] [-n]` | 版號讀 Script A 檔頭,複製成 `publish/Script_ABC_Blanton_<Ver>/`,排除 `*.log`,可另出 zip | | **打包交付** | `publish/publish.sh [-v Ver] [-f] [-z] [-n]` | 版號讀 Script A 檔頭,複製成 `publish/Script_ABC_Blanton_<Ver>/`,排除 `*.log`,可另出 zip |
--- ---
@@ -711,10 +712,11 @@ margin status、全部 VRM 軌的 Vin/Vout/Iout/Temp、CB + ICB 溫感讀值,
文件已加過期警告但內文未動。第 1~4 章不受影響。 文件已加過期警告但內文未動。第 1~4 章不受影響。
- **`wait_init.ttl` 沒有逾時出口**:若 pmon 一直起不來,`result` 既非 1 也非 2,迴圈會無限重試, - **`wait_init.ttl` 沒有逾時出口**:若 pmon 一直起不來,`result` 既非 1 也非 2,迴圈會無限重試,
Script A 既不往下走也不報錯。可加最大重試次數 + `messagebox` 提示。 Script A 既不往下走也不報錯。可加最大重試次數 + `messagebox` 提示。
- **腳本的執行權**`mgmt_ping_monitor.sh`(以及保留但已停用的 `bmc_monitor*.sh`)需人工 `chmod +x` - **腳本的執行權**`mgmt_ping_monitor.sh``bmc_monitor.sh`(以及已停用的 `bmc_monitor_ddr.sh`)需人工 `chmod +x`
可在 Script B 起它之前補一行(比照既有的 `chmod +x ~/hammer/tools/amd/mlucas-avx2`),或在 publish 時處理。 可在 Script B 起它之前補一行(比照既有的 `chmod +x ~/hammer/tools/amd/mlucas-avx2`),或在 publish 時處理。
- **`bmc_monitor.sh` / `bmc_monitor_ddr.sh` 已停用**B/C 內的呼叫已註解,改由 `bgctl` + - **`bmc_monitor_ddr.sh` 已停用**B/C 內的呼叫已註解,DDR 壓力改由 `bgctl run bmc-manager run memtester`
`bmc-manager run memtester` 取代。檔案還留在 repo,待確認不再需要後刪除,或在檔頭標註停用。 取代。檔案還留在 repo,待確認不再需要後刪除,或在檔頭標註停用。
`bmc_monitor.sh` 已於 2026-08-25 重新啟用,並加上 I2C 寫入/讀回圖樣測試。)
- **`rm /host/hw-eval/current/jobs/*` 清不掉子目錄**:Script A 用的是不帶 `-r``rm`,但 Script C - **`rm /host/hw-eval/current/jobs/*` 清不掉子目錄**:Script A 用的是不帶 `-r``rm`,但 Script C
`cp -r` 那個目錄,代表裡面可能有子目錄。清不乾淨的話,上一輪的 job log 會被當成這輪的一起 `cp -r` 那個目錄,代表裡面可能有子目錄。清不乾淨的話,上一輪的 job log 會被當成這輪的一起
複製到 USB。要確認 `bgctl` 的產出結構,必要時改成 `rm -rf .../jobs/*` 複製到 USB。要確認 `bgctl` 的產出結構,必要時改成 `rm -rf .../jobs/*`
+2 -1
View File
@@ -85,7 +85,8 @@ chmod +x ~/Blanton_Script/*.sh # 佈署後必做一次
~/Blanton_Script/mgmt_ping_monitor.sh stop ~/Blanton_Script/mgmt_ping_monitor.sh stop
~/Blanton_Script/mgmt_ping_monitor.sh summary # 只看每段的 RESULT 行 ~/Blanton_Script/mgmt_ping_monitor.sh summary # 只看每段的 RESULT 行
# ⚠️ tail 子命令是 tail -f,會卡住不返回;TTL 裡要倒 log 請用 cat # ⚠️ tail 子命令是 tail -f,會卡住不返回;TTL 裡要倒 log 請用 cat
# ⚠️ bmc_monitor.sh / bmc_monitor_ddr.sh 已停用(改走 bgctl + bmc-manager),檔案仍在 # bmc_monitor.sh start/stop # free -m + BMC I2C 寫入/讀回圖樣測試,log/bmc_poll.log
# ⚠️ bmc_monitor_ddr.sh 已停用(DDR 壓力改走 bgctl + bmc-manager),檔案仍在
# ── 背景壓力(平台工具)────────────────────────────── # ── 背景壓力(平台工具)──────────────────────────────
bgctl run /usr/sbin/memtester 1G 100 bgctl run /usr/sbin/memtester 1G 100
+61
View File
@@ -0,0 +1,61 @@
# v1.0.8 — Both management links tested at once, and ARP stops looking like packet loss
## ✨ New features
**Both management NICs are pinged simultaneously and continuously**
* `mgmt_ping_monitor.sh` no longer alternates fixed bursts. `start` configures both NICs, then pings from each at the same time and keeps accumulating until stopped — so a soak-long run is one continuous measurement rather than a series of snapshots.
* `stop` renders a report into the log: the last 20 entries per NIC, each one's ping statistics and PASS/FAIL, then `ip -s link show` for both interfaces.
* Every line is timestamped, and a request that got no reply prints a marker instead of merely being absent — a drop is visible in the tail, not inferred from a gap in the sequence numbers.
* `status` shows both pids and how many replies each NIC has received so far, which is the quick way to see one side is dead without waiting for the report.
**The USB stress target is detected, not assumed**
* New `usb_target.sh` reports an inserted USB device's node, mount point or by-id path. Script B asks it for the node and builds the stress command from the answer, so a stick that enumerates as `sdb` no longer sends a write test at whatever `/dev/sda1` happens to be.
* It excludes whatever disk backs `/` and `/host` — this platform can boot from a USB DOM, which also reports as USB — refuses to guess when several USB disks are present, and when it mounts, verifies the result is actually writable rather than trusting that `mount` succeeded.
**BMC I2C integrity is exercised through the soak**
* `bmc_monitor.sh` is back in the run and now writes two complementary patterns to a BMC scratch register and reads each back. One pattern alone cannot catch a bit stuck the same way it was written; repeating the pair through the soak turns an intermittent I2C fault into something the log records rather than something the tester has to witness.
**Test artefacts leave the DUT with the run**
* Script C copies `bmc_poll.log` and `mgmt_ping.log` into the job directory before it is archived to USB, so the monitors' output travels with the `bgctl` job logs.
* DDR stress now runs continuously instead of stopping after 100 passes, matching the other soak loads.
## 🐛 Bug fixes
**Three FAILs that were not link faults**
* A bench run reported `FAIL=3` with zero NIC errors, zero drops and sub-millisecond replies. Every failure was missing exactly `icmp_seq=1` and nothing else: once the neighbour entry for the target expires, the first echo request is spent resolving ARP and `ping` counts it as loss. A discarded warm-up ping per NIC now absorbs that, which is what lets the loss threshold stay at zero and still mean something. Raising the threshold instead would have hidden genuine single-packet loss.
**Job logs were being cleared before they were collected**
* Script C ran `bgctl reset --yes` while killing processes — before the job directory is copied to USB. Moved to after the archive, so a run's own logs are collected before anything clears them.
**The fan controller was bound twice**
* One of the two MAX31790 controllers received a second `bind` immediately after the first, which could only fail because the driver was already attached.
## 📦 Downloads
| File | Contents |
|---|---|
| `Script_ABC_Blanton_V1.0.8.zip` | The full Tera Term working directory, including `Blanton_Script/` to copy onto the DUT |
**After copying to the DUT:**
```bash
chmod +x ~/Blanton_Script/*.sh # see below
grep -rlU $'\r' ~/Blanton_Script # expect no output
```
`mgmt_ping_monitor.sh` no longer re-execs itself, so `bash mgmt_ping_monitor.sh start` works without the execute bit. `bmc_monitor.sh` and `usb_target.sh` still need it — and neither git (mode 100644) nor a Windows/USB copy carries it. Without the `chmod`, `start` fails with `Permission denied` and the later `cat` finds nothing: **the section ends up empty and nothing reports an error.**
## ⚠️ Before you run
* **Declare the switch population.** `SWB_UNIT0` / `SWB_UNIT1` in `config.ttl` say which switch units this DUT has. Traffic and the readiness gate both follow them; with neither set the traffic stage is skipped rather than failing against hardware that is not there.
* **Set the fan speed you want.** `FAN_SPEED` in `config.ttl` is applied at the start of every run.
* **Management connectivity is in use, not preserved.** Both NICs are up and pinging — drive the run from the serial console.
* **If the two management NICs share a subnet, keep `ARP_STRICT=1`.** Otherwise the target's ARP can be answered by either NIC and a reply may arrive on the one that did not send. Check `nic_tx` / `nic_rx` on the RESULT line: they are that interface's own counter delta.
* **Two settings persist to `config_db.json`** and survive a reboot: LLDP is left disabled and the 100G uplinks are left configured. Restore them before the DUT moves on.
## 🔗 Links
* [ARCHITECTURE.md](https://etgit.et-wen.com/etwen/Blanton_TTL_Script/src/branch/main/ARCHITECTURE.md) — test flow, data models, constraints
* [CLAUDE.md](https://etgit.et-wen.com/etwen/Blanton_TTL_Script/src/branch/main/CLAUDE.md) — commands and the bench gotchas
**Full changelog:** [V1.0.7...V1.0.8](https://etgit.et-wen.com/etwen/Blanton_TTL_Script/compare/V1.0.7...V1.0.8)
+19 -2
View File
@@ -1,6 +1,6 @@
; ============================================================================= ; =============================================================================
; Script A for Blanton ; Script A for Blanton
; Version : V1.0.7 ; Version : V1.0.8
; Date : 2026-08-25 ; Date : 2026-08-25
; Author : ETWen ; Author : ETWen
; ============================================================================= ; =============================================================================
@@ -43,6 +43,21 @@
; ScriptC Copy stress jobs log to USB, timestamped ; ScriptC Copy stress jobs log to USB, timestamped
; utils/wait_init.ttl WT_INTERVAL 10 -> 60, fix WT_MIN comments ; utils/wait_init.ttl WT_INTERVAL 10 -> 60, fix WT_MIN comments
; ScriptC Disable BMC USB journalctl dump ; ScriptC Disable BMC USB journalctl dump
; V1.0.8 2026-08-25 Blanton_Script/bmc_monitor.sh Add BMC I2C write/read-back pattern test
; ScriptB Re-enable bmc_monitor.sh start
; ScriptC Re-enable bmc_monitor.sh stop + cat bmc_poll.log
; ScriptC Copy bmc_poll.log + mgmt_ping.log into jobs/ before the USB archive
; Blanton_Script/usb_target.sh Created - auto-detect USB device node / mount point
; ScriptB USB stress uses the detected node, not a hard-coded /dev/sda1
; ScriptB DDR memtester runs continuously (drop the 100-pass limit)
; ScriptC Move bgctl reset --yes after the USB archive
; Blanton_Script/mgmt_ping_monitor.sh V3.0.0 - both NICs ping simultaneously, accumulating
; + ARP warm-up so a stale neighbour entry is not counted as loss
; + stop renders last-N per NIC, stats, and both ip -s link show
; .gitignore Ignore monitor *.raw / *.pid / *.state
; ScriptA MGMT ping baseline 45s -> 30s, clear the log afterwards
; ScriptB Show mgmt_ping status after start
; ScriptC Show mgmt_ping status before stop, cat the report after
; ============================================================================= ; =============================================================================
include "config.ttl" include "config.ttl"
@@ -194,10 +209,12 @@ include "utils/show_dmesg.ttl"
wait prompt_sonic_root wait prompt_sonic_root
sendln "./Blanton_Script/mgmt_ping_monitor.sh start" sendln "./Blanton_Script/mgmt_ping_monitor.sh start"
wait prompt_sonic_root wait prompt_sonic_root
pause 45 pause 30
sendln "~/Blanton_Script/mgmt_ping_monitor.sh stop" sendln "~/Blanton_Script/mgmt_ping_monitor.sh stop"
wait prompt_sonic_root wait prompt_sonic_root
sendln "cat ~/Blanton_Script/log/mgmt_ping.log" sendln "cat ~/Blanton_Script/log/mgmt_ping.log"
wait prompt_sonic_root
sendln "~/Blanton_Script/mgmt_ping_monitor.sh clear"
; ========== 100G Port Status ========== ; ========== 100G Port Status ==========
+23 -3
View File
@@ -28,17 +28,34 @@ sendln "~/hammer/tools/amd/mlucas-avx2 -s l -cpu 0:15 > ~/hammer/tools/amd/mluca
wait prompt_sonic_root wait prompt_sonic_root
sendln "bgctl run bmc-manager run '/usr/bin/memtester 64M 1'" sendln "bgctl run bmc-manager run '/usr/bin/memtester 64M 1'"
; ========== DDR MEMORY STRESS TEST ========== ; ========== BMC Monitor Test ==========
wait prompt_sonic_root wait prompt_sonic_root
sendln "bgctl run /usr/sbin/memtester 1G 100" sendln "./Blanton_Script/bmc_monitor.sh start"
; ========== DDR MEMORY STRESS TEST ==========
;wait prompt_sonic_root
;sendln "bgctl run /usr/sbin/memtester 1G 100"
wait prompt_sonic_root
sendln "bgctl run /usr/sbin/memtester 1G" ; Continue Execture
; ========== SSD read/write ========== ; ========== SSD read/write ==========
wait prompt_sonic_root wait prompt_sonic_root
sendln "bgctl run qfx5252-stress-ssd 1G --runtime 3600 --passes 5 --write --force --log /host/hw-eval/current/qfx5252-stress-ssd.log" sendln "bgctl run qfx5252-stress-ssd 1G --runtime 3600 --passes 5 --write --force --log /host/hw-eval/current/qfx5252-stress-ssd.log"
; ========== USB read/write ========== ; ========== USB read/write ==========
; ---- Get USB partition node -> usb_dev ----
usb_dev = ''
wait prompt_sonic_root wait prompt_sonic_root
sendln "bgctl run qfx5252-stress-usb /dev/sda1 256M --runtime 60 --passes 1 --write --log /host/hw-eval/qfx5252-stress-usb.log" sendln 'D=$(bash ~/Blanton_Script/usb_target.sh -o dev 2>/dev/null); echo "<<D""EV=${D:-NONE}>>"'
waitregex '<<DEV=([^>]*)>>'
usb_dev = groupmatchstr1
wait prompt_sonic_root
sprintf2 cmd "bgctl run qfx5252-stress-usb %s 256M --runtime 60 --passes 1 --write --log /host/hw-eval/qfx5252-stress-usb.log" usb_dev
sendln cmd
;wait prompt_sonic_root
;sendln "bgctl run qfx5252-stress-usb /dev/sda1 256M --runtime 60 --passes 1 --write --log /host/hw-eval/qfx5252-stress-usb.log"
; ========== SHOW Background job ========== ; ========== SHOW Background job ==========
wait prompt_sonic_root wait prompt_sonic_root
@@ -79,6 +96,9 @@ timeout = 0
; ========== 10G/1G MGMT Ping Test ========== ; ========== 10G/1G MGMT Ping Test ==========
wait prompt_sonic_root wait prompt_sonic_root
sendln "./Blanton_Script/mgmt_ping_monitor.sh start" sendln "./Blanton_Script/mgmt_ping_monitor.sh start"
wait prompt_sonic_root
pause 3
sendln "./Blanton_Script/mgmt_ping_monitor.sh status"
; ========== 100G Port ========== ; ========== 100G Port ==========
+18 -4
View File
@@ -17,15 +17,16 @@ wait prompt_sonic_root
sendln "bgctl stop --all" sendln "bgctl stop --all"
wait prompt_sonic_root wait prompt_sonic_root
sendln "bgctl stop --all" sendln "bgctl stop --all"
wait prompt_sonic_root
sendln "bgctl reset --yes"
; BMC USB Test STOP ; BMC USB Test STOP
wait prompt_sonic_root wait prompt_sonic_root
sendln "systemctl stop qfx5252-bmc-usb-net-test.service" sendln "systemctl stop qfx5252-bmc-usb-net-test.service"
; 10G/1G MGMT Test STOP ; 10G/1G MGMT Test STOP
wait prompt_sonic_root wait prompt_sonic_root
sendln "~/Blanton_Script/mgmt_ping_monitor.sh status"
wait prompt_sonic_root
sendln "~/Blanton_Script/mgmt_ping_monitor.sh stop" sendln "~/Blanton_Script/mgmt_ping_monitor.sh stop"
wait prompt_sonic_root
sendln "cat ~/Blanton_Script/log/mgmt_ping.log"
; ========== TAKE DATA ========== ; ========== TAKE DATA ==========
include "utils/setup_pmon.ttl" include "utils/setup_pmon.ttl"
@@ -53,6 +54,12 @@ sendln "/usr/sbin/ras-mc-ctl --summary"
;wait prompt_sonic_root ;wait prompt_sonic_root
;sendln "cat ./Blanton_Script/log/bmc_ddr.log" ;sendln "cat ./Blanton_Script/log/bmc_ddr.log"
; ========== BMC Monitor TAKE DATA ==========
wait prompt_sonic_root
sendln "./Blanton_Script/bmc_monitor.sh stop"
wait prompt_sonic_root
sendln "cat ./Blanton_Script/log/bmc_poll.log"
; ========== BMC USB Test result ========== ; ========== BMC USB Test result ==========
;wait prompt_sonic_root ;wait prompt_sonic_root
;sendln "journalctl -u qfx5252-bmc-usb-net-test.service --no-pager" ;sendln "journalctl -u qfx5252-bmc-usb-net-test.service --no-pager"
@@ -103,15 +110,22 @@ wait prompt_sonic_root
sendln "show uptime" sendln "show uptime"
; ========== Copy Stress Log to USB ========== ; ========== Copy Stress Log to USB ==========
wait prompt_sonic_root
sendln "cp ./Blanton_Script/log/bmc_poll.log /host/hw-eval/current/jobs/"
wait prompt_sonic_root
sendln "cp ./Blanton_Script/log/mgmt_ping.log /host/hw-eval/current/jobs/"
wait prompt_sonic_root
getdate ts_date "%Y%m%d" getdate ts_date "%Y%m%d"
gettime ts_time "%H%M" gettime ts_time "%H%M"
sprintf2 usb_dst "/mnt/usb/jobs-%s_%s" ts_date ts_time sprintf2 usb_dst "/mnt/usb/jobs-%s_%s" ts_date ts_time
sprintf2 cmd "cp -r /host/hw-eval/current/jobs/ %s" usb_dst sprintf2 cmd "cp -r /host/hw-eval/current/jobs/ %s" usb_dst
wait prompt_sonic_root
sendln cmd sendln cmd
; ========== HW Test Session ========== ; ========== HW Test Session ==========
wait prompt_sonic_root wait prompt_sonic_root
sendln "hw-test-session finish" sendln "hw-test-session finish"
wait prompt_sonic_root
sendln "bgctl reset --yes"
messagebox 'GOOD JOB! Test Case DONE' 'teraterm' messagebox 'GOOD JOB! Test Case DONE' 'teraterm'
@@ -34,6 +34,10 @@ set -u
# Add a line to add a command; comment it out to disable it. # Add a line to add a command; comment it out to disable it.
COMMANDS=( COMMANDS=(
"free -m" "free -m"
"i2ctransfer -f -y 1 w6@0x41 0x00 0xC0 0x55 0xAA 0x55 0xAA"
"i2ctransfer -f -y 1 w2@0x41 0x00 0xC0 r4"
"i2ctransfer -f -y 1 w6@0x41 0x00 0xC0 0xAA 0x55 0xAA 0x55"
"i2ctransfer -f -y 1 w2@0x41 0x00 0xC0 r4"
#"uptime" #"uptime"
#"cat /proc/loadavg" #"cat /proc/loadavg"
#"cat /proc/meminfo" #"cat /proc/meminfo"
@@ -2,48 +2,41 @@
############################################################################### ###############################################################################
# mgmt_ping_monitor.sh # mgmt_ping_monitor.sh
# #
# Version : V1.0.0 # Version : V3.0.0
# Author : ETWen # Author : ETWen
# Date : 20260824 # Date : 20260826
# #
# Purpose : Exercise both management NICs and ping each one's own target, # Purpose : Ping from BOTH management NICs at the same time, continuously, and
# appending everything to a rotating log. One round is # keep accumulating until stopped:
# #
# 10G (eth0) : ip link up + address/route replace -> settle # 10G (eth0) -> TARGET_10G
# -> ping N x TARGET_10G -> ip -s link show # 1G (eth1) -> TARGET_1G
# 1G (eth1) : ip link up + address/route replace -> settle
# -> ping N x TARGET_1G -> ip -s link show
# #
# Designed to run detached in the background, same sub-commands as # `stop` renders a report into the log: the last TAIL_LINES entries
# bmc_monitor.sh: start/stop/status/fg/tail/clear. # per NIC, each one's ping statistics and verdict, then
# `ip -s link show` for both interfaces.
# #
# Notes : - Both NICs are brought up and LEFT up. Addresses and default # Notes : - Both NICs are configured with iproute2 `replace`, which is
# routes are applied with iproute2 `replace`, which is idempotent: # idempotent, and both are left up. On a shared subnet the target's
# re-running a round does not stack duplicate addresses or routes. # ARP can be answered by either NIC, so ARP_STRICT keeps each one
# - The default routes carry different metrics (100 / 200), so the # to its own address.
# lower one wins for off-subnet traffic. They do NOT decide which # - ARP WARM-UP: the first packet after a neighbour entry expires is
# NIC a ping leaves by: `ping -I <iface>` binds to the device. # spent resolving ARP and is counted as loss. On the bench this
# - !! SAME-SUBNET WARNING !! With both NICs up on one subnet the # produced three FAILs whose only missing packet was icmp_seq=1,
# target ARPs for the address and Linux may answer from either # every time, with zero NIC errors or drops. One discarded ping
# NIC, so replies can arrive on the wrong one and the result is # before the measured run removes that artefact, which is what lets
# not attributable. ARP_STRICT=1 sets arp_ignore/arp_announce to # MAX_LOSS_PCT stay at 0 and still mean something.
# stop that. Leave it on unless the two NICs are on separate # - MANAGEMENT CONNECTIVITY IS IN USE while this runs. Drive it from
# subnets, as in the iproute2 example this was modelled on. # the serial console.
# - Each leg logs `ip -s link show <iface>` and the TX/RX packet
# delta across the burst, which is the direct evidence of which
# NIC actually carried the traffic.
# - The first pings after bringing a link up are lost to ARP, not to
# the link: a bench capture showed 4x "Destination Host
# Unreachable" on the first burst and 0% loss on the very next
# one. LINK_SETTLE_SEC absorbs that; it is not packet loss.
# #
# Version History # Version History
# V1.0.0 20260824 Initial Version # V1.0.0 20260824 Initial Version (ifconfig, one NIC at a time)
# V2.0.0 20260824 Switch from ifconfig to iproute2 (ip link / address # V2.0.0 20260824 iproute2; both NICs stay up; per-NIC target;
# replace / route replace with per-NIC metric). Both # ip -s link show + TX/RX delta per leg
# NICs now stay up -- no link down between legs. Add # V3.0.0 20260826 Both NICs ping SIMULTANEOUSLY and continuously instead
# `ip -s link show` per leg with a TX/RX delta, and # of alternating fixed bursts. Add the ARP warm-up.
# ARP_STRICT for the same-subnet case. # `stop` renders last-N per NIC + statistics + both
# ip -s link outputs into the log.
############################################################################### ###############################################################################
set -u set -u
@@ -63,149 +56,67 @@ METRIC_10G=100 # lower metric wins for off-subnet traffic
# --- 1G management port --- # --- 1G management port ---
IF_1G="eth1" IF_1G="eth1"
IP_1G="192.168.1.101" IP_1G="192.168.1.101"
TARGET_1G="192.168.1.31" # host this NIC pings TARGET_1G="192.168.1.31"
PLEN_1G=24 PLEN_1G=24
GW_1G="" GW_1G=""
METRIC_1G=200 METRIC_1G=200
# --- ping --- # --- ping ---
PING_COUNT=10 # echo requests per leg PING_INTERVAL_SEC=1 # seconds between echo requests (ping -i)
MAX_LOSS_PCT=0 # loss above this marks the leg FAIL MAX_LOSS_PCT=0 # loss above this marks the NIC FAIL
TAIL_LINES=20 # how many recent entries per NIC the report shows
# Both NICs stay up. If they share a subnet the target's ARP can be answered # Extra ping flags. -D timestamps every line, -O prints a marker for a request
# by either one, so a reply may arrive on the NIC that did not send. Setting # that got no reply, so a drop is visible in the tail instead of just missing.
# this applies arp_ignore=1 / arp_announce=2 to both, which keeps each NIC to # Clear this if the platform's ping does not accept them.
# its own address. RAM only -- reverts on reboot. Set 0 for separate subnets. PING_EXTRA_OPTS="-D -O"
# One discarded ping per NIC before the measured run, to resolve ARP. Without
# it the first packet of the run is lost to neighbour resolution and looks
# exactly like a link fault. 0 disables.
ARP_WARMUP=1
ARP_WARMUP_TIMEOUT=2 # seconds to wait for the warm-up reply
# Both NICs stay up. If they share a subnet the target's ARP can be answered by
# either one, so a reply may arrive on the NIC that did not send. This applies
# arp_ignore=1 / arp_announce=2 to both. RAM only -- reverts on reboot.
ARP_STRICT=1 ARP_STRICT=1
# Prefix for the ip commands; empty because the daemon already runs as root. # Prefix for the ip commands; empty because this already runs as root.
SUDO="" SUDO=""
# Seconds to wait after bringing a link up before pinging, so ARP is resolved # Seconds to wait after bringing a link up before pinging.
# and the PHY has finished autoneg. The bench needed a few seconds on the 1G.
LINK_SETTLE_SEC=5 LINK_SETTLE_SEC=5
# Seconds between rounds (0 = start the next round immediately)
INTERVAL_SEC=10
# Per-ping-burst timeout in seconds; stops a wedged ping from freezing the
# loop. 0 disables. Must exceed PING_COUNT (one packet per second).
PING_TIMEOUT_SEC=60
# Stop after this many rounds. 0 = run until stopped manually.
MAX_ROUNDS=0
# Logging # Logging
LOG_DIR="" # empty = <script directory>/log LOG_DIR="" # empty = <script directory>/log
LOG_NAME="mgmt_ping.log" LOG_NAME="mgmt_ping.log"
MAX_LOG_SIZE_MB=100 # rotate above this size; 0 = never rotate
MAX_LOG_KEEP=5 # keep .1 .. .N
# What `start` does with the log left behind by the previous run: # What `start` does with the log left behind by the previous run:
# new = begin with an empty log; previous content and rotated copies are # new = discard it (default -- one run, one log)
# discarded (default -- one run, one log) # archive = rename it to <log>.YYYYmmdd-HHMMSS first
# archive = rename the previous log to <log>.YYYYmmdd-HHMMSS, then begin empty # append = keep it
# append = keep appending to the existing log
START_LOG_MODE="new" START_LOG_MODE="new"
############################################################################### ###############################################################################
# Internal # Internal
############################################################################### ###############################################################################
SCRIPT_NAME="$(basename -- "${BASH_SOURCE[0]}")" SCRIPT_NAME="$(basename -- "${BASH_SOURCE[0]}")"
SCRIPT_PATH="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd)/${SCRIPT_NAME}" SCRIPT_DIR="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd)"
[ -n "${LOG_DIR}" ] || LOG_DIR="$(dirname -- "${SCRIPT_PATH}")/log" [ -n "${LOG_DIR}" ] || LOG_DIR="${SCRIPT_DIR}/log"
LOG_FILE="${LOG_DIR}/${LOG_NAME}"
PID_FILE="${LOG_DIR}/${LOG_NAME%.log}.pid"
LOG_TO_FILE=0 LOG_FILE="${LOG_DIR}/${LOG_NAME}"
STOP=0 BASE="${LOG_DIR}/${LOG_NAME%.log}"
SLEEP_PID="" PID_FILE="${BASE}.pid"
STATE_FILE="${BASE}.state"
RAW_10G="${BASE}_${IF_10G}.raw"
RAW_1G="${BASE}_${IF_1G}.raw"
ts() { date '+%Y-%m-%d %H:%M:%S'; } ts() { date '+%Y-%m-%d %H:%M:%S'; }
die() { printf '[ERROR] %s\n' "$*" >&2; exit 1; } die() { printf '[ERROR] %s\n' "$*" >&2; exit 1; }
on_signal() { # "<rx_packets> <tx_packets>" for an interface. ip -s link orders the columns
STOP=1 # "bytes packets errors ...", so packets is $2, not $1.
[ -n "${SLEEP_PID}" ] && kill "${SLEEP_PID}" 2>/dev/null
}
interruptible_sleep() {
local sec="$1"
[ "${sec}" -gt 0 ] 2>/dev/null || return 0
sleep "${sec}" &
SLEEP_PID=$!
wait "${SLEEP_PID}" 2>/dev/null
SLEEP_PID=""
}
is_running() {
local pid
[ -f "${PID_FILE}" ] || return 1
pid="$(cat "${PID_FILE}" 2>/dev/null)"
[[ "${pid}" =~ ^[0-9]+$ ]] || return 1
kill -0 "${pid}" 2>/dev/null || return 1
grep -qa "${SCRIPT_NAME}" "/proc/${pid}/cmdline" 2>/dev/null || return 1
printf '%s' "${pid}"
return 0
}
rotate_log_if_needed() {
local max_bytes size i
[ "${LOG_TO_FILE}" -eq 1 ] || return 0
[ "${MAX_LOG_SIZE_MB}" -gt 0 ] || return 0
[ -f "${LOG_FILE}" ] || return 0
max_bytes=$(( MAX_LOG_SIZE_MB * 1024 * 1024 ))
size="$(stat -c %s "${LOG_FILE}" 2>/dev/null || echo 0)"
[ "${size}" -ge "${max_bytes}" ] || return 0
for (( i = MAX_LOG_KEEP - 1; i >= 1; i-- )); do
[ -f "${LOG_FILE}.${i}" ] && mv -f "${LOG_FILE}.${i}" "${LOG_FILE}.$(( i + 1 ))"
done
mv -f "${LOG_FILE}" "${LOG_FILE}.1"
exec >>"${LOG_FILE}" 2>&1
printf '[%s] INFO : log rotated at %s MB\n' "$(ts)" "${MAX_LOG_SIZE_MB}"
}
remove_rotated_logs() {
local i
for (( i = 1; i <= MAX_LOG_KEEP + 1; i++ )); do
rm -f "${LOG_FILE}.${i}"
done
}
prepare_log_on_start() {
local stamp
case "${START_LOG_MODE}" in
append)
return 0
;;
archive)
if [ -s "${LOG_FILE}" ]; then
stamp="$(date '+%Y%m%d-%H%M%S')"
mv -f "${LOG_FILE}" "${LOG_FILE}.${stamp}" \
|| die "cannot archive ${LOG_FILE}"
printf 'previous log archived: %s.%s\n' "${LOG_FILE}" "${stamp}"
fi
remove_rotated_logs
;;
new)
# Truncate rather than unlink: the inode is preserved, so a tail -f
# that is already attached keeps following the new run.
if [ -f "${LOG_FILE}" ]; then
: > "${LOG_FILE}" || die "cannot truncate ${LOG_FILE}"
fi
remove_rotated_logs
;;
*)
die "invalid START_LOG_MODE: ${START_LOG_MODE}"
;;
esac
}
# Cumulative "<rx_packets> <tx_packets>" for an interface, from ip -s link.
# The numbers sit on the line after each RX:/TX: header, and the column order
# is "bytes packets errors ..." -- so packets is $2, not $1.
if_counters() { if_counters() {
${SUDO} ip -s link show "$1" 2>/dev/null | awk ' ${SUDO} ip -s link show "$1" 2>/dev/null | awk '
/RX:/ { getline; rx = $2 } /RX:/ { getline; rx = $2 }
@@ -213,11 +124,9 @@ if_counters() {
END { printf "%s %s", (rx == "" ? 0 : rx), (tx == "" ? 0 : tx) }' END { printf "%s %s", (rx == "" ? 0 : rx), (tx == "" ? 0 : tx) }'
} }
# Bring the NIC up and (re)apply its address and default route. `replace` is # Bring the NIC up and (re)apply its address and default route.
# idempotent, so repeated rounds do not stack duplicates.
setup_iface() { setup_iface() {
local iface="$1" ipaddr="$2" plen="$3" gw="$4" metric="$5" local iface="$1" ipaddr="$2" plen="$3" gw="$4" metric="$5"
${SUDO} ip link set "${iface}" up 2>&1 || return 1 ${SUDO} ip link set "${iface}" up 2>&1 || return 1
${SUDO} ip address replace "${ipaddr}/${plen}" dev "${iface}" 2>&1 || return 1 ${SUDO} ip address replace "${ipaddr}/${plen}" dev "${iface}" 2>&1 || return 1
if [ -n "${gw}" ]; then if [ -n "${gw}" ]; then
@@ -227,74 +136,6 @@ setup_iface() {
return 0 return 0
} }
# One leg: configure <if>, settle, ping its own target, report. The NIC is
# LEFT UP -- nothing is brought down between legs.
# Usage: ping_leg <label> <if> <ip> <plen> <gw> <metric> <target>
ping_leg() {
local label="$1" iface="$2" ipaddr="$3" plen="$4" gw="$5" metric="$6" target="$7"
local out rc tx rx loss avg verdict
local c0 c1 rx0 tx0 rx1 tx1 drx dtx
printf -- '----- [%s] %s : %s (%s/%s) -> %s -----\n' \
"$(ts)" "${label}" "${iface}" "${ipaddr}" "${plen}" "${target}"
if ! setup_iface "${iface}" "${ipaddr}" "${plen}" "${gw}" "${metric}"; then
printf '[%s] FAIL : cannot configure %s\n\n' "$(ts)" "${iface}"
return 1
fi
interruptible_sleep "${LINK_SETTLE_SEC}"
[ "${STOP}" -eq 0 ] || return 1
c0="$(if_counters "${iface}")"; rx0="${c0% *}"; tx0="${c0#* }"
if [ "${PING_TIMEOUT_SEC}" -gt 0 ]; then
out="$(timeout "${PING_TIMEOUT_SEC}" ping -I "${iface}" -c "${PING_COUNT}" "${target}" 2>&1)"
else
out="$(ping -I "${iface}" -c "${PING_COUNT}" "${target}" 2>&1)"
fi
rc=$?
printf '%s\n' "${out}"
c1="$(if_counters "${iface}")"; rx1="${c1% *}"; tx1="${c1#* }"
drx=$(( rx1 - rx0 )); dtx=$(( tx1 - tx0 ))
printf -- '--- ip -s link show %s ---\n' "${iface}"
${SUDO} ip -s link show "${iface}" 2>&1
# "10 packets transmitted, 10 received, 0% packet loss"
tx="$(printf '%s' "${out}" | sed -n 's/^\([0-9]\+\) packets transmitted.*/\1/p' | tail -1)"
rx="$(printf '%s' "${out}" | sed -n 's/.*[, ]\([0-9]\+\) received.*/\1/p' | tail -1)"
loss="$(printf '%s' "${out}" | sed -n 's/.*[, ]\([0-9]\+\)% packet loss.*/\1/p' | tail -1)"
# "rtt min/avg/max/mdev = 1.044/1.743/2.387/0.437 ms"
avg="$(printf '%s' "${out}" | sed -n 's|.*= [0-9.]*/\([0-9.]*\)/.*|\1|p' | tail -1)"
[ -n "${tx}" ] || tx=0
[ -n "${rx}" ] || rx=0
[ -n "${loss}" ] || loss=100
[ -n "${avg}" ] || avg="-"
if [ "${rc}" -eq 124 ]; then
verdict="FAIL(timeout)"
elif [ "${loss}" -le "${MAX_LOSS_PCT}" ]; then
verdict="PASS"
else
verdict="FAIL"
fi
# one greppable line per leg: grep RESULT <log>
# nic_tx/nic_rx are this interface's own counter delta across the burst --
# if they stay near zero while the ping succeeded, the traffic went out a
# different NIC and the result is not attributable to this one.
printf '[%s] RESULT %-3s %-6s -> %-15s tx=%s rx=%s loss=%s%% rtt_avg=%sms nic_tx=%s nic_rx=%s %s\n\n' \
"$(ts)" "${label}" "${iface}" "${target}" "${tx}" "${rx}" "${loss}" "${avg}" \
"${dtx}" "${drx}" "${verdict}"
[ "${verdict}" = "PASS" ]
}
# arp_ignore=1 : answer ARP only for addresses configured on the incoming NIC
# arp_announce=2: pick the source address from the outgoing NIC's own subnet
# Both are RAM-only sysctls and revert on reboot.
apply_arp_strict() { apply_arp_strict() {
local i local i
[ "${ARP_STRICT}" -eq 1 ] || return 0 [ "${ARP_STRICT}" -eq 1 ] || return 0
@@ -302,143 +143,245 @@ apply_arp_strict() {
${SUDO} sysctl -qw "net.ipv4.conf.${i}.arp_ignore=1" 2>/dev/null ${SUDO} sysctl -qw "net.ipv4.conf.${i}.arp_ignore=1" 2>/dev/null
${SUDO} sysctl -qw "net.ipv4.conf.${i}.arp_announce=2" 2>/dev/null ${SUDO} sysctl -qw "net.ipv4.conf.${i}.arp_announce=2" 2>/dev/null
done done
printf '[%s] INFO : arp_ignore=1 arp_announce=2 applied to %s,%s\n' \
"$(ts)" "${IF_10G}" "${IF_1G}"
} }
main_loop() { # Resolve the neighbour so the measured run does not spend its first packet on
local round=0 pass=0 fail=0 # ARP. Result is deliberately discarded.
arp_warmup() {
local iface="$1" target="$2"
[ "${ARP_WARMUP}" -eq 1 ] || return 0
ping -I "${iface}" -c 1 -W "${ARP_WARMUP_TIMEOUT}" "${target}" >/dev/null 2>&1
return 0
}
command -v ip >/dev/null 2>&1 || die "ip (iproute2) not found" # echo "pid_10g pid_1g" and return 0 when both are alive
command -v ping >/dev/null 2>&1 || die "ping not found" is_running() {
[ "${PING_COUNT}" -gt 0 ] 2>/dev/null || die "PING_COUNT must be > 0" local p0 p1
[ -f "${PID_FILE}" ] || return 1
p0="$(sed -n '1p' "${PID_FILE}" 2>/dev/null)"
p1="$(sed -n '2p' "${PID_FILE}" 2>/dev/null)"
[[ "${p0}" =~ ^[0-9]+$ ]] || return 1
[[ "${p1}" =~ ^[0-9]+$ ]] || return 1
kill -0 "${p0}" 2>/dev/null || kill -0 "${p1}" 2>/dev/null || return 1
printf '%s %s' "${p0}" "${p1}"
return 0
}
trap on_signal INT TERM ###############################################################################
# Report
###############################################################################
# The recent entries for one NIC: replies and, thanks to -O, the requests that
# got none.
tail_entries() {
grep -aE 'bytes from|no answer|Unreachable|Time to live' "$1" 2>/dev/null \
| tail -n "${TAIL_LINES}"
}
# The trailing "--- x ping statistics ---" block.
stats_block() {
sed -n '/ping statistics ---/,$p' "$1" 2>/dev/null
}
# render_leg <label> <iface> <target> <raw> <rx0> <tx0>
render_leg() {
local label="$1" iface="$2" target="$3" raw="$4" rx0="$5" tx0="$6"
local st tx rx loss avg verdict c1 rx1 tx1 drx dtx
printf -- '----- %s : %s -> %s : last %s entries -----\n' \
"${label}" "${iface}" "${target}" "${TAIL_LINES}"
tail_entries "${raw}"
printf '\n'
st="$(stats_block "${raw}")"
printf '%s\n' "${st}"
tx="$(printf '%s' "${st}" | sed -n 's/^\([0-9]\+\) packets transmitted.*/\1/p' | tail -1)"
rx="$(printf '%s' "${st}" | sed -n 's/.*[, ]\([0-9]\+\) received.*/\1/p' | tail -1)"
loss="$(printf '%s' "${st}" | sed -n 's/.*[, ]\([0-9]\+\)% packet loss.*/\1/p' | tail -1)"
avg="$(printf '%s' "${st}" | sed -n 's|.*= [0-9.]*/\([0-9.]*\)/.*|\1|p' | tail -1)"
[ -n "${tx}" ] || tx=0
[ -n "${rx}" ] || rx=0
[ -n "${loss}" ] || loss=100
[ -n "${avg}" ] || avg="-"
c1="$(if_counters "${iface}")"; rx1="${c1% *}"; tx1="${c1#* }"
drx=$(( rx1 - rx0 )); dtx=$(( tx1 - tx0 ))
if [ "${loss}" -le "${MAX_LOSS_PCT}" ] && [ "${tx}" -gt 0 ]; then
verdict="PASS"
else
verdict="FAIL"
fi
# nic_tx/nic_rx are this interface's own counter delta over the whole run.
# A pass with nic_tx near zero means the traffic left on the other NIC.
printf '[%s] RESULT %-3s %-6s -> %-15s tx=%s rx=%s loss=%s%% rtt_avg=%sms nic_tx=%s nic_rx=%s %s\n\n' \
"$(ts)" "${label}" "${iface}" "${target}" "${tx}" "${rx}" "${loss}" "${avg}" \
"${dtx}" "${drx}" "${verdict}"
}
write_report() {
local rx0_10g tx0_10g rx0_1g tx0_1g started
started="$(sed -n '1p' "${STATE_FILE}" 2>/dev/null)"
rx0_10g="$(sed -n '2p' "${STATE_FILE}" 2>/dev/null)"; rx0_10g="${rx0_10g:-0}"
tx0_10g="$(sed -n '3p' "${STATE_FILE}" 2>/dev/null)"; tx0_10g="${tx0_10g:-0}"
rx0_1g="$(sed -n '4p' "${STATE_FILE}" 2>/dev/null)"; rx0_1g="${rx0_1g:-0}"
tx0_1g="$(sed -n '5p' "${STATE_FILE}" 2>/dev/null)"; tx0_1g="${tx0_1g:-0}"
{
printf '#############################################################\n' printf '#############################################################\n'
printf '[%s] START: pid=%d count=%d settle=%ds interval=%ds\n' \ printf '[%s] mgmt ping report\n' "$(ts)"
"$(ts)" "$$" "${PING_COUNT}" "${LINK_SETTLE_SEC}" "${INTERVAL_SEC}" printf ' started : %s\n' "${started:-unknown}"
printf '[%s] 10G %s %s/%s -> %s (metric %s)\n' \ printf ' stopped : %s\n' "$(ts)"
"$(ts)" "${IF_10G}" "${IP_10G}" "${PLEN_10G}" "${TARGET_10G}" "${METRIC_10G}" printf ' 10G : %s %s/%s -> %s (metric %s)\n' \
printf '[%s] 1G %s %s/%s -> %s (metric %s)\n' \ "${IF_10G}" "${IP_10G}" "${PLEN_10G}" "${TARGET_10G}" "${METRIC_10G}"
"$(ts)" "${IF_1G}" "${IP_1G}" "${PLEN_1G}" "${TARGET_1G}" "${METRIC_1G}" printf ' 1G : %s %s/%s -> %s (metric %s)\n' \
printf '#############################################################\n' "${IF_1G}" "${IP_1G}" "${PLEN_1G}" "${TARGET_1G}" "${METRIC_1G}"
printf ' ping : -i %s %s, max loss %s%%\n' \
"${PING_INTERVAL_SEC}" "${PING_EXTRA_OPTS}" "${MAX_LOSS_PCT}"
printf '#############################################################\n\n'
apply_arp_strict render_leg "10G" "${IF_10G}" "${TARGET_10G}" "${RAW_10G}" "${rx0_10g}" "${tx0_10g}"
render_leg "1G" "${IF_1G}" "${TARGET_1G}" "${RAW_1G}" "${rx0_1g}" "${tx0_1g}"
while [ "${STOP}" -eq 0 ]; do printf -- '--- ip -s link show %s ---\n' "${IF_10G}"
round=$(( round + 1 )) ${SUDO} ip -s link show "${IF_10G}" 2>&1
printf -- '---------- ROUND %d @ %s ----------\n' "${round}" "$(ts)" printf '\n'
printf -- '--- ip -s link show %s ---\n' "${IF_1G}"
if ping_leg "10G" "${IF_10G}" "${IP_10G}" "${PLEN_10G}" "${GW_10G}" \ ${SUDO} ip -s link show "${IF_1G}" 2>&1
"${METRIC_10G}" "${TARGET_10G}"; then } >> "${LOG_FILE}"
pass=$(( pass + 1 )); else fail=$(( fail + 1 ))
fi
[ "${STOP}" -eq 0 ] || break
if ping_leg "1G" "${IF_1G}" "${IP_1G}" "${PLEN_1G}" "${GW_1G}" \
"${METRIC_1G}" "${TARGET_1G}"; then
pass=$(( pass + 1 )); else fail=$(( fail + 1 ))
fi
rotate_log_if_needed
if [ "${MAX_ROUNDS}" -gt 0 ] && [ "${round}" -ge "${MAX_ROUNDS}" ]; then
break
fi
[ "${STOP}" -eq 0 ] && interruptible_sleep "${INTERVAL_SEC}"
done
printf '[%s] STOP : pid=%d after %d round(s), legs PASS=%d FAIL=%d\n' \
"$(ts)" "$$" "${round}" "${pass}" "${fail}"
} }
############################################################################### ###############################################################################
# Sub-commands # Sub-commands
############################################################################### ###############################################################################
do_start() { prepare_log_on_start() {
local pid local stamp
pid="$(is_running)" && die "already running (pid=${pid})" case "${START_LOG_MODE}" in
mkdir -p "${LOG_DIR}" || die "cannot create ${LOG_DIR}" append) return 0 ;;
prepare_log_on_start archive)
if [ -s "${LOG_FILE}" ]; then
if command -v setsid >/dev/null 2>&1; then stamp="$(date '+%Y%m%d-%H%M%S')"
setsid "${SCRIPT_PATH}" __daemon >/dev/null 2>&1 & mv -f "${LOG_FILE}" "${LOG_FILE}.${stamp}" || die "cannot archive ${LOG_FILE}"
else printf 'previous log archived: %s.%s\n' "${LOG_FILE}" "${stamp}"
nohup "${SCRIPT_PATH}" __daemon >/dev/null 2>&1 &
disown 2>/dev/null
fi fi
;;
sleep 1 new)
pid="$(is_running)" || die "start failed, check ${LOG_FILE}" # Truncate rather than unlink, so a tail -f already attached keeps
printf 'started (pid=%s)\nlog: %s\n' "${pid}" "${LOG_FILE}" # following the new run.
[ -f "${LOG_FILE}" ] && { : > "${LOG_FILE}" || die "cannot truncate ${LOG_FILE}"; }
;;
*) die "invalid START_LOG_MODE: ${START_LOG_MODE}" ;;
esac
} }
do_daemon() { do_start() {
local pids p0 p1 c
pids="$(is_running)" && die "already running (pids=${pids})"
command -v ip >/dev/null 2>&1 || die "ip (iproute2) not found"
command -v ping >/dev/null 2>&1 || die "ping not found"
mkdir -p "${LOG_DIR}" || die "cannot create ${LOG_DIR}" mkdir -p "${LOG_DIR}" || die "cannot create ${LOG_DIR}"
LOG_TO_FILE=1 prepare_log_on_start
exec >>"${LOG_FILE}" 2>&1 : > "${RAW_10G}"
printf '%d\n' "$$" > "${PID_FILE}" : > "${RAW_1G}"
trap 'rm -f "${PID_FILE}"' EXIT
main_loop setup_iface "${IF_10G}" "${IP_10G}" "${PLEN_10G}" "${GW_10G}" "${METRIC_10G}" \
|| die "cannot configure ${IF_10G}"
setup_iface "${IF_1G}" "${IP_1G}" "${PLEN_1G}" "${GW_1G}" "${METRIC_1G}" \
|| die "cannot configure ${IF_1G}"
apply_arp_strict
sleep "${LINK_SETTLE_SEC}"
# Spend the ARP resolution here, not on the first measured packet.
arp_warmup "${IF_10G}" "${TARGET_10G}"
arp_warmup "${IF_1G}" "${TARGET_1G}"
# Counter baseline, taken after the warm-up so its packets are excluded.
{ c="$(if_counters "${IF_10G}")"
printf '%s\n%s\n%s\n' "$(ts)" "${c% *}" "${c#* }"
c="$(if_counters "${IF_1G}")"
printf '%s\n%s\n' "${c% *}" "${c#* }"
} > "${STATE_FILE}"
# Both NICs ping at the same time and keep accumulating until stopped.
nohup ping -I "${IF_10G}" -i "${PING_INTERVAL_SEC}" ${PING_EXTRA_OPTS} \
"${TARGET_10G}" >> "${RAW_10G}" 2>&1 &
p0=$!
nohup ping -I "${IF_1G}" -i "${PING_INTERVAL_SEC}" ${PING_EXTRA_OPTS} \
"${TARGET_1G}" >> "${RAW_1G}" 2>&1 &
p1=$!
disown "${p0}" 2>/dev/null
disown "${p1}" 2>/dev/null
printf '%s\n%s\n' "${p0}" "${p1}" > "${PID_FILE}"
sleep 1
kill -0 "${p0}" 2>/dev/null || die "10G ping failed to start, see ${RAW_10G}"
kill -0 "${p1}" 2>/dev/null || die "1G ping failed to start, see ${RAW_1G}"
printf 'started (10G pid=%s, 1G pid=%s)\nlog: %s\n' "${p0}" "${p1}" "${LOG_FILE}"
} }
do_stop() { do_stop() {
local pid i local pids p0 p1 i
pid="$(is_running)" || { printf 'not running\n'; rm -f "${PID_FILE}"; return 0; }
kill -TERM -"${pid}" 2>/dev/null || kill -TERM "${pid}" 2>/dev/null pids="$(is_running)" || {
printf 'not running\n'
rm -f "${PID_FILE}"
return 0
}
p0="${pids% *}"; p1="${pids#* }"
# SIGINT, not SIGTERM: ping prints its statistics block on interrupt, and
# that block is what the report parses.
kill -INT "${p0}" 2>/dev/null
kill -INT "${p1}" 2>/dev/null
for (( i = 0; i < 20; i++ )); do for (( i = 0; i < 20; i++ )); do
kill -0 "${pid}" 2>/dev/null || break kill -0 "${p0}" 2>/dev/null || kill -0 "${p1}" 2>/dev/null || break
sleep 0.5 sleep 0.5
done done
if kill -0 "${pid}" 2>/dev/null; then kill -KILL "${p0}" 2>/dev/null
printf 'SIGTERM ignored, sending SIGKILL\n' kill -KILL "${p1}" 2>/dev/null
kill -KILL -"${pid}" 2>/dev/null || kill -KILL "${pid}" 2>/dev/null
fi write_report
rm -f "${PID_FILE}" rm -f "${PID_FILE}"
printf 'stopped (pid=%s)\n' "${pid}" printf 'stopped (10G pid=%s, 1G pid=%s)\nreport: %s\n' "${p0}" "${p1}" "${LOG_FILE}"
} }
do_status() { do_status() {
local pid local pids
if pid="$(is_running)"; then if pids="$(is_running)"; then
printf 'status : running (pid=%s)\n' "${pid}" printf 'status : running (10G pid=%s, 1G pid=%s)\n' "${pids% *}" "${pids#* }"
else else
printf 'status : stopped\n' printf 'status : stopped\n'
fi fi
printf 'log : %s\n' "${LOG_FILE}" printf 'log : %s\n' "${LOG_FILE}"
[ -f "${LOG_FILE}" ] && printf 'size : %s\n' "$(du -h "${LOG_FILE}" | cut -f1)" printf '10G : %s %s/%s -> %s\n' "${IF_10G}" "${IP_10G}" "${PLEN_10G}" "${TARGET_10G}"
printf 'ping : count=%d, interval=%ds, settle=%ds\n' "${PING_COUNT}" "${INTERVAL_SEC}" "${LINK_SETTLE_SEC}" printf '1G : %s %s/%s -> %s\n' "${IF_1G}" "${IP_1G}" "${PLEN_1G}" "${TARGET_1G}"
printf '10G : %s %s/%s -> %s (metric %s)\n' "${IF_10G}" "${IP_10G}" "${PLEN_10G}" "${TARGET_10G}" "${METRIC_10G}" [ -f "${RAW_10G}" ] && printf '10G recv: %s\n' "$(grep -ac 'bytes from' "${RAW_10G}")"
printf '1G : %s %s/%s -> %s (metric %s)\n' "${IF_1G}" "${IP_1G}" "${PLEN_1G}" "${TARGET_1G}" "${METRIC_1G}" [ -f "${RAW_1G}" ] && printf '1G recv: %s\n' "$(grep -ac 'bytes from' "${RAW_1G}")"
} }
usage() { usage() {
cat <<EOF cat <<EOF
Usage: ${SCRIPT_NAME} {start|stop|status|fg|tail|clear|summary} Usage: ${SCRIPT_NAME} {start|stop|status|summary|clear}
start Run detached in the background; the previous log is discarded first start Configure both NICs, warm up ARP, then ping from BOTH at the same
(see START_LOG_MODE) time and keep accumulating. The previous log is discarded first.
stop Stop the background instance stop Stop both pings and render the report into the log
status Show pid / log path / configured NICs status Show pids, configured NICs and replies received so far
fg Run in the foreground, output to the terminal (for a quick check) summary Print just the RESULT lines from the log
tail tail -f the log file (does NOT return -- do not call from a macro) clear Remove the log and the raw captures (must be stopped first)
clear Remove the log file and all rotated copies (must be stopped first)
summary Print just the per-leg RESULT lines from the log
Log file : ${LOG_FILE} Log file : ${LOG_FILE}
One round = 10G leg then 1G leg, each pinging its own target. Both NICs are The report holds, per NIC, the last ${TAIL_LINES} entries and the ping
brought up and left up; addresses and routes are applied with statistics, then "ip -s link show" for both interfaces.
"ip address/route replace", so repeating a round does not stack duplicates.
If both NICs share a subnet, keep ARP_STRICT=1 -- otherwise a reply can come Both NICs are used at once, so management connectivity is in play -- drive
back on the NIC that did not send it. Check nic_tx/nic_rx on the RESULT line: this from the serial console. Edit the User Configurable Section at the top
they are that interface's own counter delta across the burst. to change interfaces, addresses, targets or the ping options.
Edit the User Configurable Section at the top to change interfaces, addresses,
targets, prefix lengths, gateways, metrics or the packet count.
EOF EOF
} }
@@ -449,15 +392,12 @@ case "${1:-}" in
start) do_start ;; start) do_start ;;
stop) do_stop ;; stop) do_stop ;;
status) do_status ;; status) do_status ;;
fg) LOG_TO_FILE=0; main_loop ;;
tail) tail -n 50 -f "${LOG_FILE}" ;;
summary) grep -a 'RESULT' "${LOG_FILE}" 2>/dev/null || printf 'no results in %s\n' "${LOG_FILE}" ;; summary) grep -a 'RESULT' "${LOG_FILE}" 2>/dev/null || printf 'no results in %s\n' "${LOG_FILE}" ;;
clear) clear)
is_running >/dev/null && die "still running, stop it first" is_running >/dev/null && die "still running, stop it first"
rm -f "${LOG_FILE}" "${LOG_FILE}".[0-9]* rm -f "${LOG_FILE}" "${LOG_FILE}".[0-9]* "${RAW_10G}" "${RAW_1G}" "${STATE_FILE}"
printf 'log cleared\n' printf 'log cleared\n'
;; ;;
__daemon) do_daemon ;;
-h|--help|"") usage ;; -h|--help|"") usage ;;
*) printf '[ERROR] unknown sub-command: %s\n\n' "$1" >&2; usage; exit 1 ;; *) printf '[ERROR] unknown sub-command: %s\n\n' "$1" >&2; usage; exit 1 ;;
esac esac
@@ -0,0 +1,203 @@
#!/bin/bash
###############################################################################
# usb_target.sh
#
# Version : V1.1.0
# Author : ETWen
# Date : 20260825
# Purpose : Auto-detect an inserted USB mass-storage device and report either
# its device node, its mount point, or both.
#
# stdout : requested value(s) only, for $(...) capture
# stderr : diagnostic messages
#
# Usage : usb_target.sh [-o dev|mnt|both|id] [-n] [-r seconds]
#
# -o dev print partition device node (e.g. /dev/sda1)
# -o mnt print mount point (e.g. /mnt/usb) [default]
# -o both print "<dev> <mnt>" on one line
# -o id print stable by-id path (/dev/disk/by-id/...)
# -n detect only, do not mount
# -r sec udev enumeration wait, default 15
#
# Exit : 0 success
# 1 no USB mass-storage found
# 2 multiple USB disks detected (refuse to guess)
# 3 no usable filesystem on the device
# 4 mount failed
# 5 mounted but not writable
#
# Version History
# V1.0.0 20260825 Initial Version
# V1.1.0 20260825 Add -o/-n/-r options; expose device node and by-id path
###############################################################################
set -u
MNT_BASE="/mnt/usb"
RETRY=15
OUTPUT="mnt"
DO_MOUNT=1
log() { echo "[usb] $*" >&2; } # diagnostics go to stderr; stdout stays clean
while getopts "o:nr:h" opt; do
case "$opt" in
o) OUTPUT="$OPTARG" ;;
n) DO_MOUNT=0 ;;
r) RETRY="$OPTARG" ;;
h) sed -n '3,30p' "$0" >&2; exit 0 ;;
*) log "invalid option"; exit 1 ;;
esac
done
shift $((OPTIND - 1))
case "$OUTPUT" in
dev|mnt|both|id) ;;
*) log "invalid -o value: $OUTPUT"; exit 1 ;;
esac
# "-o dev" / "-o id" alone does not require mounting.
[ "$OUTPUT" = "dev" ] && [ "$DO_MOUNT" = "1" ] && DO_MOUNT=0
[ "$OUTPUT" = "id" ] && [ "$DO_MOUNT" = "1" ] && DO_MOUNT=0
#------------------------------------------------------------------------------
# Identify the physical disk(s) backing rootfs / /host so they can be excluded.
# Needed because some platforms boot from a USB DOM, which also reports TRAN=usb.
#------------------------------------------------------------------------------
get_system_disk() {
local src
for mp in /host / ; do
src=$(findmnt -no SOURCE "$mp" 2>/dev/null) || continue
lsblk -no PKNAME "$src" 2>/dev/null | head -1
done | sort -u
}
#------------------------------------------------------------------------------
# List candidate USB disks (whole devices, not partitions).
#------------------------------------------------------------------------------
find_usb_disks() {
local sysdisks; sysdisks=$(get_system_disk)
lsblk -dn -o NAME,TYPE,TRAN,RM 2>/dev/null | while read -r name type tran rm; do
[ "$type" = "disk" ] || continue
case "$name" in loop*|ram*|dm-*|sr*|zram*) continue ;; esac
# Older util-linux may not expose the TRAN column; fall back to sysfs.
if [ "$tran" != "usb" ]; then
readlink -f "/sys/block/$name/device" 2>/dev/null | grep -q '/usb[0-9]' || continue
[ "$rm" = "1" ] || continue
fi
echo "$sysdisks" | grep -qx "$name" && { log "skip $name (system disk)"; continue; }
echo "/dev/$name"
done
}
#------------------------------------------------------------------------------
# Pick a mountable partition from a disk; fall back to the whole device for
# superfloppy layouts (filesystem written directly, no partition table).
#------------------------------------------------------------------------------
pick_partition() {
local disk="$1" p
p=$(lsblk -ln -o NAME,TYPE,FSTYPE "$disk" | \
awk '$2=="part" && $3!="" {print "/dev/"$1; exit}')
[ -n "$p" ] && { echo "$p"; return; }
[ -n "$(lsblk -dn -o FSTYPE "$disk")" ] && echo "$disk"
}
#------------------------------------------------------------------------------
# Resolve a device node to a stable /dev/disk/by-id path, if one exists.
#------------------------------------------------------------------------------
resolve_by_id() {
local dev="$1" real link
real=$(readlink -f "$dev")
for link in /dev/disk/by-id/*; do
[ -e "$link" ] || continue
case "$link" in *-part*|*) ;; esac
[ "$(readlink -f "$link")" = "$real" ] && { echo "$link"; return 0; }
done
return 1
}
#------------------------------------------------------------------------------
# Main
#------------------------------------------------------------------------------
disks=""
for i in $(seq 1 "$RETRY"); do
disks=$(find_usb_disks)
[ -n "$disks" ] && break
sleep 1
done
[ -z "$disks" ] && { log "no USB mass-storage found"; exit 1; }
n=$(echo "$disks" | wc -l)
if [ "$n" -gt 1 ]; then
log "multiple USB disks detected, refuse to guess:"
log "$disks"
exit 2
fi
disk="$disks"
part=$(pick_partition "$disk")
[ -z "$part" ] && { log "$disk has no usable filesystem"; exit 3; }
fstype=$(lsblk -no FSTYPE "$part")
log "found $part (disk=$disk fstype=$fstype)"
# --- device-node only: no mount needed ---------------------------------------
if [ "$OUTPUT" = "dev" ]; then
echo "$part"
exit 0
fi
if [ "$OUTPUT" = "id" ]; then
if byid=$(resolve_by_id "$part"); then
echo "$byid"
exit 0
fi
log "no by-id path for $part, falling back to device node"
echo "$part"
exit 0
fi
# --- mount point required ----------------------------------------------------
mp=$(lsblk -no MOUNTPOINT "$part" | head -1)
if [ -z "$mp" ] && [ "$DO_MOUNT" = "0" ]; then
log "$part is not mounted and -n was given"
exit 4
fi
if [ -z "$mp" ]; then
log "mounting $part -> $MNT_BASE"
mkdir -p "$MNT_BASE"
# exfat/ntfs are not always built into the platform kernel.
case "$fstype" in
exfat) grep -qw exfat /proc/filesystems || modprobe exfat 2>/dev/null ;;
ntfs) grep -qw ntfs3 /proc/filesystems || modprobe ntfs3 2>/dev/null ;;
esac
if ! mount -o rw,noatime "$part" "$MNT_BASE" 2>/dev/null; then
log "mount failed (fstype=$fstype)"
exit 4
fi
mp="$MNT_BASE"
# Verify it is actually writable (read-only media, dirty FAT, or full device).
if ! touch "$mp/.wtest" 2>/dev/null; then
log "mounted read-only or no space left"
umount "$mp"
exit 5
fi
rm -f "$mp/.wtest"
else
log "$part already mounted at $mp"
fi
if [ "$OUTPUT" = "both" ]; then
echo "$part $mp"
else
echo "$mp"
fi
exit 0