feat(dsc): Add the 100G link down/up cycle test
bcm_100g_dsc_cycle.sh tunes once - gbsyncd paused, serdes-tx-set on sys 0..7, tx_taps per lane - then repeats: port 268 en=0, wait, en=1, wait, phydiag 268 dsc on both units, and the retimer read back with serdes-tx-get / serdes-rx-get for sys 0..7. One log per run under ~/dsc_log, split by "### TEST CYCLE n/N", ending in "### DSC_CYCLE_DONE rc=<n>" for a TTL wait to key on. Waiting for a link state polls "bcmcmd ps <port>" rather than reading "show interfaces status". "port en=" changes the SDK, and the SONiC table only follows once gbsyncd and orchagent have synced it - and this test pauses gbsyncd, so the SONiC table was the wrong thing to wait on. The state is read from the row whose first field ends in "(<port>)", and "down" is tested as "not up" so !ena counts. All output is ASCII English on purpose: the DUT serial console mangles UTF-8, and a Chinese error message came back as "ERROR: -c ? 1" - an error you cannot read is worse than none. -d echoes every command as a grey [CMD] line, CMD_DELAY paces them, and -n is a dry run that touches nothing and sleeps not at all. The cycle count may be given as a bare argument, which is what people type first. Not verified on hardware. gbsyncd is unpaused again on exit, Ctrl-C included. tools/100G_dsc_cycle.txt is the hand-typed bench version of the same commands, kept the way BER_Test.txt and TR518.txt are, so the script can be checked against what a person would actually run. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014RetWKFZFG1ZHcitQAyhwM
This commit is contained in:
Claude Opus 5
@@ -0,0 +1,608 @@
|
|||||||
|
#!/bin/sh
|
||||||
|
# =============================================================================
|
||||||
|
# bcm_100g_dsc_cycle.sh
|
||||||
|
#
|
||||||
|
# Author : ETWen
|
||||||
|
# Date : 20260909
|
||||||
|
#
|
||||||
|
# Version History
|
||||||
|
# V1.0.0 20260909 Initial Version
|
||||||
|
# V1.1.0 20260909 All runtime output is ASCII English - the DUT serial
|
||||||
|
# console mangles UTF-8 (a Chinese error message came
|
||||||
|
# back as "ERROR: -c ? 1"). Cycle count may be given as
|
||||||
|
# a bare argument. Added -d to echo every command, and
|
||||||
|
# CMD_DELAY to pace them.
|
||||||
|
# V1.2.0 20260909 Wait on "bcmcmd ps <port>" instead of
|
||||||
|
# "show interfaces status". The state comes from the
|
||||||
|
# SDK, which is what "port en=" actually changes, so it
|
||||||
|
# no longer depends on gbsyncd syncing into the DB.
|
||||||
|
#
|
||||||
|
# Description:
|
||||||
|
# Link down/up cycling on the 100G uplink. After every re-link it collects a
|
||||||
|
# SerDes DSC dump and reads the retimer back, so you can see whether the
|
||||||
|
# tuned TX taps settle to the same place each time the link comes up.
|
||||||
|
#
|
||||||
|
# One run has two parts:
|
||||||
|
# 1. Tuning -- once. pause gbsyncd -> serdes-tx-set sys 0..7
|
||||||
|
# -> tx_taps set on four lanes of each unit
|
||||||
|
# 2. Cycle -- repeated N times:
|
||||||
|
# port en=0 -> wait until "ps <port>" reads !ena on both units
|
||||||
|
# port en=1 -> wait until "ps <port>" reads up on both units
|
||||||
|
# phydiag <port> dsc on each unit
|
||||||
|
# serdes-tx-get / serdes-rx-get sys 0..7
|
||||||
|
# all of it appended to one log, under a per-cycle heading
|
||||||
|
#
|
||||||
|
# Command source: tools/100G_dsc_cycle.txt (the hand-typed bench version).
|
||||||
|
# Change that file and this one together.
|
||||||
|
#
|
||||||
|
# Deployment:
|
||||||
|
# Script : /root/Blanton_Script/bcm_100g_dsc_cycle.sh
|
||||||
|
# Log : ~/dsc_log/100G_dsc_cycle_test_<YYYYmmdd_HHMMSS>.log
|
||||||
|
# One file per run, every cycle in it, split by "### TEST CYCLE n/N".
|
||||||
|
#
|
||||||
|
# Paths come from $HOME. Run as root (after sudo -i) and ~ is /root.
|
||||||
|
#
|
||||||
|
# Usage:
|
||||||
|
# ./bcm_100g_dsc_cycle.sh # print this help
|
||||||
|
# ./bcm_100g_dsc_cycle.sh 200 # 200 cycles
|
||||||
|
# ./bcm_100g_dsc_cycle.sh -c 200 # same thing
|
||||||
|
# ./bcm_100g_dsc_cycle.sh -c 200 -d # echo every command as it is issued
|
||||||
|
# ./bcm_100g_dsc_cycle.sh -c 10 -T # skip tuning, cycle only
|
||||||
|
# ./bcm_100g_dsc_cycle.sh -c 1 -n # dry run, print commands, touch nothing
|
||||||
|
#
|
||||||
|
# Exit code:
|
||||||
|
# 0 = every cycle linked down and up as expected, and all data was collected
|
||||||
|
# 1 = a cycle timed out waiting for link, or data was missing
|
||||||
|
# 2 = bad argument / path
|
||||||
|
# 3 = a required tool is missing (bcmcmd / show / blanton_barchetta_tool)
|
||||||
|
#
|
||||||
|
# The last line is always "### DSC_CYCLE_DONE rc=<n>", for a TTL wait to
|
||||||
|
# key on.
|
||||||
|
#
|
||||||
|
# Design notes:
|
||||||
|
# - POSIX sh. No bash, no Python.
|
||||||
|
# - All runtime output is plain ASCII. The DUT console is not UTF-8 clean:
|
||||||
|
# Chinese comes back as "M-^A"-style garbage, and an error you cannot read
|
||||||
|
# is worse than no error at all. Comments are English for the same reason -
|
||||||
|
# whoever debugs this on the bench is reading it through that console.
|
||||||
|
# - Link state comes from "bcmcmd -n <unit> -c 'ps <port>'". Its header is
|
||||||
|
# two stacked lines, so a column index cannot be derived from it the way
|
||||||
|
# it can from a SONiC table. Instead the data row is found by its first
|
||||||
|
# field ending in "(<port>)" - e.g. ce0(268) - and the state is the field
|
||||||
|
# after it:
|
||||||
|
# ce0(268) !ena 4 100G FD ... <- port disabled
|
||||||
|
# ce0(268) up 4 100G FD ... <- link up
|
||||||
|
# "down" is therefore tested as "not up", which covers !ena as well as an
|
||||||
|
# enabled port whose link has not come back.
|
||||||
|
# - Asking the SDK rather than "show interfaces status" matters here: "port
|
||||||
|
# en=" changes the SDK, and the SONiC table only follows once gbsyncd and
|
||||||
|
# orchagent have synced it into the DB. Since this test pauses gbsyncd,
|
||||||
|
# the SONiC table was the wrong thing to wait on.
|
||||||
|
# - bcmcmd reads stdin, and inside a while loop it will swallow the loop's
|
||||||
|
# input. blanton_traffic_linespeed.sh hit this at v0.3.1 (only the first
|
||||||
|
# VLAN was configured) and bcm_dsc_scan.sh hit it again at v1.3.0 (only
|
||||||
|
# the first port was scanned). Every external command here gets
|
||||||
|
# < /dev/null.
|
||||||
|
# - bcmcmd exits 0 even when the diag shell rejects a command, so the DSC is
|
||||||
|
# verified by counting "SERDES DISPLAY DIAG DATA END", and the retimer read
|
||||||
|
# by counting sections that actually have output under them - not by exit
|
||||||
|
# code, and not by counting headings this script prints itself.
|
||||||
|
#
|
||||||
|
# Cautions:
|
||||||
|
# - !! While gbsyncd is paused the gearbox state stops being synced into the
|
||||||
|
# DB. The wait reads the SDK directly ("ps <port>") so it is not affected,
|
||||||
|
# but anything you check by hand with "show interfaces status" during the
|
||||||
|
# run may be stale. Set PAUSE_GBSYNCD=0 (or -P) if that gets in the way.
|
||||||
|
# - !! On exit, including Ctrl-C, both gbsyncd containers are unpaused again
|
||||||
|
# so nothing is left paused. Set RESUME_GBSYNCD=0 to keep them paused.
|
||||||
|
# - !! phydiag dsc is read-only, but port en=0/1 and tx_taps set do change
|
||||||
|
# the hardware. Do not run this at the same time as blanton_ber.sh,
|
||||||
|
# blanton_traffic_linespeed or blanton_tr518.sh - they take the same ports.
|
||||||
|
# - !! NOT verified on hardware yet.
|
||||||
|
# =============================================================================
|
||||||
|
|
||||||
|
set -u
|
||||||
|
|
||||||
|
# ----------------------------------------------------------------------------
|
||||||
|
# Knobs - change these for a different DUT or port, not the flow below
|
||||||
|
# ----------------------------------------------------------------------------
|
||||||
|
PORT=268 # bcmcmd port number, not a port name
|
||||||
|
UNITS="0 1" # units to drive
|
||||||
|
SYS_LANES="0 1 2 3 4 5 6 7" # blanton_barchetta_tool sys lanes, tuning
|
||||||
|
SYS_TX="pre=-10 main=87 post=0" # TX setting applied to each of them
|
||||||
|
|
||||||
|
# tx_taps, one line per lane: "<unit> <lane> <main> <pre1> <post1>"
|
||||||
|
TX_TAPS="
|
||||||
|
0 0 31 4 13
|
||||||
|
0 1 31 4 13
|
||||||
|
0 2 27 4 13
|
||||||
|
0 3 29 4 13
|
||||||
|
1 0 29 4 13
|
||||||
|
1 1 30 4 13
|
||||||
|
1 2 31 4 13
|
||||||
|
1 3 30 4 13
|
||||||
|
"
|
||||||
|
|
||||||
|
TX_LANES="0 1 2 3 4 5 6 7" # retimer TX lanes read back each cycle
|
||||||
|
RX_LANES="0 1 2 3 4 5 6 7" # retimer RX lanes read back each cycle
|
||||||
|
|
||||||
|
SUDO="" # set to "sudo" if not running as root
|
||||||
|
PAUSE_GBSYNCD=1 # pause gbsyncd0/1 before tuning
|
||||||
|
RESUME_GBSYNCD=1 # unpause again on exit, Ctrl-C included
|
||||||
|
GBSYNCD="gbsyncd0 gbsyncd1"
|
||||||
|
|
||||||
|
LINK_WAIT=90 # give up waiting for a link state after this
|
||||||
|
POLL=2 # seconds between "ps <port>" reads while waiting
|
||||||
|
SETTLE=2 # quiet time after link up, before collecting
|
||||||
|
CMD_DELAY=0.2 # pause after every command issued to the DUT
|
||||||
|
|
||||||
|
OUTDIR="${HOME:-/root}/dsc_log"
|
||||||
|
|
||||||
|
# ----------------------------------------------------------------------------
|
||||||
|
CYCLES=0
|
||||||
|
DRYRUN=0
|
||||||
|
DOTUNE=1
|
||||||
|
DEBUG=0
|
||||||
|
RC=0
|
||||||
|
|
||||||
|
usage() {
|
||||||
|
cat >&2 << 'USAGE'
|
||||||
|
Usage: bcm_100g_dsc_cycle.sh <cycles> [options]
|
||||||
|
bcm_100g_dsc_cycle.sh -c <cycles> [options]
|
||||||
|
|
||||||
|
-c <n> how many cycles to run. May also be given as a bare argument.
|
||||||
|
-o <dir> log directory (default: ~/dsc_log)
|
||||||
|
-w <sec> how long to wait for a link state (default: 90)
|
||||||
|
-p <sec> how often to re-read the port state (default: 2)
|
||||||
|
-s <sec> quiet time after link up, before collecting (default: 2)
|
||||||
|
-d debug: echo every command as [CMD] before running it
|
||||||
|
-T skip tuning, go straight to the cycles
|
||||||
|
-P leave gbsyncd alone (no pause, no unpause)
|
||||||
|
-n dry run: print the commands, touch nothing
|
||||||
|
-h this help
|
||||||
|
|
||||||
|
One log per run:
|
||||||
|
~/dsc_log/100G_dsc_cycle_test_<YYYYmmdd_HHMMSS>.log
|
||||||
|
|
||||||
|
Each cycle is one "### TEST CYCLE n/N" section holding, in order: the "ps"
|
||||||
|
output showing the port down, the same showing it up, phydiag dsc for both
|
||||||
|
units, and the retimer serdes-tx-get / serdes-rx-get for sys 0..7.
|
||||||
|
|
||||||
|
Waiting for a port state is a poll, not a single read: "bcmcmd ps <port>" on
|
||||||
|
every unit, every -p seconds, until both agree or -w seconds have passed.
|
||||||
|
|
||||||
|
Examples:
|
||||||
|
./bcm_100g_dsc_cycle.sh 200
|
||||||
|
./bcm_100g_dsc_cycle.sh 200 -d
|
||||||
|
./bcm_100g_dsc_cycle.sh -c 20 -T -w 120
|
||||||
|
USAGE
|
||||||
|
exit 1
|
||||||
|
}
|
||||||
|
|
||||||
|
[ $# -eq 0 ] && usage
|
||||||
|
|
||||||
|
# getopts stops at the first argument that is not an option, so "200 -d" would
|
||||||
|
# leave -d unread. Parse, take the bare count, then parse whatever followed it.
|
||||||
|
# getopts inside a function reads the function's own arguments; OPTIND is
|
||||||
|
# global, so it has to be reset on each entry.
|
||||||
|
PARSED=0
|
||||||
|
parse_opts() {
|
||||||
|
OPTIND=1
|
||||||
|
while getopts "c:o:w:p:s:dTPnh" opt; do
|
||||||
|
case "$opt" in
|
||||||
|
c) CYCLES="$OPTARG" ;;
|
||||||
|
o) OUTDIR="$OPTARG" ;;
|
||||||
|
w) LINK_WAIT="$OPTARG" ;;
|
||||||
|
p) POLL="$OPTARG" ;;
|
||||||
|
s) SETTLE="$OPTARG" ;;
|
||||||
|
d) DEBUG=1 ;;
|
||||||
|
T) DOTUNE=0 ;;
|
||||||
|
P) PAUSE_GBSYNCD=0; RESUME_GBSYNCD=0 ;;
|
||||||
|
n) DRYRUN=1 ;;
|
||||||
|
*) usage ;;
|
||||||
|
esac
|
||||||
|
done
|
||||||
|
PARSED=$((OPTIND - 1))
|
||||||
|
}
|
||||||
|
|
||||||
|
parse_opts "$@"
|
||||||
|
shift "$PARSED"
|
||||||
|
|
||||||
|
# Allow the count as a bare argument: "... 200" == "... -c 200". That is what
|
||||||
|
# most people type first, and answering it with an error helps nobody.
|
||||||
|
if [ "$CYCLES" = "0" ] && [ $# -ge 1 ]; then
|
||||||
|
case "$1" in
|
||||||
|
-*) : ;; # an option; leave it for the second pass
|
||||||
|
*) CYCLES="$1"; shift ;; # anything else is meant to be the count,
|
||||||
|
esac # and the check below says so if it is not
|
||||||
|
fi
|
||||||
|
|
||||||
|
# Anything after the count is options again: "200 -d -T"
|
||||||
|
if [ $# -gt 0 ]; then
|
||||||
|
parse_opts "$@"
|
||||||
|
shift "$PARSED"
|
||||||
|
fi
|
||||||
|
|
||||||
|
if [ $# -gt 0 ]; then
|
||||||
|
echo "ERROR: unexpected argument '$1'" >&2
|
||||||
|
echo " run with no argument to see the usage" >&2
|
||||||
|
exit 2
|
||||||
|
fi
|
||||||
|
|
||||||
|
case "$CYCLES" in
|
||||||
|
''|*[!0-9]*)
|
||||||
|
echo "ERROR: cycle count must be a positive integer (got: '$CYCLES')" >&2
|
||||||
|
echo " try: $0 200 or $0 -c 200" >&2
|
||||||
|
exit 2 ;;
|
||||||
|
esac
|
||||||
|
if [ "$CYCLES" -lt 1 ]; then
|
||||||
|
echo "ERROR: cycle count must be at least 1 (got: $CYCLES)" >&2
|
||||||
|
exit 2
|
||||||
|
fi
|
||||||
|
|
||||||
|
# ----------------------------------------------------------------------------
|
||||||
|
# Colour, only when stdout is a terminal, so a redirected run stays clean
|
||||||
|
# ----------------------------------------------------------------------------
|
||||||
|
if [ -t 1 ] && [ -z "${NO_COLOR:-}" ]; then
|
||||||
|
C_INFO=$(printf '\033[1;34m') # blue
|
||||||
|
C_OK=$(printf '\033[1;32m') # green
|
||||||
|
C_WARN=$(printf '\033[1;33m') # yellow
|
||||||
|
C_ERR=$(printf '\033[1;31m') # red
|
||||||
|
C_CMD=$(printf '\033[90m') # dark grey
|
||||||
|
C_OFF=$(printf '\033[0m')
|
||||||
|
else
|
||||||
|
C_INFO=''; C_OK=''; C_WARN=''; C_ERR=''; C_CMD=''; C_OFF=''
|
||||||
|
fi
|
||||||
|
|
||||||
|
# A dry run touches nothing, so it must not sleep either - otherwise
|
||||||
|
# "-c 200 -n" sits there for as long as the real thing would.
|
||||||
|
naptime() { [ "$DRYRUN" = "1" ] && return 0; [ "${1:-0}" = "0" ] && return 0; sleep "$1"; }
|
||||||
|
|
||||||
|
info() { printf '%s[info]%s %s\n' "$C_INFO" "$C_OFF" "$*"; }
|
||||||
|
ok() { printf '%s[ OK ]%s %s\n' "$C_OK" "$C_OFF" "$*"; }
|
||||||
|
warn() { printf '%s[warn]%s %s\n' "$C_WARN" "$C_OFF" "$*" >&2; }
|
||||||
|
err() { printf '%s[ERR ]%s %s\n' "$C_ERR" "$C_OFF" "$*" >&2; }
|
||||||
|
|
||||||
|
# Both -d and -n print commands. During collection the whole block is
|
||||||
|
# redirected into a temp file, so those lines would end up in the file with
|
||||||
|
# nothing on screen. Bind fd 9 to the terminal up front and print there.
|
||||||
|
if { [ "$DEBUG" = "1" ] || [ "$DRYRUN" = "1" ]; } && ( true > /dev/tty ) 2>/dev/null; then
|
||||||
|
exec 9> /dev/tty
|
||||||
|
else
|
||||||
|
exec 9>&1
|
||||||
|
fi
|
||||||
|
# ':' is a special built-in in dash (the DUT's /bin/sh); a failed redirection
|
||||||
|
# on one kills the whole shell with no message. Hence (true > /dev/tty) in a
|
||||||
|
# subshell above - if that cannot open, only the subshell dies.
|
||||||
|
|
||||||
|
# Print a command the way you would type it: quote any argument that needs it,
|
||||||
|
# so the line can be copied back to the prompt as is.
|
||||||
|
# NOTE: sh has no local variables - every name assigned here is global. All
|
||||||
|
# helper variables are therefore prefixed. _show_cmd used to assign "tag",
|
||||||
|
# which silently overwrote the caller's cycle label and turned every later
|
||||||
|
# message into "[info] [CMD] : Wait Port Link Down".
|
||||||
|
_show_cmd() {
|
||||||
|
_sc_tag="$1"; _sc_col="$2"; shift 2
|
||||||
|
{
|
||||||
|
printf '%s%s%s' "$_sc_col" "$_sc_tag" "$C_OFF"
|
||||||
|
for a in "$@"; do
|
||||||
|
case "$a" in
|
||||||
|
''|*[!A-Za-z0-9_./=:,+-]*) printf " '%s'" "$a" ;;
|
||||||
|
*) printf ' %s' "$a" ;;
|
||||||
|
esac
|
||||||
|
done
|
||||||
|
printf '\n'
|
||||||
|
} >&9
|
||||||
|
}
|
||||||
|
|
||||||
|
# Every external command goes through here: never reads stdin, echoed when
|
||||||
|
# asked, and paced by CMD_DELAY so the DUT is not hammered.
|
||||||
|
run() {
|
||||||
|
[ "$DEBUG" = "1" ] && [ "$DRYRUN" = "0" ] && _show_cmd '[CMD]' "$C_CMD" "$@"
|
||||||
|
if [ "$DRYRUN" = "1" ]; then
|
||||||
|
_show_cmd ' +' "$C_CMD" "$@"
|
||||||
|
return 0
|
||||||
|
fi
|
||||||
|
"$@" < /dev/null
|
||||||
|
_cmd_rc=$?
|
||||||
|
naptime "$CMD_DELAY"
|
||||||
|
return $_cmd_rc
|
||||||
|
}
|
||||||
|
|
||||||
|
# Same, but discards output. Do not write "> /dev/null" at the call site or
|
||||||
|
# the [CMD] line goes with it.
|
||||||
|
runq() {
|
||||||
|
[ "$DEBUG" = "1" ] && [ "$DRYRUN" = "0" ] && _show_cmd '[CMD]' "$C_CMD" "$@"
|
||||||
|
if [ "$DRYRUN" = "1" ]; then
|
||||||
|
_show_cmd ' +' "$C_CMD" "$@"
|
||||||
|
return 0
|
||||||
|
fi
|
||||||
|
"$@" < /dev/null > /dev/null 2>&1
|
||||||
|
_cmd_rc=$?
|
||||||
|
naptime "$CMD_DELAY"
|
||||||
|
return $_cmd_rc
|
||||||
|
}
|
||||||
|
|
||||||
|
# ----------------------------------------------------------------------------
|
||||||
|
# Preflight
|
||||||
|
# ----------------------------------------------------------------------------
|
||||||
|
if [ "$DRYRUN" = "0" ]; then
|
||||||
|
command -v bcmcmd > /dev/null 2>&1 || { err "not found: bcmcmd"; exit 3; }
|
||||||
|
# -T only skips tuning; the per-cycle retimer read still needs this tool
|
||||||
|
if [ ! -x /usr/sbin/blanton_barchetta_tool ]; then
|
||||||
|
err "not found: /usr/sbin/blanton_barchetta_tool (tuning and the per-cycle retimer read both use it)"
|
||||||
|
exit 3
|
||||||
|
fi
|
||||||
|
fi
|
||||||
|
|
||||||
|
# busybox sleep may not take a fraction
|
||||||
|
if [ "$DRYRUN" = "0" ] && [ "$CMD_DELAY" != "0" ] && ! sleep "$CMD_DELAY" 2>/dev/null; then
|
||||||
|
warn "sleep '$CMD_DELAY' not supported, falling back to 1"
|
||||||
|
CMD_DELAY=1
|
||||||
|
fi
|
||||||
|
|
||||||
|
mkdir -p "$OUTDIR" 2>/dev/null || { err "cannot create outdir: $OUTDIR"; exit 2; }
|
||||||
|
[ -w "$OUTDIR" ] || { err "outdir not writable: $OUTDIR"; exit 2; }
|
||||||
|
|
||||||
|
TS=$(date '+%Y%m%d_%H%M%S')
|
||||||
|
LOG="${OUTDIR}/100G_dsc_cycle_test_${TS}.log"
|
||||||
|
: > "$LOG" || { err "cannot write $LOG"; exit 2; }
|
||||||
|
|
||||||
|
STATUS_TMP=$(mktemp) || { err "mktemp failed"; exit 2; }
|
||||||
|
CYCLE_TMP=$(mktemp) || { err "mktemp failed"; exit 2; }
|
||||||
|
|
||||||
|
# ----------------------------------------------------------------------------
|
||||||
|
# gbsyncd must go back the way it was found, or the container stays paused
|
||||||
|
# ----------------------------------------------------------------------------
|
||||||
|
PAUSED=0
|
||||||
|
resume_gbsyncd() {
|
||||||
|
[ "$PAUSED" = "1" ] || return 0
|
||||||
|
[ "$RESUME_GBSYNCD" = "1" ] || return 0
|
||||||
|
PAUSED=0
|
||||||
|
for _rg_c in $GBSYNCD; do
|
||||||
|
if runq $SUDO docker unpause "$_rg_c"; then
|
||||||
|
info "gbsyncd resumed: $_rg_c"
|
||||||
|
else
|
||||||
|
warn "docker unpause $_rg_c failed - check it by hand"
|
||||||
|
fi
|
||||||
|
done
|
||||||
|
}
|
||||||
|
on_exit() { rm -f "$STATUS_TMP" "$CYCLE_TMP"; resume_gbsyncd; }
|
||||||
|
on_int() {
|
||||||
|
echo ""
|
||||||
|
warn "interrupted by user"
|
||||||
|
resume_gbsyncd
|
||||||
|
printf '\n### DSC_CYCLE_DONE rc=130 (INTERRUPTED)\n' >> "$LOG"
|
||||||
|
printf '### DSC_CYCLE_DONE rc=130 (INTERRUPTED)\n'
|
||||||
|
rm -f "$STATUS_TMP" "$CYCLE_TMP"
|
||||||
|
exit 130
|
||||||
|
}
|
||||||
|
trap on_exit EXIT
|
||||||
|
trap on_int INT TERM
|
||||||
|
|
||||||
|
tee_log() { tee -a "$LOG"; }
|
||||||
|
|
||||||
|
# ----------------------------------------------------------------------------
|
||||||
|
# Link state. Find which field "Oper" is from the header row, then read that
|
||||||
|
# field on every data row. Returns 0 only when every interface matches.
|
||||||
|
# The table itself is left in $STATUS_TMP for the caller to print.
|
||||||
|
# ----------------------------------------------------------------------------
|
||||||
|
# $1 = "up" or "down". Runs "ps <port>" on every unit, leaves the combined
|
||||||
|
# output in $STATUS_TMP for the caller to print, and returns 0 only when every
|
||||||
|
# unit agrees.
|
||||||
|
link_is() {
|
||||||
|
_li_want="$1"
|
||||||
|
: > "$STATUS_TMP"
|
||||||
|
for _li_u in $UNITS; do
|
||||||
|
printf '\n[unit %s] bcmcmd -n %s -c "ps %s"\n' "$_li_u" "$_li_u" "$PORT" >> "$STATUS_TMP"
|
||||||
|
if [ "$DRYRUN" = "1" ]; then
|
||||||
|
_show_cmd ' +' "$C_CMD" bcmcmd -n "$_li_u" -c "ps $PORT"
|
||||||
|
printf ' ce0(%s) %s 4 100G FD (dry-run)\n' \
|
||||||
|
"$PORT" "$([ "$_li_want" = up ] && echo 'up ' || echo '!ena')" >> "$STATUS_TMP"
|
||||||
|
continue
|
||||||
|
fi
|
||||||
|
[ "$DEBUG" = "1" ] && _show_cmd '[CMD]' "$C_CMD" bcmcmd -n "$_li_u" -c "ps $PORT"
|
||||||
|
bcmcmd -n "$_li_u" -c "ps $PORT" < /dev/null >> "$STATUS_TMP" 2>&1
|
||||||
|
naptime "$CMD_DELAY"
|
||||||
|
done
|
||||||
|
|
||||||
|
# The data row is the one whose first field ends in "(<port>)"; the state
|
||||||
|
# is the field right after it. "down" is "not up", so !ena counts.
|
||||||
|
awk -v port="$PORT" -v want="$_li_want" -v units="$(echo "$UNITS" | wc -w)" '
|
||||||
|
$1 ~ ("\\(" port "\\)$") {
|
||||||
|
n++
|
||||||
|
if (want == "up") { if ($2 != "up") bad++ }
|
||||||
|
else { if ($2 == "up") bad++ }
|
||||||
|
}
|
||||||
|
END { exit (n >= units && bad == 0) ? 0 : 1 }
|
||||||
|
' "$STATUS_TMP"
|
||||||
|
}
|
||||||
|
|
||||||
|
# $1 = wanted state (Down/up), $2 = label for the messages
|
||||||
|
wait_link() {
|
||||||
|
_wl_want="$1"; _wl_tag="$2"
|
||||||
|
_wl_waited=0
|
||||||
|
while :; do
|
||||||
|
if link_is "$_wl_want"; then
|
||||||
|
{
|
||||||
|
printf '\n--- port %s after %ss ---\n' "$_wl_want" "$_wl_waited"
|
||||||
|
cat "$STATUS_TMP"
|
||||||
|
} | tee_log
|
||||||
|
ok "$_wl_tag : port $_wl_want after ${_wl_waited}s"
|
||||||
|
return 0
|
||||||
|
fi
|
||||||
|
[ "$_wl_waited" -ge "$LINK_WAIT" ] && break
|
||||||
|
naptime "$POLL"
|
||||||
|
_wl_waited=$((_wl_waited + POLL))
|
||||||
|
done
|
||||||
|
{
|
||||||
|
printf '\n--- TIMEOUT waiting for port %s (%ss) ---\n' "$_wl_want" "$LINK_WAIT"
|
||||||
|
cat "$STATUS_TMP"
|
||||||
|
} | tee_log
|
||||||
|
err "$_wl_tag : TIMEOUT waiting for port $_wl_want (${LINK_WAIT}s)"
|
||||||
|
return 1
|
||||||
|
}
|
||||||
|
|
||||||
|
port_en() {
|
||||||
|
for _pe_u in $UNITS; do
|
||||||
|
runq bcmcmd -n "$_pe_u" -c "port ${PORT} en=$1"
|
||||||
|
done
|
||||||
|
}
|
||||||
|
|
||||||
|
# ----------------------------------------------------------------------------
|
||||||
|
# Log header
|
||||||
|
# ----------------------------------------------------------------------------
|
||||||
|
{
|
||||||
|
printf '=====================================================================\n'
|
||||||
|
printf '100G DSC cycle test\n'
|
||||||
|
printf ' script : bcm_100g_dsc_cycle.sh V1.2.0\n'
|
||||||
|
printf ' started : %s\n' "$(date '+%Y-%m-%d %H:%M:%S')"
|
||||||
|
printf ' cycles : %s\n' "$CYCLES"
|
||||||
|
printf ' port : %s units : %s\n' "$PORT" "$UNITS"
|
||||||
|
printf ' tuning : %s\n' "$([ "$DOTUNE" = 1 ] && echo yes || echo 'skipped (-T)')"
|
||||||
|
printf ' tx lanes : %s\n' "$TX_LANES"
|
||||||
|
printf ' rx lanes : %s\n' "$RX_LANES"
|
||||||
|
printf ' link wait: %ss poll %ss settle %ss cmd delay %ss\n' \
|
||||||
|
"$LINK_WAIT" "$POLL" "$SETTLE" "$CMD_DELAY"
|
||||||
|
printf '=====================================================================\n'
|
||||||
|
} >> "$LOG"
|
||||||
|
|
||||||
|
info "log : $LOG"
|
||||||
|
|
||||||
|
# ----------------------------------------------------------------------------
|
||||||
|
# 1. Tuning, once
|
||||||
|
# ----------------------------------------------------------------------------
|
||||||
|
if [ "$DOTUNE" = "1" ]; then
|
||||||
|
info "Tuning"
|
||||||
|
printf '\n### TUNING %s\n' "$(date '+%Y-%m-%d %H:%M:%S')" >> "$LOG"
|
||||||
|
|
||||||
|
if [ "$PAUSE_GBSYNCD" = "1" ]; then
|
||||||
|
for c in $GBSYNCD; do
|
||||||
|
if runq $SUDO docker pause "$c"; then
|
||||||
|
PAUSED=1
|
||||||
|
info "gbsyncd paused: $c"
|
||||||
|
else
|
||||||
|
warn "docker pause $c failed (container not running?)"
|
||||||
|
fi
|
||||||
|
done
|
||||||
|
fi
|
||||||
|
|
||||||
|
run $SUDO /usr/sbin/blanton_barchetta_tool --serdes-tx-get sys 0 2>&1 | tee_log
|
||||||
|
|
||||||
|
for l in $SYS_LANES; do
|
||||||
|
# shellcheck disable=SC2086
|
||||||
|
run $SUDO /usr/sbin/blanton_barchetta_tool --serdes-tx-set sys "$l" $SYS_TX 2>&1 | tee_log
|
||||||
|
done
|
||||||
|
|
||||||
|
echo "$TX_TAPS" | while read -r u lane main pre1 post1; do
|
||||||
|
[ -n "${u:-}" ] || continue
|
||||||
|
run bcmcmd -n "$u" -c \
|
||||||
|
" dsh -c 'phydiag ${PORT} tx_taps set lane=${lane} main=${main} pre1=${pre1} post1=${post1}' " \
|
||||||
|
2>&1 | tee_log
|
||||||
|
done
|
||||||
|
|
||||||
|
ok "Tuning done"
|
||||||
|
fi
|
||||||
|
|
||||||
|
# ----------------------------------------------------------------------------
|
||||||
|
# 2. Test cycles
|
||||||
|
# ----------------------------------------------------------------------------
|
||||||
|
n=1
|
||||||
|
while [ "$n" -le "$CYCLES" ]; do
|
||||||
|
tag="Test Cycle $n / $CYCLES"
|
||||||
|
|
||||||
|
printf '\n\n### TEST CYCLE %s/%s %s\n' "$n" "$CYCLES" \
|
||||||
|
"$(date '+%Y-%m-%d %H:%M:%S')" >> "$LOG"
|
||||||
|
|
||||||
|
# ---- Disable Port ------------------------------------------------------
|
||||||
|
info "$tag : Disable Port"
|
||||||
|
port_en 0
|
||||||
|
|
||||||
|
info "$tag : Wait Port Link Down"
|
||||||
|
wait_link down "$tag" || RC=1
|
||||||
|
|
||||||
|
# ---- Enable Port -------------------------------------------------------
|
||||||
|
info "$tag : Enable Port"
|
||||||
|
port_en 1
|
||||||
|
|
||||||
|
info "$tag : Wait Link up"
|
||||||
|
if wait_link up "$tag"; then
|
||||||
|
naptime "$SETTLE"
|
||||||
|
else
|
||||||
|
RC=1
|
||||||
|
fi
|
||||||
|
|
||||||
|
# ---- Collect -----------------------------------------------------------
|
||||||
|
# Written to a per-cycle temp file first, then appended. Appending to the
|
||||||
|
# log and then re-reading it to check would mean re-reading an ever larger
|
||||||
|
# file every round - O(n^2) over 200 cycles.
|
||||||
|
info "$tag : Collect Data"
|
||||||
|
: > "$CYCLE_TMP"
|
||||||
|
for u in $UNITS; do
|
||||||
|
{
|
||||||
|
printf '\n--- phydiag %s dsc : unit %s ---\n' "$PORT" "$u"
|
||||||
|
run bcmcmd -n "$u" -c "dsh -c 'phydiag ${PORT} dsc'" 2>&1
|
||||||
|
} >> "$CYCLE_TMP"
|
||||||
|
done
|
||||||
|
|
||||||
|
info "$tag : Read Retimer TX"
|
||||||
|
for l in $TX_LANES; do
|
||||||
|
{
|
||||||
|
printf '\n--- serdes-tx-get sys %s ---\n' "$l"
|
||||||
|
run $SUDO /usr/sbin/blanton_barchetta_tool --serdes-tx-get sys "$l" 2>&1
|
||||||
|
} >> "$CYCLE_TMP"
|
||||||
|
done
|
||||||
|
|
||||||
|
info "$tag : Read Retimer RX"
|
||||||
|
for l in $RX_LANES; do
|
||||||
|
{
|
||||||
|
printf '\n--- serdes-rx-get sys %s ---\n' "$l"
|
||||||
|
run $SUDO /usr/sbin/blanton_barchetta_tool --serdes-rx-get sys "$l" 2>&1
|
||||||
|
} >> "$CYCLE_TMP"
|
||||||
|
done
|
||||||
|
|
||||||
|
cat "$CYCLE_TMP" >> "$LOG"
|
||||||
|
|
||||||
|
if [ "$DRYRUN" = "0" ]; then
|
||||||
|
# bcmcmd exits 0 even when it failed, so count the end marker instead
|
||||||
|
got=$(grep -c 'SERDES DISPLAY DIAG DATA END' "$CYCLE_TMP" 2>/dev/null || true)
|
||||||
|
want=$(echo "$UNITS" | wc -w)
|
||||||
|
if [ "${got:-0}" -lt "$want" ]; then
|
||||||
|
err "$tag : got only ${got:-0}/${want} DSC dumps"
|
||||||
|
RC=1
|
||||||
|
fi
|
||||||
|
|
||||||
|
# The section headings are printed by this script, so counting them
|
||||||
|
# only proves the loop ran. Count sections that have output under them.
|
||||||
|
for d in tx rx; do
|
||||||
|
case "$d" in
|
||||||
|
tx) wantl="$TX_LANES" ;;
|
||||||
|
rx) wantl="$RX_LANES" ;;
|
||||||
|
esac
|
||||||
|
got_l=$(awk -v pat="--- serdes-${d}-get sys " '
|
||||||
|
index($0, pat) == 1 { if (insec && body) ok++; insec = 1; body = 0; next }
|
||||||
|
/^--- / || /^### / { if (insec && body) ok++; insec = 0; next }
|
||||||
|
insec && NF { body = 1 }
|
||||||
|
END { if (insec && body) ok++; print ok + 0 }
|
||||||
|
' "$CYCLE_TMP")
|
||||||
|
want_l=$(echo "$wantl" | wc -w)
|
||||||
|
if [ "${got_l:-0}" -lt "$want_l" ]; then
|
||||||
|
err "$tag : retimer $(echo "$d" | tr 'a-z' 'A-Z') returned data for only ${got_l:-0}/${want_l} lanes"
|
||||||
|
RC=1
|
||||||
|
fi
|
||||||
|
done
|
||||||
|
fi
|
||||||
|
|
||||||
|
n=$((n + 1))
|
||||||
|
done
|
||||||
|
|
||||||
|
resume_gbsyncd
|
||||||
|
|
||||||
|
printf '\n### DSC_CYCLE_DONE rc=%s\n' "$RC" >> "$LOG"
|
||||||
|
if [ "$RC" = "0" ]; then
|
||||||
|
ok "all $CYCLES cycles done -> $LOG"
|
||||||
|
else
|
||||||
|
err "some cycles did not complete as expected - see $LOG"
|
||||||
|
fi
|
||||||
|
printf '### DSC_CYCLE_DONE rc=%s\n' "$RC"
|
||||||
|
exit "$RC"
|
||||||
@@ -0,0 +1,90 @@
|
|||||||
|
cd /usr/sbin
|
||||||
|
sudo docker pause gbsyncd1
|
||||||
|
sudo docker pause gbsyncd0
|
||||||
|
sudo /usr/sbin/blanton_barchetta_tool --serdes-tx-get sys 0
|
||||||
|
|
||||||
|
//Tuning
|
||||||
|
sudo /usr/sbin/blanton_barchetta_tool --serdes-tx-set sys 0 pre=-10 main=87 post=0
|
||||||
|
sudo /usr/sbin/blanton_barchetta_tool --serdes-tx-set sys 1 pre=-10 main=87 post=0
|
||||||
|
sudo /usr/sbin/blanton_barchetta_tool --serdes-tx-set sys 2 pre=-10 main=87 post=0
|
||||||
|
sudo /usr/sbin/blanton_barchetta_tool --serdes-tx-set sys 3 pre=-10 main=87 post=0
|
||||||
|
sudo /usr/sbin/blanton_barchetta_tool --serdes-tx-set sys 4 pre=-10 main=87 post=0
|
||||||
|
sudo /usr/sbin/blanton_barchetta_tool --serdes-tx-set sys 5 pre=-10 main=87 post=0
|
||||||
|
sudo /usr/sbin/blanton_barchetta_tool --serdes-tx-set sys 6 pre=-10 main=87 post=0
|
||||||
|
sudo /usr/sbin/blanton_barchetta_tool --serdes-tx-set sys 7 pre=-10 main=87 post=0
|
||||||
|
bcmcmd -n 0 -c " dsh -c 'phydiag 268 tx_taps set lane=0 main=31 pre1=4 post1=13' "
|
||||||
|
bcmcmd -n 0 -c " dsh -c 'phydiag 268 tx_taps set lane=1 main=31 pre1=4 post1=13' "
|
||||||
|
bcmcmd -n 0 -c " dsh -c 'phydiag 268 tx_taps set lane=2 main=27 pre1=4 post1=13' "
|
||||||
|
bcmcmd -n 0 -c " dsh -c 'phydiag 268 tx_taps set lane=3 main=29 pre1=4 post1=13' "
|
||||||
|
bcmcmd -n 1 -c " dsh -c 'phydiag 268 tx_taps set lane=0 main=29 pre1=4 post1=13' "
|
||||||
|
bcmcmd -n 1 -c " dsh -c 'phydiag 268 tx_taps set lane=1 main=30 pre1=4 post1=13' "
|
||||||
|
bcmcmd -n 1 -c " dsh -c 'phydiag 268 tx_taps set lane=2 main=31 pre1=4 post1=13' "
|
||||||
|
bcmcmd -n 1 -c " dsh -c 'phydiag 268 tx_taps set lane=3 main=30 pre1=4 post1=13' "
|
||||||
|
|
||||||
|
以下Test Cycle 目標200次
|
||||||
|
//Disable Port
|
||||||
|
bcmcmd -n 0 -c "port 268 en=0"
|
||||||
|
bcmcmd -n 1 -c "port 268 en=0"
|
||||||
|
|
||||||
|
//Wait Link Down
|
||||||
|
bcmcmd -n 0 -c "ps 268"
|
||||||
|
bcmcmd -n 1 -c "ps 268"
|
||||||
|
root@sonic:~# bcmcmd -n 0 -c "ps 268"
|
||||||
|
ps 268
|
||||||
|
ena/ speed/ link auto STP lrn max cut loop
|
||||||
|
port link Lns duplex scan neg? state pause discrd ops medium frame thru FEC back
|
||||||
|
ce0(268) !ena 4 100G FD SW No Forward Untag FA Backplane 9122 No RS528
|
||||||
|
drivshell>
|
||||||
|
root@sonic:~# bcmcmd -n 1 -c "ps 268"
|
||||||
|
ps 268
|
||||||
|
ena/ speed/ link auto STP lrn max cut loop
|
||||||
|
port link Lns duplex scan neg? state pause discrd ops medium frame thru FEC back
|
||||||
|
ce0(268) !ena 4 100G FD SW No Forward Untag FA Backplane 9122 No RS528
|
||||||
|
drivshell>
|
||||||
|
root@sonic:~#
|
||||||
|
|
||||||
|
|
||||||
|
//Enable Port
|
||||||
|
bcmcmd -n 0 -c "port 268 en=1"
|
||||||
|
bcmcmd -n 1 -c "port 268 en=1"
|
||||||
|
|
||||||
|
//Wait Link Up
|
||||||
|
bcmcmd -n 0 -c "ps 268"
|
||||||
|
bcmcmd -n 1 -c "ps 268"
|
||||||
|
root@sonic:~# bcmcmd -n 0 -c "ps 268"
|
||||||
|
ps 268
|
||||||
|
ena/ speed/ link auto STP lrn max cut loop
|
||||||
|
port link Lns duplex scan neg? state pause discrd ops medium frame thru FEC back
|
||||||
|
ce0(268) up 4 100G FD SW No Forward Untag FA Backplane 9122 No RS528
|
||||||
|
drivshell>
|
||||||
|
root@sonic:~# bcmcmd -n 1 -c "ps 268"
|
||||||
|
ps 268
|
||||||
|
ena/ speed/ link auto STP lrn max cut loop
|
||||||
|
port link Lns duplex scan neg? state pause discrd ops medium frame thru FEC back
|
||||||
|
ce0(268) up 4 100G FD SW No Forward Untag FA Backplane 9122 No RS528
|
||||||
|
drivshell>
|
||||||
|
root@sonic:~#
|
||||||
|
|
||||||
|
|
||||||
|
//Collect DSC Data
|
||||||
|
bcmcmd -n 0 -c "dsh -c 'phydiag 268 dsc'"
|
||||||
|
bcmcmd -n 1 -c "dsh -c 'phydiag 268 dsc'"
|
||||||
|
|
||||||
|
//讀Retimer TX/RX
|
||||||
|
sudo /usr/sbin/blanton_barchetta_tool --serdes-tx-get sys 0
|
||||||
|
sudo /usr/sbin/blanton_barchetta_tool --serdes-tx-get sys 1
|
||||||
|
sudo /usr/sbin/blanton_barchetta_tool --serdes-tx-get sys 2
|
||||||
|
sudo /usr/sbin/blanton_barchetta_tool --serdes-tx-get sys 3
|
||||||
|
sudo /usr/sbin/blanton_barchetta_tool --serdes-tx-get sys 4
|
||||||
|
sudo /usr/sbin/blanton_barchetta_tool --serdes-tx-get sys 5
|
||||||
|
sudo /usr/sbin/blanton_barchetta_tool --serdes-tx-get sys 6
|
||||||
|
sudo /usr/sbin/blanton_barchetta_tool --serdes-tx-get sys 7
|
||||||
|
|
||||||
|
sudo /usr/sbin/blanton_barchetta_tool --serdes-rx-get sys 0
|
||||||
|
sudo /usr/sbin/blanton_barchetta_tool --serdes-rx-get sys 1
|
||||||
|
sudo /usr/sbin/blanton_barchetta_tool --serdes-rx-get sys 2
|
||||||
|
sudo /usr/sbin/blanton_barchetta_tool --serdes-rx-get sys 3
|
||||||
|
sudo /usr/sbin/blanton_barchetta_tool --serdes-rx-get sys 4
|
||||||
|
sudo /usr/sbin/blanton_barchetta_tool --serdes-rx-get sys 5
|
||||||
|
sudo /usr/sbin/blanton_barchetta_tool --serdes-rx-get sys 6
|
||||||
|
sudo /usr/sbin/blanton_barchetta_tool --serdes-rx-get sys 7
|
||||||
Reference in New Issue
Block a user