Attachment 874674 Details for Bug 1223620

test logs

test-5.14.21-150500.55.59 (text/plain), 37.54 KB, created by Muralidhara MK on 2024-05-06 06:27:28 UTC

(hide)

Description:

Filename:

MIME Type:

Creator: Muralidhara MK

Created: 2024-05-06 06:27:28 UTC

Size: 37.54 KB

patch

obsolete

>Tested in 1 socket system:
>
>KERNEL:
>====================
>muralimk@sh5-pla-rr10-48:~> uname -a
>Linux sh5-pla-rr10-48 5.14.21-150500.55.59-default #1 SMP PREEMPT_DYNAMIC Thu Apr 18 12:59:33 UTC 2024 (e8ae24a) x86_64 x86_64 x86_64 GNU/Linux
>
>BIOS:
>====================
>muralimk@sh5-pla-rr10-48:~> sudo dmidecode -t bios | grep Version
>        Version: TMO0071B
>muralimk@sh5-pla-rr10-48:~>
>
>boot parameters:
>====================
>muralimk@sh5-pla-rr10-48:~> sudo cat /proc/cmdline
>BOOT_IMAGE=/boot/vmlinuz-5.14.21-150500.55.59-default root=UUID=4b7b7014-1f1d-46a0-b59a-7560783d0900 panic=0 nowatchdog msr.allow_writes=on nokaslr amdgpu.noretry=1 pci=realloc=off modprobe.blacklist=amdgpu console=ttyS1,115200 console=tty0 autoyast.debug=0 splash=silent quiet security=apparmor crashkernel=203M,high mitigations=auto
>
>
>Distro:
>=======
>muralimk@sh5-pla-rr10-48:~> sudo cat /etc/os-release
>NAME="SLES"
>VERSION="15-SP5"
>VERSION_ID="15.5"
>PRETTY_NAME="SUSE Linux Enterprise Server 15 SP5"
>ID="sles"
>ID_LIKE="suse"
>ANSI_COLOR="0;32"
>CPE_NAME="cpe:/o:suse:sles:15:sp5"
>DOCUMENTATION_URL="https://documentation.suse.com/"
>
>
>Accelerators present:
>============
>muralimk@sh5-pla-rr10-48:~> sudo lspci | grep -i Accel
>01:00.0 Processing accelerators: Advanced Micro Devices, Inc. [AMD/ATI] Aqua Vanjaram [Instinct MI300A]
>
>
>kernel configs:
>================
>muralimk@sh5-pla-rr10-48:~> sudo cat /boot/config-5.14.21-150500.55.59-default | grep HSMP
>CONFIG_AMD_HSMP=m
>muralimk@sh5-pla-rr10-48:~> sudo cat /boot/config-5.14.21-150500.55.59-default | grep  MCE
>CONFIG_X86_MCE=y
>CONFIG_X86_MCELOG_LEGACY=y
>CONFIG_X86_MCE_INTEL=y
>CONFIG_X86_MCE_AMD=y
>CONFIG_X86_MCE_THRESHOLD=y
>CONFIG_X86_MCE_INJECT=m
>CONFIG_IR_MCE_KBD_DECODER=m
>CONFIG_IR_MCEUSB=m
>CONFIG_EDAC_DECODE_MCE=m
>CONFIG_XEN_MCE_LOG=y
>muralimk@sh5-pla-rr10-48:~> sudo cat /boot/config-5.14.21-150500.55.59-default | grep AMD64
>CONFIG_AGP_AMD64=y
>CONFIG_EDAC_AMD64=m
>muralimk@sh5-pla-rr10-48:~> sudo cat /boot/config-5.14.21-150500.55.59-default | grep ERST
># CONFIG_ACPI_APEI_ERST_DEBUG is not set
>muralimk@sh5-pla-rr10-48:~> sudo cat /boot/config-5.14.21-150500.55.59-default | grep AER
>CONFIG_ACPI_APEI_PCIEAER=y
>CONFIG_PCIEAER=y
>CONFIG_PCIEAER_INJECT=m
>muralimk@sh5-pla-rr10-48:~> sudo cat /boot/config-5.14.21-150500.55.59-default | grep RAS_CEC
>CONFIG_RAS_CEC=y
># CONFIG_RAS_CEC_DEBUG is not set
>muralimk@sh5-pla-rr10-48:~> sudo cat /boot/config-5.14.21-150500.55.59-default | grep RAS_FMPM
>CONFIG_RAS_FMPM=m
>muralimk@sh5-pla-rr10-48:~> sudo cat /boot/config-5.14.21-150500.55.59-default | grep RAS_ATL
>muralimk@sh5-pla-rr10-48:~> sudo cat /boot/config-5.14.21-150500.55.59-default | grep RAS_AMD_ATL
>muralimk@sh5-pla-rr10-48:~> sudo cat /boot/config-5.14.21-150500.55.59-default | grep DEBUG_FS
>CONFIG_XEN_DEBUG_FS=y
>CONFIG_BLK_DEBUG_FS=y
>CONFIG_BLK_DEBUG_FS_ZONED=y
>CONFIG_SCSI_SNIC_DEBUG_FS=y
># CONFIG_SCSI_LPFC_DEBUG_FS is not set
># CONFIG_USB_GADGET_DEBUG_FS is not set
># CONFIG_OCFS2_DEBUG_FS is not set
>CONFIG_DEBUG_FS=y
>CONFIG_DEBUG_FS_ALLOW_ALL=y
># CONFIG_DEBUG_FS_DISALLOW_MOUNT is not set
># CONFIG_DEBUG_FS_ALLOW_NONE is not set
>CONFIG_FAULT_INJECTION_DEBUG_FS=y
>
>Node sysfs entries:
>================================
>muralimk@sh5-pla-rr10-48:~>  ls /sys/devices/system/edac/mc/
>mc0  power  subsystem  uevent
>muralimk@sh5-pla-rr10-48:~>  ls /sys/devices/system/edac/mc/mc0
>ce_count         csrow0  csrow2  max_location  power  rank1   rank11  rank13  rank15  rank17  rank19  rank20  rank22  rank24  rank26  rank28  rank3   rank31  rank5  rank7  rank9           seconds_since_reset  ue_count         uevent
>ce_noinfo_count  csrow1  csrow3  mc_name       rank0  rank10  rank12  rank14  rank16  rank18  rank2   rank21  rank23  rank25  rank27  rank29  rank30  rank4   rank6  rank8  reset_counters  size_mb              ue_noinfo_count
>
>muralimk@sh5-pla-rr10-48:~> cat /sys/devices/system/edac/mc/mc0/mc_name
>F19h_M90h
>muralimk@sh5-pla-rr10-48:~> cat /sys/devices/system/edac/mc/mc0/size_mb
>32768
>
>MCE entries:
>===============================
>muralimk@sh5-pla-rr10-48:~> ls /sys/devices/system/machinecheck/machinecheck0
>bank0   bank14  bank2   bank25  bank30  bank36  bank41  bank47  bank52  bank58  bank63          coherent_slave_0   coherent_slave_4  decode_unit     l2_cache    l3_cache_5  ma_llc_10  ma_llc_7         nbio           print_all  umc_0   umc_5  usr_dp_0
>bank1   bank15  bank20  bank26  bank31  bank37  bank42  bank48  bank53  bank59  bank7           coherent_slave_1   coherent_slave_5  dont_log_ce     l3_cache_0  l3_cache_6  ma_llc_2   ma_llc_8         param_block_0  smu        umc_1   umc_6  usr_dp_1
>bank10  bank16  bank21  bank27  bank32  bank38  bank43  bank49  bank54  bank6   bank8           coherent_slave_10  coherent_slave_6  execution_unit  l3_cache_1  l3_cache_7  ma_llc_3   ma_llc_9         param_block_1  subsystem  umc_10  umc_7  xgmi_phy
>bank11  bank17  bank22  bank28  bank33  bank39  bank44  bank5   bank55  bank60  bank9           coherent_slave_11  coherent_slave_7  floating_point  l3_cache_2  load_store  ma_llc_4   monarch_timeout  pcie           tolerant   umc_2   umc_8
>bank12  bank18  bank23  bank29  bank34  bank4   bank45  bank50  bank56  bank61  check_interval  coherent_slave_2   coherent_slave_8  ignore_ce       l3_cache_3  ma_llc_0    ma_llc_5   mp5              pie            trigger    umc_3   umc_9
>bank13  bank19  bank24  bank3   bank35  bank40  bank46  bank51  bank57  bank62  cmci_disabled   coherent_slave_3   coherent_slave_9  insn_fetch      l3_cache_4  ma_llc_1    ma_llc_6   nbif             power          uevent     umc_4   usb
>
>Persistence feature test:
>==========================
>dmesg logs:
>==========
>..
>[    3.676587] AMD Address Translation Library initialized
>[    3.677937] fmpm: max FRUs: 1, max entries: 8, max record length: 496
>[    3.716409] fmpm: FRU Memory Poison Manager initialized
>...
>
>debugfs:
>===========
>muralimk@sh5-pla-rr10-48:~> sudo ls /sys/kernel/debug/ras/fmpm/entries
>/sys/kernel/debug/ras/fmpm/entries
>muralimk@sh5-pla-rr10-48:~> sudo cat /sys/kernel/debug/ras/fmpm/entries
>fru_idx fru_id                  entry   timestamp               hw_id                   addr                    spa
>0       0xabcdef0000000000      0       *                       *                       *                       *
>0       0xabcdef0000000000      1       *                       *                       *                       *
>0       0xabcdef0000000000      2       *                       *                       *                       *
>0       0xabcdef0000000000      3       *                       *                       *                       *
>0       0xabcdef0000000000      4       *                       *                       *                       *
>0       0xabcdef0000000000      5       *                       *                       *                       *
>0       0xabcdef0000000000      6       *                       *                       *                       *
>0       0xabcdef0000000000      7       *                       *                       *                       *
>
>
>Error Injection:
>=================
>muralimk@sh5-pla-rr10-48:~> cd /usr/local/sbin/amdraserrorinjectiontest/
>muralimk@sh5-pla-rr10-48:/usr/local/sbin/amdraserrorinjectiontest> sudo ./cmdline_amdraserrorinjectiontest --help
>
>      AMD RAS Error Injection Test - 2.3.9
>..
>
>muralimk@sh5-pla-rr10-48:/usr/local/sbin/amdraserrorinjectiontest> sudo bash -c "echo 1 > /sys/devices/system/machinecheck/machinecheck0/check_interval"
>muralimk@sh5-pla-rr10-48:/usr/local/sbin/amdraserrorinjectiontest> cat /sys/devices/system/machinecheck/machinecheck0/check_interval
>1
>
>
>muralimk@sh5-pla-rr10-48:/usr/local/sbin/amdraserrorinjectiontest> sudo cat /proc/iomem | grep RAM
>00001000-0009ffff : System RAM
>00100000-03ffffff : System RAM
>04083000-75cfffff : System RAM
>76000000-9c747017 : System RAM
>9c747018-9c776c57 : System RAM
>9c776c58-9c777017 : System RAM
>9c777018-9c787057 : System RAM
>9c787058-9d9dffff : System RAM
>9da33000-9e8edfff : System RAM
>9e8ef000-a174efff : System RAM
>a61ff000-a7ffdfff : System RAM
>100000000-22fffffff : System RAM
>250000000-204fffffff : System RAM
>muralimk@sh5-pla-rr10-48:/usr/local/sbin/amdraserrorinjectiontest>
>
>muralimk@sh5-pla-rr10-48:/usr/local/sbin/amdraserrorinjectiontest> sudo ./cmdline_amdraserrorinjectiontest --gpuhbmdramecc --address 0x90800 --errortype 0 --errormode 1 --enablemcapolling 1 --ospollingduration 1000
>[gpuhbmdramecc]
>injectonly=0
>errormode=1
>errortype=0
>address=0x90800
>socketdieumc=0:0:0
>channel=0
>maskregenable=0
>beatselection=0
>errorinjectionword0=1
>errorinjectionvalue0=1
>errorinjectionword1=0
>errorinjectionvalue1=0
>errorinjectionword2=0
>errorinjectionvalue2=0
>errorinjectionword3=0
>errorinjectionvalue3=0
>setinjectionmode=0
>setloopduration=1
>duration=1
>enablescrubberredirection=1
>enableperiodicscrubber=1
>clearmcaregister=0
>enablemcapolling=1
>mcapollingduration=50
>delayafterinjection=1000
>ospollingduration=1000
>
>Error Injection location:
>-----------------
>Explicit Error Address :0x90800
>Socket 0 Die 0 UMC 2 CH 6
>Beat 0x0
>Beat Word 0x1
>Beat Value 0x1
>
>Error Detection :
>-----------------
>Error detected in OS log file.
>
>Error Detection Additional Info :
>---------------------------------
>OS log file register values :
>MCA_STATUS_UMC : 0xdc2040000000011b
>MCA_IPID_UMC   : 0x9600592f00
>MCA_SYND_UMC  : 0xa000000
>OS log (dmesg) : Hardware Error: CPU:2 (19:90:0) MC17_STATUSOver|CE|MiscV|AddrV|-|-|SyndV|CECC|-|-|-: 0xdc2040000000011b Hardware Error: IPID: 0x0000009600592f00, Syndrome: 0x000000000a000000
>
>MCA Register read values:
>MCA_STATUS_UMC : 0x0
>MCA_ADDR_UMC   : 0x50
>MCA_SYND_UMC   : 0xa000000
>MCA_IPID_UMC   : 0x9600592f00
>MCA_MISC0_UMC  : 0xd01a000201000000
>MCA_DEADDR_UMC : 0x0
>MCA_DESTAT_UMC : 0x0
>
>Test Outcome :
>--------------
>Pass. (Error detected in OS log file)
>
>HTML result file :
>---------------------------------
>/root//AMD_RAS_INJECTION_LOGS/HBM_DRAM_ECC_LOG.log_1_3_40.html
>
>muralimk@sh5-pla-rr10-48:/usr/local/sbin/amdraserrorinjectiontest>
>
>muralimk@sh5-pla-rr10-48:/usr/local/sbin/amdraserrorinjectiontest> sudo dmesg
>..
>[   71.868098] mce: [Hardware Error]: Machine check events logged
>[   71.868106] [Hardware Error]: Corrected error, no action required.
>[   71.876711] [Hardware Error]: CPU:2 (19:90:0) MC17_STATUS[Over|CE|MiscV|AddrV|-|-|SyndV|CECC|-|-|-]: 0xdc2040000000011b
>[   71.891254] [Hardware Error]: Error Addr: 0x0000000000000050
>[   71.899190] [Hardware Error]: PPIN: 0xabcdef0000000000
>[   71.906524] [Hardware Error]: IPID: 0x0000009600592f00, Syndrome: 0x000000000a000000
>[   71.916766] [Hardware Error]: Unified Memory Controller Ext. Error Code: 0
>[   71.916802] EDAC MC0: 1 CE Unknown syndrome - possible error reporting race on mc#0csrow#2channel#6 (csrow:2 channel:6 page:0x90 offset:0x800 grain:64 syndrome:0x0)
>[   71.926067] [Hardware Error]: cache level: L3/GEN, tx: GEN, mem-tx: RD
>[   72.032727] Memory failure: 0x80: recovery action for unknown page: Ignored
>[   72.052904] Memory failure: 0x88: recovery action for unknown page: Ignored
>[   72.073907] Memory failure: 0x90: recovery action for unknown page: Ignored
>[   72.091308] Memory failure: 0x98: recovery action for unknown page: Ignored
>[   72.108874] Memory failure: 0x282: recovery action for free buddy page: Recovered
>[   72.126749] Memory failure: 0x28a: recovery action for free buddy page: Recovered
>[   72.144658] Memory failure: 0x292: recovery action for free buddy page: Recovered
>[   72.162498] Memory failure: 0x29a: recovery action for free buddy page: Recovered
>
>check peristence storage:
>======================
>muralimk@sh5-pla-rr10-48:/usr/local/sbin/amdraserrorinjectiontest> sudo cat /sys/kernel/debug/ras/fmpm/entries
>fru_idx fru_id                  entry   timestamp               hw_id                   addr                    spa
>0       0xabcdef0000000000      0       2024-05-06T06:03:45     0x0000009600592f00      0x0000000000000050      0x0000000000090800
>0       0xabcdef0000000000      1       *                       *                       *                       *
>0       0xabcdef0000000000      2       *                       *                       *                       *
>0       0xabcdef0000000000      3       *                       *                       *                       *
>0       0xabcdef0000000000      4       *                       *                       *                       *
>0       0xabcdef0000000000      5       *                       *                       *                       *
>0       0xabcdef0000000000      6       *                       *                       *                       *
>0       0xabcdef0000000000      7       *                       *                       *                       *
>
>
>Inject error on other address:
>==============================
>muralimk@sh5-pla-rr10-48:/usr/local/sbin/amdraserrorinjectiontest> sudo ./cmdline_amdraserrorinjectiontest --gpuhbmdramecc --address 0x252000000 --errortype 0 --errormode 1 --enablemcapolling 1 --ospollingduration 1000
>[gpuhbmdramecc]
>injectonly=0
>errormode=1
>errortype=0
>address=0x252000000
>socketdieumc=0:0:0
>channel=0
>maskregenable=0
>beatselection=0
>errorinjectionword0=1
>errorinjectionvalue0=1
>errorinjectionword1=0
>errorinjectionvalue1=0
>errorinjectionword2=0
>errorinjectionvalue2=0
>errorinjectionword3=0
>errorinjectionvalue3=0
>setinjectionmode=0
>setloopduration=1
>duration=1
>enablescrubberredirection=1
>enableperiodicscrubber=1
>clearmcaregister=0
>enablemcapolling=1
>mcapollingduration=50
>delayafterinjection=1000
>ospollingduration=1000
>
>Error Injection location:
>-----------------
>Explicit Error Address :0x252000000
>Socket 0 Die 2 UMC 2 CH 5
>Beat 0x0
>Beat Word 0x1
>Beat Value 0x1
>
>Error Detection :
>-----------------
>Error detected in OS log file.
>
>Error Detection Additional Info :
>---------------------------------
>OS log file register values :
>MCA_STATUS_UMC : 0xdc2040000000011b
>MCA_IPID_UMC   : 0x809600591f00
>MCA_SYND_UMC  : 0xa000000
>OS log (dmesg) : Hardware Error: CPU:7 (19:90:0) MC27_STATUSOver|CE|MiscV|AddrV|-|-|SyndV|CECC|-|-|-: 0xdc2040000000011b Hardware Error: IPID: 0x0000809600591f00, Syndrome: 0x000000000a000000
>
>MCA Register read values:
>MCA_STATUS_UMC : 0x0
>MCA_ADDR_UMC   : 0x202000
>MCA_SYND_UMC   : 0xa000000
>MCA_IPID_UMC   : 0x809600591f00
>MCA_MISC0_UMC  : 0xd01a000201000000
>MCA_DEADDR_UMC : 0x0
>MCA_DESTAT_UMC : 0x0
>
>Test Outcome :
>--------------
>Pass. (Error detected in OS log file)
>
>HTML result file :
>---------------------------------
>/root//AMD_RAS_INJECTION_LOGS/HBM_DRAM_ECC_LOG.log_1_6_1.html
>
>muralimk@sh5-pla-rr10-48:/usr/local/sbin/amdraserrorinjectiontest>
>
>muralimk@sh5-pla-rr10-48:/usr/local/sbin/amdraserrorinjectiontest> sud odmesg
>-bash: sud: command not found
>muralimk@sh5-pla-rr10-48:/usr/local/sbin/amdraserrorinjectiontest> sudo dmesg
>[  213.820130] mce: [Hardware Error]: Machine check events logged
>[  213.820136] [Hardware Error]: Corrected error, no action required.
>[  213.828586] [Hardware Error]: CPU:7 (19:90:0) MC27_STATUS[Over|CE|MiscV|AddrV|-|-|SyndV|CECC|-|-|-]: 0xdc2040000000011b
>[  213.842795] [Hardware Error]: Error Addr: 0x0000000000202000
>[  213.850576] [Hardware Error]: PPIN: 0xabcdef0000000000
>[  213.857764] [Hardware Error]: IPID: 0x0000809600591f00, Syndrome: 0x000000000a000000
>[  213.867867] [Hardware Error]: Unified Memory Controller Ext. Error Code: 0
>[  213.867901] EDAC MC0: 1 CE Unknown syndrome - possible error reporting race on mc#0csrow#1channel#5 (csrow:1 channel:5 page:0x252000 offset:0xc00 grain:64 syndrome:0x0)
>[  213.877040] [Hardware Error]: cache level: L3/GEN, tx: GEN, mem-tx: RD
>[  213.894346] Memory failure: 0x252000: recovery action for free buddy page: Recovered
>[  213.912922] Memory failure: 0x252008: recovery action for free buddy page: Recovered
>[  213.931126] Memory failure: 0x252010: recovery action for free buddy page: Recovered
>[  213.949264] Memory failure: 0x252018: recovery action for free buddy page: Recovered
>[  213.967370] Memory failure: 0x252202: recovery action for free buddy page: Recovered
>[  213.985464] Memory failure: 0x25220a: recovery action for free buddy page: Recovered
>[  214.003552] Memory failure: 0x252212: recovery action for free buddy page: Recovered
>[  214.021659] Memory failure: 0x25221a: recovery action for free buddy page: Recovered
>
>muralimk@sh5-pla-rr10-48:/usr/local/sbin/amdraserrorinjectiontest> sudo cat /sys/kernel/debug/ras/fmpm/entries                                                fru_idx fru_id                  entry   timestamp               hw_id                   addr                    spa
>0       0xabcdef0000000000      0       2024-05-06T06:03:45     0x0000009600592f00      0x0000000000000050      0x0000000000090800
>0       0xabcdef0000000000      1       2024-05-06T06:06:07     0x0000809600591f00      0x0000000000202000      0x0000000252000c00
>0       0xabcdef0000000000      2       *                       *                       *                       *
>0       0xabcdef0000000000      3       *                       *                       *                       *
>0       0xabcdef0000000000      4       *                       *                       *                       *
>0       0xabcdef0000000000      5       *                       *                       *                       *
>0       0xabcdef0000000000      6       *                       *                       *                       *
>0       0xabcdef0000000000      7       *                       *                       *                       *
>muralimk@sh5-pla-rr10-48:/usr/local/sbin/amdraserrorinjectiontest>
>
>
>Reboot and Check the persisted poison entries:
>============================================
>dmesg:
>...
>[    3.677386] AMD Address Translation Library initialized
>[    3.678599] fmpm: max FRUs: 1, max entries: 8, max record length: 496
>[    3.717959] intel_rapl_common: Found RAPL domain package
>[    3.801325] bpfilter: Loaded bpfilter_umh pid 1238
>[    3.801862] Started bpfilter
>[    4.026246] Memory failure: 0x80: recovery action for unknown page: Ignored
>[    4.036346] Memory failure: 0x88: recovery action for unknown page: Ignored
>[    4.048373] Memory failure: 0x90: recovery action for unknown page: Ignored
>[    4.061902] Memory failure: 0x98: recovery action for unknown page: Ignored
>[    4.070899] Memory failure: 0x282: recovery action for free buddy page: Recovered
>[    4.080524] Memory failure: 0x28a: recovery action for free buddy page: Recovered
>[    4.090269] Memory failure: 0x292: recovery action for free buddy page: Recovered
>[    4.099804] Memory failure: 0x29a: recovery action for free buddy page: Recovered
>[    4.111789] Memory failure: 0x252000: recovery action for free buddy page: Recovered
>[    4.121587] Memory failure: 0x252008: recovery action for free buddy page: Recovered
>[    4.131976] Memory failure: 0x252010: recovery action for free buddy page: Recovered
>[    4.141636] Memory failure: 0x252018: recovery action for free buddy page: Recovered
>[    4.151769] Memory failure: 0x252202: recovery action for free buddy page: Recovered
>[    4.161788] Memory failure: 0x25220a: recovery action for free buddy page: Recovered
>[    4.171620] Memory failure: 0x252212: recovery action for free buddy page: Recovered
>[    4.181421] Memory failure: 0x25221a: recovery action for free buddy page: Recovered
>[    4.190881] fmpm: FRU Memory Poison Manager initialized
>...
>
>muralimk@sh5-pla-rr10-48:~> sudo cat /sys/kernel/debug/ras/fmpm/entries
>fru_idx fru_id                  entry   timestamp               hw_id                   addr                    spa
>0       0xabcdef0000000000      0       2024-05-06T06:03:45     0x0000009600592f00      0x0000000000000050      0x0000000000090800
>0       0xabcdef0000000000      1       2024-05-06T06:06:07     0x0000809600591f00      0x0000000000202000      0x0000000252000c00
>0       0xabcdef0000000000      2       *                       *                       *                       *
>0       0xabcdef0000000000      3       *                       *                       *                       *
>0       0xabcdef0000000000      4       *                       *                       *                       *
>0       0xabcdef0000000000      5       *                       *                       *                       *
>0       0xabcdef0000000000      6       *                       *                       *                       *
>0       0xabcdef0000000000      7       *                       *                       *                       *
>
>
>
>
>
>In-band libray and HSMP driver test:
>===================================================
>muralimk@sh5-pla-rr10-48:~> sudo modprobe amd_hsmp
>muralimk@sh5-pla-rr10-48:~> lsmod | grep hsmp
>amd_hsmp               16384  0
>muralimk@sh5-pla-rr10-48:~> ls /dev/hsmp
>/dev/hsmp
>muralimk@sh5-pla-rr10-48:~> ls /sys/devices/platform/amd_hsmp/
>driver  driver_override  misc  modalias  power  socket0  subsystem  uevent
>muralimk@sh5-pla-rr10-48:~>
>muralimk@sh5-pla-rr10-48:~> sudo modinfo amd_hsmp
>filename:       /lib/modules/5.14.21-150500.55.59-default/kernel/drivers/platform/x86/amd/amd_hsmp.ko.zst
>license:        GPL v2
>version:        2.0
>description:    AMD HSMP Platform Interface Driver
>suserelease:    SLE15-SP5
>srcversion:     F74FB11F565032398064F84
>depends:
>supported:      yes
>retpoline:      Y
>intree:         Y
>name:           amd_hsmp
>vermagic:       5.14.21-150500.55.59-default SMP preempt mod_unload modversions
>sig_id:         PKCS#7
>signer:         SUSE Linux Enterprise Secure Boot CA
>sig_key:        CA:FC:B5:D7:5E:C5:89:82
>sig_hashalgo:   sha256
>signature:      00:3B:6D:A3:AB:60:8F:A7:C4:4D:9F:B2:2B:56:56:A7:4F:ED:B1:58:
>                A6:C5:05:34:2F:5C:45:06:86:25:31:EC:AD:A1:E0:16:D2:6C:AD:A4:
>                9A:9D:80:CB:8C:2A:8D:66:2C:3B:38:8F:53:D3:EE:C3:D5:D8:1D:5E:
>                56:1A:BF:65:35:CB:DB:6B:5A:22:51:E1:9C:01:AD:59:42:FE:53:B6:
>                60:68:76:D8:93:2A:8A:95:4C:3C:FA:DC:4D:0F:97:CA:58:8E:4A:25:
>                20:77:FA:C5:0C:E4:8C:26:F1:7D:22:BF:34:DF:63:2B:4F:DA:13:B7:
>                8D:EB:F1:26:A7:26:E5:9F:99:2B:F6:60:44:DE:40:1F:0F:9D:86:DE:
>                DC:F8:9B:1E:77:D0:12:E2:ED:D4:13:70:31:F4:EF:56:5D:CA:38:B0:
>                6B:34:BA:7D:A3:AE:F4:A2:0A:41:A3:33:7E:F5:95:72:BD:F9:B1:5C:
>                2B:9B:B8:DB:B1:19:CB:3B:AA:6F:D8:33:5B:A2:37:4E:79:DD:EC:01:
>                7C:49:D0:07:35:2E:F9:D6:86:0B:A9:E4:6A:1E:A6:C2:69:37:5C:40:
>                FF:11:47:64:81:44:F7:17:15:45:39:B8:13:19:9E:34:7F:2B:2A:BE:
>                5A:56:1A:D2:C9:46:C5:18:03:9D:E9:55:5B:94:01:46
>muralimk@sh5-pla-rr10-48:~>
>
>Clone IN-BAND library:
>====================
>git clone https://github.com/amd/esmi_ib_library.git
>
>compile and execute:
>----------------
>muralimk@sh5-pla-rr10-48:~/esmi_ib_library/build> sudo ./e_smi_tool -h
>
>============================= E-SMI ===================================
>
>Usage: ./e_smi_tool [Option]... <INPUT>...
>
>Output Option<s>:
>  -h, --help                                                    Show this help message
>  -A, --showall                                                 Show all esmi parameter values
>  -V  --version                                                 Show e-smi library version
>  --testmailbox [SOCKET] [VALUE<0-0xFFFFFFFF>]                  Test HSMP mailbox interface
>  --writemsrallowlist                                           Write msr-safe allowlist file
>
>Get Option<s>:
>  --showcoreenergy [CORE]                                       Show energy for a given CPU (Joules)
>  --showsockenergy                                              Show energy for all sockets (KJoules)
>  --showsockpower                                               Show power metrics for all sockets (Watts)
>  --showcorebl [CORE]                                           Show Boostlimit for a given CPU (MHz)
>  --showsockc0res [SOCKET]                                      Show c0_residency for a given socket (%%)
>  --showsmufwver                                                Show SMU FW Version
>  --showhsmpprotover                                            Show HSMP Protocol Version
>  --showprochotstatus                                           Show HSMP PROCHOT status for all sockets
>  --showclocks                                                  Show Clock Metrics (MHz) for all sockets
>  --showcclkfreqlimit [CORE]                                    Show current clock frequency limit(MHz) for a given core
>  --showsvipower                                                Show svi based power telemetry of all rails for all sockets
>  --showxgmibw [LINK<G0-G7>] [BW<AGG_BW,RD_BW,WR_BW>]           Show xGMI bandwidth for a given socket, linkname and bwtype
>  --showiobw [SOCKET] [LINK<P2,P3,G0-G7>]                       Show IO aggregate bandwidth for a given socket and linkname
>  --showlclkdpmlevel [SOCKET] [NBIOID<0-3>]                     Show lclk dpm level for a given nbio in a given socket
>  --showsockclkfreqlimit [SOCKET]                               Show current clock frequency limit(MHz) for a given socket
>  --showmetrictablever                                          Show Metrics Table Version
>  --showmetrictable [SOCKET]                                    Show Metrics Table
>
>Set Option<s>:
>  --setpowerlimit [SOCKET] [POWER]                              Set power limit for a given socket (mWatts)
>  --setcorebl [CORE] [BOOSTLIMIT]                               Set boost limit for a given core (MHz)
>  --setsockbl [SOCKET] [BOOSTLIMIT]                             Set Boost limit for a given Socket (MHz)
>  --apbdisable [SOCKET] [PSTATE<0-2>]                           Set Data Fabric Pstate for a given socket
>  --apbenable [SOCKET]                                          Enable the Data Fabric performance boost algorithm for a given socket
>  --setxgmiwidth [MIN<0-2>] [MAX<0-2>]                          Set xgmi link width in a multi socket system (MAX >= MIN)
>  --setlclkdpmlevel [SOCKET] [NBIOID<0-3>] [MIN<0-3>] [MAX<0-3>]Set lclk dpm level for a given nbio in a given socket (MAX >= MIN)
>
>============================= End of E-SMI ============================
>
>muralimk@sh5-pla-rr10-48:~/esmi_ib_library/build> sudo ./e_smi_tool --showsmufwver
>
>============================= E-SMI ===================================
>
>
>------------------------------------------
>| SMU FW Version   |  85.74.1            |
>------------------------------------------
>
>============================= End of E-SMI ============================
>muralimk@sh5-pla-rr10-48:~/esmi_ib_library/build> sudo ./e_smi_tool --showhsmpprotover
>
>============================= E-SMI ===================================
>
>
>---------------------------------
>| HSMP Protocol Version  | 6    |
>---------------------------------
>
>============================= End of E-SMI ============================
>muralimk@sh5-pla-rr10-48:~/esmi_ib_library/build>
>muralimk@sh5-pla-rr10-48:~/esmi_ib_library/build> sudo ./e_smi_tool --showsockpower
>
>============================= E-SMI ===================================
>
>
>-----------------------------------------------------
>| Sensor Name                    | Socket 0         |
>-----------------------------------------------------
>| Power (Watts)                  | 132.938          |
>| PowerLimit (Watts)             | 550.000          |
>| PowerLimitMax (Watts)          | 550.000          |
>-----------------------------------------------------
>
>
>============================= End of E-SMI ============================
>
>
>muralimk@sh5-pla-rr10-48:~/esmi_ib_library/build> sudo ./e_smi_tool --showmetrictable 0
>
>============================= E-SMI ===================================
>
>-------------------------------------------------------------------------
>|                       METRICS TABLE (FAMILY:0x19,MODEL:0x90)          |
>-------------------------------------------------------------------------
>
>-------------------------------------------------------------------------
>| ACCUMULATOR COUNTER                   |  2911997                      |
>-------------------------------------------------------------------------
>
>-------------------------------------------------------------------------
>| MAX SOCKET TEMP                       |              37.000 Â°C        |
>| MAX VR TEMP                           |              32.000 Â°C        |
>| MAX HBM TEMP                          |              38.000 Â°C        |
>| MAX SOCKET TEMP ACC                   |       104819645.125 Â°C        |
>| MAX VR TEMP ACC                       |        99991882.000 Â°C        |
>| MAX HBM TEMP ACC                      |       111031422.000 Â°C        |
>-------------------------------------------------------------------------
>
>-----------------------------------------------------------------
>| SOCKET POWER LIMIT                    |  550.000 W            |
>| MAX SOCKET POWER LIMIT                |  550.000 W            |
>| SOCKET POWER                          |  133.766 W            |
>-----------------------------------------------------------------
>
>-------------------------------------------------------------------------
>| TIMESTAMP Raw                         |          295992961932         |
>| TIMESTAMP Readable                    |  Mon May  6 00:47:07 2024
>
>| SOCKET ENERGY ACC                     |          340.143 kJ           |
>| CCD ENERGY ACC                        |            8.646 kJ           |
>| XCD ENERGY ACC                        |          165.252 kJ           |
>| AID ENERGY ACC                        |          137.514 kJ           |
>| HBM ENERGY ACC                        |           28.732 kJ           |
>-------------------------------------------------------------------------
>
>-----------------------------------------------------------------
>| CCLK frequency limit                  |  2.467 GHz            |
>| GFXCLK frequency limit                |  0.000 MHz            |
>| Effective FCLK frequency              |  1494.632 MHz         |
>| Effective UCLK frequency              |  1031.028 MHz         |
>-----------------------------------------------------------------
>
>-------------------------------------------------------------------------
>| Effective frequency per AID:                                          |
>-------------------------------------------------------------------------
>| AID | SOCCLK          | VCLK          | DCLK          | LCLK          |
>-------------------------------------------------------------------------
>| [0] | 23.161 MHz      | 0.000 MHz     | 7.143 MHz     | 18.772 MHz    |
>| [1] | 23.173 MHz      | 33.884 MHz    | 25.948 MHz    | 18.750 MHz    |
>| [2] | 23.214 MHz      | 33.938 MHz    | 25.986 MHz    | 363.283 MHz   |
>| [3] | 23.214 MHz      | 33.938 MHz    | 25.987 MHz    | 835.105 MHz   |
>-------------------------------------------------------------------------
>
>---------------------------------------------------------------------------------------------------------
>| List of supported frequencies(0 means state is not supported):                                        |
>---------------------------------------------------------------------------------------------------------
>| AID | FCLK            | UCLK          | SOCCLK        | VCLK          | DCLK          | LCLK          |
>---------------------------------------------------------------------------------------------------------
>| [0] |1200.000 MHz     |900.000 MHz    |615.385 MHz    |914.286 MHz    |711.111 MHz    |600.000 MHz    |
>| [1] |1600.000 MHz     |1100.000 MHz   |800.000 MHz    |1114.286 MHz   |866.667 MHz    |844.444 MHz    |
>| [2] |1900.000 MHz     |1200.000 MHz   |888.889 MHz    |1300.000 MHz   |975.000 MHz    |1150.000 MHz   |
>| [3] |2100.000 MHz     |1300.000 MHz   |1000.000 MHz   |1433.333 MHz   |1075.000 MHz   |1150.000 MHz   |
>--------------------------------------------------------------------------------------------------------
>
>------------------------------------------------------------------------------------------------------------------
>| CCLK frequency accumulated for target CPUs:                                                                    |
>-----------------------------------------------------------------------------------------------------------------
>| CPU[00] :          4557519.538 GHz  | CPU[08] :          4573013.189 GHz  | CPU[16] :          4571081.411 GHz |
>| CPU[01] :          4559820.905 GHz  | CPU[09] :          4559000.000 GHz  | CPU[17] :          4562807.360 GHz |
>| CPU[02] :          4774842.289 GHz  | CPU[10] :          4569911.198 GHz  | CPU[18] :          4563463.961 GHz |
>| CPU[03] :          4690756.633 GHz  | CPU[11] :          4570757.727 GHz  | CPU[19] :          4560539.646 GHz |
>| CPU[04] :          4562386.721 GHz  | CPU[12] :          4562552.958 GHz  | CPU[20] :          4559223.128 GHz |
>| CPU[05] :          4564526.087 GHz  | CPU[13] :          4560658.474 GHz  | CPU[21] :          4558797.465 GHz |
>| CPU[06] :          4556683.144 GHz  | CPU[14] :          4559935.987 GHz  | CPU[22] :          4566151.046 GHz |
>| CPU[07] :          4555876.834 GHz  | CPU[15] :          4570279.410 GHz  | CPU[23] :          4580673.705 GHz |
>------------------------------------------------------------------------------------------------------------------
>
>---------------------------------------------------------
>| Frequency per target XCC:                             |
>---------------------------------------------------------
>| XCC | GFXCLK ACC                      | GFXCLK        |
>---------------------------------------------------------
>| [0] |                 0.000 MHz       | 0.000 MHz     |
>| [1] |                 0.000 MHz       | 0.000 MHz     |
>| [2] |                 0.000 MHz       | 0.000 MHz     |
>| [3] |                 0.000 MHz       | 0.000 MHz     |
>| [4] |                 0.000 MHz       | 0.000 MHz     |
>| [5] |                 0.000 MHz       | 0.000 MHz     |
>| [6] |                 0.000 MHz       | 0.000 MHz     |
>| [7] |                 0.000 MHz       | 0.000 MHz     |
>---------------------------------------------------------
>
>-----------------------------------------------------------------
>| Max CCLK frequency supported by CPU   |  3.700 GHz            |
>| Min CCLK frequency supported by CPU   |  0.400 GHz            |
>| Max GFXCLK supported by accelerator   |  2225.000 MHz         |
>| Min GFXCLK supported by accelerator   |  500.000 MHz          |
>| Max LCLK DPM state range              |  2                    |
>| Min LCLK DPM state range              |  0                    |
>------------------------------------------------------------------
>
>-----------------------------------------------------------------
>| Current operating XGMI link width     |  0.000                 |
>| Current operating XGMI link bitrate   |  0.000 Gbps            |
>------------------------------------------------------------------
>
>---------------------------------------------------------------------------
>| XGMI Bandwidth accumulated per XGMI link in local socket                |
>---------------------------------------------------------------------------
>| Link  |       XGMI Read BW            |       XGMI Write BW             |
>---------------------------------------------------------------------------
>| [0]   |               0.000 Gbps      |              0.000 Gbps         |
>| [1]   |               0.000 Gbps      |              0.000 Gbps         |
>| [2]   |               0.000 Gbps      |              0.000 Gbps         |
>| [3]   |               0.000 Gbps      |              0.000 Gbps         |
>| [4]   |               0.000 Gbps      |              0.000 Gbps         |
>| [5]   |               0.000 Gbps      |              0.000 Gbps         |
>| [6]   |               0.000 Gbps      |              0.000 Gbps         |
>| [7]   |               0.000 Gbps      |              0.000 Gbps         |
>--------------------------------------------------------------------------
>
>--------------------------------------------------------------------------
>| Avg C0 residency of all enabled cores |               0.008 %          |
>| Avg XCC busy for all enabled XCCs     |               0.000 %          |
>| HBM BW utilization for all HBM stacks |               0.000 %          |
>| Acc value of SocketC0Residency        |           12099.022            |
>| Acc value of SocketGfxBusy            |               0.000            |
>| HBM BW for all socket HBM stacks      |           26970.388 Gbps       |
>| Max HBM BW running at max UCLK freq   |            5324.801 Gbps       |
>| Acc value of Dram BW Utilization      |               2.509            |
>--------------------------------------------------------------------------
>
>--------------------------------------------------------------------------------
>| PCIe BW for devs connected to AID[0]  |               0.000 Gbps              |
>| PCIe BW for devs connected to AID[1]  |               0.000 Gbps              |
>| PCIe BW for devs connected to AID[2]  |               0.000 Gbps              |
>| PCIe BW for devs connected to AID[3]  |               0.000 Gbps              |
>--------------------------------------------------------------------------------
>
>---------------------------------------------------------
>| Active controllers                    | Acc value     |
>---------------------------------------------------------
>| Prochot                               |  0            |
>| PPT controller                        |  2911996      |
>| Socket thermal throttling controller  |  2911997      |
>| VR thermal throttling controller      |  1            |
>| HBM thermal throttling controller     |  2911997      |
>---------------------------------------------------------
>
>
>============================= End of E-SMI ============================

Actions: View

Attachments on bug 1223620: 874672 | 874674