H100 GPUs with Protected PCIe mode enabled are not working in TDX instance #124
Description
In an Ubuntu 25.04 host environment with TDX enabled, I tried to use run_td.sh in canonical/tdx to activate eight H100 GPUs in Protected PCIe (PPCIe) mode, but the NVIDIA driver fails to recognize them and shows an error.
[ 14.817960] nvidia-nvswitch0: using MSI
[ 19.857274] nvidia-nvswitch: Probing device 0000:02:00.0, Vendor Id = 0x10de, Device Id = 0x22a3, Class = 0x68000
[ 20.898856] nvidia-nvswitch1: using MSI
[ 25.942622] nvidia-nvswitch: Probing device 0000:03:00.0, Vendor Id = 0x10de, Device Id = 0x22a3, Class = 0x68000
[ 25.948274] nvidia-nvswitch1: SXid (PCI:0000:02:00.0): 22013, Non-fatal, Link 00 Minion Link DLREQ interrupt
[ 25.950244] nvidia-nvswitch1: SXid (PCI:0000:02:00.0): 22013, Severity 0 Engine instance 00 Sub-engine instance 00
[ 25.952303] nvidia-nvswitch1: SXid (PCI:0000:02:00.0): 22013, Data {0x00000000, 0x00000000, 0x00000000, 0x00000000, 0x80005302, 0x00000000, 0x00000000, 0x00000000, 0x00000000}
[ 25.955391] nvidia-nvswitch1: SXid (PCI:0000:02:00.0): 22013, Non-fatal, Link 01 Minion Link DLREQ interrupt
[ 25.957361] nvidia-nvswitch1: SXid (PCI:0000:02:00.0): 22013, Severity 0 Engine instance 01 Sub-engine instance 00
[ 25.959397] nvidia-nvswitch1: SXid (PCI:0000:02:00.0): 22013, Data {0x00000001, 0x00000001, 0x00000000, 0x00000001, 0x80005302, 0x00000000, 0x00000000, 0x00000000, 0x00000000}
[ 25.962516] nvidia-nvswitch1: SXid (PCI:0000:02:00.0): 22013, Non-fatal, Link 02 Minion Link DLREQ interrupt
[ 25.964487] nvidia-nvswitch1: SXid (PCI:0000:02:00.0): 22013, Severity 0 Engine instance 02 Sub-engine instance 00
[ 25.966532] nvidia-nvswitch1: SXid (PCI:0000:02:00.0): 22013, Data {0x00000002, 0x00000002, 0x00000000, 0x00000002, 0x80005302, 0x00000000, 0x00000000, 0x00000000, 0x00000000}
[ 25.969612] nvidia-nvswitch1: SXid (PCI:0000:02:00.0): 22013, Non-fatal, Link 03 Minion Link DLREQ interrupt
[ 25.971614] nvidia-nvswitch1: SXid (PCI:0000:02:00.0): 22013, Severity 0 Engine instance 03 Sub-engine instance 00
[ 25.973626] nvidia-nvswitch1: SXid (PCI:0000:02:00.0): 22013, Data {0x00000003, 0x00000003, 0x00000000, 0x00000003, 0x80005302, 0x00000000, 0x00000000, 0x00000000, 0x00000000}
[ 25.976765] nvidia-nvswitch1: SXid (PCI:0000:02:00.0): 22013, Non-fatal, Link 04 Minion Link DLREQ interrupt
[ 25.978738] nvidia-nvswitch1: SXid (PCI:0000:02:00.0): 22013, Severity 0 Engine instance 04 Sub-engine instance 00
[ 25.980753] nvidia-nvswitch1: SXid (PCI:0000:02:00.0): 22013, Data {0x00000004, 0x00000004, 0x00000000, 0x00000004, 0x80005302, 0x00000000, 0x00000000, 0x00000000, 0x00000000}
[ 25.983859] nvidia-nvswitch1: SXid (PCI:0000:02:00.0): 22013, Non-fatal, Link 05 Minion Link DLREQ interrupt
[ 25.985828] nvidia-nvswitch1: SXid (PCI:0000:02:00.0): 22013, Severity 0 Engine instance 05 Sub-engine instance 00
[ 25.987837] nvidia-nvswitch1: SXid (PCI:0000:02:00.0): 22013, Data {0x00000005, 0x00000005, 0x00000000, 0x00000005, 0x80005302, 0x00000000, 0x00000000, 0x00000000, 0x00000000}
[ 25.990929] nvidia-nvswitch1: SXid (PCI:0000:02:00.0): 22013, Non-fatal, Link 06 Minion Link DLREQ interrupt
[ 25.992929] nvidia-nvswitch1: SXid (PCI:0000:02:00.0): 22013, Severity 0 Engine instance 06 Sub-engine instance 00
[ 25.994967] nvidia-nvswitch1: SXid (PCI:0000:02:00.0): 22013, Data {0x00000006, 0x00000006, 0x00000000, 0x00000006, 0x80005302, 0x00000000, 0x00000000, 0x00000000, 0x00000000}
[ 25.998059] nvidia-nvswitch1: SXid (PCI:0000:02:00.0): 22013, Non-fatal, Link 07 Minion Link DLREQ interrupt
[ 26.000051] nvidia-nvswitch1: SXid (PCI:0000:02:00.0): 22013, Severity 0 Engine instance 07 Sub-engine instance 00
[ 26.002084] nvidia-nvswitch1: SXid (PCI:0000:02:00.0): 22013, Data {0x00000007, 0x00000007, 0x00000000, 0x00000007, 0x80005302, 0x00000000, 0x00000000, 0x00000000, 0x00000000}
[ 26.005241] nvidia-nvswitch1: SXid (PCI:0000:02:00.0): 22013, Non-fatal, Link 32 Minion Link DLREQ interrupt
[ 26.007198] nvidia-nvswitch1: SXid (PCI:0000:02:00.0): 22013, Severity 0 Engine instance 32 Sub-engine instance 00
[ 26.009219] nvidia-nvswitch1: SXid (PCI:0000:02:00.0): 22013, Data {0x00000020, 0x00000020, 0x00000000, 0x00000020, 0x80005302, 0x00000000, 0x00000000, 0x00000000, 0x00000000}
[ 26.012302] nvidia-nvswitch1: SXid (PCI:0000:02:00.0): 22013, Non-fatal, Link 33 Minion Link DLREQ interrupt
[ 26.014286] nvidia-nvswitch1: SXid (PCI:0000:02:00.0): 22013, Severity 0 Engine instance 33 Sub-engine instance 00
[ 26.016332] nvidia-nvswitch1: SXid (PCI:0000:02:00.0): 22013, Data {0x00000021, 0x00000021, 0x00000000, 0x00000021, 0x80005302, 0x00000000, 0x00000000, 0x00000000, 0x00000000}
[ 26.019413] nvidia-nvswitch1: SXid (PCI:0000:02:00.0): 22013, Non-fatal, Link 34 Minion Link DLREQ interrupt
[ 26.021371] nvidia-nvswitch1: SXid (PCI:0000:02:00.0): 22013, Severity 0 Engine instance 34 Sub-engine instance 00
...
[ 26.977920] nvidia-nvswitch2: using MSI
[ 32.016590] nvidia-nvswitch: Probing device 0000:04:00.0, Vendor Id = 0x10de, Device Id = 0x22a3, Class = 0x68000
[ 33.044193] nvidia-nvswitch3: using MSI
[ 38.089059] nvidia-nvswitch3: SXid (PCI:0000:04:00.0): 22013, Non-fatal, Link 32 Minion Link DLREQ interrupt
[ 38.091306] nvidia-nvswitch3: SXid (PCI:0000:04:00.0): 22013, Severity 0 Engine instance 32 Sub-engine instance 00
...
[ 38.168211] nvidia-nvswitch3: SXid (PCI:0000:04:00.0): 22013, Data {0x0000002a, 0x0000002a, 0x00000000, 0x0000002a, 0x80005302, 0x00000000, 0x00000000, 0x00000000, 0x00000000}
[ 38.171374] nvidia-nvswitch3: SXid (PCI:0000:04:00.0): 22013, Non-fatal, Link 43 Minion Link DLREQ interrupt
[ 38.173420] nvidia-nvswitch3: SXid (PCI:0000:04:00.0): 22013, Severity 0 Engine instance 43 Sub-engine instance 00
[ 38.174242] NVRM: loading NVIDIA UNIX Open Kernel Module for x86_64 570.195.03 Release Build (dvs-builder@U22-I3-H04-03-1) Sat Sep 20 00:47:16 UTC 2025
[ 38.175561] nvidia-nvswitch3: SXid (PCI:0000:04:00.0): 22013, Data {0x0000002b, 0x0000002b, 0x00000000, 0x0000002b, 0x80005302, 0x00000000, 0x00000000, 0x00000000, 0x00000000}
[ 38.178720] nvidia-nvswitch3: SXid (PCI:0000:04:00.0): 22013, Non-fatal, Link 44 Minion Link DLREQ interrupt
[ 38.180708] nvidia-nvswitch3: SXid (PCI:0000:04:00.0): 22013, Severity 0 Engine instance 44 Sub-engine instance 00
[ 38.183583] nvidia-nvswitch3: SXid (PCI:0000:04:00.0): 22013, Data {0x0000002c, 0x0000002c, 0x00000000, 0x0000002c, 0x80005302, 0x00000000, 0x00000000, 0x00000000, 0x00000000}
[ 38.187830] nvidia-nvswitch3: SXid (PCI:0000:04:00.0): 22013, Non-fatal, Link 45 Minion Link DLREQ interrupt
[ 38.189818] nvidia-nvswitch3: SXid (PCI:0000:04:00.0): 22013, Severity 0 Engine instance 45 Sub-engine instance 00
[ 38.191863] nvidia-nvswitch3: SXid (PCI:0000:04:00.0): 22013, Data {0x0000002d, 0x0000002d, 0x00000000, 0x0000002d, 0x80005302, 0x00000000, 0x00000000, 0x00000000, 0x00000000}
[ 38.194959] nvidia-nvswitch3: SXid (PCI:0000:04:00.0): 22013, Non-fatal, Link 46 Minion Link DLREQ interrupt
[ 38.197002] nvidia-nvswitch3: SXid (PCI:0000:04:00.0): 22013, Severity 0 Engine instance 46 Sub-engine instance 00
[ 38.199112] nvidia-nvswitch3: SXid (PCI:0000:04:00.0): 22013, Data {0x0000002e, 0x0000002e, 0x00000000, 0x0000002e, 0x80005302, 0x00000000, 0x00000000, 0x00000000, 0x00000000}
[ 38.202291] nvidia-nvswitch3: SXid (PCI:0000:04:00.0): 22013, Non-fatal, Link 47 Minion Link DLREQ interrupt
[ 38.204329] nvidia-nvswitch3: SXid (PCI:0000:04:00.0): 22013, Severity 0 Engine instance 47 Sub-engine instance 00
[ 38.206415] nvidia-nvswitch3: SXid (PCI:0000:04:00.0): 22013, Data {0x0000002f, 0x0000002f, 0x00000000, 0x0000002f, 0x80005302, 0x00000000, 0x00000000, 0x00000000, 0x00000000}
[ 38.209606] nvidia-nvswitch3: SXid (PCI:0000:04:00.0): 22013, Non-fatal, Link 48 Minion Link DLREQ interrupt
[ 38.211633] nvidia-nvswitch3: SXid (PCI:0000:04:00.0): 22013, Severity 0 Engine instance 48 Sub-engine instance 00
[ 38.213740] nvidia-nvswitch3: SXid (PCI:0000:04:00.0): 22013, Data {0x00000030, 0x00000030, 0x00000000, 0x00000030, 0x80005302, 0x00000000, 0x00000000, 0x00000000, 0x00000000}
[ 38.216877] nvidia-nvswitch3: SXid (PCI:0000:04:00.0): 22013, Non-fatal, Link 49 Minion Link DLREQ interrupt
[ 38.218883] nvidia-nvswitch3: SXid (PCI:0000:04:00.0): 22013, Severity 0 Engine instance 49 Sub-engine instance 00
[ 38.220944] nvidia-nvswitch3: SXid (PCI:0000:04:00.0): 22013, Data {0x00000031, 0x00000031, 0x00000000, 0x00000031, 0x80005302, 0x00000000, 0x00000000, 0x00000000, 0x00000000}
[ 38.224182] nvidia-nvswitch3: SXid (PCI:0000:04:00.0): 22013, Non-fatal, Link 50 Minion Link DLREQ interrupt
[ 38.226224] nvidia-nvswitch3: SXid (PCI:0000:04:00.0): 22013, Severity 0 Engine instance 50 Sub-engine instance 00
[ 38.228284] nvidia-nvswitch3: SXid (PCI:0000:04:00.0): 22013, Data {0x00000032, 0x00000032, 0x00000000, 0x00000032, 0x80005302, 0x00000000, 0x00000000, 0x00000000, 0x00000000}
[ 38.231436] nvidia-nvswitch3: SXid (PCI:0000:04:00.0): 22013, Non-fatal, Link 51 Minion Link DLREQ interrupt
[ 38.233398] nvidia-nvswitch3: SXid (PCI:0000:04:00.0): 22013, Severity 0 Engine instance 51 Sub-engine instance 00
[ 38.235455] nvidia-nvswitch3: SXid (PCI:0000:04:00.0): 22013, Data {0x00000033, 0x00000033, 0x00000000, 0x00000033, 0x80005302, 0x00000000, 0x00000000, 0x00000000, 0x00000000}
[ 38.238560] nvidia-nvswitch3: SXid (PCI:0000:04:00.0): 22013, Non-fatal, Link 52 Minion Link DLREQ interrupt
[ 38.240550] nvidia-nvswitch3: SXid (PCI:0000:04:00.0): 22013, Severity 0 Engine instance 52 Sub-engine instance 00
[ 38.242613] nvidia-nvswitch3: SXid (PCI:0000:04:00.0): 22013, Data {0x00000034, 0x00000034, 0x00000000, 0x00000034, 0x80005302, 0x00000000, 0x00000000, 0x00000000, 0x00000000}
[ 38.245689] nvidia-nvswitch3: SXid (PCI:0000:04:00.0): 22013, Non-fatal, Link 53 Minion Link DLREQ interrupt
[ 38.247670] nvidia-nvswitch3: SXid (PCI:0000:04:00.0): 22013, Severity 0 Engine instance 53 Sub-engine instance 00
[ 38.247715] nvidia-modeset: Loading NVIDIA UNIX Open Kernel Mode Setting Driver for x86_64 570.195.03 Release Build (dvs-builder@U22-I3-H04-03-1) Sat Sep 20 00:39:49 UTC 2025
[ 38.249713] nvidia-nvswitch3: SXid (PCI:0000:04:00.0): 22013, Data {0x00000035, 0x00000035, 0x00000000, 0x00000035, 0x80005302, 0x00000000, 0x00000000, 0x00000000, 0x00000000}
[ 38.252818] nvidia-nvswitch3: SXid (PCI:0000:04:00.0): 22013, Non-fatal, Link 54 Minion Link DLREQ interrupt
[ 38.254793] nvidia-nvswitch3: SXid (PCI:0000:04:00.0): 22013, Severity 0 Engine instance 54 Sub-engine instance 00
[ 38.256810] nvidia-nvswitch3: SXid (PCI:0000:04:00.0): 22013, Data {0x00000036, 0x00000036, 0x00000000, 0x00000036, 0x80005302, 0x00000000, 0x00000000, 0x00000000, 0x00000000}
[ 38.259920] nvidia-nvswitch3: SXid (PCI:0000:04:00.0): 22013, Non-fatal, Link 55 Minion Link DLREQ interrupt
[ 38.261884] nvidia-nvswitch3: SXid (PCI:0000:04:00.0): 22013, Severity 0 Engine instance 55 Sub-engine instance 00
[ 38.263912] nvidia-nvswitch3: SXid (PCI:0000:04:00.0): 22013, Data {0x00000037, 0x00000037, 0x00000000, 0x00000037, 0x80005302, 0x00000000, 0x00000000, 0x00000000, 0x00000000}
[ 38.267035] nvidia-nvswitch3: SXid (PCI:0000:04:00.0): 22013, Non-fatal, Link 56 Minion Link DLREQ interrupt
[ 38.268110] NVRM: confComputeConstructEngine_IMPL: GPU confidential compute capability is not enabled.
[ 38.268947] NVRM: nvAssertFailedNoLog: Assertion failed: 0 @ conf_compute.c:118
[ 38.268964] NVRM: osInitNvMapping: *** Cannot attach gpu
[ 38.268966] NVRM: RmInitAdapter: osInitNvMapping failed, bailing out of RmInitAdapter
[ 38.268983] NVRM: GPU 0000:05:00.0: RmInitAdapter failed! (0x22:0x3f:744)
[ 38.269013] nvidia-nvswitch3: SXid (PCI:0000:04:00.0): 22013, Severity 0 Engine instance 56 Sub-engine instance 00
[ 38.269689] NVRM: GPU 0000:05:00.0: rm_init_adapter failed, device minor number 0
[ 38.271047] nvidia-nvswitch3: SXid (PCI:0000:04:00.0): 22013, Data {0x00000038, 0x00000038, 0x00000000, 0x00000038, 0x80005302, 0x00000000, 0x00000000, 0x00000000, 0x00000000}
[ 38.279408] nvidia-nvswitch3: SXid (PCI:0000:04:00.0): 22013, Non-fatal, Link 57 Minion Link DLREQ interrupt
[ 38.281613] nvidia-nvswitch3: SXid (PCI:0000:04:00.0): 22013, Severity 0 Engine instance 57 Sub-engine instance 00
[ 38.283653] nvidia-nvswitch3: SXid (PCI:0000:04:00.0): 22013, Data {0x00000039, 0x00000039, 0x00000000, 0x00000039, 0x80005302, 0x00000000, 0x00000000, 0x00000000, 0x00000000}
[ 38.286738] nvidia-nvswitch3: SXid (PCI:0000:04:00.0): 22013, Non-fatal, Link 58 Minion Link DLREQ interrupt
[ 38.288716] nvidia-nvswitch3: SXid (PCI:0000:04:00.0): 22013, Severity 0 Engine instance 58 Sub-engine instance 00
[ 38.288927] [drm] [nvidia-drm] [GPU ID 0x00000500] Loading driver
[ 38.290771] nvidia-nvswitch3: SXid (PCI:0000:04:00.0): 22013, Data {0x0000003a, 0x0000003a, 0x00000000, 0x0000003a, 0x80005302, 0x00000000, 0x00000000, 0x00000000, 0x00000000}
[ 38.293851] nvidia-nvswitch3: SXid (PCI:0000:04:00.0): 22013, Non-fatal, Link 59 Minion Link DLREQ interrupt
[ 38.295819] nvidia-nvswitch3: SXid (PCI:0000:04:00.0): 22013, Severity 0 Engine instance 59 Sub-engine instance 00
[ 38.297872] nvidia-nvswitch3: SXid (PCI:0000:04:00.0): 22013, Data {0x0000003b, 0x0000003b, 0x00000000, 0x0000003b, 0x80005302, 0x00000000, 0x00000000, 0x00000000, 0x00000000}
[ 38.301002] nvidia-nvswitch3: SXid (PCI:0000:04:00.0): 22013, Non-fatal, Link 60 Minion Link DLREQ interrupt
[ 38.303008] nvidia-nvswitch3: SXid (PCI:0000:04:00.0): 22013, Severity 0 Engine instance 60 Sub-engine instance 00
[ 38.305037] nvidia-nvswitch3: SXid (PCI:0000:04:00.0): 22013, Data {0x0000003c, 0x0000003c, 0x00000000, 0x0000003c, 0x80005302, 0x00000000, 0x00000000, 0x00000000, 0x00000000}
[ 38.308143] nvidia-nvswitch3: SXid (PCI:0000:04:00.0): 22013, Non-fatal, Link 61 Minion Link DLREQ interrupt
[ 38.310127] nvidia-nvswitch3: SXid (PCI:0000:04:00.0): 22013, Severity 0 Engine instance 61 Sub-engine instance 00
[ 38.312135] nvidia-nvswitch3: SXid (PCI:0000:04:00.0): 22013, Data {0x0000003d, 0x0000003d, 0x00000000, 0x0000003d, 0x80005302, 0x00000000, 0x00000000, 0x00000000, 0x00000000}
[ 38.315223] nvidia-nvswitch3: SXid (PCI:0000:04:00.0): 22013, Non-fatal, Link 62 Minion Link DLREQ interrupt
[ 38.317230] nvidia-nvswitch3: SXid (PCI:0000:04:00.0): 22013, Severity 0 Engine instance 62 Sub-engine instance 00
[ 38.319260] nvidia-nvswitch3: SXid (PCI:0000:04:00.0): 22013, Data {0x0000003e, 0x0000003e, 0x00000000, 0x0000003e, 0x80005302, 0x00000000, 0x00000000, 0x00000000, 0x00000000}
[ 38.322340] nvidia-nvswitch3: SXid (PCI:0000:04:00.0): 22013, Non-fatal, Link 63 Minion Link DLREQ interrupt
[ 38.324321] nvidia-nvswitch3: SXid (PCI:0000:04:00.0): 22013, Severity 0 Engine instance 63 Sub-engine instance 00
[ 38.326340] nvidia-nvswitch3: SXid (PCI:0000:04:00.0): 22013, Data {0x0000003f, 0x0000003f, 0x00000000, 0x0000003f, 0x80005302, 0x00000000, 0x00000000, 0x00000000, 0x00000000}
[ 38.329647] nvidia-nvswitch0: open (major=238)
[ 38.329901] nvidia-nvswitch1: open (major=238)
[ 38.330143] nvidia-nvswitch2: open (major=238)
[ 38.330347] nvidia-nvswitch3: open (major=238)
[ 38.331219] nvidia-nvlink: nvlink driver close
[ 38.331221] nvidia-nvlink: nvlink driver open
[ 38.331375] nvidia-nvswitch0: open (major=238)
[ 38.331380] nvidia-nvswitch0: open (major=238)
[ 38.331382] nvidia-nvswitch0: open (major=238)
[ 38.331384] nvidia-nvswitch0: open (major=238)
[ 38.331387] nvidia-nvswitch0: open (major=238)
[ 38.331389] nvidia-nvswitch1: open (major=238)
[ 38.331392] nvidia-nvswitch1: open (major=238)
[ 38.331394] nvidia-nvswitch1: open (major=238)
[ 38.331395] nvidia-nvswitch1: open (major=238)
[ 38.331397] nvidia-nvswitch1: open (major=238)
[ 38.331400] nvidia-nvswitch2: open (major=238)
[ 38.331402] nvidia-nvswitch2: open (major=238)
[ 38.331404] nvidia-nvswitch2: open (major=238)
[ 38.331406] nvidia-nvswitch2: open (major=238)
[ 38.331408] nvidia-nvswitch2: open (major=238)
[ 38.331411] nvidia-nvswitch3: open (major=238)
[ 38.331413] nvidia-nvswitch3: open (major=238)
[ 38.331415] nvidia-nvswitch3: open (major=238)
[ 38.331417] nvidia-nvswitch3: open (major=238)
[ 38.331419] nvidia-nvswitch3: open (major=238)
[ 38.332880] NVRM: confComputeConstructEngine_IMPL: GPU confidential compute capability is not enabled.
[ 38.332884] NVRM: nvAssertFailedNoLog: Assertion failed: 0 @ conf_compute.c:118
[ 38.332901] NVRM: osInitNvMapping: *** Cannot attach gpu
[ 38.332903] NVRM: RmInitAdapter: osInitNvMapping failed, bailing out of RmInitAdapter
[ 38.332919] NVRM: GPU 0000:05:00.0: RmInitAdapter failed! (0x22:0x3f:744)
[ 38.333634] NVRM: GPU 0000:05:00.0: rm_init_adapter failed, device minor number 0
[ 38.396639] NVRM: confComputeConstructEngine_IMPL: GPU confidential compute capability is not enabled.
[ 38.396643] NVRM: nvAssertFailedNoLog: Assertion failed: 0 @ conf_compute.c:118
[ 38.396658] NVRM: osInitNvMapping: *** Cannot attach gpu
[ 38.396659] NVRM: RmInitAdapter: osInitNvMapping failed, bailing out of RmInitAdapter
[ 38.396669] NVRM: GPU 0000:05:00.0: RmInitAdapter failed! (0x22:0x3f:744)
[ 38.397399] NVRM: GPU 0000:05:00.0: rm_init_adapter failed, device minor number 0
[ 38.400699] [drm:nv_drm_load [nvidia_drm]] *ERROR* [nvidia-drm] [GPU ID 0x00000500] Failed to allocate NvKmsKapiDevice
[ 38.402997] [drm:nv_drm_register_drm_device [nvidia_drm]] *ERROR* [nvidia-drm] [GPU ID 0x00000500] Failed to register device
[ 38.405408] [drm] [nvidia-drm] [GPU ID 0x00000600] Loading driver
[ 38.450276] NVRM: confComputeConstructEngine_IMPL: GPU confidential compute capability is not enabled.
[ 38.450281] NVRM: nvAssertFailedNoLog: Assertion failed: 0 @ conf_compute.c:118
[ 38.450295] NVRM: osInitNvMapping: *** Cannot attach gpu
[ 38.450297] NVRM: RmInitAdapter: osInitNvMapping failed, bailing out of RmInitAdapter
[ 38.450307] NVRM: GPU 0000:06:00.0: RmInitAdapter failed! (0x22:0x3f:744)
[ 38.450948] NVRM: GPU 0000:06:00.0: rm_init_adapter failed, device minor number 1
[ 38.513222] NVRM: confComputeConstructEngine_IMPL: GPU confidential compute capability is not enabled.
[ 38.513230] NVRM: nvAssertFailedNoLog: Assertion failed: 0 @ conf_compute.c:118
[ 38.513251] NVRM: osInitNvMapping: *** Cannot attach gpu
[ 38.513252] NVRM: RmInitAdapter: osInitNvMapping failed, bailing out of RmInitAdapter
[ 38.513272] NVRM: GPU 0000:06:00.0: RmInitAdapter failed! (0x22:0x3f:744)
[ 38.513918] NVRM: GPU 0000:06:00.0: rm_init_adapter failed, device minor number 1
[ 38.517826] [drm:nv_drm_load [nvidia_drm]] *ERROR* [nvidia-drm] [GPU ID 0x00000600] Failed to allocate NvKmsKapiDevice
[ 38.521766] [drm:nv_drm_register_drm_device [nvidia_drm]] *ERROR* [nvidia-drm] [GPU ID 0x00000600] Failed to register device
[ 38.524270] [drm] [nvidia-drm] [GPU ID 0x00000700] Loading driver
[ 38.576767] NVRM: confComputeConstructEngine_IMPL: GPU confidential compute capability is not enabled.
[ 38.576771] NVRM: nvAssertFailedNoLog: Assertion failed: 0 @ conf_compute.c:118
[ 38.576836] NVRM: osInitNvMapping: *** Cannot attach gpu
[ 38.576838] NVRM: RmInitAdapter: osInitNvMapping failed, bailing out of RmInitAdapter
[ 38.576847] NVRM: GPU 0000:06:00.0: RmInitAdapter failed! (0x22:0x3f:744)
[ 38.577566] NVRM: GPU 0000:06:00.0: rm_init_adapter failed, device minor number 1
[ 38.640593] NVRM: confComputeConstructEngine_IMPL: GPU confidential compute capability is not enabled.
[ 38.640597] NVRM: nvAssertFailedNoLog: Assertion failed: 0 @ conf_compute.c:118
[ 38.640610] NVRM: osInitNvMapping: *** Cannot attach gpu
[ 38.640612] NVRM: RmInitAdapter: osInitNvMapping failed, bailing out of RmInitAdapter
[ 38.640621] NVRM: GPU 0000:07:00.0: RmInitAdapter failed! (0x22:0x3f:744)
[ 38.641364] NVRM: GPU 0000:07:00.0: rm_init_adapter failed, device minor number 2
For reference, I’ve already verified on the host that both nvgpu and nvswitch are configured in PPCIe mode, and the setup also works correctly when CC mode is enabled.
VBIOS
GPU 96.00.D0.00.02
NVSwitch 96.10.6D.00.01
QEMU commandline
qemu-system-x86_64 -accel kvm -m 100G -smp 32 -name td,process=td,debug-threads=on -cpu host,-avx10 -object {"qom-type":"tdx-guest","id":"tdx","quote-generation-socket":{"type": "vsock", "cid":"2","port":"4050"}} -object memory-backend-ram,id=mem0,size=100G -machine q35,kernel_irqchip=split,confidential-guest-support=tdx,memory-backend=mem0 -bios /usr/share/ovmf/OVMF.fd -nographic -nodefaults -vga none -device virtio-net-pci,netdev=nic0_td -netdev user,id=nic0_td,hostfwd=tcp::10022-:22 -drive file=/home/joongyu.sun/tdx/guest-tools/image/tdx-guest-ubuntu-24.04-generic.qcow2,if=none,id=virtio-disk0 -device virtio-blk-pci,drive=virtio-disk0 -pidfile /tmp/tdx-demo-td-pid.pid -daemonize -device vhost-vsock-pci,guest-cid=3 -object iommufd,id=iommufd0 -device pcie-root-port,port=16,chassis=1,id=pci.1,bus=pcie.0,multifunction=on,addr=0x5 -device vfio-pci,host=19:00.0,bus=pci.1,addr=0x0,iommufd=iommufd0 -fw_cfg name=opt/ovmf/X-PciMmio64Mb1,string=262144 -device pcie-root-port,port=17,chassis=2,id=pci.2,bus=pcie.0,addr=0x5.0x1 -device vfio-pci,host=3b:00.0,bus=pci.2,addr=0x0,iommufd=iommufd0 -fw_cfg name=opt/ovmf/X-PciMmio64Mb2,string=262144 -device pcie-root-port,port=18,chassis=3,id=pci.3,bus=pcie.0,addr=0x5.0x2 -device vfio-pci,host=4c:00.0,bus=pci.3,addr=0x0,iommufd=iommufd0 -fw_cfg name=opt/ovmf/X-PciMmio64Mb3,string=262144 -device pcie-root-port,port=19,chassis=4,id=pci.4,bus=pcie.0,addr=0x5.0x3 -device vfio-pci,host=5d:00.0,bus=pci.4,addr=0x0,iommufd=iommufd0 -fw_cfg name=opt/ovmf/X-PciMmio64Mb4,string=262144 -device pcie-root-port,port=20,chassis=5,id=pci.5,bus=pcie.0,addr=0x5.0x4 -device vfio-pci,host=9b:00.0,bus=pci.5,addr=0x0,iommufd=iommufd0 -fw_cfg name=opt/ovmf/X-PciMmio64Mb5,string=262144 -device pcie-root-port,port=21,chassis=6,id=pci.6,bus=pcie.0,addr=0x5.0x5 -device vfio-pci,host=bb:00.0,bus=pci.6,addr=0x0,iommufd=iommufd0 -fw_cfg name=opt/ovmf/X-PciMmio64Mb6,string=262144 -device pcie-root-port,port=22,chassis=7,id=pci.7,bus=pcie.0,addr=0x5.0x6 -device vfio-pci,host=cb:00.0,bus=pci.7,addr=0x0,iommufd=iommufd0 -fw_cfg name=opt/ovmf/X-PciMmio64Mb7,string=262144 -device pcie-root-port,port=23,chassis=8,id=pci.8,bus=pcie.0,addr=0x5.0x7 -device vfio-pci,host=db:00.0,bus=pci.8,addr=0x0,iommufd=iommufd0 -fw_cfg name=opt/ovmf/X-PciMmio64Mb8,string=262144 -device pcie-root-port,port=24,chassis=9,id=pci.9,bus=pcie.0,multifunction=on,addr=0x4 -device vfio-pci,host=0000:83:00.0,bus=pci.9,addr=0x0,iommufd=iommufd0 -device pcie-root-port,port=25,chassis=10,id=pci.10,bus=pcie.0,addr=0x4.0x1 -device vfio-pci,host=0000:84:00.0,bus=pci.10,addr=0x0,iommufd=iommufd0 -device pcie-root-port,port=26,chassis=11,id=pci.11,bus=pcie.0,addr=0x4.0x2 -device vfio-pci,host=0000:85:00.0,bus=pci.11,addr=0x0,iommufd=iommufd0 -device pcie-root-port,port=27,chassis=12,id=pci.12,bus=pcie.0,addr=0x4.0x3 -device vfio-pci,host=0000:86:00.0,bus=pci.12,addr=0x0,iommufd=iommufd0
root@tdx-guest:~# lspci -nn | grep -i nvidia
01:00.0 Bridge [0680]: NVIDIA Corporation GH100 [H100 NVSwitch] [10de:22a3] (rev a1)
02:00.0 Bridge [0680]: NVIDIA Corporation GH100 [H100 NVSwitch] [10de:22a3] (rev a1)
03:00.0 Bridge [0680]: NVIDIA Corporation GH100 [H100 NVSwitch] [10de:22a3] (rev a1)
04:00.0 Bridge [0680]: NVIDIA Corporation GH100 [H100 NVSwitch] [10de:22a3] (rev a1)
05:00.0 3D controller [0302]: NVIDIA Corporation GH100 [H100 SXM5 80GB] [10de:2330] (rev a1)
06:00.0 3D controller [0302]: NVIDIA Corporation GH100 [H100 SXM5 80GB] [10de:2330] (rev a1)
07:00.0 3D controller [0302]: NVIDIA Corporation GH100 [H100 SXM5 80GB] [10de:2330] (rev a1)
08:00.0 3D controller [0302]: NVIDIA Corporation GH100 [H100 SXM5 80GB] [10de:2330] (rev a1)
09:00.0 3D controller [0302]: NVIDIA Corporation GH100 [H100 SXM5 80GB] [10de:2330] (rev a1)
0a:00.0 3D controller [0302]: NVIDIA Corporation GH100 [H100 SXM5 80GB] [10de:2330] (rev a1)
0b:00.0 3D controller [0302]: NVIDIA Corporation GH100 [H100 SXM5 80GB] [10de:2330] (rev a1)
0c:00.0 3D controller [0302]: NVIDIA Corporation GH100 [H100 SXM5 80GB] [10de:2330] (rev a1)
root@tdx-guest:~# nvidia-smi
No devices were found
Is there anything I might be missing?
System report
The below information captured from host machine
Operating system details
Distributor ID: Ubuntu
Description: Ubuntu 25.04
Release: 25.04
Codename: plucky
Kernel version
6.14.0-1005-intel #5-Ubuntu SMP PREEMPT_DYNAMIC Wed Jul 2 17:00:29 UTC 2025 x86_64 x86_64 GNU/Linux
TDX kernel logs
[ 5.157012] virt/tdx: BIOS enabled: private KeyID range [32, 64)
[ 5.157013] virt/tdx: Disable ACPI S3. Turn off TDX in the BIOS to use ACPI S3.
[ 28.064263] virt/tdx: TDX module 1.5.16.00, build number 869, build date 0134fe6b
[ 32.120910] virt/tdx: 8388612 KB allocated for PAMT
[ 32.120915] virt/tdx: module initialized
...
[ 5.157012] virt/tdx: BIOS enabled: private KeyID range [32, 64)
[ 5.157013] virt/tdx: Disable ACPI S3. Turn off TDX in the BIOS to use ACPI S3.
[ 28.064263] virt/tdx: TDX module 1.5.16.00, build number 869, build date 0134fe6b
[ 32.120910] virt/tdx: 8388612 KB allocated for PAMT
[ 32.120915] virt/tdx: module initialized
TDX CPU instruction support
CPU supports TDX according to /proc/cpuinfo
Model specific registers (MSRs)
MK_TME_ENABLED bit: 1 (expected value: 1)
SEAM_RR bit: 1 (expected value: 1)
NUM_TDX_PRIV_KEYS: 20
SGX_AND_MCHECK_STATUS: 0 (expected value: 0)
Production platform: Production (expected value: Production)
CPU details
INTEL(R) XEON(R) PLATINUM 8562Y+