HydraIssues

Mocht je nog tijd hebben nu, zou je eens naar de Mercator kunnen kijken, hij start altijd goed maar na enkele zinnen bevriest het scherm en krijg ik deze melding

Machine: wobbly

Investigation 2026-08-02

Node: wobbly-llama-92 / node-5f8b7b59 / hostname airborne-two, online, hydrabody v2.0.65.
Build: C:/experiences/mercator-talks/MercatorV0.9/Mercator56/Binaries/Win64/ (UE 5.6.1, Shipping).
Launch args: Mercator56 -Language=NL -PushToTalkEnabled=False -SkipStartMenu=True.

What the dialog is

UE CrashReportClient. CrashType = GPUCrash, ErrorMessage = "GPU Crash dump Triggered", RHI = D3D12,
Aftermath dump present (D3D12.*.nv-gpudmp). This is a GPU device removal / engine timeout, not a
Blueprint or content exception.

Crash history (C:\Users\airborne-two-user\AppData\Local\Mercator56\Saved\Crashes)

03-06 08:43  Hang (GameThread waited 120s on RenderThread) + GPUCrash   drv 576.88
04-22 18:08  GPUCrash  after   59s   drv 576.88
04-24 17:41  GPUCrash  after  245s   drv 596.21
05-02 15:56  GPUCrash  after   53s   drv 596.21
05-02 15:59  EXCEPTION_ACCESS_VIOLATION after 28s
05-03 17:27  GPUCrash  after  124s   drv 596.21
07-25 14:14  GPUCrash  after   24s
07-25 14:15  GPUCrash  after   13s
07-25 14:19  GPUCrash  after  163s
08-02 13:47  GPUCrash  after   11s
08-02 13:50  GPUCrash  after   37s
08-02 13:56  GPUCrash  after   24s   <-- the one in the photo (11:56 UTC, ~5 min before this report)

"Starts fine, freezes after a few sentences" matches: the crash lands 11 to 37 seconds after launch.

It is not a Mercator bug

The same machine also has GPU crash dumps for the Rupelmonde experience (04-24, 05-02, 05-10) and
6 Sunshine.exe APPCRASH reports. Two unrelated UE apps failing the same way points at the machine.

The machine itself is unstable

System event log and WER on airborne-two:

• 39 x LiveKernelEvent 0x141 (VIDEO_ENGINE_TIMEOUT_DETECTED), plus a WATCHDOG live kernel dump at 08-02 13:50.
• nvlddmkm Event ID 153 ("Error occurred on GPUID: 100", \Device\Video3) at 13:47:35, 13:50:02, 13:50:03
  and 13:56:44, exactly matching each UE GPU crash.
• Kernel bugchecks (BSOD): 07-25 0x00000001, 08-01 0x00000139, 08-02 0x0000000A, 08-02 0x00000139.
  WER also holds 0x9F, 0xD1, 0x13A buckets. Minidumps in C:\Windows\Minidump (5 files).
• Boot history shows 9 restarts on 08-01 and 08-02 alone.

Varied, inconsistent bugcheck codes plus repeated GPU engine timeouts is the signature of failing
hardware or a corrupted display driver stack, not of one bad application.

Environment notes

• GPU: RTX 5070 Ti, driver 596.21 (32.0.15.9621, dated 2026-04-13), installed 2026-04-22. Idle temp 46 C,
  power limit 300 W, nothing anomalous at idle.
• OS: Windows 25H2 build 26200.8875. KB5120102 (07-17) and KB5101650 / KB5100998 (07-19) went in shortly
  before the crash rate jumped on 07-25. Correlation only, not proven.
• Two Virtual Display Driver adapter instances are present (ROOT\DISPLAY\0000 and ROOT\DISPLAY\0001),
  giving 3 active monitors (1 x SAM0B96 physical + 2 x MTT1337 virtual). This is the duplicate VDD condition
  from #428 and should be cleaned up regardless.

Suggested next steps (in order, all need physical access to wobbly)

1. Remove the duplicate VDD adapter instance so only one virtual display remains.
2. Clean-reinstall the NVIDIA driver (DDU, then a known good branch), and consider rolling back to a driver
   older than 596.21.
3. If GPU timeouts persist, stress test (memtest for system RAM, an OCCT/FurMark VRAM pass) and treat the
   5070 Ti as suspect for RMA.
4. Only after the machine survives a stress pass should Mercator V0.9 be retested.

Not fixable remotely. Mercator itself needs no code change based on this evidence.

Update 2026-08-02 16:40 local, after the two new comments

Analysed the added photo (boot recovery screen) and both comments. This changes the reading of the
earlier section, so treat the assessment below as superseding it.

What the second photo shows

Windows Boot Manager recovery, not a BSOD:

Your PC couldn't turn off properly
The application or operating system couldn't be loaded because a required file is missing or contains errors.
File: \windows\system32\winload.efi
Error code: 0xc0000098

0xc0000098 means the BCD entry or a boot-critical file is damaged. At that moment the machine could not
boot Windows at all.

New crash data since the first pass

Two more bugchecks today, both 0x0000003B SYSTEM_SERVICE_EXCEPTION with 0xc0000005 (access violation):

14:18:02  0x3B (c0000005, fffff8009e3196c0, ffffb486c5eee5b0, 0)
14:53:26  0x3B (c0000005, fffff802ab3196c0, fffff58b0f7265b0, 0)

The module bases differ (kernel ASLR randomises them per boot) but the faulting instruction lands on the
same offset ...3196c0 both times. That is a deterministic fault in one specific kernel module, which
argues against random hardware degradation and for a corrupted or buggy driver.

Boot history today, 9 starts: 06:42:21, 13:25:14, 13:48:52, 13:52:16, 14:16:48, 14:18:03, 14:53:26,
15:31:32, 16:25:21.

The update churn is real

Matching "hij is al de hele tijd aan het updaten":

• KB5007651 (Windows Security platform) installed 13:50:55, i.e. between the second and third GPU crash.
• Defender security intelligence updates at 13:35:27 and 15:41:43.

A machine that bugchecks part-way through update servicing is a good way to end up with a damaged boot
loader, which is exactly the 0xc0000098 in the photo.

Still no CPU or memory-controller evidence

Zero WHEA-Logger events, all time. No machine-check exceptions, no corrected PCIe errors. A degrading
CPU or memory controller almost always leaves a WHEA trail well before it starts bugchecking. Nothing here.

Revised assessment

• CPU: ruled out for practical purposes. No WHEA, no evidence.
• GPU hardware: demoted to secondary. Still possible, but the repeatable 0x3B offset and the boot-file
  corruption both point at software state rather than silicon.
• Primary hypothesis: corrupted system and driver state on this Windows install, with a Windows Update
  servicing loop grinding against it. The 39 video engine timeouts may be a symptom of a broken display
  driver stack rather than a dying card.

On "Working as expected again"

This does not close the issue. The box has recovered and re-broken at least four times today, and it is
currently offline from hydracluster (last heartbeat lost around 16:31 local, still offline at 16:40)
even though the screen was reported working at 16:32. hydranode is not heartbeating after the 16:25 boot.

Revised next steps (physical access to wobbly required)

1. Do not treat the current working state as a fix.
2. Repair system state first, in this order, rebooting between: chkdsk /f (the boot corruption),
   sfc /scannow, DISM /Online /Cleanup-Image /RestoreHealth.
3. Let Windows Update drain to a clean state, then pause updates on this body machine. Body machines
   auto-installing updates during show hours is an operational hazard that deserves its own issue.
4. Remove the duplicate VDD adapter instance (ROOT\DISPLAY\0001) so one virtual display remains.
5. DDU clean reinstall of the NVIDIA driver.
6. Only if the GPU engine timeouts survive steps 2 to 5: memtest plus a VRAM/compute stress pass, and then
   treat the 5070 Ti as an RMA candidate.
7. Separately, find out why hydranode is not heartbeating after the last boot.

Location correction

Operator confirms the machine is physically at Rupelmonde. The hydracluster node record still says
venue nerdland, district bxl1-test. Worth correcting separately, since venue feeds body discovery.