1671264312516829184

2026-08-25 01:23:59 -04:00
1 parent 9e994c42e8
commit 6539243163
@@ -0,0 +1,232 @@
+---
+title: "Or for an APU, a way to have the storage device write direct to DRAM in pages setup for use on the GPU, because anything the CPU writes that the GPU reads is stuck on the strangled snooping bus or the strangled WC buffer limit on store."
+type: archive
+source: twitter
+source_url: "https://x.com/NOTimothyLottes/status/1671279693104914434"
+author: "NOTimothyLottes"
+handle: NOTimothyLottes
+post_id: "1671279693104914434"
+date: 2023-06-20
+archived: 2026-08-25
+draft: false
+tags:
+  - archive
+  - twitter
+  - NOTimothyLottes
+description: "Or for an APU, a way to have the storage device write direct to DRAM in pages setup for use on the GPU, because anything the CPU writes that the GPU reads is stuck on the strangled snooping bus or the strangled WC buffer limit on store."
+in_reply_to: ""
+parent_post_id: "1671276990987370501"
+---
+
+## Source
+
+- URL: https://x.com/NOTimothyLottes/status/1671279693104914434
+- Author: NOTimothyLottes (@NOTimothyLottes)
+- Posted: 2023-06-20 22:11:44
+
+## Branch
+
+**1/**
+
+Or for an APU, a way to have the storage device write direct to DRAM in pages setup for use on the GPU, because anything the CPU writes that the GPU reads is stuck on the strangled snooping bus or the strangled WC buffer limit on store.
+
+**2/**
+
+If the app tried to manage GPU memory oversubscription via these strangled CPU mappings you'd literally see many second stalls. One can only assume the kernel can do storage<->DRAM DMA for managing GPU DRAM oversubscription?
+
+**3/**
+
+Even if could fill VRAM in 1 sec, just 2x oversubscription of VRAM implies 1 sec delay to context switch (assuming 2 apps accessing all meM). So classic VM multitasking is useless. Only single app focus model with pinned memory makes any useful sense. Release pin on focus change.
+
+**4/**
+
+The trend is massive res massive VRAM paired with tiny IO bus. So the case for pinned memory and bus master transfers only grows importance as things evolve.
+
+**5/**
+
+Back to SteamDeck numbers: GTT (HOST_VISIBLE) + USWC (non-HOST_CACHED) takes 1 sec for 1st 4 GiB BO alloc, but *30 sec* for 2nd 4 GiB alloc. Kernel driver time for memory allocation (maybe page table related) can be brutal (general comment for PCs too).
+
+**6/**
+
+Related, I think Chips&Cheese CPU/GPU link DMA and Compute Bandwidths are only measured to a GTT+USWC buffer (only supporting useless uncached CPU reads). Bandwidth exceeds the CPU's bus capacity, implies it is 'garlic' or GPU bus only accesses, direct to DRAM ...
+
+![](https://pbs.twimg.com/media/FzHOnZ_X0AEW7CE?format=png&name=orig)
+
+**7/**
+
+Since RADV+AMDVLK don't support user-space CPU mapped flush | invalidate, this implies the only supported mapping to read from DRAM direct is USWC (uncached R and write+combine W) ...
+
+**8/**
+
+Doing GPU stores to a CPU mapped GTT without USWC would be crippingly slow (limited by the snooping bus rate). But this is unfortunately the only option available for CPU read back. So if doing a shader store, it better be only a few waves and running in parallel.
+
+**9/**
+
+"Use GTT because it's as fast as VRAM on the Deck", could only work if GTT+USWC, as that would be only way to get Garlic (direct to DRAM high bandwidth bus). GTT without USWC would need Onion (slow snooping bus) because AMDgpu's only no-CPU map option is for VRAM!
+
+![](https://pbs.twimg.com/media/FzHTDxCX0AADoM1?format=png&name=orig)
+
+**10/**
+
+Summary of the theory on best Steam Deck practices. This is the plan for my deck-compute-only driver too. Theory -> as in I haven't yet verified the GPU-side parts (my driver isn't that far along yet).
+
+![](https://pbs.twimg.com/media/FzHZ_ZiWwAAdEkd?format=png&name=orig)
+
+**11/**
+
+Possible to do better? MAYBE! CPU readback actually has 2 problems, 1st the slow GPU-side copy (4 GiB copy via snooping bus could be almost 4 sec, but direct to DRAM via USWC might be just 160 ms). Also CPU only has 4 MiB of L3, so the majority of 4 GiB will be uncached later ...
+
+**12/**
+
+Believe UC MTYPE (uncached) forces the CPU into serialized behavior. My test was single thread 8-byte/access reads. https://uops.info/html-instr/VMOVDQA_M256_YMM.html#ZEN2 Looks like Zen2 might be able to get 32-byte/access via VMOVDQA, and going multi-threaded (8 thread), that might be a 32x speed up ..
+
+**13/**
+
+If so might be able to approach under a GB/s for the CPU-side part (UC multithread via VMOVDQA), which would be close enough to the non-USWC running single threaded using the cache.
+
+**14/**
+
+What you'd really want here as a band-aid workaround is ability for the GPU to act as if the CPU map was USWC (so go direct to DRAM), but have the CPU map act as non-USWC, so it goes through the cache. Then some kind of hack to flush the tiny 4 MiB L3 and lower caches on the CPU.
+
+**15/**
+
+CPU readback (reading and summing 8-byte) GTT without USWC:
+
+1.16 GiB/s  1 thread
+1.42 GiB/s  4 threads
+1.48 GiB/s  8 threads
+
+Going multi-core on cached readback doesn't really help much. Proper test, parked threads waiting on futex, signal, last-1st active core timing.
+
+**16/**
+
+Now CPU readback GTT+USWC
+
+0.13 GiB/s 4 threads 8-byte reads
+0.24 GiB/s 8 threads 8-byte reads
+1.15 GiB/s 8 threads 32-byte MOVNTDQA
+
+So measurements match theory, going multi-core with MOVNTDQA uncached read on GTT+USWC can be made to match 1 thread GTT (without USWC)
+
+**17/**
+
+Both those results above had been using a pair of 4 GiB allocations. The GTT+USWC one used one 4 GiB GTT+USWC for timing, and one 4 GiB GTT (unused). And that test suffered from a 30 sec GTT BO allocation. So something was going very wrong in the page mapping ...
+
+**18/**
+
+When I rerun same test with just one 4 GiB GTT+USWC allocation, the 30 sec stall is gone, and the performance also changes:
+
+18.34 GiB/s - 8 threads x 32-byte MOVNTDQA (UC)
+
+Oh!
+
+Perhaps there is some resource limit that kills perf if too much memory gets mapped, page faults?
+
+**19/**
+
+Top with thread cumulative results doesn't show anything significantly different between the 4 GiB and 8 GiB runs in terms of page faults ... suggests it must be something else
+
+![](https://pbs.twimg.com/media/FzJ9YrHXsAIfCOg?format=jpg&name=orig)
+![](https://pbs.twimg.com/media/FzJ9Y5LWwAYyyjt?format=jpg&name=orig)
+
+**20/**
+
+And yet there is obviously a bug in my multi-core tests, you can tell directly from the page fault numbers, only one thread is taking all the faults. So it is back to finding my coding error (fail).
+
+**21/**
+
+Lunch break and fixed the bug. Two runs now and leaving threads open to get TOP results. First run definitely soaks up the page faults, second run is page fault free (expected). Both around only 7 GB/s.
+
+![](https://pbs.twimg.com/media/FzKPZxUWABQ6oQJ?format=jpg&name=orig)
+
+**22/**
+
+And the 8 GiB of BO mapped, but only 4 GiB used run. The first pass gets only 1 GiB/s and the second gets 7 GiB/s. Page fault number is similar to last run, can only conclude page fault costs exploded?
+
+![](https://pbs.twimg.com/media/FzKRF1OXsAE7c0-?format=jpg&name=orig)
+
+**23/**
+
+30 sec BO alloc time + super low bandwidth on 1st pass only (where page faults happen) suggest that Linux Kernel logic explodes in cost if too many pages are used in this way ...
+
+**24/**
+
+~8K faults for 512 MiB accessed / thread = 64 KiB/fault ... X86-64 has either 4 KiB or 2 MiB for page size. So not using large pages (fail). Probably mapping 16 pages per fault. Not sure if this implies anything about GPU page size (but certainly hoping it isn't 4 KiB, ouch).
+
+**25/**
+
+Some other very rough measured numbers of GTT+USWC with 8 cores splitting 4 GiB of BO.
+
+~7 GiB/s R
+~10 GiB/s W then R
+~15 GiB/s W
+
+I think these seem plausable now (so maybe no more code bugs).
+
+**26/**
+
+One takeaways of all this, is that you need to pre-warm the page tables for large mapped buffers (by touching all pages) when the user isn't waiting on results. And if you are doing batch jobs that {open device, send data to GPU, get data back, close device} you are screwed!
+
+**27/**
+
+And lastly (maybe) comparison of GTT and GTT+USWC both 8 threads splitting streaming through 4 GiB of mapping (second pass, no page fault issues):
+
+R> ~14 GiB/s (GTT) and ~7 GiB/s (GTT+USWC)
+W+R> ~14 GiB/s (GTT) and ~10 GiB/s (GTT+USWC)
+W> ~13 GiB/s (GTT) and ~15 GiB/s (GTT+USWC)
+
+**28/**
+
+So an alternative option summary for those who don't want the extra GPU-side GPU to CPU mapped buffer copy step.
+
+![](https://pbs.twimg.com/media/FzKlE0NXoAI74IA?format=png&name=orig)
+
+**29/**
+
+If anyone is looking to repro the 30 sec stall: amdgpu_bo_alloc() one 4 GiB GTT+USWC, then one 4 GiB GTT buffer, the second alloc causes the Deck to become unresponsive for 30 seconds
+
+**30/**
+
+madvise() with MADV_HUGEPAGE on mapped 4 GiB region doesn't do anything (still faults at 64 KiB granularity), and none of these MADV_{WILLNEED|POPULATE_READ|POPULATE_WRITE} have any effect either (still waits until use before faulting, causing low initial effective bandwidth)
+
+**31/**
+
+64KiB strided write through 4 GiB GTT+USWC (to pre-fault) costs the same as writing full 4 GiB, roughly 3 seconds. So it is quite literally massive page fault overhead for 1st access. No possible workaround found at this time for initial load time problems.
+
+**32/**
+
+If doing two 4 GiB GTT+USWC allocations, there is also a 30 second BO allocate cost on the 2nd one. And this makes the initial page fault cost for access to the first 4 GiB take another 30 seconds. Effectively hangs the machine for a full minute.
+
+**33/**
+
+Doing two 4 GiB GTT allocations (without USWC), doesn't incure any of the 30 second stalls. So that problem is specific to big USWC allocations. However the initial access page fault problem (extra 3 seconds) is there, so a 2ndary problem with just mapping lots of APU memory.
+
+**34/**
+
+Allocation of one 4 GiB GTT first then a 4 GiB GTT+USWC doesn't see the 30 second allocation stall. Almost like anything post a big USWC alloc is poisioned. And after mapping both, then accessing the GTT only, the 30 second time for page faulting comes back.
+
+**35/**
+
+Continuing from before, even if you don't map the 2nd GTT+USWC, the 30 second initial page faulting time is still there, so the act of mapping doesn't matter, simply doing the BO allocation had already doomed the Linux page management.
+
+**36/**
+
+Despite header docs which imply flag only works on DOMAIN_VRAM, using DOMAIN_GTT+AMDGPU_GEM_CREATE_NO_CPU_ACCESS is apparently what you want for non-mapped GTT allocations. Mapped 4 GiB GTT with 4 GiB GTT+NO_CPU, drops the initial page fault time from 30 seconds to 3 sec.
+
+![](https://pbs.twimg.com/media/FzOqwUfXsAEsmC9?format=png&name=orig)
+
+**37/**
+
+@NOTimothyLottes the problem there is usually tiling, GPUs expect graphicy things in VRAM to be tiled in very specific ways, it's hard to have a storage device do that. Using a transfer engine that can tile/detile makes a lot more sense.
+
+**38/**
+
+@DaveAirlie The obvious elephant is an engine using only buffers for resource streaming and doing its own GPU-side decompression. This is practical NOW and portable too. But the lack of a bus master no-CPU direct storage transfer is a serious problem for throughput.
+
+**39/**
+
+@DaveAirlie See raw texture streaming (even with lossless compression) is way less interesting now given modern massive data tech direction, u want substantially better lossy compression simply to fit in launch constraints. So really, buffers are the way to go anyway :)
+
+## Related
+
+- Spine: [[archive/threads/NOTimothyLottes/2023-06-20-how-about-a-mega-thread-on-the-mechanics-of-cpu]]
@@ -0,0 +1,35 @@
+---
+title: "@NOTimothyLottes Great thread!"
+type: archive
+source: twitter
+source_url: "https://x.com/SheriefFYI/status/1671279374622924802"
+author: "Sherief, FYI"
+handle: SheriefFYI
+post_id: "1671279374622924802"
+date: 2023-06-20
+archived: 2026-08-25
+draft: false
+tags:
+  - archive
+  - twitter
+  - NOTimothyLottes
+description: "@NOTimothyLottes Great thread!"
+in_reply_to: ""
+parent_post_id: "1671276990987370501"
+---
+
+## Source
+
+- URL: https://x.com/SheriefFYI/status/1671279374622924802
+- Author: Sherief, FYI (@SheriefFYI)
+- Posted: 2023-06-20 22:10:28
+
+## Branch
+
+**1/**
+
+@NOTimothyLottes Great thread!
+
+## Related
+
+- Spine: [[archive/threads/NOTimothyLottes/2023-06-20-how-about-a-mega-thread-on-the-mechanics-of-cpu]]
@@ -0,0 +1,37 @@
+---
+title: "@NOTimothyLottes On Linux, NVIDIA has a very good UVM."
+type: archive
+source: twitter
+source_url: "https://x.com/never_released/status/1671278880219447298"
+author: "Longhorn"
+handle: never_released
+post_id: "1671278880219447298"
+date: 2023-06-20
+archived: 2026-08-25
+draft: false
+tags:
+  - archive
+  - twitter
+  - NOTimothyLottes
+description: "@NOTimothyLottes On Linux, NVIDIA has a very good UVM."
+in_reply_to: ""
+parent_post_id: "1671276990987370501"
+---
+
+## Source
+
+- URL: https://x.com/never_released/status/1671278880219447298
+- Author: Longhorn (@never_released)
+- Posted: 2023-06-20 22:08:30
+
+## Branch
+
+**1/**
+
+@NOTimothyLottes On Linux, NVIDIA has a very good UVM.
+
+However, the infrastructure for doing that on Windows is quite lacking to say the least...
+
+## Related
+
+- Spine: [[archive/threads/NOTimothyLottes/2023-06-20-how-about-a-mega-thread-on-the-mechanics-of-cpu]]
@@ -0,0 +1,80 @@
+---
+title: "How about a mega thread on the mechanics of CPU/GPU communication."
+type: archive
+source: twitter
+source_url: "https://x.com/NOTimothyLottes/status/1671264312516829184"
+author: "NOTimothyLottes"
+handle: NOTimothyLottes
+post_id: "1671264312516829184"
+date: 2023-06-20
+archived: 2026-08-25
+draft: false
+tags:
+  - archive
+  - twitter
+  - NOTimothyLottes
+description: "How about a mega thread on the mechanics of CPU/GPU communication."
+in_reply_to: ""
+---
+
+## Source
+
+- URL: https://x.com/NOTimothyLottes/status/1671264312516829184
+- Author: NOTimothyLottes (@NOTimothyLottes)
+- Posted: 2023-06-20 21:10:37
+
+## Thread
+
+**1/**
+
+How about a mega thread on the mechanics of CPU/GPU communication. Using AMDgpu based timing  results on the SteamDeck an example, but relating to the larger picture of PC GFX APIs like Vulkan/etc. I don't claim to know all, so experts feel free to add your wisdom :)
+
+![](https://pbs.twimg.com/media/FzGF4owWwAMsc2I?format=png&name=orig)
+
+**2/**
+
+Both RADV and AMDVLK: Flush/invalidate mapped memory ranges is a NOP. So bus-crossing dGPU traffic to HOST_VISIBLE is automatically snooping CPU caches. The one without HOST_CACHED, is Write+Combined [WC] on store, and Uncached [UC] on read.  The one with HOST_CACHED is non-WC/UC
+
+![](https://pbs.twimg.com/media/FzGG4OnWwAEu19H?format=png&name=orig)
+
+**3/**
+
+In AMDgpu (the kernel driver), likely DEVICE_LOCAL maps to AMDGPU_GEM_DOMAIN_VRAM (also the carve out on APUs) and the non-DEVICE_LOCAL maps to AMDGPU_GEM_DOMAIN_GTT
+
+![](https://pbs.twimg.com/media/FzGJkkFWwAEBowR?format=png&name=orig)
+
+**4/**
+
+AMD+RADV added {DEVICE_COHERENT_BIT_AMD,
+DEVICE_UNCACHED_BIT_AMD} variations to the core 4 memory types. Likely to support GPU crash debug. But also provides a way to avoid needing to write-back (flush) GPU caches before CPU read. Likely AMDgpu kernel flag mapping below.
+
+![](https://pbs.twimg.com/media/FzGKZ_bWcAE3cGh?format=png&name=orig)
+
+**5/**
+
+This AMDGPU_GEM_CREATE_CPU_GTT_USWC appear to toggle on WriteCombine [WC] for CPU store, and Uncached [US] for CPU reads (cases of HOST_VISIBLE without HOST_CACHED)
+
+![](https://pbs.twimg.com/media/FzGL03LXgAEeO0K?format=png&name=orig)
+
+**6/**
+
+For review from https://chipsandcheese.com/2023/03/05/van-gogh-amds-steam-deck-apu/ Deck bandwidths: ~71 GB/s GPU, ~43 GB/s DMA, ~34 GB/s shader copy CPU<->GPU, ~25 GB/s CPU/CPU, and damn, brutal 0.27 GB/s CPU mapped GPU buffer reads, 0.71 GB/s CPU mapped GPU buffer writes
+
+![](https://pbs.twimg.com/media/FzGNGBuWAAA6wlr?format=png&name=orig)
+![](https://pbs.twimg.com/media/FzGNlFvWwAANhH4?format=png&name=orig)
+
+**7/**
+
+And going direct to AMDgpu instead of VK on the Deck shows these kinds of bandwidths (non-DEVICE_LOCAL,  HOST_VISIBLE with HOST_CACHED and without). So using Write-Combined is amazingly painful for stores.
+
+![](https://pbs.twimg.com/media/FzGO3XdWIAAQl3V?format=png&name=orig)
+
+**8/**
+
+Implies that the choices one might make on dGPU PC don't necessarily port over to APUs at all. Another challenge: it takes almost 7 seconds to zero-fill using a 64-bit store for() loop the 8-GiB of mapped memory. Hints at why load times are such a challenge even in the best case.
+
+**9/**
+
+This all hints at why PC OS derived systems are lacking in stuffing GPU VRAM. Really need some kind of bus mastered DMA (zero-copy) between non-volatile storage (disk) and GPU DRAM to avoid this CPU-touching performance tax.
+
+Branches: [[archive/threads/NOTimothyLottes/2023-06-20-how-about-a-mega-thread-on-the-mechanics-of-cpu/2023-06-20-never_released-on-linux-nvidia-has-a-very-good-uvm-however-the]], [[archive/threads/NOTimothyLottes/2023-06-20-how-about-a-mega-thread-on-the-mechanics-of-cpu/2023-06-20-SheriefFYI-great-thread]], [[archive/threads/NOTimothyLottes/2023-06-20-how-about-a-mega-thread-on-the-mechanics-of-cpu/2023-06-20-NOTimothyLottes-or-for-an-apu-a-way-to-have-the-storage-device]]
@@ -0,0 +1,35 @@
+---
+title: "@NOTimothyLottes It's what debuggers should be."
+type: archive
+source: twitter
+source_url: "https://x.com/kenpex/status/1743086413736976462"
+author: "c0de517e/AngeloPesce"
+handle: kenpex
+post_id: "1743086413736976462"
+date: 2024-01-05
+archived: 2026-08-25
+draft: false
+tags:
+  - archive
+  - twitter
+  - NOTimothyLottes
+description: "@NOTimothyLottes It's what debuggers should be."
+in_reply_to: ""
+parent_post_id: "1742743018929557805"
+---
+
+## Source
+
+- URL: https://x.com/kenpex/status/1743086413736976462
+- Author: c0de517e/AngeloPesce (@kenpex)
+- Posted: 2024-01-05 01:46:00
+
+## Branch
+
+**1/**
+
+@NOTimothyLottes It's what debuggers should be. Debuggers made zero progress in the last decade, but fundamentally they patch running code... just they are limited to patch an interrupt instead of allowing us to do... anything. Even tracepoints, while nice, are a castrated version of livecode.
+
+## Related
+
+- Spine: [[archive/threads/NOTimothyLottes/2024-01-04-debug-for-majority-gpu-side-game-dev-run-time]]
@@ -0,0 +1,38 @@
+---
+title: "Debug for majority-GPU-side game dev? Run-time recompile and hex viewer."
+type: archive
+source: twitter
+source_url: "https://x.com/NOTimothyLottes/status/1742742039169573054"
+author: "NOTimothyLottes"
+handle: NOTimothyLottes
+post_id: "1742742039169573054"
+date: 2024-01-04
+archived: 2026-08-25
+draft: false
+tags:
+  - archive
+  - twitter
+  - NOTimothyLottes
+description: "Debug for majority-GPU-side game dev? Run-time recompile and hex viewer."
+in_reply_to: ""
+---
+
+## Source
+
+- URL: https://x.com/NOTimothyLottes/status/1742742039169573054
+- Author: NOTimothyLottes (@NOTimothyLottes)
+- Posted: 2024-01-04 02:57:35
+
+## Thread
+
+**1/**
+
+Debug for majority-GPU-side game dev? Run-time recompile and hex viewer. If I want to see a value it is 'ramWI1.hxd[pos]=value;' (write to the memory that the hex dump reads from). Everything else is automatic, just save source and the results show up on screen. Zero's not drawn.
+
+![](https://pbs.twimg.com/media/GC91mNxXEAAaovO?format=png&name=orig)
+
+**2/**
+
+IMO this is preferred to a 'debugger'. I can fill values from all workgroups simply by placing them at different positions, etc. Can get timings via shader clock for profiling even. It is transformative in dev productivity.
+
+Branches: [[archive/threads/NOTimothyLottes/2024-01-04-debug-for-majority-gpu-side-game-dev-run-time/2024-01-05-kenpex-its-what-debuggers-should-be-debuggers-made-zero]]
@@ -0,0 +1,35 @@
+---
+title: "@NOTimothyLottes That’s what I do."
+type: archive
+source: twitter
+source_url: "https://x.com/AgileJebrim/status/1976730663018491918"
+author: "Jebrim"
+handle: AgileJebrim
+post_id: "1976730663018491918"
+date: 2025-10-10
+archived: 2026-08-25
+draft: false
+tags:
+  - archive
+  - twitter
+  - NOTimothyLottes
+description: "@NOTimothyLottes That’s what I do."
+in_reply_to: ""
+parent_post_id: "1976637149387587994"
+---
+
+## Source
+
+- URL: https://x.com/AgileJebrim/status/1976730663018491918
+- Author: Jebrim (@AgileJebrim)
+- Posted: 2025-10-10 19:24:50
+
+## Branch
+
+**1/**
+
+@NOTimothyLottes That’s what I do. One pinned thread per core. SCHED_FIFO, highest priority, and reconfigure various settings within the OS to avoid preemptions/interrupts/hyperthreading.
+
+## Related
+
+- Spine: [[archive/threads/NOTimothyLottes/2025-10-10-thread-about-this-from-rfleury-https-www-rfleury]]
@@ -0,0 +1,35 @@
+---
+title: "@NOTimothyLottes Pinning works well only if your program is the only one working."
+type: archive
+source: twitter
+source_url: "https://x.com/lectem/status/1976935182289805554"
+author: "Clément Grégoire"
+handle: lectem
+post_id: "1976935182289805554"
+date: 2025-10-11
+archived: 2026-08-25
+draft: false
+tags:
+  - archive
+  - twitter
+  - NOTimothyLottes
+description: "@NOTimothyLottes Pinning works well only if your program is the only one working."
+in_reply_to: ""
+parent_post_id: "1976637149387587994"
+---
+
+## Source
+
+- URL: https://x.com/lectem/status/1976935182289805554
+- Author: Clément Grégoire (@lectem)
+- Posted: 2025-10-11 08:57:31
+
+## Branch
+
+**1/**
+
+@NOTimothyLottes Pinning works well only if your program is the only one working. Any modern computer most likely have things running in the background. If any thread gets preempted in the middle of an execution block (that happens even with job stealing) you're screwed.
+
+## Related
+
+- Spine: [[archive/threads/NOTimothyLottes/2025-10-10-thread-about-this-from-rfleury-https-www-rfleury]]
@@ -0,0 +1,59 @@
+---
+title: "@NOTimothyLottes Do you have a source from which one could learn how to achieve the equivalent flexibility by following this approach? Sometimes on gaming devices you only have a handful of threads to deal with unlike on PCs."
+type: archive
+source: twitter
+source_url: "https://x.com/techno_bog/status/1977096425059320169"
+author: "TECHNOBOG"
+handle: techno_bog
+post_id: "1977096425059320169"
+date: 2025-10-11
+archived: 2026-08-25
+draft: false
+tags:
+  - archive
+  - twitter
+  - NOTimothyLottes
+description: "@NOTimothyLottes Do you have a source from which one could learn how to achieve the equivalent flexibility by following this approach? Sometimes on gaming devices you only have a handful of threads to deal with unlike on PCs."
+in_reply_to: ""
+parent_post_id: "1976637149387587994"
+---
+
+## Source
+
+- URL: https://x.com/techno_bog/status/1977096425059320169
+- Author: TECHNOBOG (@techno_bog)
+- Posted: 2025-10-11 19:38:15
+
+## Branch
+
+**1/**
+
+@NOTimothyLottes Do you have a source from which one could learn how to achieve the equivalent flexibility by following this approach? Sometimes on gaming devices you only have a handful of threads to deal with unlike on PCs.
+
+**2/**
+
+@techno_bog Oh my conclusion was to abandon using the CPU for anything other than program init-time work and some IO processing, move everything else to the GPU where one has access to a good parallel programming model. Also enable CPU to power down so GPU gets {power,thermal,bandwidth}
+
+**3/**
+
+@NOTimothyLottes That’s not what you say in the post? Regardless, re the GPU, I struggle to understand how I’d do that within reasonable time. Ryan’s approach seems to be ergonomic enough to apply generically.
+
+**4/**
+
+@techno_bog Oh Ryan's approach isn't bad per say, it's way better than what most people do, it just wasn't something that yielded guaranteed quality of service that I was looking for. Nothing on the CPU running Windows can do that..
+
+**5/**
+
+@NOTimothyLottes Then it must be going completely over my head. What are you criticizing exactly? Programming using the CPU in general?  Why pick Ryan's post specifically then?
+
+**6/**
+
+@techno_bog Criticizing the Windows lack of allowing pined exclusive core use for the foreground application. If I play a PC game, as the user, I don't want background garbage to preempt what I'm playing. Ryan's post because it is a smart way to approach the best one can do on Windows today
+
+**7/**
+
+@NOTimothyLottes OK, I definitely had a reading comprehension issue here. Very new to this, thanks for clarifying.
+
+## Related
+
+- Spine: [[archive/threads/NOTimothyLottes/2025-10-10-thread-about-this-from-rfleury-https-www-rfleury]]
@@ -0,0 +1,60 @@
+---
+title: "Thread about this from @rfleury - https://www.rfleury.com/p/multi-core-by-default - because I used to do that too on the CPU ..."
+type: archive
+source: twitter
+source_url: "https://x.com/NOTimothyLottes/status/1976631453749535059"
+author: "NOTimothyLottes"
+handle: NOTimothyLottes
+post_id: "1976631453749535059"
+date: 2025-10-10
+archived: 2026-08-25
+draft: false
+tags:
+  - archive
+  - twitter
+  - NOTimothyLottes
+description: "Thread about this from @rfleury - https://www.rfleury.com/p/multi-core-by-default - because I used to do that too on the CPU ..."
+in_reply_to: ""
+---
+
+## Source
+
+- URL: https://x.com/NOTimothyLottes/status/1976631453749535059
+- Author: NOTimothyLottes (@NOTimothyLottes)
+- Posted: 2025-10-10 12:50:37
+
+## Thread
+
+**1/**
+
+Thread about this from @rfleury - https://www.rfleury.com/p/multi-core-by-default - because I used to do that too on the CPU ...
+
+**2/**
+
+Same basic idea, all CPU threads ran the same program, each thread entry point gets an index of which thread they are. Same style as GPU programming but on the CPU! Also did runtime C code editing by exiting all threads and re-entering via new DLL ...
+
+**3/**
+
+There are some obvious problems though: CPU "random" preemptions -> if any thread grabs work through an atomic or even by just dividing up work according to thread index, if that thread gets preempted, the future sync will stall introducing bubbles ...
+
+**4/**
+
+Some CPU schedulers will preempt long running CPU threads at times of which one has no control. About the only way to try to improve this situation is to have a pair of threads instead of one, where they round robin execution using system context switches ...
+
+**5/**
+
+Meaning giving the OS places that it becomes better to context switch a thread (at the point the thread has no ownership of a work item). This of course introduces overhead and complexity and breaks the manual threadIndex approach to work carving
+
+**6/**
+
+Meaning patching the problem by speculative context switches doesn't guarantee the OS won't preempt during a bad time, it just decreases the probability
+
+**7/**
+
+There are other options though: allow work duplication, meaning make the work such that if 2 or more threads all compute the data, they get the same answer ... then if a thread gets to a dependency, it can start to finish work that gets stalled due to preemption
+
+**8/**
+
+This unfortunately also adds complexity and overhead, but at least one can continue to make forward progress. The RIGHT solution is pinned explicit CPU cores, an in modern times with huge thread count CPUs, there is no real excuse not to do this other than stupidity
+
+Branches: [[archive/threads/NOTimothyLottes/2025-10-10-thread-about-this-from-rfleury-https-www-rfleury/2025-10-10-AgileJebrim-thats-what-i-do-one-pinned-thread-per-core-sched]], [[archive/threads/NOTimothyLottes/2025-10-10-thread-about-this-from-rfleury-https-www-rfleury/2025-10-11-lectem-pinning-works-well-only-if-your-program-is-the]], [[archive/threads/NOTimothyLottes/2025-10-10-thread-about-this-from-rfleury-https-www-rfleury/2025-10-11-techno_bog-do-you-have-a-source-from-which-one-could-learn]]
@@ -1,30 +1,38 @@
 ---
-title: "hits the easy to debug CPU or GPU stuff in the same framework (simply write and it just appears) - it's always a HEX dump, but can show decimal by pretending it's a 4-bit/character terminal with a hex font :)"
+title: "no-debugger-debug / console / printf-style - prior evolved to one mmapped fixed-width-height log file with an atomic for multi-thread|process messages - but even that seems too complexy ..."
 type: archive
 source: twitter
-source_url: "https://x.com/NOTimothyLottes/status/2061124942968545433"
+source_url: "https://x.com/NOTimothyLottes/status/2061123768429211767"
 author: "NOTimothyLottes"
 handle: NOTimothyLottes
-post_id: "2061124942968545433"
+post_id: "2061123768429211767"
 date: 2026-05-31
-archived: 2026-08-23
-draft: true
+archived: 2026-08-25
+draft: false
 tags:
  - archive
  - twitter
  - NOTimothyLottes
-description: "hits the easy to debug CPU or GPU stuff in the same framework (simply write and it just appears) - it's always a HEX dump, but can show decimal by pretending it's a 4-bit/character terminal with a hex font :)"
+description: "no-debugger-debug / console / printf-style - prior evolved to one mmapped fixed-width-height log file with an atomic for multi-thread|process messages - but even that seems too complexy ..."
 in_reply_to: ""
 ---

 ## Source

- URL: https://x.com/NOTimothyLottes/status/2061124942968545433
+- URL: https://x.com/NOTimothyLottes/status/2061123768429211767
 - Author: NOTimothyLottes (@NOTimothyLottes)
- Posted: 2026-05-31 16:37:34
+- Posted: 2026-05-31 16:32:54

 ## Thread

 **1/**

+no-debugger-debug / console / printf-style - prior evolved to one mmapped fixed-width-height log file with an atomic for multi-thread|process messages - but even that seems too complexy ...
+
+**2/**
+
+moving to this >>> CART file is mmapped on CPU with mapped GPU access (no file IO) - background page walker to ensure OS won't page it out - beginning of CART file is a grid of 32-bit unsigned values - hex dumping that as my 'log file' - either to term or GPU render
+
+**3/**
+
 hits the easy to debug CPU or GPU stuff in the same framework (simply write and it just appears) - it's always a HEX dump, but can show decimal by pretending it's a 4-bit/character terminal with a hex font :)
@@ -16,6 +16,7 @@ tags:
 - [[archive/threads/NOTimothyLottes/2022-11-13-my-prefered-method-of-managing-application-data]]
 - [[archive/threads/NOTimothyLottes/2023-04-28-https-gpuopen-com-learn-rgp-1-15-enhanced-isa]]
 - [[archive/threads/NOTimothyLottes/2023-05-03-found-some-of-my-old-blogs-thread-looking-back-an]]
+- [[archive/threads/NOTimothyLottes/2023-06-20-how-about-a-mega-thread-on-the-mechanics-of-cpu]]
 - [[archive/threads/NOTimothyLottes/2023-06-30-nothing-oriented-programming-nop-where-the-goal]]
 - [[archive/threads/NOTimothyLottes/2023-08-03-spc-using-8-deep-swap-all-aliasing-same-bo-image]]
 - [[archive/threads/NOTimothyLottes/2023-08-04-for-anyone-tuning-in-now-spc-is-kind-of-like-a]]
@@ -27,6 +28,7 @@ tags:
 - [[archive/threads/NOTimothyLottes/2023-12-15-long-thread]]
 - [[archive/threads/NOTimothyLottes/2023-12-16-if-there-is-anything-to-learn-for-programming-it]]
 - [[archive/threads/NOTimothyLottes/2023-12-19-multi-session-is-nice-since-the-binary-auto]]
+- [[archive/threads/NOTimothyLottes/2024-01-04-debug-for-majority-gpu-side-game-dev-run-time]]
 - [[archive/threads/NOTimothyLottes/2024-04-06-on-https-mastodon-gamedev-place-notimothylottes]]
 - [[archive/threads/NOTimothyLottes/2024-05-04-it-was-only-later-when-systems-forced-c-abi]]
 - [[archive/threads/NOTimothyLottes/2024-11-16-0-what-a-horrible-term-printf-debugging]]
@@ -47,6 +49,7 @@ tags:
 - [[archive/threads/NOTimothyLottes/2025-07-31-people-claim-assembly-is-hard-a-good-counter-to]]
 - [[archive/threads/NOTimothyLottes/2025-08-01-also-run-from-gpu-side-editing-tools-for-shader]]
 - [[archive/threads/NOTimothyLottes/2025-08-02-so-far-the-plan-is-to-have-the-data-read-write]]
+- [[archive/threads/NOTimothyLottes/2025-10-10-thread-about-this-from-rfleury-https-www-rfleury]]
 - [[archive/threads/NOTimothyLottes/2025-10-13-short-talk-on-how-i-do-cpu-side-debugging-without]]
 - [[archive/threads/NOTimothyLottes/2025-11-17-i-wish-everyone-would-just-adopt-reversed-with]]
 - [[archive/threads/NOTimothyLottes/2026-01-14-can-do-that-but-at-least-these-kinds-of-systems]]
@@ -0,0 +1,55 @@
+---
+title: "@Triang3l That clause based arch was interesting in many ways, mem clause articulates the boundary for write combining."
+type: archive
+source: twitter
+source_url: "https://x.com/NOTimothyLottes/status/2084996454175612983"
+author: "NOTimothyLottes"
+handle: NOTimothyLottes
+post_id: "2084996454175612983"
+date: 2026-08-05
+archived: 2026-08-25
+draft: true
+tags:
+  - archive
+  - twitter
+  - Triang3l
+description: "@Triang3l That clause based arch was interesting in many ways, mem clause articulates the boundary for write combining."
+in_reply_to: ""
+parent_post_id: "2084985800836477433"
+---
+
+## Source
+
+- URL: https://x.com/NOTimothyLottes/status/2084996454175612983
+- Author: NOTimothyLottes (@NOTimothyLottes)
+- Posted: 2026-08-05 13:34:26
+
+## Branch
+
+**1/**
+
+@Triang3l That clause based arch was interesting in many ways, mem clause articulates the boundary for write combining. Was also easy to engineer slow perf by fine granularity predication too.
+
+**2/**
+
+@NOTimothyLottes But what is even a memory clause when MEM_RAT is a control flow instruction, or did the GPU group consecutive MEM_RATs into an implicit "clause"? I also wonder when you can mix RAT and RAT_CACHELESS (the OpenCL compiler apparently just stops using DB at all if CB is ever needed).
+
+**3/**
+
+@Triang3l My understanding is that youd need UNCACHED for both write and read. So wouldn’t be all that useful perf wise. My guess is the transition from MEM_EXPORT to MEM_RD (where youd do UNCACHED) implicitly waits for uncached write visibility.
+
+**4/**
+
+@NOTimothyLottes Yeah, I really hope so… But I guess I should just look at D3D11/GL/CL shader dumps. So many scary things involved, like WAIT_ACK, and that mention of SCOption_R800_UAV_NONUAV_SYNC_WORKAROUND_BUG216513_1 in disassembly. Unless they just fall back to RAT NOP_RTN for all loads 🤷‍♂️
+
+**5/**
+
+@Triang3l Haha oops, very foggy, sounds like it is explicit via WAIT_ACK, rather than implicit. Fees like the predecessor to s_waitcnt in many respects.
+
+**6/**
+
+@Triang3l There are a lot of good cases to have the same memory bound as read and write simultaneously, but under an implicit contract of no r/w overlap for a given cache line . Then dont need uncached reads, but probly would need WAIT_ACK still if write order visibility matters.
+
+## Related
+
+- Spine: [[archive/threads/Triang3l/2026-08-05-writes-have-write-only-caches-that-are]]
@@ -0,0 +1,48 @@
+---
+title: "\"writes … have write-only caches that are invalidated, and all update bits are sent to memory AT THE END OF A CLAUSE\""
+type: archive
+source: twitter
+source_url: "https://x.com/Triang3l/status/2084985800836477433"
+author: "🔺 TriΔng3l 🔺 🐸"
+handle: Triang3l
+post_id: "2084985800836477433"
+date: 2026-08-05
+archived: 2026-08-25
+draft: false
+tags:
+  - archive
+  - twitter
+  - Triang3l
+description: "\"writes … have write-only caches that are invalidated, and all update bits are sent to memory AT THE END OF A CLAUSE\""
+in_reply_to: ""
+---
+
+## Source
+
+- URL: https://x.com/Triang3l/status/2084985800836477433
+- Author: 🔺 TriΔng3l 🔺 🐸 (@Triang3l)
+- Posted: 2026-08-05 12:52:06
+
+## Thread
+
+**1/**
+
+"writes … have write-only caches that are invalidated, and all update bits are sent to memory AT THE END OF A CLAUSE"
+
+Like whaaat, do I need an ALU NOP to read what I've just written to an SSBO?
+
+Far easier to list what makes sense in TeraScale compute shaders than what doesn't
+
+Branches: [[archive/threads/Triang3l/2026-08-05-writes-have-write-only-caches-that-are/2026-08-05-NOTimothyLottes-that-clause-based-arch-was-interesting-in-many]]
+
+**2/**
+
+A clause is a sequence of ALU, buffer/texture SRV fetch, or LDS/GDS instructions.
+
+However, UAV writes are "control flow" export instructions outside clauses.
+
+But you can read from buffer UAVs via uncached SRVs (quicker), but what if you read in the first clause after the write?
+
+**3/**
+
+(Though it's pretty pointless to read back the value just written to an SSBO, but with storage texel buffers that may be done for format conversion.)
@@ -0,0 +1,12 @@
+---
+title: Triang3l
+type: note
+draft: false
+description: Archived threads by Triang3l.
+tags:
+  - archive
+  - twitter
+  - Triang3l
+---
+
+- [[archive/threads/Triang3l/2026-08-05-writes-have-write-only-caches-that-are]]
@@ -19,6 +19,7 @@ Example of the note shape (single spine, no branches): [[archive/threads/How to
 - [[archive/threads/MyNameIsMJP]]
 - [[archive/threads/NOTimothyLottes]]
 - [[archive/threads/SebAaltonen]]
+- [[archive/threads/Triang3l]]
 - [[archive/threads/VPCOMPRESSB]]
 - [[archive/threads/VictorTaelin]]
 - [[archive/threads/kenpex]]