re-render threads with user handle of reply.

2026-08-25 14:56:39 -04:00
1 parent 1ceb4b1583
commit f8380db665
@@ -26,199 +26,199 @@ parent_post_id: "1869436199884222515"

 ## Branch

-**1/**
+**1/** @NOTimothyLottes

@AgileJebrim @SebAaltonen @munohikari (3.) If you had multiple line stores collecting on a given line with ECC, you'd have the same bottleneck as fire-and-forget atomics. A throughput of one line modification per clock.

-**2/**
+**2/** @NOTimothyLottes

@AgileJebrim @SebAaltonen @munohikari (4.) Note the prior assumes a read hit! 

 So at the point you get anti-atomic, you are also anti-store and anti-ECC, and anti-etc. It's the same pipelines with the same variability due to on-chip networking, different clock domain crossing, etc.

-**3/**
+**3/** @NOTimothyLottes

@AgileJebrim @SebAaltonen @munohikari (5.) Good GPUs have multiple ALUs doing the atomic by the last level cache. So it scales very well as a function of line collisions. Meaning if all lanes access separate addresses on a cacheline, those go in parallel.

-**4/**
+**4/** @NOTimothyLottes

@AgileJebrim @SebAaltonen @munohikari (6.) Atomics and regular stores have the same memory controller base-camping behavior. If all SIMDs store to addresses that hash to the same last level cache partition, they will drop efficiency just like doing all atomics to the same address.

-**5/**
+**5/** @NOTimothyLottes

@AgileJebrim @SebAaltonen @munohikari (7.) Anyway, while CPU atomics are serious design fail, GPU atomics on good GPUs are your best friend. They are actually a fantastic tool for getting more regular performance in many parallel algorithms.

-**6/**
+**6/** @SebAaltonen

@NOTimothyLottes @AgileJebrim @munohikari Indeed. Many CPU programmers have the wrong belief that GPU atomics are slow and unusable. They are fantastic. Bump allocator is your best friend on CPU side. On GPU you do atomic add to implement a bump allocator. As simple as that. It's super fast.

-**7/**
+**7/** @AgileJebrim

@SebAaltonen @NOTimothyLottes @munohikari I know NVIDIA did a really good atomicAdd, especially if you don’t use the result of the atomicAdd(). If you use it, then it forces it down a significantly slower path. But it’s all implementation-specific and different vendors will do different things yes?

-**8/**
+**8/** @NOTimothyLottes

@AgileJebrim @SebAaltonen @munohikari Atomics come in two forms
 (1.) fire-and-forget (aka you don't use the return) - the compiler will flag it for no-return, so it behaves like a store
 (2.) read the return - it behaves like a {store, and read} simultaneously with layout "coherent"

-**9/**
+**9/** @NOTimothyLottes

@AgileJebrim @SebAaltonen @munohikari Good GPUs have address coalescing on writes and atomics. So a wave wide store is reduced to a cost that scales with the number of total cachelines all waves touch, with byte-write-mask so there is no read-modify-write

-**10/**
+**10/** @NOTimothyLottes

@AgileJebrim @SebAaltonen @munohikari ... atomics have the same but if any wave lane's addresses collide it will bump those to another request (for the same cacheline). Possible that some HW implements a HW reduction for collisions of some kinds of atomics locally before the requests go out, but don't assume that

-**11/**
+**11/** @NOTimothyLottes

@AgileJebrim @SebAaltonen @munohikari ... it's more likely that some compilers try to fix stupid behavior by doing a wave-reduction in software, and that causes hell for those of us who already write good code

-**12/**
+**12/** @AgileJebrim

@NOTimothyLottes @SebAaltonen @munohikari Thoughts on atomic compare exchange? I use similar functionality on CPUs to globally synchronize threads and I figured I’d give it a shot attempting to do that on GPUs.

-**13/**
+**13/** @NOTimothyLottes

@AgileJebrim @SebAaltonen @munohikari Compare and exchange since it takes 2 values, it could have half the peak rate through on-chip network. Personally I have always used algorithms that do better things without the cmpswap atomic. You always can get the return (value pre-atomic), so cmpswap utility is less

-**14/**
+**14/** @AgileJebrim

@NOTimothyLottes @SebAaltonen My CPU sync mechanism works by stalling each thread until it reaches a multiple of the total number of threads to be synced, only incrementing an integer (swapping) when it reaches its specific thread index.

-**15/**
+**15/** @NOTimothyLottes

@AgileJebrim @SebAaltonen I assume you are using a futex then on Linux to block? But the GPU doesn't have such a thing exposed in software. Instead you'd be effectively polling using serial global memory atomic latency as your rate limiter and 'blocking'

-**16/**
+**16/** @NOTimothyLottes

@AgileJebrim @SebAaltonen Serializing workgroups on the GPU in this way has a very high latency granularity. There is also the issue of preemption on PCs, there is no guarantee of forward progress due to no guarantee all workgroups get restored

-**17/**
+**17/** @NOTimothyLottes

@AgileJebrim @SebAaltonen So you can deadlock post preemption if the workgroup that needs to do the atomic to unblock those spinning cannot get back on the machine

-**18/**
+**18/** @NOTimothyLottes

@AgileJebrim @SebAaltonen Now, the important part, there is one compromise rule that has the highest chance of being possible, that is an ask that if there is a partial restore, that it restores the oldest launched workgroups first ...

-**19/**
+**19/** @NOTimothyLottes

@AgileJebrim @SebAaltonen And with that you can have some forward progress if an algorithm only has dependencies on workgroups launched before it (like lower kernel global workgroup index), food for thought for algorithm design

-**20/**
+**20/** @NOTimothyLottes

@AgileJebrim @SebAaltonen on the CPU side, you'd futex block to get the thread off being scheduled so the dependent one could get on the machine to avoid deadlock, it isn't efficient by any means but works, the GPU's exposed only voluntary method to get off the machine is to exit the workgroup ...

-**21/**
+**21/** @NOTimothyLottes

@AgileJebrim @SebAaltonen So if you uber-task, meaning just keep launching the same dispatch, and the workgroup finds what it needs to do (instead of working off kernel global index), then you can workaround preemption that way too ...

-**22/**
+**22/** @NOTimothyLottes

@AgileJebrim @SebAaltonen Blocking becomes just workgroup exit. And the next workgroup gets lower schedule priority than existing active workgroups which helps solve of the scheduling issues

-**23/**
+**23/** @AgileJebrim

@NOTimothyLottes @SebAaltonen I’m not using a Linux futex. Could you look over my code here?

-**24/**
+**24/** @simplex_fx

@AgileJebrim @NOTimothyLottes @SebAaltonen Hey Jeb, have you considered hitting the funny L shaped button on the right of your keyboard more frequently? 😅

-**25/**
+**25/** @AgileJebrim

@simplex_fx @NOTimothyLottes @SebAaltonen Enter? Nah

-**26/**
+**26/** @simplex_fx

@AgileJebrim @NOTimothyLottes @SebAaltonen

 ![](https://pbs.twimg.com/media/GfG1cCJW8AAXcDg?format=jpg&name=orig)

-**27/**
+**27/** @AgileJebrim

@simplex_fx @NOTimothyLottes @SebAaltonen If I don’t have to scroll right…

-**28/**
+**28/** @simplex_fx

@AgileJebrim @NOTimothyLottes @SebAaltonen Dude, stop with that WQHD bullshit, buy a proper tv and limit yourself to 80-120

-**29/**
+**29/** @AgileJebrim

@simplex_fx @NOTimothyLottes @SebAaltonen Nty. The hell do I need a tv for? I have several monitors.

-**30/**
+**30/** @simplex_fx

@AgileJebrim @NOTimothyLottes @SebAaltonen How big?

-**31/**
+**31/** @AgileJebrim

@simplex_fx @NOTimothyLottes @SebAaltonen Big enough to maximize a window on each.

-**32/**
+**32/** @simplex_fx

@AgileJebrim @NOTimothyLottes @SebAaltonen If you go too wide, you will just kill your neck on the long run

-**33/**
+**33/** @AgileJebrim

@simplex_fx @NOTimothyLottes @SebAaltonen I have 4 monitors

-**34/**
+**34/** @simplex_fx

@AgileJebrim @NOTimothyLottes @SebAaltonen How big?

-**35/**
+**35/** @NOTimothyLottes

@simplex_fx @AgileJebrim @SebAaltonen Ideally one of these

 ![](https://pbs.twimg.com/media/GfHntpaWYAAGnOk?format=png&name=orig)

-**36/**
+**36/** @AgileJebrim

@NOTimothyLottes @simplex_fx @SebAaltonen Not big enough. Evans & Sutherland still makes planetariums. We have smaller scale 360 degree domes. I think they’re about 14ft high.

-**37/**
+**37/** @SebAaltonen

@AgileJebrim @NOTimothyLottes Same is true for everything. All memory loads on GPUs are highly vendor specific. On GTX 1080 (Pascal) your memory load goes through sampler, which is around 85 cycles of latency. While on Turing your raw load is only 28 cycles of latency (CUDA benchmarks).

-**38/**
+**38/** @AgileJebrim

@SebAaltonen @NOTimothyLottes For sure, but my focus is more on how deterministic and predictable these are. I’ll take 85 if it’s always 85 over something that might vary unpredictably between 28 and 100.

-**39/**
+**39/** @SebAaltonen

@AgileJebrim @NOTimothyLottes Nvidia has big L1$, so they are more predictable. Texture samplers are shared with all waves running on the same compute unit. They are not really predictable. You wait until the unit is ready. There's quite a lot of fluctuation. That's why you have 8x-10x hyperthreading per CU.

-**40/**
+**40/** @SebAaltonen

@AgileJebrim @NOTimothyLottes The idea is to keep the SIMD running all the time. Wave execution time variance hasn't been a big optimization goal. There's always more work coming, so it's not a problem if some wave takes 10x time to execute compared to others.

-**41/**
+**41/** @SebAaltonen

@AgileJebrim @NOTimothyLottes Basically if the SIMD is executing instructions all the time, then your workload runs at predictable amount of instructions per cycle. The problem is barriers. You wait for longest wave to finish. You can't to batch barriers to avoid turning latency into real lost SIMD cycles.

-**42/**
+**42/** @AgileJebrim

@SebAaltonen @NOTimothyLottes If they all take the same amount of time then there shouldn’t be much idling of hardware for waiting on the longest wave.

-**43/**
+**43/** @SebAaltonen

@AgileJebrim @NOTimothyLottes That's true. But they don't take the same amount of time due to caches. That's where the 10x hyper-threading kicks in. As long as the SIMD is fed all the time, there's no visible issue with fluctuating wave lengths. Barriers make that visible. Minimal amount of barriers = good.

-**44/**
+**44/** @AgileJebrim

@NOTimothyLottes @SebAaltonen @munohikari Would you implement a prefix sum compaction algorithm using atomicAdd?

-**45/**
+**45/** @NOTimothyLottes

@AgileJebrim @SebAaltonen @munohikari Standard optimizations apply, like doing wave-reductions first.

-**46/**
+**46/** @AgileJebrim

@NOTimothyLottes @SebAaltonen @munohikari Okay so you don’t just blanket throw an atomicAdd on it then.

-**47/**
+**47/** @NOTimothyLottes

@AgileJebrim @SebAaltonen @munohikari No. For things like scanning a 1-bit value, it's a wave masked bit count op in software, followed by predicating the atomic to just lane 0 of the wave. For scanning uint32s it's a log2 wave reduction, then predicating the add to lane 0. And so on

@@ -26,13 +26,13 @@ parent_post_id: "1869424048607993895"

 ## Branch

-**1/**
+**1/** @SebAaltonen

@AgileJebrim @NOTimothyLottes @munohikari Yes, but it's going to be MUCH slower. You can do sums and reductions in groupshared memory, but if that mobile GPU doesn't have real groupshared memory, it's going to be very slow. For example GMEM based groupshared is this slow on Qualcomm (and 730 is much better than 610):

 ![](https://pbs.twimg.com/media/GfGISwfWgAI6UuY?format=jpg&name=orig)

-**2/**
+**2/** @SebAaltonen

@AgileJebrim @NOTimothyLottes @munohikari X1 is the brand new laptop chip. This architecture is only on latest mobile phones. Low end is Adreno 610 and similar.

@@ -25,56 +25,56 @@ in_reply_to: ""

 ## Thread

-**1/**
+**1/** @AgileJebrim

@munohikari @SebAaltonen You left out the bit where I also said no graphics pipeline. :P

-**2/**
+**2/** @SebAaltonen

@AgileJebrim @munohikari Computer shaders are a can of worms for old mobile GPUs too. Some have fast groupshared memory (could be tiny), some emulate it. Some have wave intrinsics, some don't. 64-bit atomic support is nonexistent (for MM Dreams / Native style Z+payload packing).

-**3/**
+**3/** @SebAaltonen

@AgileJebrim @munohikari Also compute shader writes are not compressed (pixel shader framebuffer writes have HW lossless compressor). You pay full memory bandwidth cost, which causes phone to heat (drain battery) and eventually throttle.

-**4/**
+**4/** @AgileJebrim

@SebAaltonen @munohikari Alternative approach is to do all rendering in screen space in a fragment shader.

-**5/**
+**5/** @SebAaltonen

@AgileJebrim @munohikari If you intend to do distance field ray-tracing using volume textures, worth noting that most mobile GPUs don't do proper 3d tiling layout for volume textures. Instead they use legacy 2d sliced layout, which results in significantly more L1$ trashing. Can 5x your mem BW use.

-**6/**
+**6/** @AgileJebrim

@SebAaltonen @munohikari Raycasting, no bouncing, using cubic textured bricks that emulate voxel surfaces via a parallax occlusion map approach measuring distances to each voxel surface.

 If I’m writing something for an avionics cockpit, I’d be either using an Arm Mali-G78AE or NVIDIA DRIVE AGX Thor GPU.

-**7/**
+**7/** @AgileJebrim

@SebAaltonen @munohikari I’d also be using Vulkan SC, which I believe disables atomics entirely. :P

-**8/**
+**8/** @NOTimothyLottes

@AgileJebrim @SebAaltonen @munohikari Disabling atomics is like chopping off both legs because someone thinks one tow has a blister when it actually doesn't.

-**9/**
+**9/** @AgileJebrim

@NOTimothyLottes @SebAaltonen @munohikari You can still implement a comparable feature yourself.

 Branches: [[archive/threads/AgileJebrim/2024-12-18-you-left-out-the-bit-where-i-also-said-no/2024-12-18-SebAaltonen-yes-but-its-going-to-be-much-slower-you-can-do]]

-**10/**
+**10/** @NOTimothyLottes

@AgileJebrim @SebAaltonen @munohikari No actually you cannot. No method of software (or even HW) implemented lock-line algorithm survives the kind of latency in the GPU memory system. There is a big reason atomics on GPUs are done right by the last level of cache (the coherent memory domain)

-**11/**
+**11/** @NOTimothyLottes

@AgileJebrim @SebAaltonen @munohikari Something you really need to understand for your own benefit:
 (1.) There are a lot of more expensive things happening in the GPU memory system than atomics. For instance DCC compression. It requires metadata access (to get the state of memory just to do the compression on store).

-**12/**
+**12/** @NOTimothyLottes

@AgileJebrim @SebAaltonen @munohikari (2.) Think of GPU atomics as a read-modify-write op. It's effectively the same behavior as a partial cacheline write-through STORE operation on HW using ECC. Because ECC eats the bits used for byte-write-mask.

@@ -26,7 +26,7 @@ parent_post_id: "2010303501851877740"

 ## Branch

-**1/**
+**1/** @Alecazam123

@Dispatch_Graph Hard to believe I’m still writing games in Win32 25 years later.  But it runs on SteamOS, so can never be replaced now.

@@ -26,7 +26,7 @@ parent_post_id: "2010303501851877740"

 ## Branch

-**1/**
+**1/** @BowTiedTails

@Dispatch_Graph Yep. Win32 is heaven when you see what else is out there.

@@ -26,7 +26,7 @@ parent_post_id: "2010303501851877740"

 ## Branch

-**1/**
+**1/** @VPCOMPRESSB

@Dispatch_Graph coincidentally, i made a Vulkan Wayland compute engine yesterday in ~600 LoC.

@@ -26,32 +26,32 @@ parent_post_id: "2010303501851877740"

 ## Branch

-**1/**
+**1/** @VPCOMPRESSB

 yeah. i actually prefer Wayland because if you implement something super simple, you get something super simple back (e.g., if i create a window, i get the bare minimum window - no decorations, no resizing, no keyboard or mouse input). with Win32, i'd have fight it and move through its weeds to force it to make something simple.
 another problem with Win32 is that it modularizes too much, and make everything seem irrelevant from each other. in Wayland, it's all unified, where if just learn 1 pattern from 1 thing, you can apply that pattern to everything else, from windows to input (e.g., all interfaces are just a simple server-client messaging model). basically, you can take a step back and look at everything as a whole, and clearly see a pattern.

-**2/**
+**2/** @NOTimothyLottes

@VPCOMPRESSB @Dispatch_Graph You didn't get the true Wayland experience until you try interfacing to Wayland without the Wayland user-space library. Meaning do it with just the kernel calls required. Then you will have taken the gold off the poop, and understood how bad it actually is.

-**3/**
+**3/** @NOTimothyLottes

@VPCOMPRESSB @Dispatch_Graph WIN32 base system DLLs are a substantially cleaner interface. I only do non-decorated full-screen (foreground or background) window. Majority of WIN32 code fits on screen grab with mostly comments. I'm not a fan of their "WndProc" but at least it is simple.

 ![](https://pbs.twimg.com/media/G-Zd9wLWUAE9kZj?format=png&name=orig)

-**4/**
+**4/** @NOTimothyLottes

@VPCOMPRESSB @Dispatch_Graph Some like to hate on X11, but for what it was designed to do (support both local and remote displays), it is at least very well documented and actually quite easy to roll an interface for with just Linux kernel calls (no external libraries). Docs here: https://x.org/releases/X11R7.7/doc/xproto/x11protocol.html

-**5/**
+**5/** @NOTimothyLottes

@VPCOMPRESSB @Dispatch_Graph Duckduckgo Wayland protocol and you get this garbage as documentation (below). Nothing there actually explains how it works, it's just high-level OOP obfustication speak. My vomit translator cannot even process this stuff.

 ![](https://pbs.twimg.com/media/G-Zjy3uXgAETWbE?format=jpg&name=orig)

-**6/**
+**6/** @NOTimothyLottes

@VPCOMPRESSB @Dispatch_Graph Lets continue. It's built over unix domain sockets and doesn't support remote rendering. Haha, WFT. Just because they want to transfer file descriptors between processes for "bulk data". Wayland looses network displays and is SLOWER (fail): https://www.dedoimedo.com/computers/plasma-6-4-performance-wayland-x11-power-cpu-kernel.html

@@ -59,33 +59,33 @@ another problem with Win32 is that it modularizes too much, and make everything
 ![](https://pbs.twimg.com/media/G-Zla8PXYAAdRoE?format=png&name=orig)
 ![](https://pbs.twimg.com/media/G-ZmyPCWQAAKtYR?format=png&name=orig)

-**7/**
+**7/** @NOTimothyLottes

@VPCOMPRESSB @Dispatch_Graph Don't understand why Linux people had to take what was working, X11, and instead of just improving it, they had to make something that {removed the remote desktop, made it slower, and forces an OOP garbage design}. There is nothing good here.

-**8/**
+**8/** @NOTimothyLottes

@VPCOMPRESSB @Dispatch_Graph Another great thing about X11, is that it was trivial to both write and use your own window manager, using the same stable system for DECADES. I had a 350 line minwm.c window manager. Dead easy.

 ![](https://pbs.twimg.com/media/G-ZrxUJXwAAINS7?format=png&name=orig)

-**9/**
+**9/** @NOTimothyLottes

@VPCOMPRESSB @Dispatch_Graph Unix was about KISS (keep it simple stupid) then Linux user-space people transformed that into MISC {make it stupid complex}. Seen this trend with Audio, from OSS to ALSA, and so on.

-**10/**
+**10/** @NOTimothyLottes

@VPCOMPRESSB @Dispatch_Graph I worked in the {SunOS,Solaris,HPUX,SGI,AXI,etc} workstation era and started on Linux before ELF and before pthreads. Meaning I build things based on fork() [instead of pthreads] and SHM (shared page mappings) when doing local-only interfaces (no high kernel call granularity).

-**11/**
+**11/** @NOTimothyLottes

@VPCOMPRESSB @Dispatch_Graph If Linux people wanted to replace X11 with a local only interface, at a minimum it should have been simple SHM for the majority, and then only go outside SHM when needing to say kick a doorbell (make wake another process), etc.

-**12/**
+**12/** @o__boga

@NOTimothyLottes @VPCOMPRESSB @Dispatch_Graph Obfuscation is the secret for its "security!"!

-**13/**
+**13/** @VPCOMPRESSB

@NOTimothyLottes @Dispatch_Graph i've never actually used X11. this is much easier.

@@ -26,7 +26,7 @@ parent_post_id: "2010303501851877740"

 ## Branch

-**1/**
+**1/** @hadal101dev

@Dispatch_Graph Yea windows makes some interesting choices lol. Calling CreateFile when you want to OPEN a file? very weird lol.

@@ -26,7 +26,7 @@ parent_post_id: "2010303501851877740"

 ## Branch

-**1/**
+**1/** @smolus

@Dispatch_Graph one thing I appreciate about wayland is that it's just a simple protocol spec so you are not married to any particular API/library but yeah it can be a lot of work to get basic things working

@@ -26,7 +26,7 @@ parent_post_id: "2010303501851877740"

 ## Branch

-**1/**
+**1/** @Chris_Linux

@Dispatch_Graph Apples and oranges comparison here

@@ -26,7 +26,7 @@ parent_post_id: "2010303501851877740"

 ## Branch

-**1/**
+**1/** @FoundryOfLies

@Dispatch_Graph It's KWin

@@ -26,7 +26,7 @@ parent_post_id: "2010303501851877740"

 ## Branch

-**1/**
+**1/** @JoaoBapt

@Dispatch_Graph I suggest you to stick with GLFW if you’re only doing Vulkan.

@@ -26,13 +26,13 @@ parent_post_id: "2010303501851877740"

 ## Branch

-**1/**
+**1/** @Oka7071

@Dispatch_Graph

 ![](https://pbs.twimg.com/media/G-fL2X9WAAARbpz?format=jpg&name=orig)

-**2/**
+**2/** @Dispatch_Graph

@Oka4902 if you give me 3 trillion dollars I can elaborate

@@ -26,7 +26,7 @@ parent_post_id: "2010303501851877740"

 ## Branch

-**1/**
+**1/** @PervWizz

@Dispatch_Graph Well. Win32 began as a bad API and then organically got reshaped into a worse one. Same goes for X11

@@ -26,11 +26,11 @@ parent_post_id: "2010303501851877740"

 ## Branch

-**1/**
+**1/** @Rami36217810

@Dispatch_Graph What about x11

-**2/**
+**2/** @Dispatch_Graph

@Rami36217810 its aight

@@ -26,7 +26,7 @@ parent_post_id: "2010303501851877740"

 ## Branch

-**1/**
+**1/** @SyntaxError2505

@Dispatch_Graph Why? Sdl works great

@@ -26,7 +26,7 @@ parent_post_id: "2010303501851877740"

 ## Branch

-**1/**
+**1/** @f82james

@Dispatch_Graph fuck x11

@@ -26,7 +26,7 @@ parent_post_id: "2010303501851877740"

 ## Branch

-**1/**
+**1/** @linux_version

@Dispatch_Graph Oddly enough, with how old the wine project is, Win32 might be the most stable API on Linux.

@@ -26,7 +26,7 @@ parent_post_id: "2010303501851877740"

 ## Branch

-**1/**
+**1/** @mizanyx

@Dispatch_Graph Elaborate please, I'm still an X11 user

@@ -26,7 +26,7 @@ parent_post_id: "2010303501851877740"

 ## Branch

-**1/**
+**1/** @platonvin

@Dispatch_Graph windows.h is like a nightmare but real. It is so bad, that many people manually paste parts of it to avoid including entire thing

@@ -26,7 +26,7 @@ parent_post_id: "2010303501851877740"

 ## Branch

-**1/**
+**1/** @ruminaik

@Dispatch_Graph And let us say Amen

@@ -26,7 +26,7 @@ parent_post_id: "2010303501851877740"

 ## Branch

-**1/**
+**1/** @thedarkhog

@Dispatch_Graph Get Xlibre instead. If you're using arch, btw., it's in the AUR.

@@ -25,16 +25,16 @@ in_reply_to: ""

 ## Thread

-**1/**
+**1/** @Dispatch_Graph

 Ok, the Win32 API sure is something, but WOW wayland is awful

 Branches: [[archive/threads/Dispatch_Graph/2026-01-11-ok-the-win32-api-sure-is-something-but-wow/2026-01-11-VPCOMPRESSB-coincidentally-i-made-a-vulkan-wayland-compute]], [[archive/threads/Dispatch_Graph/2026-01-11-ok-the-win32-api-sure-is-something-but-wow/2026-01-11-hadal101dev-yea-windows-makes-some-interesting-choices-lol]], [[archive/threads/Dispatch_Graph/2026-01-11-ok-the-win32-api-sure-is-something-but-wow/2026-01-11-VPCOMPRESSB-yeah-i-actually-prefer-wayland-because-if-you]], [[archive/threads/Dispatch_Graph/2026-01-11-ok-the-win32-api-sure-is-something-but-wow/2026-01-11-Alecazam123-hard-to-believe-im-still-writing-games-in-win32]], [[archive/threads/Dispatch_Graph/2026-01-11-ok-the-win32-api-sure-is-something-but-wow/2026-01-11-smolus-one-thing-i-appreciate-about-wayland-is-that-its]], [[archive/threads/Dispatch_Graph/2026-01-11-ok-the-win32-api-sure-is-something-but-wow/2026-01-11-BowTiedTails-yep-win32-is-heaven-when-you-see-what-else-is-out]], [[archive/threads/Dispatch_Graph/2026-01-11-ok-the-win32-api-sure-is-something-but-wow/2026-01-12-thedarkhog-get-xlibre-instead-if-youre-using-arch-btw-its-in]], [[archive/threads/Dispatch_Graph/2026-01-11-ok-the-win32-api-sure-is-something-but-wow/2026-01-12-Chris_Linux-apples-and-oranges-comparison-here]], [[archive/threads/Dispatch_Graph/2026-01-11-ok-the-win32-api-sure-is-something-but-wow/2026-01-12-mizanyx-elaborate-please-im-still-an-x11-user]], [[archive/threads/Dispatch_Graph/2026-01-11-ok-the-win32-api-sure-is-something-but-wow/2026-01-12-SyntaxError2505-why-sdl-works-great]], [[archive/threads/Dispatch_Graph/2026-01-11-ok-the-win32-api-sure-is-something-but-wow/2026-01-12-FoundryOfLies-its-kwin]], [[archive/threads/Dispatch_Graph/2026-01-11-ok-the-win32-api-sure-is-something-but-wow/2026-01-12-platonvin-windows-h-is-like-a-nightmare-but-real-it-is-so]], [[archive/threads/Dispatch_Graph/2026-01-11-ok-the-win32-api-sure-is-something-but-wow/2026-01-12-ruminaik-and-let-us-say-amen]], [[archive/threads/Dispatch_Graph/2026-01-11-ok-the-win32-api-sure-is-something-but-wow/2026-01-12-f82james-fuck-x11]], [[archive/threads/Dispatch_Graph/2026-01-11-ok-the-win32-api-sure-is-something-but-wow/2026-01-12-linux_version-oddly-enough-with-how-old-the-wine-project-is]], [[archive/threads/Dispatch_Graph/2026-01-11-ok-the-win32-api-sure-is-something-but-wow/2026-01-12-PervWizz-well-win32-began-as-a-bad-api-and-then]], [[archive/threads/Dispatch_Graph/2026-01-11-ok-the-win32-api-sure-is-something-but-wow/2026-01-12-JoaoBapt-i-suggest-you-to-stick-with-glfw-if-youre-only]], [[archive/threads/Dispatch_Graph/2026-01-11-ok-the-win32-api-sure-is-something-but-wow/2026-01-12-Rami36217810-what-about-x11]], [[archive/threads/Dispatch_Graph/2026-01-11-ok-the-win32-api-sure-is-something-but-wow/2026-01-12-Oka7071-thread]]

-**2/**
+**2/** @ocarina588

@Dispatch_Graph 😂😂😂, what are you using? Hyprland ?

-**3/**
+**3/** @Dispatch_Graph

@ocarina588 KDE Plasma
@@ -26,27 +26,27 @@ parent_post_id: "1884730404587909544"

 ## Branch

-**1/**
+**1/** @mu_chrinovic

@FFmpeg I know a person with a PhD that might disagree with you

-**2/**
+**2/** @NOTimothyLottes

@mu_chrinovic @FFmpeg Yeah PTX is not assembly, it's a mid level language that is closer to a 1:1 match for the HW ISA --- BUT --- the actual register allocation and management is still done by a high-level compiler ...

-**3/**
+**3/** @janekm

@NOTimothyLottes @mu_chrinovic @FFmpeg That's almost semantics at this point though? If you're writing x86 assembly it gets translated to microcode in the CPU as well, and will do register remapping etc. Only difference in Nvidia case that's happening in the driver, right?

-**4/**
+**4/** @NOTimothyLottes

@janekm @mu_chrinovic @FFmpeg Sounds like you are trying to claim software compiler register allocation is the same as HW runtime register renaming? No. CPU register renaming maps a small set of registers to a larger set to get IPC. Compiler GPU reg allocation maps a massive SSA mess to a small set.

-**5/**
+**5/** @janekm

@NOTimothyLottes @mu_chrinovic @FFmpeg Fair point, my argument was merely that both cases of assembly language are really an intermediate language that gets translated to the final instructions executed by the CPU/GPU. Feels quite pedantic to claim that PTX assembly is not assembly, merely because of reg allocation 🤷‍♂️

-**6/**
+**6/** @NOTimothyLottes

@janekm @mu_chrinovic @FFmpeg SOFTWARE
 Assembler = human writes the physical HW instructions
@@ -56,11 +56,11 @@ HW
 Instruction-reordering / wave-scheduling
 Register renaming

-**7/**
+**7/** @janekm

@NOTimothyLottes @mu_chrinovic @FFmpeg We've been calling it Assembly since at least when JVM byte code was first a thing. Doesn't matter that a JIT compiler will still do register allocation.

-**8/**
+**8/** @NOTimothyLottes

@janekm @mu_chrinovic @FFmpeg Sure people have been incorrectly labeling stuff for a long time

@@ -25,12 +25,12 @@ in_reply_to: ""

 ## Thread

-**1/**
+**1/** @FFmpeg

 PTX is a form of assembly language

 Branches: [[archive/threads/FFmpeg/2025-01-29-ptx-is-a-form-of-assembly-language/2025-01-29-mu_chrinovic-i-know-a-person-with-a-phd-that-might-disagree]]

-**2/**
+**2/** @FFmpeg

 https://docs.nvidia.com/cuda/inline-ptx-assembly/index.html
@@ -26,7 +26,7 @@ parent_post_id: "1781724279798686001"

 ## Branch

-**1/**
+**1/** @jaap_null

@KostasAAA It's a silly thing on many levels. The entire arch. limitation is awkward, then the way compilers deal (or not deal) with it is just broken. Sad to see it's still around.

@@ -26,7 +26,7 @@ parent_post_id: "1781724279798686001"

 ## Branch

-**1/**
+**1/** @matiasgoldberg

@KostasAAA What's the value of shaderSampledImageArrayDynamicIndexing?

@@ -25,19 +25,19 @@ in_reply_to: ""

 ## Thread

-**1/**
+**1/** @KostasAAA

 When using arrays of resources/bindless, it's worth paying attention to index divergence. Even if using thread divergent indices, the compiler will use the index from the first active thread to access a resource, for all threads in the wave, which can cause hard to find bugs 1/3

 ![](https://pbs.twimg.com/media/GLnsaWeWoAA0c-x?format=jpg&name=orig)

-**2/**
+**2/** @KostasAAA

 This happens even when compiler knows that the index varies per thread. To fix this you need to use the NonUniformResourceIndex qualifier. The compiler will then add a waterfall loop to batch wave threads by resource index and ensure that each thread accesses the correct one. 2/3

 ![](https://pbs.twimg.com/media/GLntvM6W0AEVe0L?format=jpg&name=orig)

-**3/**
+**3/** @KostasAAA

 This can have a performance impact though, with the worst case being each thread indexing a different resource, both in terms of thread coherence (looping over each thread individually) and texture cache. For more info https://www.asawicki.info/news_1734_which_values_are_scalar_in_a_shader 3/3

@@ -25,71 +25,71 @@ in_reply_to: ""

 ## Thread

-**1/**
+**1/** @MyNameIsMJP

@BelgianRenderer @KostasAAA @Nicolas_Lopez_ Apparently uniformity analysis is hard. 🙁

-**2/**
+**2/** @KostasAAA

@MyNameIsMJP @BelgianRenderer @Nicolas_Lopez_ If uniformity can't be deduced, the compiler could be conservative and assume divergence. If it guessed wrong, the waterfall loop won't be executed and all threads will use the same index. Also, the shader author could help the compiler with a UniformResourceIndex qualifier.

-**3/**
+**3/** @MyNameIsMJP

@KostasAAA @BelgianRenderer @Nicolas_Lopez_ Yeah I tend to agree that assuming non-uniformity by default for descriptor indexing would have been a better path, with some better language support for explicit tagging of uniform variables. Maybe in HLSL 2027. 🙂

-**4/**
+**4/** @NOTimothyLottes

@MyNameIsMJP @KostasAAA @BelgianRenderer @Nicolas_Lopez_ Uniformity is the fast path, it would be horrible to tax the fast path for the sake of things designed to be slow (aka using resource divergence). The right thing to do is place in explicit uniform/nonuniform qualifiers into the languages so it's explicit.

-**5/**
+**5/** @JasperBekkers

@NOTimothyLottes @MyNameIsMJP @KostasAAA @BelgianRenderer @Nicolas_Lopez_ We do nuri for almost all our access (we sometimes turn it off because of badly readable Isa on amd). However the performance impact is in the noise, not all hw resolves this in the shader cores.

-**6/**
+**6/** @NOTimothyLottes

@JasperBekkers @MyNameIsMJP @KostasAAA @BelgianRenderer @Nicolas_Lopez_ NV doesn't advertise this: 32-byte sector size (even with 128-byte lines) mixed with per lane IPs (so you can pipeline latency of divergent accesses) is all about shifting the perf cliff of bad behavior enough to win benchmarks, it is still a cliff (look at ALU utilization) ...

-**7/**
+**7/** @NOTimothyLottes

@JasperBekkers @MyNameIsMJP @KostasAAA @BelgianRenderer @Nicolas_Lopez_ AMD never invested in pipelining divergent latency, and their compiler compounds this by serializing a lot of other things that shouldn't be serialized ...

-**8/**
+**8/** @NOTimothyLottes

@JasperBekkers @MyNameIsMJP @KostasAAA @BelgianRenderer @Nicolas_Lopez_ If one is already into the perf cliff, it's easy on AMD to compound perf debt to the point something can never be optimized, and maybe that's ok if time-to-market/ease-of-use/etc is the primary goal ...

-**9/**
+**9/** @NOTimothyLottes

@JasperBekkers @MyNameIsMJP @KostasAAA @BelgianRenderer @Nicolas_Lopez_ There are cases where AMD's compiler is still failing hard on UNIFORM (aka wave coherent) multi-resource access: zero latency hiding -> every resource is a worst case serialization of {fetch descriptor, wait, sample} with zero pipelining ... example below

 ![](https://pbs.twimg.com/media/GL7p5vJXUAECnVi?format=png&name=orig)

-**10/**
+**10/** @NOTimothyLottes

@JasperBekkers @MyNameIsMJP @KostasAAA @BelgianRenderer @Nicolas_Lopez_ Or this case where the images get batched, but the descriptors accesses are serialized (below) ... so it's easy sometimes to assume NURI is not that expensive because the compiler already shot itself in the face (which is quite common in high complexity material shading).

 ![](https://pbs.twimg.com/media/GL7rS7uWsAEebjc?format=png&name=orig)

-**11/**
+**11/** @JasperBekkers

@NOTimothyLottes @MyNameIsMJP @KostasAAA @BelgianRenderer @Nicolas_Lopez_ Most of the time for us the resource access is uniform,  so the overhead is just a few waves to test this though.

-**12/**
+**12/** @NOTimothyLottes

@JasperBekkers @MyNameIsMJP @KostasAAA @BelgianRenderer @Nicolas_Lopez_ If the whole shader coherently branches into uniform vs non-uniform material paths, sure that can work if the non-uniform path is almost never used, and the compiler doesn't choak on the register pressure of the non-uniform code path.

-**13/**
+**13/** @NOTimothyLottes

@JasperBekkers @MyNameIsMJP @KostasAAA @BelgianRenderer @Nicolas_Lopez_ At least for the sake of learnings for others, I'll continue down the compiler rabbit hole a little more ... even serialized latency of K$ (constant) HITs (all in the cache) will destroy shader perf (a single hit is like 30 some clocks). This pattern happens on AMD today still.

 ![](https://pbs.twimg.com/media/GL7uT1wXoAA7CQj?format=png&name=orig)

-**14/**
+**14/** @NOTimothyLottes

@JasperBekkers @MyNameIsMJP @KostasAAA @BelgianRenderer @Nicolas_Lopez_ And as for 'fixing' bad programmer behavior via the compiler, lets look at atomicAdd() behavior. First the compiler will burn the overhead to elect an active lane even when the compiler statically knows lane zero is active (simple example below).

 ![](https://pbs.twimg.com/media/GL7wLb8WgAEicQD?format=png&name=orig)

-**15/**
+**15/** @NOTimothyLottes

@JasperBekkers @MyNameIsMJP @KostasAAA @BelgianRenderer @Nicolas_Lopez_ Try to out smart it like this,

@@ -100,12 +100,12 @@ It fights back, and makes something even worse. [RED] ... see the multiply by ac

 ![](https://pbs.twimg.com/media/GL7xh8ZWsAArlkO?format=png&name=orig)

-**16/**
+**16/** @NOTimothyLottes

@JasperBekkers @MyNameIsMJP @KostasAAA @BelgianRenderer @Nicolas_Lopez_ And sometimes it gets really squirrly and throws in a full wave reduction (see the DPP ops). When the good dev is just trying to do an already predicated to lane zero atomic ADD ...

 ![](https://pbs.twimg.com/media/GL7yB5cXIAACo5E?format=png&name=orig)

-**17/**
+**17/** @NOTimothyLottes

@JasperBekkers @MyNameIsMJP @KostasAAA @BelgianRenderer @Nicolas_Lopez_ So if any solution depends on trusting a compiler to implicitly do the right thing, I'd be very very concerned, because they don't. Compiler size perf 'fixes' are brittle, break easy, and become massive nightmares for programmers who know what they are doing.
@@ -26,7 +26,7 @@ parent_post_id: "1588515348223254528"

 ## Branch

-**1/**
+**1/** @rianflo

@NOTimothyLottes Welcome Not Back! 🤘🏻♥️

@@ -28,7 +28,7 @@ in_reply_to: "1588331007622930432"

 ## Thread

-**1/**
+**1/** @NOTimothyLottes

@rianflo Vulkan: ~1500 lines of overhead (including header src, as I don't include) for complete hotrod bind-everything-only-once compute-only gfx engine. Not a fan of VK bloatware API complexity, but frankly it can be minimized, and exposes enough to do wonderfull things to the hardware.

@@ -25,86 +25,86 @@ in_reply_to: ""

 ## Thread

-**1/**
+**1/** @NOTimothyLottes

 GPU Programming Tip Line Thread /

-**2/**
+**2/** @NOTimothyLottes

 [0] Normalization fail case is 'rsq(0)=INF*0=NaN', trim out intermediate INF to fix, 'normalize_safe(x){return x*min(MAX_FLOAT,rsq(dot(x,x)));}'

-**3/**
+**3/** @NOTimothyLottes

 [1] PC FP16 1/denormals generates INFs which can easily eventually result in NaNs, fix positives with 'rcp(max(x,SMALLEST_NORMAL))'

-**4/**
+**4/** @NOTimothyLottes

 [2] 'spirv-opt -Os' (optimize for size) is your weapon against "back-before-its-done--getting-groceries" IHV compile times

-**5/**
+**5/** @NOTimothyLottes

 [3] Driver ignoring your '[[dont_unroll]]', 'int eatThisBuddy=1+/*hidden-zero*/constantBuffer.zero[0];for(i=0;i<1024;i+=eatThisBuddy){...'

-**6/**
+**6/** @NOTimothyLottes

 [4] 'Ship-One-Shader' = One SPIR-V binary, use specialization constants to select shader at PSO generation time, minimizes released shader binary size

-**7/**
+**7/** @NOTimothyLottes

 [5] 'Ship-One-Shader' requires 'spirv-opt -Os'

-**8/**
+**8/** @NOTimothyLottes

 [6] Use signed 'mask=bitfieldExtract(int(v),bit,1)' to turn bit into all 0's or 1's mask

-**9/**
+**9/** @NOTimothyLottes

 [7] On PC for AMD's native V_BFI_B32 (to select bits based on mask) use 'Bfi(int src,int ins,int mask){return (ins&mask)|(src&(~mask));}'

-**10/**
+**10/** @NOTimothyLottes

 [8] Use 'bitfieldInsert(,,0,compileTimeImmediate)' hits fast V_BFI_B32 on AMD, and portable to other vendors

-**11/**
+**11/** @NOTimothyLottes

 [9] AMD ONLY: clamp(a,b,c) is implemented as med3(a,b,c), so can get V_MED3_* without an extension (in a non-portable way)

-**12/**
+**12/** @NOTimothyLottes

 [10] Bools as 0|1 floats '(a&b)|c' can be done via 'saturate(a*b+c)'

-**13/**
+**13/** @NOTimothyLottes

 [11] Bools as 0|1 floats '!(a&b)' can be done via '(-a)*b+1.0'

-**14/**
+**14/** @NOTimothyLottes

 [12] Convert INFs to NaNs via 'x*0.0+x'

-**15/**
+**15/** @NOTimothyLottes

 [13] Semi-persistent workgroup opt = reusing the workgroup for more work before exit (ie processing four 8x8 tiles in a 16x16 footprint using a 64-wide group)

-**16/**
+**16/** @NOTimothyLottes

 [14] Semi-persistent workgroups can be good for up to 10% perf on AMD (YMMV) ... assuming compiler doesn't fail VGPR allocation, check your disassembly

-**17/**
+**17/** @NOTimothyLottes

 [15] Semi-persistent workgroups gain by keeping more local work on the same L0, by factoring out wait for store on wave exit, and better scheduling

-**18/**
+**18/** @NOTimothyLottes

 [16] Merge passes to avoid round trip through DRAM, often huge wins there

-**19/**
+**19/** @NOTimothyLottes

 [17] Sometimes serial dependent passes can be merged into one shader to keep work in L2 for >10% gains, requires "unsafe you-shouldnt-do-that" logic that works

-**20/**
+**20/** @NOTimothyLottes

 [18] Proper double rate "packed" 16-bit can provides gains up to 30% depending on workload/platform (except NV)

-**21/**
+**21/** @NOTimothyLottes

 [19] Even without double rate 16-bit, 16-bit is the most important tool for managing register pressure problems, esp with smaller HW register limits or compiler troubles
@@ -25,7 +25,7 @@ in_reply_to: ""

 ## Thread

-**1/**
+**1/** @NOTimothyLottes

 Example of "Bind-Everything-Once" Layout Aliasing in Vulkan. Storage image format by aliased name, also bound as sampled image. Choose buffer {readonly,writeonly,volatile (read uncached),atomic} access by aliased name, as either {uint,uint4,float,float4}.

@@ -25,6 +25,6 @@ in_reply_to: "1590003266129383424"

 ## Thread

-**1/**
+**1/** @NOTimothyLottes

@olson_dan If only PC vendors with open ISAs (AMD/Intel) would allow loading binary compute shaders in VK built using non-driver toolchains. If Valve enabled binary shaders on Steam Deck in Linux, that would be my new exclusive dev platform, instead of just a possible porting target
@@ -25,6 +25,6 @@ in_reply_to: ""

 ## Thread

-**1/**
+**1/** @NOTimothyLottes

 My prefered method of managing application data was the 'self-modifying-binary': (a.) on launch copy binary to temp file, (b.) launch temp file, (c.) exit, (d.) now temp file execution can freely modify origional binary :)
@@ -25,22 +25,22 @@ in_reply_to: ""

 ## Thread

-**1/**
+**1/** @NOTimothyLottes

 https://gpuopen.com/learn/rgp-1-15-enhanced-isa-view/ Nice Disasm View :) - However op  latency isn't useful. Need to see multiwave instruction trace across time, colored by instruction type, enhanced with mark if wave is waiting to issue a given functional unit. Only then will great optimization be possible.

-**2/**
+**2/** @NOTimothyLottes

 It would also be best to see overlay of board power (because !/W is the end goal), and an overlay of exactly where there are bubbles in DRAM traffic. Because anything memory bound, the goal is to minimize DRAM bubbles. Need to see how they form dynamically over time to fix ...

-**3/**
+**3/** @NOTimothyLottes

 For GPU profilers, might see say {92% ALU, 50% TEX, 21% L2}, then wonder, where is 8% I'm missing. And only way to start to understand that, is to be seeing how waves are executing instruction wise against each other. CPU-profiler-like averages per disassembly line won't cut it.

-**4/**
+**4/** @NOTimothyLottes

 Parallel machines can only be understood by seeing the parallel execution. I still have examples today of taking a vanila 32-bit shader (already designed to be fast) and getting 30% returns on all micro optimizations.

-**5/**
+**5/** @NOTimothyLottes

 Lastly, side with enabling. Remember people can learn too. And there is typically super skilled people seeded at all companies that teach others how to use advanced tools when they are available.
@@ -26,11 +26,11 @@ parent_post_id: "1653556990302470151"

 ## Branch

-**1/**
+**1/** @BrianKaris

@NOTimothyLottes Great to see these return. Still have any of the videos?

-**2/**
+**2/** @NOTimothyLottes

@BrianKaris I might, but will have to build a machine to attach the old drives. Too busy making new stuff

@@ -26,7 +26,7 @@ parent_post_id: "1653556990302470151"

 ## Branch

-**1/**
+**1/** @BrookeHodgman

@NOTimothyLottes I remember watching all of these blog entries fondly ❤

@@ -26,15 +26,15 @@ parent_post_id: "1653556990302470151"

 ## Branch

-**1/**
+**1/** @Karyuutensei

@NOTimothyLottes Is the blog still up somewhere?

-**2/**
+**2/** @NOTimothyLottes

@Karyuutensei No but I have archives in pieces on old hard drives. So there is a chance I might restore it some day.

-**3/**
+**3/** @Karyuutensei

@NOTimothyLottes Speaking of archives, http://archive.org  has some from the blogspot here https://web.archive.org/web/20160427173127/http://timothylottes.blogspot.com/ but basically nothing from the github page :S

@@ -26,11 +26,11 @@ parent_post_id: "1653556990302470151"

 ## Branch

-**1/**
+**1/** @oconnorkg

@NOTimothyLottes Did you write an engine called Atom? Talked about it on the http://OpenGL.org forum?

-**2/**
+**2/** @NOTimothyLottes

@oconnorkg There was also Atomontage or something like that by another person. I had a project called Atom, but I don't remember if I was on the OGL message board

@@ -26,7 +26,7 @@ parent_post_id: "1653556990302470151"

 ## Branch

-**1/**
+**1/** @won3d

@NOTimothyLottes I remember all those fondly. I also remembered the trick you did to batch logical draw calls with branching in the shader

@@ -26,15 +26,15 @@ parent_post_id: "1653559205360553985"

 ## Branch

-**1/**
+**1/** @won3d

@NOTimothyLottes My former thesis advisor was a big proponent of forth. Apparently SunOS used forth in preboot and you could do all sorts of wacky things. He interfaced with custom compute accelerators this way. I agree that concatenantive programming languages are underexplored

-**2/**
+**2/** @NOTimothyLottes

@won3d I think it is less the exact form of the language but more 'the way to make things simple' that truely important now.

-**3/**
+**3/** @won3d

@NOTimothyLottes There's something about Forth-style composition that is has a very nice appeal. It really seems to marry syntax and semantics very directly (but I can appreciate it at a distance, having not done any real work in it as you have)

@@ -26,7 +26,7 @@ parent_post_id: "1653570742762479620"

 ## Branch

-**1/**
+**1/** @Laxer3A

@NOTimothyLottes This is when I learned about your work and found it incredible. Nowadays with 4K / 8K, people are really craving for a very good shader/model to emulate as close as possible the CRT feel for arcade / old console etc. (I am serious)

@@ -26,7 +26,7 @@ parent_post_id: "1653556990302470151"

 ## Branch

-**1/**
+**1/** @arm1n

@NOTimothyLottes Thanks for digging those up! Great stuff, very inspiring.

@@ -25,7 +25,7 @@ in_reply_to: ""

 ## Thread

-**1/**
+**1/** @NOTimothyLottes

 Found some of my old blogs: thread looking back an image at a time. 2007, ellipsoid l-system sprites with temporal feedback, yes temporal reprojection in 2007.

@@ -33,13 +33,13 @@ Found some of my old blogs: thread looking back an image at a time. 2007, ellips

 Branches: [[archive/threads/NOTimothyLottes/2023-05-03-found-some-of-my-old-blogs-thread-looking-back-an/2023-05-03-won3d-i-remember-all-those-fondly-i-also-remembered-the]], [[archive/threads/NOTimothyLottes/2023-05-03-found-some-of-my-old-blogs-thread-looking-back-an/2023-05-03-BrianKaris-great-to-see-these-return-still-have-any-of-the]], [[archive/threads/NOTimothyLottes/2023-05-03-found-some-of-my-old-blogs-thread-looking-back-an/2023-05-03-Karyuutensei-is-the-blog-still-up-somewhere]], [[archive/threads/NOTimothyLottes/2023-05-03-found-some-of-my-old-blogs-thread-looking-back-an/2023-05-03-BrookeHodgman-i-remember-watching-all-of-these-blog-entries]], [[archive/threads/NOTimothyLottes/2023-05-03-found-some-of-my-old-blogs-thread-looking-back-an/2023-05-03-oconnorkg-did-you-write-an-engine-called-atom-talked-about]], [[archive/threads/NOTimothyLottes/2023-05-03-found-some-of-my-old-blogs-thread-looking-back-an/2023-05-06-arm1n-thanks-for-digging-those-up-great-stuff-very]]

-**2/**
+**2/** @NOTimothyLottes

 2007 particle based fluid sim mixed with spring model, this was 2D to work out ideas for later 3D implementation, rastered particle properties along particle paths into mip tree to enable high-speed collision

 ![](https://pbs.twimg.com/media/FvKd2pZWcAArMRq?format=jpg&name=orig)

-**3/**
+**3/** @NOTimothyLottes

 2007 crazy custom language and live editor for x86-64, color forth is like a virus of the mind, no going back once the understanding is there, and the evolutions of those ideas are ground breaking

@@ -47,7 +47,7 @@ Branches: [[archive/threads/NOTimothyLottes/2023-05-03-found-some-of-my-old-blog

 Branches: [[archive/threads/NOTimothyLottes/2023-05-03-found-some-of-my-old-blogs-thread-looking-back-an/2023-05-03-won3d-my-former-thesis-advisor-was-a-big-proponent-of]]

-**4/**
+**4/** @NOTimothyLottes

 2010 digital photography, was compositing exposure brackets to get 16-bit tiffs with good dynamic range and did my own development tools (which had some major flaws I'd only learn about much later in life)

@@ -56,51 +56,51 @@ Branches: [[archive/threads/NOTimothyLottes/2023-05-03-found-some-of-my-old-blog
 ![](https://pbs.twimg.com/media/FvKgJSwWIAIUavB?format=jpg&name=orig)
 ![](https://pbs.twimg.com/media/FvKgU0NXoAAaXK4?format=jpg&name=orig)

-**5/**
+**5/** @NOTimothyLottes

 2011 Was using temporal reprojection in my own engines for a while. Posted TSSAA source, an early TAA using viewport jitter for tri raster. I was mostly convinved this was a tech dead end due to the challenges of thin flicker. Decade+ later, and this story has a lot more to it ..

 ![](https://pbs.twimg.com/media/FvKhxrpXgAEmVaS?format=png&name=orig)
 ![](https://pbs.twimg.com/media/FvKhzBoXgAABF7I?format=png&name=orig)

-**6/**
+**6/** @NOTimothyLottes

 2011 A simple GIF showing the challenges of {perceptual vs linear} filtering with temporal aliasing high frequency content with {small to large kernel windows}. No negative lobes here.

 <video controls src="https://video.twimg.com/tweet_video/FvKi92TWcAA19un.mp4"></video>

-**7/**
+**7/** @NOTimothyLottes

 2014 simple anti-aliased point based CS rendering engine with fisheye output using stocastic/stratified visiblity and GPU-side scene graph

 ![](https://pbs.twimg.com/media/FvKj76BXwAEDBNO?format=png&name=orig)

-**8/**
+**8/** @NOTimothyLottes

 2014 simple non-temporal spatial AA using rotated viewport to provide a good sample distribution with standard non-AA triangle rendering

 ![](https://pbs.twimg.com/media/FvKkiT4WAAU2Z3a?format=jpg&name=orig)

-**9/**
+**9/** @NOTimothyLottes

 2014 monochrome 120 hz laptop ray marcher desiged for VR, fixed cost trace, leaving scene with holes, temporal holefilling reprojection, faked 'lighting' using hierarchical + directional distance probing in lower LOD scene representations, was amazing to just fly around in there

 ![](https://pbs.twimg.com/media/FvKk3NoXwAApx-w?format=png&name=orig)
 ![](https://pbs.twimg.com/media/FvKk42ZXoAInjtk?format=png&name=orig)

-**10/**
+**10/** @NOTimothyLottes

 2014 was amazing how nice it was possible to get the geometry even when tracing only 1/8 the screen pixels per frame, stratified sample pattern really does wonders to neighborhood stability, no flicker at all

 ![](https://pbs.twimg.com/media/FvKltaSWcAMg5ke?format=jpg&name=orig)

-**11/**
+**11/** @NOTimothyLottes

 2014 still distracted by custom code generation and languages, running stuff on CRTs with custom live editors (color forth like), generating binaries with ELF header generation inside the code itself

 ![](https://pbs.twimg.com/media/FvKmJ1aXgAI9Qt9?format=png&name=orig)

-**12/**
+**12/** @NOTimothyLottes

 2015 some more images inside the live fractal tracer

@@ -108,13 +108,13 @@ Branches: [[archive/threads/NOTimothyLottes/2023-05-03-found-some-of-my-old-blog
 ![](https://pbs.twimg.com/media/FvKmtisXsAIC4v-?format=jpg&name=orig)
 ![](https://pbs.twimg.com/media/FvKmu0XXsAEP-PW?format=jpg&name=orig)

-**13/**
+**13/** @NOTimothyLottes

 2015 crazy mini x86 OS project (bootloader source below), using codeless binary editor with annotation. Everyone should try generating binaries with no assembler, learn alot in the process

 ![](https://pbs.twimg.com/media/FvKnKcEXoAA-pjq?format=png&name=orig)

-**14/**
+**14/** @NOTimothyLottes

 2015 live realtime 120+ hz on laptop tracer now with spatial/temporal reconstruction on 1 shaded/sample per ray for 'fake' lit fog, did a lot of z based logic in the spatial/temporal reconstruction, realized later I was relying on the 120 Hz to hide perceptual artifacts

@@ -123,14 +123,14 @@ Branches: [[archive/threads/NOTimothyLottes/2023-05-03-found-some-of-my-old-blog
 ![](https://pbs.twimg.com/media/FvKoL0ZXoAAkwWj?format=jpg&name=orig)
 ![](https://pbs.twimg.com/media/FvKoNDuXgAAbXBM?format=jpg&name=orig)

-**15/**
+**15/** @NOTimothyLottes

 2015 still doing the crazy custom languages and tiny OS projects. I feel like there is a real loss in computer evolution, imagine a machine with C64 simplicity that evolved to have a massively parallel vector machine. It would be amazing. It would be approachable by one person.

 ![](https://pbs.twimg.com/media/FvKotj4XwAEO-pU?format=png&name=orig)
 ![](https://pbs.twimg.com/media/FvKovAaWcAADk-H?format=png&name=orig)

-**16/**
+**16/** @NOTimothyLottes

 2015 did some custom CPU-side CRT shader tools for high resolution physical paper prints for someone's book. Wasn't trying to be physically accurate, but rather provide the feeling you'd get from a CRT in paper print form

@@ -26,69 +26,69 @@ parent_post_id: "1671276990987370501"

 ## Branch

-**1/**
+**1/** @NOTimothyLottes

 Or for an APU, a way to have the storage device write direct to DRAM in pages setup for use on the GPU, because anything the CPU writes that the GPU reads is stuck on the strangled snooping bus or the strangled WC buffer limit on store.

-**2/**
+**2/** @NOTimothyLottes

 If the app tried to manage GPU memory oversubscription via these strangled CPU mappings you'd literally see many second stalls. One can only assume the kernel can do storage<->DRAM DMA for managing GPU DRAM oversubscription?

-**3/**
+**3/** @NOTimothyLottes

 Even if could fill VRAM in 1 sec, just 2x oversubscription of VRAM implies 1 sec delay to context switch (assuming 2 apps accessing all meM). So classic VM multitasking is useless. Only single app focus model with pinned memory makes any useful sense. Release pin on focus change.

-**4/**
+**4/** @NOTimothyLottes

 The trend is massive res massive VRAM paired with tiny IO bus. So the case for pinned memory and bus master transfers only grows importance as things evolve.

-**5/**
+**5/** @NOTimothyLottes

 Back to SteamDeck numbers: GTT (HOST_VISIBLE) + USWC (non-HOST_CACHED) takes 1 sec for 1st 4 GiB BO alloc, but *30 sec* for 2nd 4 GiB alloc. Kernel driver time for memory allocation (maybe page table related) can be brutal (general comment for PCs too).

-**6/**
+**6/** @NOTimothyLottes

 Related, I think Chips&Cheese CPU/GPU link DMA and Compute Bandwidths are only measured to a GTT+USWC buffer (only supporting useless uncached CPU reads). Bandwidth exceeds the CPU's bus capacity, implies it is 'garlic' or GPU bus only accesses, direct to DRAM ...

 ![](https://pbs.twimg.com/media/FzHOnZ_X0AEW7CE?format=png&name=orig)

-**7/**
+**7/** @NOTimothyLottes

 Since RADV+AMDVLK don't support user-space CPU mapped flush | invalidate, this implies the only supported mapping to read from DRAM direct is USWC (uncached R and write+combine W) ...

-**8/**
+**8/** @NOTimothyLottes

 Doing GPU stores to a CPU mapped GTT without USWC would be crippingly slow (limited by the snooping bus rate). But this is unfortunately the only option available for CPU read back. So if doing a shader store, it better be only a few waves and running in parallel.

-**9/**
+**9/** @NOTimothyLottes

 "Use GTT because it's as fast as VRAM on the Deck", could only work if GTT+USWC, as that would be only way to get Garlic (direct to DRAM high bandwidth bus). GTT without USWC would need Onion (slow snooping bus) because AMDgpu's only no-CPU map option is for VRAM!

 ![](https://pbs.twimg.com/media/FzHTDxCX0AADoM1?format=png&name=orig)

-**10/**
+**10/** @NOTimothyLottes

 Summary of the theory on best Steam Deck practices. This is the plan for my deck-compute-only driver too. Theory -> as in I haven't yet verified the GPU-side parts (my driver isn't that far along yet).

 ![](https://pbs.twimg.com/media/FzHZ_ZiWwAAdEkd?format=png&name=orig)

-**11/**
+**11/** @NOTimothyLottes

 Possible to do better? MAYBE! CPU readback actually has 2 problems, 1st the slow GPU-side copy (4 GiB copy via snooping bus could be almost 4 sec, but direct to DRAM via USWC might be just 160 ms). Also CPU only has 4 MiB of L3, so the majority of 4 GiB will be uncached later ...

-**12/**
+**12/** @NOTimothyLottes

 Believe UC MTYPE (uncached) forces the CPU into serialized behavior. My test was single thread 8-byte/access reads. https://uops.info/html-instr/VMOVDQA_M256_YMM.html#ZEN2 Looks like Zen2 might be able to get 32-byte/access via VMOVDQA, and going multi-threaded (8 thread), that might be a 32x speed up ..

-**13/**
+**13/** @NOTimothyLottes

 If so might be able to approach under a GB/s for the CPU-side part (UC multithread via VMOVDQA), which would be close enough to the non-USWC running single threaded using the cache.

-**14/**
+**14/** @NOTimothyLottes

 What you'd really want here as a band-aid workaround is ability for the GPU to act as if the CPU map was USWC (so go direct to DRAM), but have the CPU map act as non-USWC, so it goes through the cache. Then some kind of hack to flush the tiny 4 MiB L3 and lower caches on the CPU.

-**15/**
+**15/** @NOTimothyLottes

 CPU readback (reading and summing 8-byte) GTT without USWC:

@@ -98,7 +98,7 @@ CPU readback (reading and summing 8-byte) GTT without USWC:

 Going multi-core on cached readback doesn't really help much. Proper test, parked threads waiting on futex, signal, last-1st active core timing.

-**16/**
+**16/** @NOTimothyLottes

 Now CPU readback GTT+USWC

@@ -108,11 +108,11 @@ Now CPU readback GTT+USWC

 So measurements match theory, going multi-core with MOVNTDQA uncached read on GTT+USWC can be made to match 1 thread GTT (without USWC)

-**17/**
+**17/** @NOTimothyLottes

 Both those results above had been using a pair of 4 GiB allocations. The GTT+USWC one used one 4 GiB GTT+USWC for timing, and one 4 GiB GTT (unused). And that test suffered from a 30 sec GTT BO allocation. So something was going very wrong in the page mapping ...

-**18/**
+**18/** @NOTimothyLottes

 When I rerun same test with just one 4 GiB GTT+USWC allocation, the 30 sec stall is gone, and the performance also changes:

@@ -122,38 +122,38 @@ Oh!

 Perhaps there is some resource limit that kills perf if too much memory gets mapped, page faults?

-**19/**
+**19/** @NOTimothyLottes

 Top with thread cumulative results doesn't show anything significantly different between the 4 GiB and 8 GiB runs in terms of page faults ... suggests it must be something else

 ![](https://pbs.twimg.com/media/FzJ9YrHXsAIfCOg?format=jpg&name=orig)
 ![](https://pbs.twimg.com/media/FzJ9Y5LWwAYyyjt?format=jpg&name=orig)

-**20/**
+**20/** @NOTimothyLottes

 And yet there is obviously a bug in my multi-core tests, you can tell directly from the page fault numbers, only one thread is taking all the faults. So it is back to finding my coding error (fail).

-**21/**
+**21/** @NOTimothyLottes

 Lunch break and fixed the bug. Two runs now and leaving threads open to get TOP results. First run definitely soaks up the page faults, second run is page fault free (expected). Both around only 7 GB/s.

 ![](https://pbs.twimg.com/media/FzKPZxUWABQ6oQJ?format=jpg&name=orig)

-**22/**
+**22/** @NOTimothyLottes

 And the 8 GiB of BO mapped, but only 4 GiB used run. The first pass gets only 1 GiB/s and the second gets 7 GiB/s. Page fault number is similar to last run, can only conclude page fault costs exploded?

 ![](https://pbs.twimg.com/media/FzKRF1OXsAE7c0-?format=jpg&name=orig)

-**23/**
+**23/** @NOTimothyLottes

 30 sec BO alloc time + super low bandwidth on 1st pass only (where page faults happen) suggest that Linux Kernel logic explodes in cost if too many pages are used in this way ...

-**24/**
+**24/** @NOTimothyLottes

 ~8K faults for 512 MiB accessed / thread = 64 KiB/fault ... X86-64 has either 4 KiB or 2 MiB for page size. So not using large pages (fail). Probably mapping 16 pages per fault. Not sure if this implies anything about GPU page size (but certainly hoping it isn't 4 KiB, ouch).

-**25/**
+**25/** @NOTimothyLottes

 Some other very rough measured numbers of GTT+USWC with 8 cores splitting 4 GiB of BO.

@@ -163,11 +163,11 @@ Some other very rough measured numbers of GTT+USWC with 8 cores splitting 4 GiB

 I think these seem plausable now (so maybe no more code bugs).

-**26/**
+**26/** @NOTimothyLottes

 One takeaways of all this, is that you need to pre-warm the page tables for large mapped buffers (by touching all pages) when the user isn't waiting on results. And if you are doing batch jobs that {open device, send data to GPU, get data back, close device} you are screwed!

-**27/**
+**27/** @NOTimothyLottes

 And lastly (maybe) comparison of GTT and GTT+USWC both 8 threads splitting streaming through 4 GiB of mapping (second pass, no page fault issues):

@@ -175,55 +175,55 @@ R> ~14 GiB/s (GTT) and ~7 GiB/s (GTT+USWC)
 W+R> ~14 GiB/s (GTT) and ~10 GiB/s (GTT+USWC)
 W> ~13 GiB/s (GTT) and ~15 GiB/s (GTT+USWC)

-**28/**
+**28/** @NOTimothyLottes

 So an alternative option summary for those who don't want the extra GPU-side GPU to CPU mapped buffer copy step.

 ![](https://pbs.twimg.com/media/FzKlE0NXoAI74IA?format=png&name=orig)

-**29/**
+**29/** @NOTimothyLottes

 If anyone is looking to repro the 30 sec stall: amdgpu_bo_alloc() one 4 GiB GTT+USWC, then one 4 GiB GTT buffer, the second alloc causes the Deck to become unresponsive for 30 seconds

-**30/**
+**30/** @NOTimothyLottes

 madvise() with MADV_HUGEPAGE on mapped 4 GiB region doesn't do anything (still faults at 64 KiB granularity), and none of these MADV_{WILLNEED|POPULATE_READ|POPULATE_WRITE} have any effect either (still waits until use before faulting, causing low initial effective bandwidth)

-**31/**
+**31/** @NOTimothyLottes

 64KiB strided write through 4 GiB GTT+USWC (to pre-fault) costs the same as writing full 4 GiB, roughly 3 seconds. So it is quite literally massive page fault overhead for 1st access. No possible workaround found at this time for initial load time problems.

-**32/**
+**32/** @NOTimothyLottes

 If doing two 4 GiB GTT+USWC allocations, there is also a 30 second BO allocate cost on the 2nd one. And this makes the initial page fault cost for access to the first 4 GiB take another 30 seconds. Effectively hangs the machine for a full minute.

-**33/**
+**33/** @NOTimothyLottes

 Doing two 4 GiB GTT allocations (without USWC), doesn't incure any of the 30 second stalls. So that problem is specific to big USWC allocations. However the initial access page fault problem (extra 3 seconds) is there, so a 2ndary problem with just mapping lots of APU memory.

-**34/**
+**34/** @NOTimothyLottes

 Allocation of one 4 GiB GTT first then a 4 GiB GTT+USWC doesn't see the 30 second allocation stall. Almost like anything post a big USWC alloc is poisioned. And after mapping both, then accessing the GTT only, the 30 second time for page faulting comes back.

-**35/**
+**35/** @NOTimothyLottes

 Continuing from before, even if you don't map the 2nd GTT+USWC, the 30 second initial page faulting time is still there, so the act of mapping doesn't matter, simply doing the BO allocation had already doomed the Linux page management.

-**36/**
+**36/** @NOTimothyLottes

 Despite header docs which imply flag only works on DOMAIN_VRAM, using DOMAIN_GTT+AMDGPU_GEM_CREATE_NO_CPU_ACCESS is apparently what you want for non-mapped GTT allocations. Mapped 4 GiB GTT with 4 GiB GTT+NO_CPU, drops the initial page fault time from 30 seconds to 3 sec.

 ![](https://pbs.twimg.com/media/FzOqwUfXsAEsmC9?format=png&name=orig)

-**37/**
+**37/** @DaveAirlie

@NOTimothyLottes the problem there is usually tiling, GPUs expect graphicy things in VRAM to be tiled in very specific ways, it's hard to have a storage device do that. Using a transfer engine that can tile/detile makes a lot more sense.

-**38/**
+**38/** @NOTimothyLottes

@DaveAirlie The obvious elephant is an engine using only buffers for resource streaming and doing its own GPU-side decompression. This is practical NOW and portable too. But the lack of a bus master no-CPU direct storage transfer is a serious problem for throughput.

-**39/**
+**39/** @NOTimothyLottes

@DaveAirlie See raw texture streaming (even with lossless compression) is way less interesting now given modern massive data tech direction, u want substantially better lossy compression simply to fit in launch constraints. So really, buffers are the way to go anyway :)

@@ -26,7 +26,7 @@ parent_post_id: "1671276990987370501"

 ## Branch

-**1/**
+**1/** @SheriefFYI

@NOTimothyLottes Great thread!

@@ -26,7 +26,7 @@ parent_post_id: "1671276990987370501"

 ## Branch

-**1/**
+**1/** @never_released

@NOTimothyLottes On Linux, NVIDIA has a very good UVM.

@@ -25,55 +25,55 @@ in_reply_to: ""

 ## Thread

-**1/**
+**1/** @NOTimothyLottes

 How about a mega thread on the mechanics of CPU/GPU communication. Using AMDgpu based timing  results on the SteamDeck an example, but relating to the larger picture of PC GFX APIs like Vulkan/etc. I don't claim to know all, so experts feel free to add your wisdom :)

 ![](https://pbs.twimg.com/media/FzGF4owWwAMsc2I?format=png&name=orig)

-**2/**
+**2/** @NOTimothyLottes

 Both RADV and AMDVLK: Flush/invalidate mapped memory ranges is a NOP. So bus-crossing dGPU traffic to HOST_VISIBLE is automatically snooping CPU caches. The one without HOST_CACHED, is Write+Combined [WC] on store, and Uncached [UC] on read.  The one with HOST_CACHED is non-WC/UC

 ![](https://pbs.twimg.com/media/FzGG4OnWwAEu19H?format=png&name=orig)

-**3/**
+**3/** @NOTimothyLottes

 In AMDgpu (the kernel driver), likely DEVICE_LOCAL maps to AMDGPU_GEM_DOMAIN_VRAM (also the carve out on APUs) and the non-DEVICE_LOCAL maps to AMDGPU_GEM_DOMAIN_GTT

 ![](https://pbs.twimg.com/media/FzGJkkFWwAEBowR?format=png&name=orig)

-**4/**
+**4/** @NOTimothyLottes

 AMD+RADV added {DEVICE_COHERENT_BIT_AMD,
 DEVICE_UNCACHED_BIT_AMD} variations to the core 4 memory types. Likely to support GPU crash debug. But also provides a way to avoid needing to write-back (flush) GPU caches before CPU read. Likely AMDgpu kernel flag mapping below.

 ![](https://pbs.twimg.com/media/FzGKZ_bWcAE3cGh?format=png&name=orig)

-**5/**
+**5/** @NOTimothyLottes

 This AMDGPU_GEM_CREATE_CPU_GTT_USWC appear to toggle on WriteCombine [WC] for CPU store, and Uncached [US] for CPU reads (cases of HOST_VISIBLE without HOST_CACHED)

 ![](https://pbs.twimg.com/media/FzGL03LXgAEeO0K?format=png&name=orig)

-**6/**
+**6/** @NOTimothyLottes

 For review from https://chipsandcheese.com/2023/03/05/van-gogh-amds-steam-deck-apu/ Deck bandwidths: ~71 GB/s GPU, ~43 GB/s DMA, ~34 GB/s shader copy CPU<->GPU, ~25 GB/s CPU/CPU, and damn, brutal 0.27 GB/s CPU mapped GPU buffer reads, 0.71 GB/s CPU mapped GPU buffer writes

 ![](https://pbs.twimg.com/media/FzGNGBuWAAA6wlr?format=png&name=orig)
 ![](https://pbs.twimg.com/media/FzGNlFvWwAANhH4?format=png&name=orig)

-**7/**
+**7/** @NOTimothyLottes

 And going direct to AMDgpu instead of VK on the Deck shows these kinds of bandwidths (non-DEVICE_LOCAL,  HOST_VISIBLE with HOST_CACHED and without). So using Write-Combined is amazingly painful for stores.

 ![](https://pbs.twimg.com/media/FzGO3XdWIAAQl3V?format=png&name=orig)

-**8/**
+**8/** @NOTimothyLottes

 Implies that the choices one might make on dGPU PC don't necessarily port over to APUs at all. Another challenge: it takes almost 7 seconds to zero-fill using a 64-bit store for() loop the 8-GiB of mapped memory. Hints at why load times are such a challenge even in the best case.

-**9/**
+**9/** @NOTimothyLottes

 This all hints at why PC OS derived systems are lacking in stuffing GPU VRAM. Really need some kind of bus mastered DMA (zero-copy) between non-volatile storage (disk) and GPU DRAM to avoid this CPU-touching performance tax.

@@ -26,13 +26,13 @@ parent_post_id: "1674757854471806977"

 ## Branch

-**1/**
+**1/** @SebAaltonen

@NOTimothyLottes Would be nice to talk a full presentation about this topic, but people want me to show cool tech and pretty pixels instead. Maybe one day.

 I will save your image for that day (if I get approval to use it), since Twitter search is super bad.

-**2/**
+**2/** @NOTimothyLottes

@SebAaltonen "Approval", haha, sure it's covered under my favorite license

@@ -26,12 +26,12 @@ parent_post_id: "1674757854471806977"

 ## Branch

-**1/**
+**1/** @_spolsh

@NOTimothyLottes Once in a while people need to be reminded to stop the bloat, understand the machine guts and be self aware about productive process
 https://tonsky.me/blog/disenchantment/

-**2/**
+**2/** @NOTimothyLottes

@_spolsh That was a fantastic read, thanks

@@ -26,7 +26,7 @@ parent_post_id: "1674757854471806977"

 ## Branch

-**1/**
+**1/** @domipheus

@NOTimothyLottes I like this, given I never write code any more.
 sadly i am still a grumpy git.
@@ -26,12 +26,12 @@ parent_post_id: "1674757854471806977"

 ## Branch

-**1/**
+**1/** @leonard_coder

@NOTimothyLottes $4e71
 if you know, you know

-**2/**
+**2/** @NOTimothyLottes

@leonard_coder ${EA, 1A, 3A, 5A, 7A, DA, or FA}

@@ -26,7 +26,7 @@ parent_post_id: "1674757854471806977"

 ## Branch

-**1/**
+**1/** @CodingFiend

@NOTimothyLottes @Jonathan_Blow i designed a language Beads, which uses a graph DB, has no memory allocation, super modular, approaches this minimalism as close as humanly possible, but billable hour / job security seeking programmers refuse to try it

@@ -26,7 +26,7 @@ parent_post_id: "1674757854471806977"

 ## Branch

-**1/**
+**1/** @DefenceForceOrg

@NOTimothyLottes Many games used to be like that, at least the ones without dynamic content, mods, etc...

@@ -26,7 +26,7 @@ parent_post_id: "1674757854471806977"

 ## Branch

-**1/**
+**1/** @PierreTerdiman

@NOTimothyLottes I was calling it "disoriented programming" just the other day.

@@ -26,7 +26,7 @@ parent_post_id: "1674757854471806977"

 ## Branch

-**1/**
+**1/** @SimonNordon

@NOTimothyLottes NOP is ideal.

@@ -26,7 +26,7 @@ parent_post_id: "1674757854471806977"

 ## Branch

-**1/**
+**1/** @TimSweeneyEpic

@NOTimothyLottes Can teach it in abstinence-only programming classes!

@@ -26,7 +26,7 @@ parent_post_id: "1674757854471806977"

 ## Branch

-**1/**
+**1/** @asparagiii

@NOTimothyLottes Looks like a koala

@@ -26,7 +26,7 @@ parent_post_id: "1674757854471806977"

 ## Branch

-**1/**
+**1/** @azeke1984

@NOTimothyLottes Best software is absense of it:

@@ -26,7 +26,7 @@ parent_post_id: "1674757854471806977"

 ## Branch

-**1/**
+**1/** @claudiogonzalo

@NOTimothyLottes c++ to √c

@@ -26,7 +26,7 @@ parent_post_id: "1674757854471806977"

 ## Branch

-**1/**
+**1/** @helpful_signage

@NOTimothyLottes Folks, they're calling it "computer oriented programming". It's where you stop trying to impress peers and just tell the computer the best way to do things

@@ -26,7 +26,7 @@ parent_post_id: "1674757854471806977"

 ## Branch

-**1/**
+**1/** @mike_acton

@NOTimothyLottes

@@ -26,7 +26,7 @@ parent_post_id: "1674757854471806977"

 ## Branch

-**1/**
+**1/** @wyattscarpenter

@NOTimothyLottes https://github.com/wyattscarpenter/wu

@@ -25,7 +25,7 @@ in_reply_to: ""

 ## Thread

-**1/**
+**1/** @NOTimothyLottes

 "Nothing Oriented Programming" (NOP). Where the goal is to achieve minimal enlightenment by writing as little possible code: no dynamic memory allocation, no thousand files of glue, no includes, no objects. Write nothing, be happy. Even applied it to the logo, zero effort.

@@ -25,38 +25,38 @@ in_reply_to: ""

 ## Thread

-**1/**
+**1/** @NOTimothyLottes

 SPC: Left off with KEY and GAMEPAD input GPU-accessable. NOP design, no functions, no API, just a fixed VA address and data format to read, dead easy.

 ![](https://pbs.twimg.com/media/F2pRdu9WgAABCLE?format=jpg&name=orig)

 ![](https://pbs.twimg.com/media/F2pReKsXoAAPvow?format=jpg&name=orig)
-**2/**
+**2/** @NOTimothyLottes

 Wrote some console data dumping to aid in bringing up HID parsers for Deck and three generations of PlayStation controllers. Apparently XBox controllers aren't HIDs so not sure how one is supposed to access those.

 ![](https://pbs.twimg.com/media/F2pSfIHXcAMI-IQ?format=jpg&name=orig)

 ![](https://pbs.twimg.com/media/F2pSfzAWcAMo6t7?format=jpg&name=orig)
-**3/**
+**3/** @NOTimothyLottes

 Supporting hot plug was a bit of a mess, just trying to open devices every second. Adding open devices to an epoll. {Product, vendor} isn't enough to ID a device, too much aliasing, so I use packet size as well.

 ![](https://pbs.twimg.com/media/F2pTfekWYAAAwSi?format=jpg&name=orig)
-**4/**
+**4/** @NOTimothyLottes

 Open devices with matching packet size get set in a bit array, and I collect up to 4 of those device outputs to sent to the GPU. Special logic to always place Steam Deck controls last to make docking work well.

 ![](https://pbs.twimg.com/media/F2pUEG0XwAEwYvP?format=jpg&name=orig)
-**5/**
+**5/** @NOTimothyLottes

 I'll have to tackle audio at some point, but I'm skipping networking for version 1. But otherwise SPC is good to move on to the next step, which is to build out the GPU side assembler & data editor. Which will be written in binary at first.

-**6/**
+**6/** @NOTimothyLottes

 Project is roughly 2000 lines of C at this point. With no includes. Anyone have any suggestions for hosting let me know. Source will be released.

-**7/**
+**7/** @NOTimothyLottes

 For anyone tuning in now, SPC is kind of like a Pico8, but instead of being a portable fantasy console, SPC turns the Steam Deck into an actual console for exclusive GPU assembly projects. The way consoles had been done before silly portability became the only focus.
@@ -25,26 +25,26 @@ in_reply_to: ""

 ## Thread

-**1/**
+**1/** @NOTimothyLottes

 SPC: Using 8 deep swap all aliasing same BO (image) fixes Game Mode crash with external display. Pipelined load gets to black screen in 2 ms. Takes 460 ms to allocate 4 GiB GTT video ram, 558 ms to copy 256 MiB cart from page cache to USWC (page faults), all in parallel.

 ![](https://pbs.twimg.com/media/F2mmqYXWoAAG3fj?format=jpg&name=orig)

-**2/**
+**2/** @NOTimothyLottes

 Clearing page cache to simulate a cold launch: Takes 1.5 sec to read 256 MiB from the stock SSD. This is worst case load time in desktop mode. Won't be any compile/etc after this. So literally in game at this point. Speed of light. This is how it is done.

-**3/**
+**3/** @NOTimothyLottes

 Forgot the timing capture ... Load from SSD.

 ![](https://pbs.twimg.com/media/F2mpszoXMAE00Hy?format=jpg&name=orig)

-**4/**
+**4/** @NOTimothyLottes

 There are only 3 threads on load: (1.) to open X11 then spin doing present/keyboard, (2.) to open the AMDgpu device, SDMA copy after CART read, and then eventually spin doing dispatchs, (3.) one to load the CART from disk to mapped USWC GTT via read().

-**5/**
+**5/** @NOTimothyLottes

 Something I don't know: does read() since it's kernel- side automatically get some multi-core parallelism? Not a good idea to manually thread the read() because SSD needs linear reads to be fast. Not possible to know if the reads are page cache hits in advance either ...
@@ -25,22 +25,22 @@ in_reply_to: ""

 ## Thread

-**1/**
+**1/** @NOTimothyLottes

 1st SPC project: to be based on vintage 640x480 render (VGA style), +fallback to 640x240 for 15 KHz arcade CRTs (PGM-esk), and CRT shader scaling on Deck and OLED/LCD. Use mix of render pixel aspect changes, running less than 480 lines, and letter-box to keep integer Y scaling.

 ![](https://pbs.twimg.com/media/F20LgFtWQAAMtcv?format=png&name=orig)

-**2/**
+**2/** @NOTimothyLottes

 My 15 KHz CRT output path when docked is HD Fury Nano (won't do <480p) doing HDMI to VGA, with a sync combiner (to workaround GBS-C bug), into GBS-C which line drops 480p ~60Hz to 240p ~60Hz and outputs component. The 640 width gets maintained even if some CRTs cannot resolve it.

-**3/**
+**3/** @NOTimothyLottes

 Presents issues for UI, namely that my bitmap fonts need to look good in both 480p and 240p modes, so for editor (assembler) I'm going with a font that uses right angles and designed around 2x2 stroke width, in a 8x16 (pow2) bitmap, so 640x240p still gets a readable 8x8 (below)

 ![](https://pbs.twimg.com/media/F20Ogh8XoAAq7hR?format=png&name=orig)

-**4/**
+**4/** @NOTimothyLottes

 The next challenge is overscan safety, since real non-VGA CRTs will be used in development. The Y axis has an easy fix, simply keep cursor pinned in the 'safe' zone, but still print lines in the overscan area (so they can be seen when attached to non-overscan CRTs/etc too).
@@ -25,75 +25,75 @@ in_reply_to: ""

 ## Thread

-**1/**
+**1/** @NOTimothyLottes

 SPC: RDNA2 ASSEMBLER [thread] ... Did initial design plan this weekend. The aim is a GPU-side assembler+editor, building RDNA2 binary images for Steam Deck. Live edit, live data view, etc. Assembler and editor running in massive vectorized shader code.

-**2/**
+**2/** @NOTimothyLottes

 Good reason not to use text input for assembler, instead will use a custom binary format, and treat all 32-bit words in the binary the exact same way (so massively parallel assembly step). This is parallel design 101.

-**3/**
+**3/** @NOTimothyLottes

 Some basics: RDNA2 instructions are 1-N 32-bit words. Each of the 32-bit components have a bunch of fields. Will typically need 1-6 fields per 32-bit word, as will be joining some fields into one argument (like DLC/GLC) below.

 ![](https://pbs.twimg.com/media/F232wWkX0AAZWPh?format=jpg&name=orig)
 ![](https://pbs.twimg.com/media/F233F_sWMAAqHBo?format=png&name=orig)

-**4/**
+**4/** @NOTimothyLottes

 Could argue that MIMG has the most fields, but in practice many of those fields will be joined (see example), and I'm not likely to be using NSA encoding. So still will only need maximum 6 arguments to build a 32-bit value.

 ![](https://pbs.twimg.com/media/F233qJyWQAAjmOo?format=jpg&name=orig)
 ![](https://pbs.twimg.com/media/F2333WKXEAAc2fZ?format=png&name=orig)

-**5/**
+**5/** @NOTimothyLottes

 Custom editor, using a custom font, designed for vintage CRTs. Wanted 5-character symbol names for easy packing in 30-bit values. So setup a 6-bit character table. This is using a modified 'right-angle-only' font (like the one below). So some glyphs alias, and only upper case.

 ![](https://pbs.twimg.com/media/F234Y1IXcAECWTB?format=png&name=orig)

-**6/**
+**6/** @NOTimothyLottes

 Will need some kind of symbol table, what ends up providing the arguments to the rules that build the 32-bit words that comprise the final binary. The symbol indirection provides a way to name registers for example, and then change the register mapping later.

 ![](https://pbs.twimg.com/media/F2357mwXUAAC6O6?format=png&name=orig)

-**7/**
+**7/** @NOTimothyLottes

 Notice SMEM ops have 21-bit immediate offsets. I'll use 20-bits of that for an absolute positive offset from KART base. 1 MiB of easy data. The editor will relink tagged symbols that represent offsets automatically.

 ![](https://pbs.twimg.com/media/F236eA_XkAAc8-_?format=jpg&name=orig)

-**8/**
+**8/** @NOTimothyLottes

 Next will need a collection of rules, how to join the 6 possible symbol arguments together. Rules for example for each of the RDNA2 instruction encodings (VOPC, etc). This reduces to just a 6 entry 5-bit left shift amount associated with each argument.

 ![](https://pbs.twimg.com/media/F237AEiWgAAkzD0?format=png&name=orig)

-**9/**
+**9/** @NOTimothyLottes

 But will need a 7th argument, hard coded to a signed 16-bit *4 relative offset as this is how RDNA2 does all it's relative branches. This will be setup inside the editor to auto relink as source is edited.

 ![](https://pbs.twimg.com/media/F237uqaXAAAKAFl?format=jpg&name=orig)

-**10/**
+**10/** @NOTimothyLottes

 Source then is simple, two 128-bit values per 32-bit word in the binary. One to select {RULE, SYMBOLS, RELATIVE BRANCH}, and another for {LABEL, COMMENT}.

 ![](https://pbs.twimg.com/media/F238Q-zXYAAO1Eo?format=png&name=orig)

-**11/**
+**11/** @NOTimothyLottes

 And then the source editor, shows a line per 32-bit value (so VOP3's for example would take 2 lines). So the 'assembler' is deceptively simple, but powerful enough to do everything required to generate GPU binary code. And all the opcode generation is in the data itself :)

 ![](https://pbs.twimg.com/media/F238wJ8WoAAtEfw?format=png&name=orig)

-**12/**
+**12/** @NOTimothyLottes

 The editor itself will have 3 views {symbol table, rule table, source table}. The editor itself will use 2 copies of everything, so each edit, it's one entry/lane, reading prior state and updating to the new. Dead simple, dead parallel (not necessarily efficient) :)

-**13/**
+**13/** @NOTimothyLottes

 Yeah, perhaps wildly alien. But for those who grew up with MOD-like trackers (like Impulse Tracker below), the interface will almost feel like home (those had simple {instrument, pattern, arangement} editors, with a very spreadsheet feel).

@@ -26,33 +26,33 @@ parent_post_id: "1688491417109196800"

 ## Branch

-**1/**
+**1/** @BlurBusters

@NOTimothyLottes Very interesting "crosstalk from voltage inversion" artifact. Some repeating patterns or flicker patterns can trigger crosstalk artifacts too. Behavior varies a lot between TN, IPS, and VA, but all of them have to alternate voltage polarity spatially and temporally.

 ![](https://pbs.twimg.com/media/F3I8AZmW0AEy6jn?format=png&name=orig)
 ![](https://pbs.twimg.com/media/F3I8BVlXoAAnuJB?format=png&name=orig)

-**2/**
+**2/** @BlurBusters

@NOTimothyLottes Related reads:
 https://www.lagom.nl/lcd-test/inversion.php
 https://forums.blurbusters.com/viewtopic.php?t=7539
 http://www.techmind.org/lcd/#:~:text=Inversion

-**3/**
+**3/** @NOTimothyLottes

@BlurBusters I appreciate the links. Came to the same conclusion, for LCDs when doing low scaling like 1:2x2 'CRT' emulation, I'll have to do a stylized blue noise phosphor pattern instead.

-**4/**
+**4/** @Ak4115x

@NOTimothyLottes @BlurBusters "VCOM inversion" -if wasn't line2line but frame2frame that'd "flicker" instead at half the refresh rate so that's why it's preferred& there's also another peculiarity of this scheme. Worth looking into, spent a few hours afternoon,it's late evening now maybe share another time.

-**5/**
+**5/** @BlurBusters

@Ak4115x @NOTimothyLottes Inversion algorithm is both spatial and temporal on all of them, regardless of spatial pattern of voltage inversion. Line to line is also frame to frame. (lines swaps voltage polarity in next frame)

-**6/**
+**6/** @Ak4115x

@BlurBusters @NOTimothyLottes LCOS has singular VCOM there's no pattern to it and no such artifact.

@@ -25,7 +25,7 @@ in_reply_to: ""

 ## Thread

-**1/**
+**1/** @NOTimothyLottes

 CRT emulation on LCDs can be hard. For example it's often easy to generate a pattern that causes a LCD hardware bug. NOT joking. Here is an example on the Steam Deck. The pattern in the window affects the scan (Deck scan is rotated) outside the window.

@@ -34,22 +34,22 @@ CRT emulation on LCDs can be hard. For example it's often easy to generate a pat

 Branches: [[archive/threads/NOTimothyLottes/2023-08-07-crt-emulation-on-lcds-can-be-hard/2023-08-10-BlurBusters-very-interesting-crosstalk-from-voltage-inversion]]

-**2/**
+**2/** @NOTimothyLottes

 The actual pattern is this (it's a 2x1 pixel checker of {G,RB}).

 ![](https://pbs.twimg.com/media/F266epnW4AAdCiK?format=png&name=orig)

-**3/**
+**3/** @NOTimothyLottes

 The theory: Deck's 90deg scan results in {(bottom)R, G, B (top)} sub-pixel components. So could alternate {G,RB} only per pixel in a checker pattern to generate a new sub-pixel pattern at a different resolution. In this case trying for 640x400 virtual resolution.

 ![](https://pbs.twimg.com/media/F267UYuXAAAO50A?format=png&name=orig)

-**4/**
+**4/** @NOTimothyLottes

 For virtual pixel, shift energy into masked virtual phosphor. So 50% brightness red, would be {100%, 0%} for {in, out} of phosphor pixels. Over 50% one starts to increase non-phosphor pixels until some maximum so mask is still visible, but brightness isn't too compromised.

-**5/**
+**5/** @NOTimothyLottes

 Of course this is done in linear, so it's energy conserving in theory. But when the panel fails, it breaks down and doesn't work. I see this problem on a bunch of different devices. My personal AMD laptop has the exact same fail case (but rotated differently).
@@ -26,15 +26,15 @@ parent_post_id: "1692560907988029769"

 ## Branch

-**1/**
+**1/** @tomcr2100

@NOTimothyLottes i am not quite sure what do you mean by subpixel scan? do you override the normal (refresh) scanout process?

-**2/**
+**2/** @NOTimothyLottes

@tomcr2100 It's like "clear-type" font rendering where if you know the LCD sub-pixel configuration for the RGB components of a pixel, you can do a much better job of computing and outputing the value at the specific RGB sub-pixel element. So i'm doing this for CRT-like emulation on the Deck

-**3/**
+**3/** @tomcr2100

@NOTimothyLottes just curious, i started doing something similar, but this year i have focused more on rest&recover. i went with a trilinear style aperture grill approximation, and just allow prim components to leak into the two others.

@@ -25,14 +25,14 @@ in_reply_to: ""

 ## Thread

-**1/**
+**1/** @NOTimothyLottes

 SPC: CRT shader on the Deck. This time not doing subpix render. Yet RGB vs BGR order matters still. One way it becomes possible to simulate even spaced aperture grill (removing scan fx), and the other more of the scan effect.

 ![](https://pbs.twimg.com/media/F30tppDWwAAA-8J?format=jpg&name=orig)
 ![](https://pbs.twimg.com/media/F30tqE8XYAAgrFv?format=jpg&name=orig)

-**2/**
+**2/** @NOTimothyLottes

 Comparing all methods (grille, bad convergence scan, subpix scan)

@@ -40,13 +40,13 @@ Comparing all methods (grille, bad convergence scan, subpix scan)
 ![](https://pbs.twimg.com/media/F30uKJdXkAA4PV8?format=jpg&name=orig)
 ![](https://pbs.twimg.com/media/F30uKouWQAAvY0q?format=jpg&name=orig)

-**3/**
+**3/** @NOTimothyLottes

 The non-subpix scan has 1-pixel separation of RGB (intentional bad convergence) to help hide scaling. The subpix scan uses 1/3-pixel separation (less visible misaligned convergence). Pixel as in output pixel.

 Branches: [[archive/threads/NOTimothyLottes/2023-08-18-spc-crt-shader-on-the-deck/2023-08-18-tomcr2100-i-am-not-quite-sure-what-do-you-mean-by-subpixel]]

-**4/**
+**4/** @NOTimothyLottes

 The subpix version is definitely better. Because of the almost 3x doubling of scanline width detail, it just perceptually feels a lot cleaner and easier to reconstruct in the mind.

@@ -26,11 +26,11 @@ parent_post_id: "1692397318471926054"

 ## Branch

-**1/**
+**1/** @Karyuutensei

@NOTimothyLottes How many passes do you need to produce this result?

-**2/**
+**2/** @NOTimothyLottes

@Karyuutensei 1 pass: 12 fetches/outpix, in 2 batches. I use 6 gather4's to grab a 4-wide(x)2-line region for the {R,G,B} channels. Then 6 more fetches to do energy conserving computation of output for each {nearest 2 lines}x{R,G,B} channel based {intensity, position vertically on scan}.

@@ -25,7 +25,7 @@ in_reply_to: ""

 ## Thread

-**1/**
+**1/** @NOTimothyLottes

 SPC: Last CRT shader was broken. Corrected it. It's energy conserving now, and correctly doing sub-pixel line width raster using the individual channels. Cell camera  shots below at different zooms.

@@ -33,16 +33,16 @@ SPC: Last CRT shader was broken. Corrected it. It's energy conserving now, and c
 ![](https://pbs.twimg.com/media/F3yYZQwXcAABFvi?format=jpg&name=orig)
 ![](https://pbs.twimg.com/media/F3yYZo5WAAA0aRS?format=jpg&name=orig)

-**2/**
+**2/** @NOTimothyLottes

 Each scan line is simulated with only 3 pixels. But you get 9 sub-pixels to play with, so that is where the quality comes from. It reaches 7/8th of the display peak brightness, this compromise was so there is always some amount of perceptual scan effect.

-**3/**
+**3/** @NOTimothyLottes

 The scan line thickness is adaptive proportional to linear brightness. When the scan thins out it is energy conserving and increases the thin pixel brightness to compensate. When it needs to go bright, it blooms the line. Lines have some overlap in bloom.

 Branches: [[archive/threads/NOTimothyLottes/2023-08-18-spc-last-crt-shader-was-broken/2023-08-18-Karyuutensei-how-many-passes-do-you-need-to-produce-this-result]]

-**4/**
+**4/** @NOTimothyLottes

 One human perceptual reason CRTs feel like they have more contrast than they do, is because an individual scan line is super bright when in focus. So the black surround feels darker to the mind.
@@ -26,11 +26,11 @@ parent_post_id: "1735631826427732305"

 ## Branch

-**1/**
+**1/** @SebAaltonen

@NOTimothyLottes They design one API for each separate use case instead of making general purpose flexible SIMD hardware and API for the developers to implement what ever algorithm they need. CPU side got this right. PS2 had correct design, but Nvidia choose hardware T&L for desktop :(

-**2/**
+**2/** @SebAaltonen

@NOTimothyLottes I kind of understand hardware T&L back then because it was a new field and programmable hardware didn't exist. But now we have massive amount of wide SIMDs and bandwidth and big caches too. We can do better!

@@ -25,106 +25,106 @@ in_reply_to: ""

 ## Thread

-**1/**
+**1/** @NOTimothyLottes

 <long thread> 
 Holiday GPU thoughts as a recovering pc/mobile/etc cross platform dev-holic ...

-**2/**
+**2/** @NOTimothyLottes

 [0] Early STP prototypes: had standard-PC-practice multi-dispatch 32-bit version, and hyper-optimized packed-16-bit semi-persistent single-dispatch-ubershader that got L2 reuse. Optimized one was 40% faster in profiling (VK RNDA2 GPU)! Industry has huge untapped opt potential

-**3/**
+**3/** @NOTimothyLottes

 [1] Even for PC, 16-bit is required for optimization because: almost double the amount of register state a shader can leverage. Holds true even when platform lacks double rate 16-bit. Top optimization is pass merging, PC went bandwidth starved. Mobile is actually ALU starved ...

-**4/**
+**4/** @NOTimothyLottes

 [2] Double rate 16-bit done right, is typically good for the smaller of {30% perf, return to non-ALU bound}, can make a GPU feel at least one HW generation faster.

-**5/**
+**5/** @NOTimothyLottes

 [3] For PC, I'd personally draw my compatibility line where platforms support 16-bit, and only ship explicit packed 16-bit shaders instead of a separate 32-bit/16-bit shader permutation. AMD it's Vega up, for NV note Maxwell/Pascal's HFMA2: https://docs.nvidia.com/cuda/cuda-binary-utilities/index.html#maxwell-and-pascal-instruction-set

-**6/**
+**6/** @NOTimothyLottes

 [4] Suggest doing dev work on a platform with great packed 16-bit support: AMD PC Vulkan on RDNA2, and don't filter through the DXC deoptimizer, so it is actually possible to see the value. Then hammer on vendors who still have spotty compiler related issues.

-**7/**
+**7/** @NOTimothyLottes

 [5] Gather4 is the hot optimization for packed 16-bit because data is returned in SoA form. Always explicitly pack 16-bit and alias to/from UINT32 when passing constants to shader (best to alias eight 16-bit values as uint4). Etc.

-**8/**
+**8/** @NOTimothyLottes

 [6] Trick for good 16-bit is range management. Sometimes pre-scaling is needed to avoid overflow,
 and make sure to bring max values out of denormal before using rcp.

-**9/**
+**9/** @NOTimothyLottes

 [7] Power is a function of data movement. "Touch it Once" is the top optimization, all the "Split it into a Graph, Pipeline Stage it" people are walking themselves towards high-power.

-**10/**
+**10/** @NOTimothyLottes

 [8] Majority of GPU workloads are whole-pass serially dependent, but in practice temporally independent with exception of small localized serial dependency. If you "stick it into a frame graph" you cut off your ability to optimize.

-**11/**
+**11/** @NOTimothyLottes

 [9] Resolution growth is slowing. Mobile's high DPI is beyond good enough. 8k won't be successfull because of the extra 4x scaling TAA tax, or it won't matter because people will just integer scale on scanout.

-**12/**
+**12/** @NOTimothyLottes

 [10] Scaling TAA will converge to consistent higher quality over time, thus PC is likely to locally converge to 720p|1080p rendering for 4k (for high|med FPS). This implies that the industry should hit a point where large L3 can hold full render targets ...

-**13/**
+**13/** @NOTimothyLottes

 [11] TAA scaling, full render targets in L3: Maybe RDNA2 128 MiB L3 was spot on actually, and RNDA3's L3 drop in capacity, and the no L3 consoles had this wrong Tracing and big nets probably need that L3 too.

-**14/**
+**14/** @NOTimothyLottes

 [12] Lots of value getting a good 8x area scale for 4K 120 Hz. Yet devs won't see value until they make the non-pixel costs scale well, and move post before scaling.

-**15/**
+**15/** @NOTimothyLottes

 [13] Triangles are not needed any more. Hint 8x area scaling doesn't know any connectivity. Connectivity is inferred by likeness to accumulated feedback. Shaded samples from non-triangles -> scaling TAA still works. Hint 2: scaling TAA raster is mostly just tri culling.

-**16/**
+**16/** @NOTimothyLottes

 [14] TAA gets quality limited at high scaling because it can only infer correct behavior from geometry similar to what is rendered. Geometric aliasing drops too much thin geo. Fix for that is to change the standard practice, which will obsolete all black boxes.

-**17/**
+**17/** @NOTimothyLottes

 [15] Stratified sampling is good for at least 2x the geo density compared to regular grid ... frame viewport jitter is end-of-life for quality.

-**18/**
+**18/** @NOTimothyLottes

 [16] TAA's ability to correctly infer reprojection validity on skinned objects is crippled by lack of correct forward projection. Fixing that will break the black boxes.

-**19/**
+**19/** @NOTimothyLottes

 [17] Shading density will end up variable, TAA's disocclusion and convergence logic needs to be out of the black box, and deeply integrated into shading systems too (because it points where to shade more). This evolution will break all the black boxes.

-**20/**
+**20/** @NOTimothyLottes

 [18] Per-pixel ML people have it wrong, your net is multi-pass, all the top optimizations require less passes. Don't matter if you optimize the matrix math, power is burned in the bandwidth, and you cannot reduce your bandwidth enough to compete with the better options.

-**21/**
+**21/** @NOTimothyLottes

 [19] GI needs a surface shade cache, might as well object space shade at that point, which enables NUMA, which enables more HW scaling.

-**22/**
+**22/** @NOTimothyLottes

 [20] HW RT people had it very wrong, top optimization is non-ray-traversal ordered access. Follows: occlusion testing is neighbor-coherent, no trace, just test shared occluders. GI reduces to high frequency occlusion of some lower frequency probe domain.

-**23/**
+**23/** @NOTimothyLottes

 [21] HW RT's skin-all re-tree before trace won't scale ever. Only stratified visibility gets log scaling on animated geo. HW RT has no system for bounding costs, while stratified visibility does. Think out of the black box, GPU got programmability decade back now.

-**24/**
+**24/** @NOTimothyLottes

 [22] Love HW people, but they opt for a $ train even as it is derailing: they need to build HW, make it faster, it's what they do. But leave a trail of back-compat debris TS/ROV/GS/etc after each gfx fad. Vote with your engineering, general purpose CS, just say no to black boxes.

 Branches: [[archive/threads/NOTimothyLottes/2023-12-15-long-thread/2023-12-15-SebAaltonen-they-design-one-api-for-each-separate-use-case]]

-**25/**
+**25/** @NOTimothyLottes

 [23] Scaling TAAs want sparse data each frame, more like a low-frequency pixel control cage displacing a high-frequency reprojected feedback. No pixel centers any more, no need to interpolate before shading. Rather want striped data, so the cachelines group spaced-out samples ...
@@ -26,11 +26,11 @@ parent_post_id: "1736157335058300999"

 ## Branch

-**1/**
+**1/** @AgileJebrim

@NOTimothyLottes Yep put the whole game logic in a game.glsl compute shader.

-**2/**
+**2/** @AgileJebrim

 If doing a large open world game, set up a paging system to stream data to the GPU in tiles, particularly static data. All dynamic data should be handled on the GPU. If leveraging  server, you can pass along the network packets as they are. CPU shouldn’t really be doing much for a game client. It certainly shouldn’t be varying due to scene complexity.

@@ -26,25 +26,25 @@ parent_post_id: "1736147815829565624"

 ## Branch

-**1/**
+**1/** @SheriefFYI

@NOTimothyLottes two point six seconds?!! D3D12CreateDevice() on my machine is ~300ms..

-**2/**
+**2/** @NOTimothyLottes

@SheriefFYI Yeah and that's a warm second run too. I dev on crippled laptops so when the handicaps are removed the stuff really flies.

-**3/**
+**3/** @SheriefFYI

@NOTimothyLottes still that sounds like a lot, what specs / driver? Any weird vk layers loading?

-**4/**
+**4/** @NOTimothyLottes

@SheriefFYI Ok, yeah just reprofiled, and it's down to 269 ms. Best part of the Win10 experience, never know what is actually happening in the background.

 ![](https://pbs.twimg.com/media/GBhSG-XXMAAT0FS?format=png&name=orig)

-**5/**
+**5/** @SheriefFYI

@NOTimothyLottes *whew* that sounds more normal

@@ -26,17 +26,17 @@ parent_post_id: "1736154923408953499"

 ## Branch

-**1/**
+**1/** @VPCOMPRESSB

@NOTimothyLottes so you're preallocating every type of descriptor to a max count and aliasing them with various qualifiers/attributes? all that in a single memory layout?

 if this is the case, the program identifier makes sense. you can just swap out the code, leaving the data untouched.

-**2/**
+**2/** @NOTimothyLottes

@VPCOMPRESSB I use the layout aliasing to choose things like {read-only K$ reads, vs read-write texture cache access} for buffers, for making sure GLC=1 (write through to coherent cache domain) is enabled for stores (image and buffer), as well as selecting the texel format for image access :)

-**3/**
+**3/** @VPCOMPRESSB

 i revisited last night.

@@ -48,27 +48,27 @@ i assume your code is laid in stages. each stage corresponds to a specific image

 aliasing supports this even further, with explicit intentions for resources.

-**4/**
+**4/** @NOTimothyLottes

@VPCOMPRESSB On modern desktop GPUs it is very efficient to have everything bound all the time (bind everything once/frame). On AMD on use the descriptor is loaded into SGPRs. You only pay for what is used. And ideally group descriptors used together near so they share same cache lines.

-**5/**
+**5/** @NOTimothyLottes

@VPCOMPRESSB Merging independent shaders is then easy. Merging dependent shaders is more complex. You can both do it an unsafe way or a safe way (too long for one tweet)...

-**6/**
+**6/** @NOTimothyLottes

@VPCOMPRESSB The unsafe way: launch a N+M sized 1D dispatch. Bank on having enough work in N such that when the dependent M starts that enough of N is finished in practice. This can work when early M depends on only early N workgroups.

-**7/**
+**7/** @NOTimothyLottes

@VPCOMPRESSB There are various safe ways. For instance  if the dependent work isn't done, one can conditionally duplicate the work. Or conditionally use a fallback. Or only use the pre task if it had finished to accelerate the second, for example hierarchical empty space skipping ...

-**8/**
+**8/** @NOTimothyLottes

@VPCOMPRESSB The challenge of course is that the  obvious stuff like spin waiting isn't safe because of bad API and driver design : no forward progress guarantee (on preemption, etc)

-**9/**
+**9/** @NOTimothyLottes

@VPCOMPRESSB Quite literally they just need to put in a guarantee that workgroups with lower coords in scan order are launched first and restored first in a partial preemption restore. Then you could safely spin on work in a prior numbered workgroup

@@ -25,7 +25,7 @@ in_reply_to: ""

 ## Thread

-**1/**
+**1/** @NOTimothyLottes

 Inspecting my old VK engines. On start I would warm all the pages. Code pages get a read, and data pages got an atomic add of a non-compiler known zero. An attempt to improve initial load time. But in the end nothing helps the 2.6 seconds or so required to just open the VK device

@@ -33,25 +33,25 @@ Inspecting my old VK engines. On start I would warm all the pages. Code pages ge

 Branches: [[archive/threads/NOTimothyLottes/2023-12-16-inspecting-my-old-vk-engines/2023-12-17-SheriefFYI-two-point-six-seconds-d3d12createdevice-on-my]]

-**2/**
+**2/** @NOTimothyLottes

 Since I almost never get crashes I forgot about this, but had written a system to auto relaunch the app on crash or special case exit. No complex error handling, just bank on fast relaunch instead.

 ![](https://pbs.twimg.com/media/GBgJ1y5W0AArwSt?format=png&name=orig)

-**3/**
+**3/** @NOTimothyLottes

 The hope of single PSO in VK was scrapped because IHV compile times explode. However I did maintain single SPIR-V file, and use spec constants to choose individual shaders. The threads doing compile dump compile times and disassembly in DEV mode.

 ![](https://pbs.twimg.com/media/GBgMbMEWkAAOd_d?format=png&name=orig)

-**4/**
+**4/** @NOTimothyLottes

 I'm a fan of single file applications. So when not in DEV mode the one SPIR-V file gets pulled from the EXE. Compiling PSOs is always the critical path, the moment the "bind everything once" layout is done, PSO compile goes wide on the machine.

 ![](https://pbs.twimg.com/media/GBgOQRiXoAAX2k6?format=png&name=orig)

-**5/**
+**5/** @NOTimothyLottes

 "Bind-Everything-Once" in Vulkan looks like this. Bindings {1}=CPU pushing to GPU, {2}=buffer (more on that next), {3}=as UAVs, {4}=as TEX ... No binding logic at runtime ever. Access anything in the most efficient way any time.

@@ -59,13 +59,13 @@ I'm a fan of single file applications. So when not in DEV mode the one SPIR-V fi

 Branches: [[archive/threads/NOTimothyLottes/2023-12-16-inspecting-my-old-vk-engines/2025-08-08-VPCOMPRESSB-so-youre-preallocating-every-type-of-descriptor]]

-**6/**
+**6/** @NOTimothyLottes

 Did something silly but fast for buffers. Here is an example from a dummy test app. The main SSBO gets aliased as 4 structures, each with the same data accessable with four types {uint, uint4, float, float4}. So same binding layout aliasing chooses load type and K$ or V$ access.

 ![](https://pbs.twimg.com/media/GBgQ0xlWQAA7Yks?format=png&name=orig)

-**7/**
+**7/** @NOTimothyLottes

 Literally the CPU does almost nothing. All game input gets pushed to the GPU for game logic. Some major advantages in input latency to do player logic on the GPU instead. And it's dead easy.

@@ -73,16 +73,16 @@ Literally the CPU does almost nothing. All game input gets pushed to the GPU for

 Branches: [[archive/threads/NOTimothyLottes/2023-12-16-inspecting-my-old-vk-engines/2023-12-17-AgileJebrim-yep-put-the-whole-game-logic-in-a-game-glsl]]

-**8/**
+**8/** @NOTimothyLottes

 This requires context, I build programs in a hardware style, fixed resources, fixed shaders, and all logic expressed in the data itself. No code or resource variation, no runtime issues period. So another dummy program example, I can just list my resources on load in a table ...

 ![](https://pbs.twimg.com/media/GBgTCgJXsAEPH3W?format=png&name=orig)

-**9/**
+**9/** @NOTimothyLottes

 Radically simplicity is how I roll personally. Easy to confuse the idea that one needs complexity to express a complex system. It's more like one needs complexity to express a complex solution -> abstractions and hierachical meta feeds on itself until tech debt freezes progess.

-**10/**
+**10/** @NOTimothyLottes

 If there is anything to learn for programming, it is to be more critical of your own ideas than all else, be ruthless, trim until what is left is only what is actually required, question everything constantly.
@@ -25,17 +25,17 @@ in_reply_to: ""

 ## Thread

-**1/**
+**1/** @NOTimothyLottes

 Working towards a public-domain release of a cleaned version of my Win32 Vulkan NV+AMD compute graphics platform. What I used last for rapid prototyping (which is also IMO fine for releasable software). Won't call it an 'Engine' because it only provides tools to hang yourself ...

-**2/**
+**2/** @NOTimothyLottes

 Simplified the error logger today. Memory mapped file is definitely the way to go, one atomic per message, log multiple sessions, fixed size file acts like a message ring buffer. Background thread keeps the last message updating the same line on the console (for easy debug).

 ![](https://pbs.twimg.com/media/GBvFvOXXMAAFLXS?format=png&name=orig)
 ![](https://pbs.twimg.com/media/GBvFvOWX0AAxSuU?format=png&name=orig)

-**3/**
+**3/** @NOTimothyLottes

 Multi-session is nice, since the binary auto reloads in some cases (crash, etc), want to see why the reload happened, so don't want to clear the log. Yes, I still don't use debuggers, this logger + runtime shader recompile is all I use personally for dev work.
@@ -26,7 +26,7 @@ parent_post_id: "1742743018929557805"

 ## Branch

-**1/**
+**1/** @kenpex

@NOTimothyLottes It's what debuggers should be. Debuggers made zero progress in the last decade, but fundamentally they patch running code... just they are limited to patch an interrupt instead of allowing us to do... anything. Even tracepoints, while nice, are a castrated version of livecode.

@@ -25,13 +25,13 @@ in_reply_to: ""

 ## Thread

-**1/**
+**1/** @NOTimothyLottes

 Debug for majority-GPU-side game dev? Run-time recompile and hex viewer. If I want to see a value it is 'ramWI1.hxd[pos]=value;' (write to the memory that the hex dump reads from). Everything else is automatic, just save source and the results show up on screen. Zero's not drawn.

 ![](https://pbs.twimg.com/media/GC91mNxXEAAaovO?format=png&name=orig)

-**2/**
+**2/** @NOTimothyLottes

 IMO this is preferred to a 'debugger'. I can fill values from all workgroups simply by placing them at different positions, etc. Can get timings via shader clock for profiling even. It is transformative in dev productivity.

@@ -26,7 +26,7 @@ parent_post_id: "1776597200073568623"

 ## Branch

-**1/**
+**1/** @MartinJIFuller

@NOTimothyLottes @SebAaltonen A worthwhile project, knowledge well worth sharing, but is a book the right format? Perhaps yes, just thinking out loud.

@@ -26,7 +26,7 @@ parent_post_id: "1776597200073568623"

 ## Branch

-**1/**
+**1/** @RickyAH

@NOTimothyLottes @SebAaltonen @MartinJIFuller Ha! That won’t save you from my follow

@@ -26,7 +26,7 @@ parent_post_id: "1776597200073568623"

 ## Branch

-**1/**
+**1/** @BenSimsTech

@NOTimothyLottes @SebAaltonen @MartinJIFuller Oh sick, I'm deep in optimising shaders in an AAA game for GCN hardware right now (You can probably guess the platform(s)) so this is super relevant to me. 😅
 Going to give it all a thorough read, I'm after all the GCN info I can find right now.
@@ -27,7 +27,7 @@ in_reply_to: "1776518700230721688"

 ## Thread

-**1/**
+**1/** @NOTimothyLottes

@MartinJIFuller @SebAaltonen On https://mastodon.gamedev.place/@NOTimothyLottes I'm collecting posts with AMD GPU disassembly dumps for a book on everything the shader compiler 'isn't doing'. Constant AMD GPU compiler perf bug SPAM is a key part of my online strategy to keep a low follower count.

@@ -25,6 +25,6 @@ in_reply_to: ""

 ## Thread

-**1/**
+**1/** @NOTimothyLottes

@Nerfoxingaround @SebAaltonen It was only later when systems forced C ABI library interfaces with it's stacks and junk that things got messy and only because of the C. Then C++ happened and things got much worse. Pure ASM is easy, its interfacing with the bloat world that got hard.
@@ -26,19 +26,19 @@ parent_post_id: "1857813028349128825"

 ## Branch

-**1/**
+**1/** @bmcnett

@NOTimothyLottes So each line is 63 spaces followed by a \n, and those 63 spaces get replaced by printable characters dynamically? I have long dreamed of a text format with a fixed 128 byte line width, and cache line size is gradually switching over from 64 to 128

-**2/**
+**2/** @NOTimothyLottes

@bmcnett Yes fixed character[63]='\n', string fills until then or spaces out. I did 128-char once, but still sometimes use small VGA resolutions. Also written a source editor a few times with this fixed line size, makes for a very simple implementation ...

-**3/**
+**3/** @bmcnett

@NOTimothyLottes Variable line width ASCII text files made a lot of sense when storage was 100KB or 100MB or 100GB, but now that it's in the terabytes, why even bother

-**4/**
+**4/** @NOTimothyLottes

@bmcnett That line of thinking is universal solution to most self-inflicted problems of the modern era.

@@ -26,32 +26,32 @@ parent_post_id: "1857805438151983120"

 ## Branch

-**1/**
+**1/** @datenwolf

@NOTimothyLottes Why the heck do you care about cache lines, if you write to a mmaped file? Of course there's overhead: TLBs aren't free. And TLBs matter… at lot in fact. When you map the file, initially the kernel marks some address space, but doesn't populate it with pages. 1/

-**2/**
+**2/** @datenwolf

@NOTimothyLottes On the first write to a non-faulted page, the kernel has to find a free page in memory, associate it in the page table, all the while keeping an eye on all threads process, since changes to a process virtual address space must appear atomic. 2/

-**3/**
+**3/** @datenwolf

@NOTimothyLottes Memory maps aren't free. Simple writes to a file will easily outperform and cause less overhead, than naively implemented mmaped I/O. 3/

-**4/**
+**4/** @NOTimothyLottes

@datenwolf BTW, I pre-fault my mapped files. Just like I pre-fault the binary and data segments too. But sure the pre-fault could go stale. And yes TLBs are a system perf issue, but atlas you need admin priv on Windows to play 2 MiB pages for data, so it's off the table

-**5/**
+**5/** @NOTimothyLottes

@datenwolf Why cacheline aligned and sized on the CPU? In theory store the entire line in a short enough timeframe and your Read-Modify-Write becomes just a Write! CPU doesn't have byte-write-mask (I think). If SW+HW is good. And threads keeping to isolated lines means no false sharing. Etc

-**6/**
+**6/** @datenwolf

@NOTimothyLottes (That cacheline remark was meant rhetorically; between cache misses and page faults, a page fault will largely overshadow a cache miss in performance overhead.
 But yes, if you prefault, then that problem is mitigated.)

-**7/**
+**7/** @NOTimothyLottes

@datenwolf 'Smart' people keep trying to impose (aka force) their will on others through poor system APIs, like not being able to pre-pin pages. Good look buddy, I'll just run a background task that continuously touch walks the pages I want pinned anyway.

@@ -26,15 +26,15 @@ parent_post_id: "1857803914604618029"

 ## Branch

-**1/**
+**1/** @static_assert_0

@NOTimothyLottes This is super cool, but why map a file and not just stdout?

-**2/**
+**2/** @vodangkhoa873

@static_assert_0 @NOTimothyLottes The writes can be done concurrently from many threads?

-**3/**
+**3/** @NOTimothyLottes

@vodangkhoa873 @static_assert_0 Yes, one 'atomicAdd(atomAdr,1)&65535' to get the line number (window of 65536 max lines) then write the line individually. Only contention is on that atomicAdd CPU instruction which is as minimal as possible.

@@ -42,11 +42,11 @@ I do the atomic add instruction using simple inline ASM (below)

 ![](https://pbs.twimg.com/media/GciHgZzXIAAD9by?format=png&name=orig)

-**4/**
+**4/** @static_assert_0

@NOTimothyLottes @vodangkhoa873 I love this, but are you not able to get that codegen with intrinsics or just normal C?

-**5/**
+**5/** @NOTimothyLottes

@static_assert_0 @vodangkhoa873 I've been hand rolling intrinsics CPU-side in GCC with inline asm since the dawn of support for that. So haven't kept up with the high-level-lang mess fire since.

@@ -26,15 +26,15 @@ parent_post_id: "1857804669994565771"

 ## Branch

-**1/**
+**1/** @HerrUppoHoppa

@NOTimothyLottes Curious why you memory map the 32 bit counter to the file. Also why does it get half the file as opposed to just the end of the mapped region?

-**2/**
+**2/** @NOTimothyLottes

@HerrUppoHoppa It's pow2 message lines so fixing wrap is AND(cnt,lines-1). Counter after that and needs to take a page minimum. So yes I could make the size slightly smaller, but just lazy. Counter in file because then multiple runs accumulate in the file for comparison.

-**3/**
+**3/** @HerrUppoHoppa

@NOTimothyLottes Ah I see, interesting solution!

@@ -25,7 +25,7 @@ in_reply_to: ""

 ## Thread

-**1/**
+**1/** @NOTimothyLottes

 [0] What a horrible term 'printf-debugging'
 If you are debugger-free might as well at least be libc-free
@@ -33,7 +33,7 @@ Here is how I work CPU C code without any debugger in my engines, and how it wor

 Branches: [[archive/threads/NOTimothyLottes/2024-11-16-0-what-a-horrible-term-printf-debugging/2024-11-16-static_assert_0-this-is-super-cool-but-why-map-a-file-and-not]]

-**2/**
+**2/** @NOTimothyLottes

 [1] First I map a '.log' file, so there is no other file IO
 (a.) First half of this file (log2 sized file) contains lines of a fixed 64-character size (with the \n at the end)
@@ -41,17 +41,17 @@ Branches: [[archive/threads/NOTimothyLottes/2024-11-16-0-what-a-horrible-term-pr

 Branches: [[archive/threads/NOTimothyLottes/2024-11-16-0-what-a-horrible-term-printf-debugging/2024-11-17-HerrUppoHoppa-curious-why-you-memory-map-the-32-bit-counter-to]]

-**3/**
+**3/** @NOTimothyLottes

 [2] So writing a line to this log is 100% overhead-free and lock-free, one just increments the atomic counter, and dumps the comment line in the memory mapped file. Note the 64-byte line size matches a cacheline size, so no contention on the write of the line

 Branches: [[archive/threads/NOTimothyLottes/2024-11-16-0-what-a-horrible-term-printf-debugging/2024-11-16-datenwolf-why-the-heck-do-you-care-about-cache-lines-if-you]]

-**4/**
+**4/** @NOTimothyLottes

 [3] It's ideal for a multi-thread environment, you get correct temporal ordering captured in the log file without any real effect on the scheduling of threads.

-**5/**
+**5/** @NOTimothyLottes

 [4] Lines are always in a fixed format
 r|sc.milmic|line_|hex_____|0000000000-|string...
@@ -62,14 +62,14 @@ hex ... hex print of number
 000 ... decimal print of the number
 string ... user msg

-**6/**
+**6/** @NOTimothyLottes

 [5] I write a line with a simple function call
 Tlk(__LINE__,n,"msg");
 Where 'n' is the number, "msg" is a string
 Don't need printf, this is way better

-**7/**
+**7/** @NOTimothyLottes

 [6] Here is an example with a very simple program, just a few first lines. It's nice to automatically see exact timing. Timing since start
 0.005478 sec ... mapping of the cartridge file
@@ -79,19 +79,19 @@ etc

 ![](https://pbs.twimg.com/media/GchC4NwXQAAbDfS?format=png&name=orig)

-**8/**
+**8/** @NOTimothyLottes

 [7] Another simple example of a fully pipelined startup, can instantly know where the primary cost is, 
 "Instance begin" and "Instance end" wrap setup of the Vulkan instance, a process which took 2.4 seconds. The 4 PSOs hit in the shader cache at 3k microseconds each

 ![](https://pbs.twimg.com/media/GchFfI5WcAAZ_Al?format=png&name=orig)

-**9/**
+**9/** @NOTimothyLottes

 [8] The log file keeps multiple runs, so it's easy to compare. Notepad2 it's F5 to reload the log file.

 Branches: [[archive/threads/NOTimothyLottes/2024-11-16-0-what-a-horrible-term-printf-debugging/2024-11-16-bmcnett-so-each-line-is-63-spaces-followed-by-a-n-and]]

-**10/**
+**10/** @NOTimothyLottes

 [9] The point: keep it minimal, keep it simple, and keep the number of required things that take human time to the smallest possible. If you don't fall off the 'compile-time' curve, and re-compile/re-start/re-load is 'key-press-time' this works amazingly well (interactive)
@@ -25,34 +25,34 @@ in_reply_to: ""

 ## Thread

-**1/**
+**1/** @NOTimothyLottes

 Working towards a new bring up of front-buffering on NV via VK. My last implementation had been tuned on AMD and didn't work on NV any more. NV does seem to accept a 1-deep swap with IMMEDIATE presentation at least on latest drivers, so that is a good start.

-**2/**
+**2/** @NOTimothyLottes

 Since my image resources are static after init time, only swap images change when the driver kills the swap chain (which once upon a time seemed to happen on ALT+TAB maybe). So it's one descriptor set always bound.

-**3/**
+**3/** @NOTimothyLottes

 Stopped using VK_DESCRIPTOR_SET_LAYOUT_CREATE_UPDATE_AFTER_BIND_POOL_BIT_EXT (god awful naming length people), because I think it could be a perf hit on NV due to extra indirection. NV driver is free to bake down the single set now before the command buffer is sent over.

-**4/**
+**4/** @NOTimothyLottes

 At this stage, to the point of having everything loaded and swap chain created, it's 2700 lines of engine code. That includes embedding headers (no external includes), so it's a one file compile. People claim rolling your own engine is hard? Not really if you keep it focused.

-**5/**
+**5/** @NOTimothyLottes

 So far it's a 0.25 sec hot load time on this NV dGPU laptop. Includes mapping a 4 MiB 'cart' file from pagecache, doing a 512 MiB buffer for GPU usage, hits on all PSOs, allocating some images, and kicking the command buffer that copies in the cart, and clears everything.

-**6/**
+**6/** @NOTimothyLottes

 Everything at load-time is multi-threaded to try to minimize start to in-game time. This is in sharp contrast to runtime where the only multi-threading being used is to separate things that are blocking.

-**7/**
+**7/** @NOTimothyLottes

 I get some utility on parallelizing {vulkan instance creation, mapping the cart file, warming the TLBs by walking all the pages, bringing up the window} it's about 0.08 seconds in at that point

-**8/**
+**8/** @NOTimothyLottes

 After VK device is open, I signal a background thread to load the SPIR-V module, while building the descriptor set layout, which then unblocks PSO compile on background threads. And the rest of the VK setup runs in parallel. Working towards swap creation.
@@ -26,35 +26,35 @@ parent_post_id: "1859369992355021246"

 ## Branch

-**1/**
+**1/** @AgileJebrim

@NOTimothyLottes How reliable in your experience in code like that is the compilation to avoid branches?

-**2/**
+**2/** @NOTimothyLottes

@AgileJebrim Lots of stuff can get mapped by the compiler to V_CNDMASK_B32 instead of branching by using mix(a,b,bool) syntax. I rarely use branching because of how horrible the code generation is. No way to express that the common path stay code linear either, etc.

-**3/**
+**3/** @AgileJebrim

@NOTimothyLottes I know in CUDA-land that it likes to create branches for ternaries if it contains more than 4 total instructions in a statement.

-**4/**
+**4/** @NOTimothyLottes

@AgileJebrim But the real problem is lacking of explicit controls, and lacking implicit branch hints (wave-uniform/non-uniform, expected branch hit rate, etc). Better if we could just do explicit ASM style branch instructions and labels.

-**5/**
+**5/** @AgileJebrim

@NOTimothyLottes Well I seek to ban branches entirely, so that’s the exact opposite of what I want.

-**6/**
+**6/** @NOTimothyLottes

@AgileJebrim If your shader I$ is say 32 KiB and your instructions average say 8-bytes. You have 4 K instructions before you are streaming the program not just the data. Code size is actually important. And while micro level unroll is needed, macro level duplication is a problem with tiny I$

-**7/**
+**7/** @AgileJebrim

@NOTimothyLottes I avoid the duplication of instruction patterns too. I feel it’s a failure to properly parallelize if you’re repeating the same instructions instead of data parallelizing further.

-**8/**
+**8/** @AgileJebrim

@NOTimothyLottes One more reason why I avoid functions.

@@ -25,11 +25,11 @@ in_reply_to: ""

 ## Thread

-**1/**
+**1/** @NOTimothyLottes

 [0] Been wanting to do this for a long time, a thread on hard-core shader development. Something that digs into what I've learned and applied over the years. Along with the evolution of the strange syntax I use when writing shaders ...

-**2/**
+**2/** @NOTimothyLottes

 [1] Lets start at one of the early ones, FXAA 3.11. Someone is hosting that here: https://gist.github.com/kosua20/0c506b81b3812ac900048059d2383126 ... most lines, it's an easy map to NV's arch which has predicated instructions (red is the predicate). Many lines are 1:1 mappings with physical HW instructions.

@@ -37,150 +37,150 @@ in_reply_to: ""

 Branches: [[archive/threads/NOTimothyLottes/2024-11-20-0-been-wanting-to-do-this-for-a-long-time-a/2024-11-20-AgileJebrim-how-reliable-in-your-experience-in-code-like-that]]

-**3/**
+**3/** @NOTimothyLottes

 [2] Looking at FSR1: https://github.com/GPUOpen-Effects/FidelityFX-FSR/blob/master/ffx-fsr/ffx_fsr1.h - that had 3 permutations {32-bit, 16-bit unpacked, 16-bit packed}, where the packed one maps to how AMD's packed instructions work (everything explicitly packed)

 ![](https://pbs.twimg.com/media/Gc3QjEiWYAADo14?format=png&name=orig)

-**4/**
+**4/** @NOTimothyLottes

 [3] Onto STP: https://github.com/Unity-Technologies/Graphics/blob/27341ce4f5c8853d7f15e9420bff499f1087aceb/Packages/com.unity.render-pipelines.core/Runtime/STP/Stp.hlsl - A different set of permutations {32-bit, packed 16-bit} then {implicit mediump vs explicit} in the 32-bit (generic) permutation via types like StpMF4 which aim to allow a compiler on mobile to get some 16-bit register savings

 ![](https://pbs.twimg.com/media/Gc3SFXTXoAEt4tX?format=png&name=orig)

-**5/**
+**5/** @NOTimothyLottes

 [4] Notice for 16-bit path, everything again is explicitly packed (unlike the 32-bit path). And GLSL to SPIR-V would get explicit packing to the IHV driver since SPIR-V has explicit vector types (unlike DX, which was an insane problem). Example, the reprojection filtering below.

 ![](https://pbs.twimg.com/media/Gc3SmjPWMAEMX_d?format=png&name=orig)

-**6/**
+**6/** @NOTimothyLottes

 [5] Then STP had defines to work out differences in sampling capability (like supporting MIN/MAX sampling with UINT or not, or supporting immediate offsets or not). Example in the code doing {z,motion} UINT32 packed nearest dilation:

 ![](https://pbs.twimg.com/media/Gc3TW0oWIAArDjw?format=png&name=orig)

-**7/**
+**7/** @NOTimothyLottes

 [6] The largest portability nightmare was mixing explicit FP16 with wave ops! So many compiler bugs across vendors = so many permutations in the code. The 32-bit path had to have a conversion from 'StpMF4' (possible implicit medium precision), to 'StpF4' explicit 32-bit.

 ![](https://pbs.twimg.com/media/Gc3UTC_WcAAyc5J?format=png&name=orig)

-**8/**
+**8/** @NOTimothyLottes

 [7] Some of perf death problems had been that DXIL didn't have bitfield ops (unlike SPIR-V), HLSL depends on IHV compilers somehow finding and pattern matching stuff like the crap below (which is MANY OPS to 1 OPCODE). So pre-processing through HLSL on non-Xbox platforms = fail!

 ![](https://pbs.twimg.com/media/Gc3V5wGW8AADAju?format=png&name=orig)

-**9/**
+**9/** @NOTimothyLottes

 [8] Go into the shader which is doing wave ops, and the true nightmare of platform workarounds becomes apparent https://github.com/Unity-Technologies/Graphics/blob/27341ce4f5c8853d7f15e9420bff499f1087aceb/Packages/com.unity.render-pipelines.core/Runtime/STP/StpSetup.compute - LSD/waveop permutations, 32-bit/16-bit in 16-bit path, etc

 ![](https://pbs.twimg.com/media/Gc3W143XEAEq6LS?format=png&name=orig)

-**10/**
+**10/** @NOTimothyLottes

 [9] So yeah, even in modern times you can hit paths in compilers and toolchains that well never got tested because no one had tried it and tried shipping with it. And I think this might hint that relatively few actually do this level of crazy

-**11/**
+**11/** @NOTimothyLottes

 [10] Part of the formula for getting great gains with 16-bit is simply the register savings. It wouldn't be possible to do good 4x4 kernel lanczos inside a TAA without packed 16-bit support (it's 64 values on load without thinking about all the other mid-algorithm data) ...

-**12/**
+**12/** @NOTimothyLottes

 [11] Looking at STP again, there is a dering, which works with the {min,max} of the near 2x2. If the HW has MIN/MAX sampling, it can fetch that, which means a lot more than 64 values, and if not they need intermediate ALU min/max, which is extra state ...

 ![](https://pbs.twimg.com/media/Gc3Ze31XAAA94Y1?format=png&name=orig)

-**13/**
+**13/** @NOTimothyLottes

 [12] Mobile hardware is so register starved (probably due to all that SRAM area going to tiles) that it wasn't possible to do good reprojection filtering PERIOD even with FP16. So STP had a crippled compromise that did bilinear in one direction, and lanczos in the other axis

 ![](https://pbs.twimg.com/media/Gc3afacXQAAFX-c?format=png&name=orig)

-**14/**
+**14/** @NOTimothyLottes

 [13] Huge wake up call to mobile HW vendors, if you don't match VGPR capacity of PCs, you can never take on where PC workloads will get to. Now given both NV and AMD have packed 16-bit, it is a great time to push pass merging and avoid DRAM traffic by doing more in one shader

-**15/**
+**15/** @NOTimothyLottes

 [14] Another part of the 16-bit perf puzzle is to not use 16-bit FP16 in constants explicitly, but only package them up as aliased as UINT32 binary blobs. And for portable systems that still need 32-bit fallbacks, always build both a packed 16-bit and 32-bit set of constants.

-**16/**
+**16/** @NOTimothyLottes

 [15] So many people tried to 16-bit permute things working from 32-bit float constants, and on PC that is instant perf death. Sure some mobile hardware can do the conversions for lower cost, and some mobile HW could pre-amble the conversions if not, but that isn't perf portable

-**17/**
+**17/** @NOTimothyLottes

 [16] Switching gears. So STP didn't get the most hardcore way I like to write shaders, and I've had a bunch of different permutations of ideas tried over the years ... specifically how to make the code more like GPU assembly!

-**18/**
+**18/** @NOTimothyLottes

 [17] Looking like assembly as in making it exactly clear what the lines map to in HW OPCODEs, and what the variable map to in REGISTERS. So you are designing to what the hardware could do, even if the compiler cannot keep up

-**19/**
+**19/** @NOTimothyLottes

 [18] Now it's true that NV and AMD HW are quite different in some respects (NV is predicated, and has generic 3 op logic ops, etc), so ultimately one has to pick their shader language abstraction based on what they want to target the most https://docs.nvidia.com/cuda/cuda-binary-utilities/index.html#instruction-set-reference (NV ISA docs)

-**20/**
+**20/** @NOTimothyLottes

 [19] Since AMD's docs are more complete, and I like to make sure a console port is possible and would be as fast as possible, I prefer to think through all my shaders as if they are getting direct mapped to AMD's RDNA2 these days: https://www.amd.com/content/dam/amd/en/documents/radeon-tech-docs/instruction-set-architectures/rdna2-shader-instruction-set-architecture.pdf

-**21/**
+**21/** @NOTimothyLottes

 [20] So I usually do a define that maps logic directly to what the associated instruction would be. Some of these actually map to shading language intrinsics already like fma() or bitfieldExtract(). Then write all my shader code using these defines. Example below for the defines

 ![](https://pbs.twimg.com/media/Gc3fKzqXUAAGnBQ?format=png&name=orig)

-**22/**
+**22/** @NOTimothyLottes

 [21] Like to use ASM like convention for {signed, unsigned}, that all integers are unsigned, and only get converted to signed when required inside the define for a specific opcode (like signed or unsigned bitfield extract). Bit casting (aliasing) is done via SI1_I1() style macros

 ![](https://pbs.twimg.com/media/Gc3gGUTWQAAZwyX?format=png&name=orig)

-**23/**
+**23/** @NOTimothyLottes

 [22] And I reduce all types to very simple 3 letter macros of which are described below. I use the same naming convention in CPU land as well. This helps keep code compact and easy to read IMO (highly subjective yes)

 ![](https://pbs.twimg.com/media/Gc3gna_XcAA7PXx?format=png&name=orig)

-**24/**
+**24/** @NOTimothyLottes

 [23] One thing I struggle with to this day is compiler bitcast aliasing perf bugs. Meaning once I tried to always only use UINT4 types everywhere, and then wrap all OPCODE defines in bitcasting defines, that didn't go so well (looking at disassembly on AMD PC).

-**25/**
+**25/** @NOTimothyLottes

 [24] Compilers depend a lot on pattern matching to NOP stuff, and well there are bugs, lots of bugs. So for the sake of portability, with the exception of {unsigned/signed} bitcasting, I use the native types when possible (no packed-16-bit/32-bit aliasing) exception of waveops

-**26/**
+**26/** @NOTimothyLottes

 [25] Another aspect of compiler problems is argument passing to function calls causing SSA state explosions, because it's "copy semantics" as defined in the langs. Once I did inout uint4[16] (all VGPRs) as my only argument to all functions. Didn't end well!

-**27/**
+**27/** @NOTimothyLottes

 [26] Someone on twitter (or was it bsky or mast?) had been doing a one shader game. And addressing these kinds of problems, and reminded us, YES the shader languages support global variables. No need to pass them into function! (bingo)

-**28/**
+**28/** @NOTimothyLottes

 [27] We have been so seasoned to dislike globals in the past few decades due to well probably early mutli-window programming styles, that it is easy to forget you can just do globals, and they are fast at tool/compile time.

-**29/**
+**29/** @NOTimothyLottes

 [28] I think the next level up of this design is to go to mostly global variables, but since the shader languages don't support global unions, then have different sets of globals when one needs to do type bitcast aliasing (which is very common with packed 16-bit and 64-bit logic)

-**30/**
+**30/** @NOTimothyLottes

 [29] I still think the 'fix' for the shader compiler SSA state explosion mapping back to registers problem is to pass in explicit aliasing hints to the vendor compilers, so they have a better base guess at register allocation. But I doubt vendors would get on board with that

-**31/**
+**31/** @NOTimothyLottes

 [30] My personal x86-64 languages typically worked with unions of structs on globals. Where each struct in the union was a different aliased meaning to the HW registers, and functions had been associated with a given struct in the union. So no stacks or argument passing.

-**32/**
+**32/** @NOTimothyLottes

 [31] That kind of setup for the GPU is ideal IMO (at least for the type of stuff I do). You can link. Because the union struct for the registers is the calling ABI, and never moves (no stack). So code sharing works amazingly well. And avoiding register state explosion is built in

-**33/**
+**33/** @NOTimothyLottes

 [32] oh damn [32], should probably stop now, until next time.
@@ -26,45 +26,45 @@ parent_post_id: "1859791162091110754"

 ## Branch

-**1/**
+**1/** @AgileJebrim

@NOTimothyLottes Shouldn’t uniform be a scalar?

-**2/**
+**2/** @NOTimothyLottes

@AgileJebrim Well 'uniform' in GLSL is a storage qualifier, "Uniforms are so named because they DO NOT CHANGE from one shader invocation to the next within a particular rendering cal" so you probably want another term for SGPR to avoid aliasing qualifiers

-**3/**
+**3/** @AgileJebrim

@NOTimothyLottes Within ISPC, uniform means scalar and varying means vector.

-**4/**
+**4/** @AgileJebrim

@NOTimothyLottes Looking deeper into the definition of SGPRs. They’re uniform just across each wavefront, rather than the entire dispatch yeah?

-**5/**
+**5/** @NOTimothyLottes

@AgileJebrim Yes it's a separate register file and logic unit that is per-wave instead of per-lane. Since it's introduction in Turing, NV calls them "URX for uniform registers" while AMD calls them SGPRs (scalar general purpose registers)

-**6/**
+**6/** @AgileJebrim

@NOTimothyLottes Almost sounding like we need an alternative IR to SPIR-V. One with predicated instructions and these wavefront-uniform registers/instructions. Maybe a PTX -> RDNA backend compiler?

-**7/**
+**7/** @NOTimothyLottes

@AgileJebrim As history goes, most re-writes fall into nearly the same problem as before. I'd rather have the fixes be applied to the existing stuff first (fix SPIR-V and GLSL, etc)

-**8/**
+**8/** @AgileJebrim

@NOTimothyLottes Perhaps. That existing PTX Vulkan extension does come to mind though.

-**9/**
+**9/** @CodePotrick

@AgileJebrim @NOTimothyLottes Its kinda annoying to put sgpr like logic in an ir tho as they have very specific limitations on each platform.

 Generalizing them in an ir might actually be a pessimisation.

-**10/**
+**10/** @CodePotrick

@AgileJebrim @NOTimothyLottes Amd for example can't do float ops on sgprs, so when you would allow float ops in the ir, amd has to move them to a vgpr, perform float op, move them back to sgpr for each ir-sgpr float op.

@@ -72,25 +72,25 @@ There are other examples for this.

 And nvidias have even other constrains.

-**11/**
+**11/** @AgileJebrim

@CodePotrick @NOTimothyLottes And what’s the alternative right now? Just keep it all in a VGPR all the time?

-**12/**
+**12/** @CodePotrick

@AgileJebrim @NOTimothyLottes Ideally you'd avoid any float ops as long as you can to keep things in sgprs if you need to save registers/want the uniformity.

 If you need float ops on that var its probably best to keep it in vgprs if the alternative would be to transfer between sgpr and vgpr a lot

-**13/**
+**13/** @AgileJebrim

@CodePotrick @NOTimothyLottes I wish I could work solely with integers all the time but if I did that on NV hardware, I’d be losing half the available ALUs.

-**14/**
+**14/** @AgileJebrim

@CodePotrick @NOTimothyLottes I haven’t looked closely at AMD hardware since I don’t work with it much.

-**15/**
+**15/** @CodePotrick

@AgileJebrim @NOTimothyLottes they have their actual isa public and inspectable which is really nice to know whats actually happening.

@@ -98,7 +98,7 @@ Rga is also really good. Feels like im more in control with amd compared to nvid

 Maybe thats just the low level gremlin in my speaking tho ;).

-**16/**
+**16/** @AgileJebrim

@CodePotrick @NOTimothyLottes I do have access to some of NVIDIA’s NDA documentation and tools…

@@ -25,50 +25,50 @@ in_reply_to: ""

 ## Thread

-**1/**
+**1/** @NOTimothyLottes

 Sometimes you only want to wave-coherent branch to something extremely rarely. There is no good way in GLSL to describe that. You always get one branch and one disruption of the linear path [RED]. What you want is [GREEN], common path is linear and branch-free. Gotos would enable

 ![](https://pbs.twimg.com/media/Gc9K3dEXIAA7tGy?format=png&name=orig)

-**2/**
+**2/** @NOTimothyLottes

 In theory wave-coherent branch hint and then providing say a 99% taken branch taken compile hint could enable a compiler to know how to do proper code gen here

-**3/**
+**3/** @NOTimothyLottes

 The other thing, if you know a path is always taken by some of the lanes, and thus on AMD you want NO branches, and instead just EXEC changing, there is no way to express that either. Doing mix(,,bool) could add extra overhead vs simple EXEC masking

-**4/**
+**4/** @NOTimothyLottes

 This second example is why you want to be able to mix 'always-lane-divergent' branch hint, with '50% probability' meaning the HW needs to always execute both paths, and thus use EXEC masking without physical branching (assuming a good compiler)

-**5/**
+**5/** @NOTimothyLottes

 Also the lack of an explicit 'scalar' (aka SGPR) qualifier on variables. The compiler gets this wrong, a qualifier just fixes that problem. Non-SGPR platforms can just ignore the qualifier. And sgpr=readlane(maybeSgpr,0) can sometimes actually become a readlane (fail)

 Branches: [[archive/threads/NOTimothyLottes/2024-11-22-sometimes-you-only-want-to-wave-coherent-branch/2024-11-22-AgileJebrim-shouldnt-uniform-be-a-scalar]]

-**6/**
+**6/** @NOTimothyLottes

 Other big thing on AMD's PC side, the HW has all these nice SALU ops which set the SCC (scalar flags register), and there is no good way to trick the compiler to doing basic stuff like branch on SALU overflow ...

-**7/**
+**7/** @NOTimothyLottes

 I tried using GLSL's uaddCarry() but it at the time pushed the whole operation from SALU back to VALU making the code extra horrible, and uaddCarry() and friends = basic fail, the carry is a bool in HW not an integer, silly lack of good pattern matching there applies

-**8/**
+**8/** @NOTimothyLottes

 Anyway this stuff is easily fixable if people just actually cared about it. It's not like you need to scrap the API or invent a new language. Just fix what we have, and people like me will use it, and teach others how to use it well too.

-**9/**
+**9/** @NOTimothyLottes

 Also for obvious reasons, at least provide a PRAGMA "not-stupid" that we can place in shaders to avoid all the overhead generating crap that deoptimizes code to check for junior programmer mistakes at runtime, this stuff is bloody hard (and sometimes) impossible to bypass today

-**10/**
+**10/** @NOTimothyLottes

 Lets add another thing, if(subgroupElect()){} to try to make stuff SALU is broken. Because subgroupElect() will do all the overhead to find the lowest active lane (even in cases the compiler should know otherwise). And if(gl_SubgroupID==0) for that also isn't well optimized

-**11/**
+**11/** @NOTimothyLottes

 Maybe another time we should dive into the problems with shading languages mixing branching with loads and killing all the latency hiding. But too much for this thread...
@@ -26,7 +26,7 @@ parent_post_id: "1868492851170205803"

 ## Branch

-**1/**
+**1/** @axelgneiting

@NOTimothyLottes @kechogarcia I'm well aware. It's really curious they still stick to this design after so many years.