diff --git a/archive/threads/AgileJebrim/2024-12-18-you-left-out-the-bit-where-i-also-said-no/2024-12-18-NOTimothyLottes-3-if-you-had-multiple-line-stores-collecting-on-a.md b/archive/threads/AgileJebrim/2024-12-18-you-left-out-the-bit-where-i-also-said-no/2024-12-18-NOTimothyLottes-3-if-you-had-multiple-line-stores-collecting-on-a.md new file mode 100644 index 0000000..41406cd --- /dev/null +++ b/archive/threads/AgileJebrim/2024-12-18-you-left-out-the-bit-where-i-also-said-no/2024-12-18-NOTimothyLottes-3-if-you-had-multiple-line-stores-collecting-on-a.md @@ -0,0 +1,227 @@ +--- +title: "@AgileJebrim @SebAaltonen @munohikari (3.) If you had multiple line stores collecting on a given line with ECC, you'd have the same bottleneck as fire-and-forget atomics." +type: archive +source: twitter +source_url: "https://x.com/NOTimothyLottes/status/1869436525869724013" +author: "NOTimothyLottes" +handle: NOTimothyLottes +post_id: "1869436525869724013" +date: 2024-12-18 +archived: 2026-08-25 +draft: false +tags: + - archive + - twitter + - AgileJebrim +description: "@AgileJebrim @SebAaltonen @munohikari (3.) If you had multiple line stores collecting on a given line with ECC, you'd have the same bottleneck as fire-and-forget atomics." +in_reply_to: "" +parent_post_id: "1869436199884222515" +--- + +## Source + +- URL: https://x.com/NOTimothyLottes/status/1869436525869724013 +- Author: NOTimothyLottes (@NOTimothyLottes) +- Posted: 2024-12-18 17:36:14 + +## Branch + +**1/** + +@AgileJebrim @SebAaltonen @munohikari (3.) If you had multiple line stores collecting on a given line with ECC, you'd have the same bottleneck as fire-and-forget atomics. A throughput of one line modification per clock. + +**2/** + +@AgileJebrim @SebAaltonen @munohikari (4.) Note the prior assumes a read hit! + +So at the point you get anti-atomic, you are also anti-store and anti-ECC, and anti-etc. It's the same pipelines with the same variability due to on-chip networking, different clock domain crossing, etc. + +**3/** + +@AgileJebrim @SebAaltonen @munohikari (5.) Good GPUs have multiple ALUs doing the atomic by the last level cache. So it scales very well as a function of line collisions. Meaning if all lanes access separate addresses on a cacheline, those go in parallel. + +**4/** + +@AgileJebrim @SebAaltonen @munohikari (6.) Atomics and regular stores have the same memory controller base-camping behavior. If all SIMDs store to addresses that hash to the same last level cache partition, they will drop efficiency just like doing all atomics to the same address. + +**5/** + +@AgileJebrim @SebAaltonen @munohikari (7.) Anyway, while CPU atomics are serious design fail, GPU atomics on good GPUs are your best friend. They are actually a fantastic tool for getting more regular performance in many parallel algorithms. + +**6/** + +@NOTimothyLottes @AgileJebrim @munohikari Indeed. Many CPU programmers have the wrong belief that GPU atomics are slow and unusable. They are fantastic. Bump allocator is your best friend on CPU side. On GPU you do atomic add to implement a bump allocator. As simple as that. It's super fast. + +**7/** + +@SebAaltonen @NOTimothyLottes @munohikari I know NVIDIA did a really good atomicAdd, especially if you don’t use the result of the atomicAdd(). If you use it, then it forces it down a significantly slower path. But it’s all implementation-specific and different vendors will do different things yes? + +**8/** + +@AgileJebrim @SebAaltonen @munohikari Atomics come in two forms +(1.) fire-and-forget (aka you don't use the return) - the compiler will flag it for no-return, so it behaves like a store +(2.) read the return - it behaves like a {store, and read} simultaneously with layout "coherent" + +**9/** + +@AgileJebrim @SebAaltonen @munohikari Good GPUs have address coalescing on writes and atomics. So a wave wide store is reduced to a cost that scales with the number of total cachelines all waves touch, with byte-write-mask so there is no read-modify-write + +**10/** + +@AgileJebrim @SebAaltonen @munohikari ... atomics have the same but if any wave lane's addresses collide it will bump those to another request (for the same cacheline). Possible that some HW implements a HW reduction for collisions of some kinds of atomics locally before the requests go out, but don't assume that + +**11/** + +@AgileJebrim @SebAaltonen @munohikari ... it's more likely that some compilers try to fix stupid behavior by doing a wave-reduction in software, and that causes hell for those of us who already write good code + +**12/** + +@NOTimothyLottes @SebAaltonen @munohikari Thoughts on atomic compare exchange? I use similar functionality on CPUs to globally synchronize threads and I figured I’d give it a shot attempting to do that on GPUs. + +**13/** + +@AgileJebrim @SebAaltonen @munohikari Compare and exchange since it takes 2 values, it could have half the peak rate through on-chip network. Personally I have always used algorithms that do better things without the cmpswap atomic. You always can get the return (value pre-atomic), so cmpswap utility is less + +**14/** + +@NOTimothyLottes @SebAaltonen My CPU sync mechanism works by stalling each thread until it reaches a multiple of the total number of threads to be synced, only incrementing an integer (swapping) when it reaches its specific thread index. + +**15/** + +@AgileJebrim @SebAaltonen I assume you are using a futex then on Linux to block? But the GPU doesn't have such a thing exposed in software. Instead you'd be effectively polling using serial global memory atomic latency as your rate limiter and 'blocking' + +**16/** + +@AgileJebrim @SebAaltonen Serializing workgroups on the GPU in this way has a very high latency granularity. There is also the issue of preemption on PCs, there is no guarantee of forward progress due to no guarantee all workgroups get restored + +**17/** + +@AgileJebrim @SebAaltonen So you can deadlock post preemption if the workgroup that needs to do the atomic to unblock those spinning cannot get back on the machine + +**18/** + +@AgileJebrim @SebAaltonen Now, the important part, there is one compromise rule that has the highest chance of being possible, that is an ask that if there is a partial restore, that it restores the oldest launched workgroups first ... + +**19/** + +@AgileJebrim @SebAaltonen And with that you can have some forward progress if an algorithm only has dependencies on workgroups launched before it (like lower kernel global workgroup index), food for thought for algorithm design + +**20/** + +@AgileJebrim @SebAaltonen on the CPU side, you'd futex block to get the thread off being scheduled so the dependent one could get on the machine to avoid deadlock, it isn't efficient by any means but works, the GPU's exposed only voluntary method to get off the machine is to exit the workgroup ... + +**21/** + +@AgileJebrim @SebAaltonen So if you uber-task, meaning just keep launching the same dispatch, and the workgroup finds what it needs to do (instead of working off kernel global index), then you can workaround preemption that way too ... + +**22/** + +@AgileJebrim @SebAaltonen Blocking becomes just workgroup exit. And the next workgroup gets lower schedule priority than existing active workgroups which helps solve of the scheduling issues + +**23/** + +@NOTimothyLottes @SebAaltonen I’m not using a Linux futex. Could you look over my code here? + +**24/** + +@AgileJebrim @NOTimothyLottes @SebAaltonen Hey Jeb, have you considered hitting the funny L shaped button on the right of your keyboard more frequently? 😅 + +**25/** + +@simplex_fx @NOTimothyLottes @SebAaltonen Enter? Nah + +**26/** + +@AgileJebrim @NOTimothyLottes @SebAaltonen + +![](https://pbs.twimg.com/media/GfG1cCJW8AAXcDg?format=jpg&name=orig) + +**27/** + +@simplex_fx @NOTimothyLottes @SebAaltonen If I don’t have to scroll right… + +**28/** + +@AgileJebrim @NOTimothyLottes @SebAaltonen Dude, stop with that WQHD bullshit, buy a proper tv and limit yourself to 80-120 + +**29/** + +@simplex_fx @NOTimothyLottes @SebAaltonen Nty. The hell do I need a tv for? I have several monitors. + +**30/** + +@AgileJebrim @NOTimothyLottes @SebAaltonen How big? + +**31/** + +@simplex_fx @NOTimothyLottes @SebAaltonen Big enough to maximize a window on each. + +**32/** + +@AgileJebrim @NOTimothyLottes @SebAaltonen If you go too wide, you will just kill your neck on the long run + +**33/** + +@simplex_fx @NOTimothyLottes @SebAaltonen I have 4 monitors + +**34/** + +@AgileJebrim @NOTimothyLottes @SebAaltonen How big? + +**35/** + +@simplex_fx @AgileJebrim @SebAaltonen Ideally one of these + +![](https://pbs.twimg.com/media/GfHntpaWYAAGnOk?format=png&name=orig) + +**36/** + +@NOTimothyLottes @simplex_fx @SebAaltonen Not big enough. Evans & Sutherland still makes planetariums. We have smaller scale 360 degree domes. I think they’re about 14ft high. + +**37/** + +@AgileJebrim @NOTimothyLottes Same is true for everything. All memory loads on GPUs are highly vendor specific. On GTX 1080 (Pascal) your memory load goes through sampler, which is around 85 cycles of latency. While on Turing your raw load is only 28 cycles of latency (CUDA benchmarks). + +**38/** + +@SebAaltonen @NOTimothyLottes For sure, but my focus is more on how deterministic and predictable these are. I’ll take 85 if it’s always 85 over something that might vary unpredictably between 28 and 100. + +**39/** + +@AgileJebrim @NOTimothyLottes Nvidia has big L1$, so they are more predictable. Texture samplers are shared with all waves running on the same compute unit. They are not really predictable. You wait until the unit is ready. There's quite a lot of fluctuation. That's why you have 8x-10x hyperthreading per CU. + +**40/** + +@AgileJebrim @NOTimothyLottes The idea is to keep the SIMD running all the time. Wave execution time variance hasn't been a big optimization goal. There's always more work coming, so it's not a problem if some wave takes 10x time to execute compared to others. + +**41/** + +@AgileJebrim @NOTimothyLottes Basically if the SIMD is executing instructions all the time, then your workload runs at predictable amount of instructions per cycle. The problem is barriers. You wait for longest wave to finish. You can't to batch barriers to avoid turning latency into real lost SIMD cycles. + +**42/** + +@SebAaltonen @NOTimothyLottes If they all take the same amount of time then there shouldn’t be much idling of hardware for waiting on the longest wave. + +**43/** + +@AgileJebrim @NOTimothyLottes That's true. But they don't take the same amount of time due to caches. That's where the 10x hyper-threading kicks in. As long as the SIMD is fed all the time, there's no visible issue with fluctuating wave lengths. Barriers make that visible. Minimal amount of barriers = good. + +**44/** + +@NOTimothyLottes @SebAaltonen @munohikari Would you implement a prefix sum compaction algorithm using atomicAdd? + +**45/** + +@AgileJebrim @SebAaltonen @munohikari Standard optimizations apply, like doing wave-reductions first. + +**46/** + +@NOTimothyLottes @SebAaltonen @munohikari Okay so you don’t just blanket throw an atomicAdd on it then. + +**47/** + +@AgileJebrim @SebAaltonen @munohikari No. For things like scanning a 1-bit value, it's a wave masked bit count op in software, followed by predicating the atomic to just lane 0 of the wave. For scanning uint32s it's a log2 wave reduction, then predicating the add to lane 0. And so on + +## Related + +- Spine: [[archive/threads/AgileJebrim/2024-12-18-you-left-out-the-bit-where-i-also-said-no]] diff --git a/archive/threads/AgileJebrim/2024-12-18-you-left-out-the-bit-where-i-also-said-no/2024-12-18-SebAaltonen-yes-but-its-going-to-be-much-slower-you-can-do.md b/archive/threads/AgileJebrim/2024-12-18-you-left-out-the-bit-where-i-also-said-no/2024-12-18-SebAaltonen-yes-but-its-going-to-be-much-slower-you-can-do.md new file mode 100644 index 0000000..df6495a --- /dev/null +++ b/archive/threads/AgileJebrim/2024-12-18-you-left-out-the-bit-where-i-also-said-no/2024-12-18-SebAaltonen-yes-but-its-going-to-be-much-slower-you-can-do.md @@ -0,0 +1,41 @@ +--- +title: "@AgileJebrim @NOTimothyLottes @munohikari Yes, but it's going to be MUCH slower." +type: archive +source: twitter +source_url: "https://x.com/SebAaltonen/status/1869425726128304359" +author: "Sebastian Aaltonen" +handle: SebAaltonen +post_id: "1869425726128304359" +date: 2024-12-18 +archived: 2026-08-25 +draft: false +tags: + - archive + - twitter + - AgileJebrim +description: "@AgileJebrim @NOTimothyLottes @munohikari Yes, but it's going to be MUCH slower." +in_reply_to: "" +parent_post_id: "1869424048607993895" +--- + +## Source + +- URL: https://x.com/SebAaltonen/status/1869425726128304359 +- Author: Sebastian Aaltonen (@SebAaltonen) +- Posted: 2024-12-18 16:53:20 + +## Branch + +**1/** + +@AgileJebrim @NOTimothyLottes @munohikari Yes, but it's going to be MUCH slower. You can do sums and reductions in groupshared memory, but if that mobile GPU doesn't have real groupshared memory, it's going to be very slow. For example GMEM based groupshared is this slow on Qualcomm (and 730 is much better than 610): + +![](https://pbs.twimg.com/media/GfGISwfWgAI6UuY?format=jpg&name=orig) + +**2/** + +@AgileJebrim @NOTimothyLottes @munohikari X1 is the brand new laptop chip. This architecture is only on latest mobile phones. Low end is Adreno 610 and similar. + +## Related + +- Spine: [[archive/threads/AgileJebrim/2024-12-18-you-left-out-the-bit-where-i-also-said-no]] diff --git a/archive/threads/AgileJebrim/2024-12-18-you-left-out-the-bit-where-i-also-said-no/index.md b/archive/threads/AgileJebrim/2024-12-18-you-left-out-the-bit-where-i-also-said-no/index.md new file mode 100644 index 0000000..ab4048a --- /dev/null +++ b/archive/threads/AgileJebrim/2024-12-18-you-left-out-the-bit-where-i-also-said-no/index.md @@ -0,0 +1,81 @@ +--- +title: "@munohikari @SebAaltonen You left out the bit where I also said no graphics pipeline." +type: archive +source: twitter +source_url: "https://x.com/AgileJebrim/status/1869397269956673662" +author: "Jebrim" +handle: AgileJebrim +post_id: "1869397269956673662" +date: 2024-12-18 +archived: 2026-08-25 +draft: false +tags: + - archive + - twitter + - AgileJebrim +description: "@munohikari @SebAaltonen You left out the bit where I also said no graphics pipeline." +in_reply_to: "" +--- + +## Source + +- URL: https://x.com/AgileJebrim/status/1869397269956673662 +- Author: Jebrim (@AgileJebrim) +- Posted: 2024-12-18 15:00:15 + +## Thread + +**1/** + +@munohikari @SebAaltonen You left out the bit where I also said no graphics pipeline. :P + +**2/** + +@AgileJebrim @munohikari Computer shaders are a can of worms for old mobile GPUs too. Some have fast groupshared memory (could be tiny), some emulate it. Some have wave intrinsics, some don't. 64-bit atomic support is nonexistent (for MM Dreams / Native style Z+payload packing). + +**3/** + +@AgileJebrim @munohikari Also compute shader writes are not compressed (pixel shader framebuffer writes have HW lossless compressor). You pay full memory bandwidth cost, which causes phone to heat (drain battery) and eventually throttle. + +**4/** + +@SebAaltonen @munohikari Alternative approach is to do all rendering in screen space in a fragment shader. + +**5/** + +@AgileJebrim @munohikari If you intend to do distance field ray-tracing using volume textures, worth noting that most mobile GPUs don't do proper 3d tiling layout for volume textures. Instead they use legacy 2d sliced layout, which results in significantly more L1$ trashing. Can 5x your mem BW use. + +**6/** + +@SebAaltonen @munohikari Raycasting, no bouncing, using cubic textured bricks that emulate voxel surfaces via a parallax occlusion map approach measuring distances to each voxel surface. + +If I’m writing something for an avionics cockpit, I’d be either using an Arm Mali-G78AE or NVIDIA DRIVE AGX Thor GPU. + +**7/** + +@SebAaltonen @munohikari I’d also be using Vulkan SC, which I believe disables atomics entirely. :P + +**8/** + +@AgileJebrim @SebAaltonen @munohikari Disabling atomics is like chopping off both legs because someone thinks one tow has a blister when it actually doesn't. + +**9/** + +@NOTimothyLottes @SebAaltonen @munohikari You can still implement a comparable feature yourself. + +Branches: [[archive/threads/AgileJebrim/2024-12-18-you-left-out-the-bit-where-i-also-said-no/2024-12-18-SebAaltonen-yes-but-its-going-to-be-much-slower-you-can-do]] + +**10/** + +@AgileJebrim @SebAaltonen @munohikari No actually you cannot. No method of software (or even HW) implemented lock-line algorithm survives the kind of latency in the GPU memory system. There is a big reason atomics on GPUs are done right by the last level of cache (the coherent memory domain) + +**11/** + +@AgileJebrim @SebAaltonen @munohikari Something you really need to understand for your own benefit: +(1.) There are a lot of more expensive things happening in the GPU memory system than atomics. For instance DCC compression. It requires metadata access (to get the state of memory just to do the compression on store). + +**12/** + +@AgileJebrim @SebAaltonen @munohikari (2.) Think of GPU atomics as a read-modify-write op. It's effectively the same behavior as a partial cacheline write-through STORE operation on HW using ECC. Because ECC eats the bits used for byte-write-mask. + +Branches: [[archive/threads/AgileJebrim/2024-12-18-you-left-out-the-bit-where-i-also-said-no/2024-12-18-NOTimothyLottes-3-if-you-had-multiple-line-stores-collecting-on-a]] diff --git a/archive/threads/AgileJebrim/index.md b/archive/threads/AgileJebrim/index.md new file mode 100644 index 0000000..b449e55 --- /dev/null +++ b/archive/threads/AgileJebrim/index.md @@ -0,0 +1,12 @@ +--- +title: AgileJebrim +type: note +draft: false +description: Archived threads by AgileJebrim. +tags: + - archive + - twitter + - AgileJebrim +--- + +- [[archive/threads/AgileJebrim/2024-12-18-you-left-out-the-bit-where-i-also-said-no]] diff --git a/archive/threads/NOTimothyLottes/2025-01-26-often-overlooked-aspect-of-gpu-perf-scaling/2025-01-26-bmcnett-my-take-is-similar-but-i-also-feel-that-a-lot-of.md b/archive/threads/NOTimothyLottes/2025-01-26-often-overlooked-aspect-of-gpu-perf-scaling/2025-01-26-bmcnett-my-take-is-similar-but-i-also-feel-that-a-lot-of.md new file mode 100644 index 0000000..30f5b92 --- /dev/null +++ b/archive/threads/NOTimothyLottes/2025-01-26-often-overlooked-aspect-of-gpu-perf-scaling/2025-01-26-bmcnett-my-take-is-similar-but-i-also-feel-that-a-lot-of.md @@ -0,0 +1,45 @@ +--- +title: "@NOTimothyLottes My take is similar, but I also feel that a lot of processing is best done predictively in parallel across multiple future frames, one SIMD lane per future frame" +type: archive +source: twitter +source_url: "https://x.com/bmcnett/status/1883581286838919440" +author: "bmcnett" +handle: bmcnett +post_id: "1883581286838919440" +date: 2025-01-26 +archived: 2026-08-25 +draft: false +tags: + - archive + - twitter + - NOTimothyLottes +description: "@NOTimothyLottes My take is similar, but I also feel that a lot of processing is best done predictively in parallel across multiple future frames, one SIMD lane per future frame" +in_reply_to: "" +parent_post_id: "1883580512171942335" +--- + +## Source + +- URL: https://x.com/bmcnett/status/1883581286838919440 +- Author: bmcnett (@bmcnett) +- Posted: 2025-01-26 18:22:28 + +## Branch + +**1/** + +@NOTimothyLottes My take is similar, but I also feel that a lot of processing is best done predictively in parallel across multiple future frames, one SIMD lane per future frame + +**2/** + +@bmcnett Don't necessarily want to store all predictions raw though. One of the advantages of say simple things like baked directional lightmaps is that one keeps the data for multiple "frames" (or views) in a compressed form until just right before usage, and it's mis-predict safe + +**3/** + +@NOTimothyLottes Yeah, you need to store many prediction outputs in compressed form, with the time dimension being one over which you compress, in addition to whatever spatial dimensions are relevant + +Limited hw support for that, sadly + +## Related + +- Spine: [[archive/threads/NOTimothyLottes/2025-01-26-often-overlooked-aspect-of-gpu-perf-scaling]] diff --git a/archive/threads/NOTimothyLottes/2025-01-26-often-overlooked-aspect-of-gpu-perf-scaling/2025-01-26-noop_dev-i-thought-framegen-was-just-an-offspring-of-mpeg2.md b/archive/threads/NOTimothyLottes/2025-01-26-often-overlooked-aspect-of-gpu-perf-scaling/2025-01-26-noop_dev-i-thought-framegen-was-just-an-offspring-of-mpeg2.md new file mode 100644 index 0000000..ccae505 --- /dev/null +++ b/archive/threads/NOTimothyLottes/2025-01-26-often-overlooked-aspect-of-gpu-perf-scaling/2025-01-26-noop_dev-i-thought-framegen-was-just-an-offspring-of-mpeg2.md @@ -0,0 +1,35 @@ +--- +title: "@NOTimothyLottes I thought framegen was just an offspring of mpeg2 motion prediction, at least how it was in TVs for 15 years." +type: archive +source: twitter +source_url: "https://x.com/noop_dev/status/1883594247653462329" +author: "Boris Chuprin" +handle: noop_dev +post_id: "1883594247653462329" +date: 2025-01-26 +archived: 2026-08-25 +draft: false +tags: + - archive + - twitter + - NOTimothyLottes +description: "@NOTimothyLottes I thought framegen was just an offspring of mpeg2 motion prediction, at least how it was in TVs for 15 years." +in_reply_to: "" +parent_post_id: "1883591956598845661" +--- + +## Source + +- URL: https://x.com/noop_dev/status/1883594247653462329 +- Author: Boris Chuprin (@noop_dev) +- Posted: 2025-01-26 19:13:58 + +## Branch + +**1/** + +@NOTimothyLottes I thought framegen was just an offspring of mpeg2 motion prediction, at least how it was in TVs for 15 years. + +## Related + +- Spine: [[archive/threads/NOTimothyLottes/2025-01-26-often-overlooked-aspect-of-gpu-perf-scaling]] diff --git a/archive/threads/NOTimothyLottes/2025-01-26-often-overlooked-aspect-of-gpu-perf-scaling/2025-01-27-Varaquilex-a-visualization-of-said-cold-caches-on-a-cs.md b/archive/threads/NOTimothyLottes/2025-01-26-often-overlooked-aspect-of-gpu-perf-scaling/2025-01-27-Varaquilex-a-visualization-of-said-cold-caches-on-a-cs.md new file mode 100644 index 0000000..93932e0 --- /dev/null +++ b/archive/threads/NOTimothyLottes/2025-01-26-often-overlooked-aspect-of-gpu-perf-scaling/2025-01-27-Varaquilex-a-visualization-of-said-cold-caches-on-a-cs.md @@ -0,0 +1,57 @@ +--- +title: "@NOTimothyLottes A visualization of said cold caches on a CS workload from Timothy's GDC 2018 Engine Optimization Hot Lap slides: https://gpuopen.com/wp-content/uploads/2018/05/gdc_2018_sponsored_engine_optimization_hot_lap.pptx" +type: archive +source: twitter +source_url: "https://x.com/Varaquilex/status/1883999400260931601" +author: "Volkan İlbeyli" +handle: Varaquilex +post_id: "1883999400260931601" +date: 2025-01-27 +archived: 2026-08-25 +draft: false +tags: + - archive + - twitter + - NOTimothyLottes +description: "@NOTimothyLottes A visualization of said cold caches on a CS workload from Timothy's GDC 2018 Engine Optimization Hot Lap slides: https://gpuopen.com/wp-content/uploads/2018/05/gdc_2018_sponsored_engine_optimization_hot_lap.pptx" +in_reply_to: "" +parent_post_id: "1883569959286313322" +--- + +## Source + +- URL: https://x.com/Varaquilex/status/1883999400260931601 +- Author: Volkan İlbeyli (@Varaquilex) +- Posted: 2025-01-27 22:03:54 + +## Branch + +**1/** + +@NOTimothyLottes A visualization of said cold caches on a CS workload from Timothy's GDC 2018 Engine Optimization Hot Lap slides: https://gpuopen.com/wp-content/uploads/2018/05/gdc_2018_sponsored_engine_optimization_hot_lap.pptx + +![](https://pbs.twimg.com/media/GiVPBdfbAAAhqJ8?format=jpg&name=orig) + +**2/** + +@Varaquilex Amazingly unfortunate that AMD still doesn't release the tool that those instruction trace captures had been visualized on. It's the best and most useful tool period I've ever seen at any company in the industry. + +**3/** + +@Varaquilex If they did choose to release it, I'd teach the world how to use it well, for free. For hardcore optimization it is close to ideal. RGP doesn't start to really show you what you need to know to make good hardcore engineering decisions on AMD hardware. + +**4/** + +@Varaquilex This is a hint for someone like Valve who could take their RADV SQTT (https://www.phoronix.com/news/RADV-SQTT-Profiling-Support) interface and build something truly transformative for GPU shader engineering + +**5/** + +@Varaquilex Mix that kind of visualization tool with a good shader compiler (and HW intrinsics) and basically it's the kind of crack a serious GPU shader dev cannot say no too + +**6/** + +@NOTimothyLottes That tool was one of the most insightful tools I have ever used. I do miss it. It would really be awesome to see it released to the public. + +## Related + +- Spine: [[archive/threads/NOTimothyLottes/2025-01-26-often-overlooked-aspect-of-gpu-perf-scaling]] diff --git a/archive/threads/NOTimothyLottes/2025-01-26-often-overlooked-aspect-of-gpu-perf-scaling/index.md b/archive/threads/NOTimothyLottes/2025-01-26-often-overlooked-aspect-of-gpu-perf-scaling/index.md new file mode 100644 index 0000000..acf1611 --- /dev/null +++ b/archive/threads/NOTimothyLottes/2025-01-26-often-overlooked-aspect-of-gpu-perf-scaling/index.md @@ -0,0 +1,111 @@ +--- +title: "Often overlooked aspect of GPU perf scaling" +type: archive +source: twitter +source_url: "https://x.com/NOTimothyLottes/status/1883566551120670753" +author: "NOTimothyLottes" +handle: NOTimothyLottes +post_id: "1883566551120670753" +date: 2025-01-26 +archived: 2026-08-25 +draft: false +tags: + - archive + - twitter + - NOTimothyLottes +description: "Often overlooked aspect of GPU perf scaling" +in_reply_to: "" +--- + +## Source + +- URL: https://x.com/NOTimothyLottes/status/1883566551120670753 +- Author: NOTimothyLottes (@NOTimothyLottes) +- Posted: 2025-01-26 17:23:55 + +## Thread + +**1/** + +Often overlooked aspect of GPU perf scaling +Shaders run at two rates +(1.) Cold I$ and K$ rate [significantly slower] +(2.) Warm I$ and K$ rate +You actually need jobs with small wave count to be sharing the same I$ and K$ across waves (cannot be latency sensitive) + +**2/** + +Big problem with mass amounts of artist graph authored material shaders paired with raster, +(1.) More materials +(2.) GPU kicks work as wide as possible (due to limited ROP queue size) +Both amplify the amount of cold cache work + +**3/** + +Very common to look even at full-screen passes and see quite long first waves/SIMD (cold caches), followed by later warm-cache steady state. As GPUs get huge, it pushes more and more waves into the cold-cache region. Scaling-TAAs do the same by reducing render resolution. + +Branches: [[archive/threads/NOTimothyLottes/2025-01-26-often-overlooked-aspect-of-gpu-perf-scaling/2025-01-27-Varaquilex-a-visualization-of-said-cold-caches-on-a-cs]] + +**4/** + +So quite literally the smaller the GPU (aka mobile), the more possible benefits from scaling-TAAs getting reduced resolution, because the cold-cache effects are minimized, etc. But with say the 5090, scaling-TAAs can drop whole GPU efficiency by a substantial amount + +**5/** + +NVIDIA has load prefetch hints (load more lines than required for the load), and AMD should ideally start including the same for both K$ and D$, so one can at least reduce worst case latency. This doesn't solve bandwidth duplication of wide work distribution though. + +**6/** + +There are a few different perf wins from decoupling shading from raster, one is the reduction of cold cache effects. One can loop through waves of work in the same material shader, without exit, without reload of constants, etc. + +**7/** + +Decoupled shading moves a subset of the shading data [material lookup] from interpolated (and thus distributed duplication) to non-temporal streaming (no duplication). + +**8/** + +Likewise if it's same-material same-wave, aka one wave is looping through a local collection of surface, the remaining non-material sampling (that is interpolated) has a higher likely hood of hitting in the cache [less ultimate duplication of data across the chip] + +**9/** + +In modern times with scaling-TAAs and maximum negative mip-bias, by reducing render resolution and maximizing frame rate, one is also minimizing cache reuse. So instead of say shading all the near samples at the same time, each are now divided across perhaps 32 frames + +**10/** + +Thus a massive mismatch between data layout and data usage. The data is stored (and compressed) localized and layered (multiple textures/material), but is increasingly being used in an effectively random access. Both scaling-TAAs and RT are guilty here. + +**11/** + +Games that rely on a mix of {heavy deferred shading, scaling-TAAs (reduced render res), RT noise reduction, and frame generation} are scaling poorly (bad engineering) and scale as a function of increased {latency, artifacts} + +**12/** + +Meanwhile games that are still engineered more like vintage games {high res, low try density, simple baked lightmap textures mostly, MSAA} are scaling relatively well getting high FPS and low latency + +**13/** + +Ultimately all of this hints that it would be better if the per-frame workloads are closer to the vintage games (no-TAA, no frame-gen), and instead one amortizes shading into something that "bakes" in realtime amortized across many frames into the vintage style engine data + +Branches: [[archive/threads/NOTimothyLottes/2025-01-26-often-overlooked-aspect-of-gpu-perf-scaling/2025-01-26-bmcnett-my-take-is-similar-but-i-also-feel-that-a-lot-of]] + +**14/** + +All these "AI" spatial-temporal denoise-scaling-TAAs are ultimately just doing really expensive logic to try to reconstruct the surface material properties at high res and local lighting from a blurry/noisy mess then apply the local conditions to the material + +**15/** + +Which is perhaps stupid given the games already easily know the exact high-res material properties, why try to "infer" them burning TOps of logic and heating your office. + +**16/** + +Scaling-denoise-TAAs do hint thought that perhaps the academic methods games use for BPR and shading are quite lacking in terms of amortization of costs. + +**17/** + +Frame-gen when it "works" ok is just an alpha blend between two states. Clearly an engine could be doing that too instead of reshading everything every frame. And an engine could choose when to do that vs something else to avoid artifacts, something a frame-gen cannot do. + +**18/** + +Ultimately the industry will choose if to devolve into AI generated slop-ware where IHVs black box more and more of your engine and you get more and more trapped and lazy. Or break out and do something better and control your own destiny. + +Branches: [[archive/threads/NOTimothyLottes/2025-01-26-often-overlooked-aspect-of-gpu-perf-scaling/2025-01-26-noop_dev-i-thought-framegen-was-just-an-offspring-of-mpeg2]] diff --git a/archive/threads/NOTimothyLottes/index.md b/archive/threads/NOTimothyLottes/index.md index e263353..3905f92 100644 --- a/archive/threads/NOTimothyLottes/index.md +++ b/archive/threads/NOTimothyLottes/index.md @@ -36,6 +36,7 @@ tags: - [[archive/threads/NOTimothyLottes/2024-12-16-also-i-wouldnt-just-modify-an-existing-compiler]] - [[archive/threads/NOTimothyLottes/2024-12-16-then-there-are-rough-edges-to-texel-buffer]] - [[archive/threads/NOTimothyLottes/2024-12-21-not-sure-if-this-is-the-end-of-this-story-yet-but]] +- [[archive/threads/NOTimothyLottes/2025-01-26-often-overlooked-aspect-of-gpu-perf-scaling]] - [[archive/threads/NOTimothyLottes/2025-04-16-cs-optimization-brain-dumping]] - [[archive/threads/NOTimothyLottes/2025-04-30-3-i-got-side-tracked-by-building-a-language-that]] - [[archive/threads/NOTimothyLottes/2025-04-30-i-laugh-when-people-say-c-is-like-assembly-they]] diff --git a/archive/threads/SebAaltonen/2024-12-18-fixed-all-our-vulkan-bugs/index.md b/archive/threads/SebAaltonen/2024-12-18-fixed-all-our-vulkan-bugs/index.md new file mode 100644 index 0000000..30d9435 --- /dev/null +++ b/archive/threads/SebAaltonen/2024-12-18-fixed-all-our-vulkan-bugs/index.md @@ -0,0 +1,42 @@ +--- +title: "Fixed all our Vulkan bugs:" +type: archive +source: twitter +source_url: "https://x.com/SebAaltonen/status/1869350790487523825" +author: "Sebastian Aaltonen" +handle: SebAaltonen +post_id: "1869350790487523825" +date: 2024-12-18 +archived: 2026-08-25 +draft: false +tags: + - archive + - twitter + - SebAaltonen +description: "Fixed all our Vulkan bugs:" +in_reply_to: "" +--- + +## Source + +- URL: https://x.com/SebAaltonen/status/1869350790487523825 +- Author: Sebastian Aaltonen (@SebAaltonen) +- Posted: 2024-12-18 11:55:33 + +## Thread + +**1/** + +Fixed all our Vulkan bugs: +- ARM: BGR swapchain handled incorrectly (our bug) +- Qualcomm: SSAO quad flicker (driver bug) +- PowerVR: material index rounding issue (our bug) +- Intel (iGPU): subpass barriers don't work (driver bug) + +Now we are ready to ship Vulkan for end users. + +![](https://pbs.twimg.com/media/GfFDoE5XYAAprel?format=jpg&name=orig) + +**2/** + +Our Vulkan backend has been in internal production for 1.5 years already. We were just waiting for Google to ship the Vulkan renderer on Flutter. We have a custom Flutter version that allows us to run our engine in the side. I carved holes to get their Vulkan device ptr, etc. diff --git a/archive/threads/SebAaltonen/index.md b/archive/threads/SebAaltonen/index.md index 181bf85..1342de6 100644 --- a/archive/threads/SebAaltonen/index.md +++ b/archive/threads/SebAaltonen/index.md @@ -11,5 +11,6 @@ tags: - [[archive/threads/SebAaltonen/2024-04-04-this-is-how-slow-code-feels-like]] - [[archive/threads/SebAaltonen/2024-05-03-why-should-one-be-able-to-code-without]] +- [[archive/threads/SebAaltonen/2024-12-18-fixed-all-our-vulkan-bugs]] - [[archive/threads/SebAaltonen/2025-03-08-stupid-hardware-question-why-does-amd-use-several]] - [[archive/threads/SebAaltonen/2025-03-28-indoor-scenes-in-hypehype-are-starting-to-look]] diff --git a/archive/threads/index.md b/archive/threads/index.md index 997f170..2bf4815 100644 --- a/archive/threads/index.md +++ b/archive/threads/index.md @@ -13,6 +13,7 @@ How to read a folder: [[notes/How threads are organized]]. Example of the note shape (single spine, no branches): [[archive/threads/How to archive a thread]] +- [[archive/threads/AgileJebrim]] - [[archive/threads/FFmpeg]] - [[archive/threads/NOTimothyLottes]] - [[archive/threads/SebAaltonen]]