diff --git a/archive/threads/index.md b/archive/threads/index.md index 14c113d..a7532ec 100644 --- a/archive/threads/index.md +++ b/archive/threads/index.md @@ -29,6 +29,7 @@ Example of the note shape (single spine, no branches): [[archive/threads/How to - [[archive/threads/VictorTaelin]] - [[archive/threads/cmuratori]] - [[archive/threads/hsn8086]] +- [[archive/threads/kechogarcia]] - [[archive/threads/kenpex]] - [[archive/threads/levelsio]] - [[archive/threads/m_schuetz]] diff --git a/archive/threads/kechogarcia/2023-12-20-stable-radix-sorting-in-the-gpu-8-bits-per-radix/2023-12-20-SebAaltonen-i-like-the-single-pass-per-pass-approach-but.md b/archive/threads/kechogarcia/2023-12-20-stable-radix-sorting-in-the-gpu-8-bits-per-radix/2023-12-20-SebAaltonen-i-like-the-single-pass-per-pass-approach-but.md new file mode 100644 index 0000000..5890d39 --- /dev/null +++ b/archive/threads/kechogarcia/2023-12-20-stable-radix-sorting-in-the-gpu-8-bits-per-radix/2023-12-20-SebAaltonen-i-like-the-single-pass-per-pass-approach-but.md @@ -0,0 +1,93 @@ +--- +title: "@kechogarcia I like the single pass (per pass) approach." +type: archive +source: twitter +source_url: "https://x.com/SebAaltonen/status/1737372016905204175" +author: "Sebastian Aaltonen" +handle: SebAaltonen +post_id: "1737372016905204175" +date: 2023-12-20 +archived: 2026-08-26 +draft: false +tags: + - archive + - twitter + - kechogarcia +description: "@kechogarcia I like the single pass (per pass) approach." +in_reply_to: "" +parent_post_id: "1737288039603269648" +--- + +## Source + +- URL: https://x.com/SebAaltonen/status/1737372016905204175 +- Author: Sebastian Aaltonen (@SebAaltonen) +- Posted: 2023-12-20 07:19:02 + +## Branch + +**1/** **@SebAaltonen** ^1737372016905204175 + +**@kechogarcia** + +I like the single pass (per pass) approach. But unfortunately it's not doable using general API due to tight loops not having forward progress guarantees ("sufficiently fair scheduling"). Also doable on AMD consoles using ordered count instructions. + +https://developer.download.nvidia.com/video/gputechconf/gtc/2020/presentations/s21572-a-faster-radix-sort-implementation.pdf + +**2/** **@kechogarcia** ^1737456119511973917 + +**@SebAaltonen** + +There's a very neat trick from AMD folk to sync groups. You do an interlockadd(1) to a dword in ram after group is done, and the last thread finished gets gets thread count. Have to use globalcoherent on the uavs you want to sync across. Haven't experimented w this but looks cool + +**3/** **@SebAaltonen** ^1737463338509152338 + +**@kechogarcia** + +Yes the last group gets the group count if you do that and it knows it's the last group, but other groups are dead already. They can't spin waiting until all other groups have finished. + +**4/** **@kechogarcia** ^1737464423080427897 + +**@SebAaltonen** + +Yeah spinning is fishy. In nanite culling they do sort of spin however, they just move to other work. Their persistant culling shader is basically polling a globalcoherent buffer to get more work from the intermediate nodes of the DAG🙈and hiding the spin with other work. + +**5/** **@SebAaltonen** ^1737466890753069229 + +**@kechogarcia** + +That only works if you never spin waiting for work. If work runs out and you still spin, then you can deadlock. It's possible that the same CU has the actual work producer and the consumer, and consumer wave spinning gets all the execution time in the SIMD. The other wave waits. + +**6/** **@kechogarcia** ^1737480192749187544 + +**@SebAaltonen** + +I think horizon guys, for vertex processing of vegetation, have a ring buffer in turn & don't spin, but wait on a label on the GPU. Something which can't be done on PC hlsl :( + +**7/** **@NOTimothyLottes** ^1737485387637940347 + +**@kechogarcia** **@SebAaltonen** + +Persistent waves on PC: {dispatch A, dispatch B} -> workgroups of A get on GPU before workgroups of B start. If both A and B run the same shader, if one needs a spinning wait in A, the wave could exit, then depend on relaunch in B to fix wave priorities to get forward progress... + +**8/** **@NOTimothyLottes** ^1737486229631827979 + +**@kechogarcia** **@SebAaltonen** + +... so N dispatches in theory provides ability to run N spinninging waits per wave ... the wave exit and relaunch priority shift would in theory avoid burning cost on the busy wait ... of course this all assumes the wait is at a stateless point in execution :) + +**9/** **@SebAaltonen** ^1737542645059031420 + +**@NOTimothyLottes** **@kechogarcia** + +Yes, but there's no API in DirectX or Vulkan for relaunching waves. + +**10/** **@NOTimothyLottes** ^1737555411941958063 + +**@SebAaltonen** **@kechogarcia** + +Either launch more workgroups than you need (or do extra dispatches). Either way though, this needs some dynamic adaption to avoid the overlaunch overhead. + +## Related + +- Spine: [[archive/threads/kechogarcia/2023-12-20-stable-radix-sorting-in-the-gpu-8-bits-per-radix]] diff --git a/archive/threads/kechogarcia/2023-12-20-stable-radix-sorting-in-the-gpu-8-bits-per-radix/2023-12-20-aras_p-noice-any-ideas-how-it-compares-to-say-amd.md b/archive/threads/kechogarcia/2023-12-20-stable-radix-sorting-in-the-gpu-8-bits-per-radix/2023-12-20-aras_p-noice-any-ideas-how-it-compares-to-say-amd.md new file mode 100644 index 0000000..636c1bd --- /dev/null +++ b/archive/threads/kechogarcia/2023-12-20-stable-radix-sorting-in-the-gpu-8-bits-per-radix/2023-12-20-aras_p-noice-any-ideas-how-it-compares-to-say-amd.md @@ -0,0 +1,43 @@ +--- +title: "@kechogarcia Noice! Any ideas how it compares to say AMD FidelityFX radix sort? I have ported it to unity for gaussian splat needs (hlsl https://github.com/aras-p/UnityGaussianSplatting/blob/fded9c0/package/Shaders/GpuSortFidelityFX.hlsl and c# https://github.com/aras-p/UnityGaussianSplatting/blob/fded9c0/package/Runtime/GpuSorting.cs ) but I don't know if it's \"good\" compared to any other impls." +type: archive +source: twitter +source_url: "https://x.com/aras_p/status/1737463905922830691" +author: "Aras Pranckevičius 🇺🇦🇱🇹" +handle: aras_p +post_id: "1737463905922830691" +date: 2023-12-20 +archived: 2026-08-26 +draft: false +tags: + - archive + - twitter + - kechogarcia +description: "@kechogarcia Noice! Any ideas how it compares to say AMD FidelityFX radix sort? I have ported it to unity for gaussian splat needs (hlsl https://github.com/aras-p/UnityGaussianSplatting/blob/fded9c0/package/Shaders/GpuSortFidelityFX.hlsl and c# https://github.com/aras-p/UnityGaussianSplatting/blob/fded9c0/package/Runtime/GpuSorting.cs ) but I don't know if it's \"good\" compared to any other impls." +in_reply_to: "" +parent_post_id: "1737288039603269648" +--- + +## Source + +- URL: https://x.com/aras_p/status/1737463905922830691 +- Author: Aras Pranckevičius 🇺🇦🇱🇹 (@aras_p) +- Posted: 2023-12-20 13:24:10 + +## Branch + +**1/** **@aras_p** ^1737463905922830691 + +**@kechogarcia** + +Noice! Any ideas how it compares to say AMD FidelityFX radix sort? I have ported it to unity for gaussian splat needs (hlsl https://github.com/aras-p/UnityGaussianSplatting/blob/fded9c0/package/Shaders/GpuSortFidelityFX.hlsl and c# https://github.com/aras-p/UnityGaussianSplatting/blob/fded9c0/package/Runtime/GpuSorting.cs ) but I don't know if it's "good" compared to any other impls. + +**2/** **@kechogarcia** ^1737464880964206908 + +**@aras_p** + +I haven't done a deep optimization yet. But I saw their code yesterday from your post, and from the looks is not stable, so likely theirs is faster because they use less LDS on Step 1 and give up on stableness. + +## Related + +- Spine: [[archive/threads/kechogarcia/2023-12-20-stable-radix-sorting-in-the-gpu-8-bits-per-radix]] diff --git a/archive/threads/kechogarcia/2023-12-20-stable-radix-sorting-in-the-gpu-8-bits-per-radix/index.md b/archive/threads/kechogarcia/2023-12-20-stable-radix-sorting-in-the-gpu-8-bits-per-radix/index.md new file mode 100644 index 0000000..5472745 --- /dev/null +++ b/archive/threads/kechogarcia/2023-12-20-stable-radix-sorting-in-the-gpu-8-bits-per-radix/index.md @@ -0,0 +1,118 @@ +--- +title: "Stable radix sorting in the GPU (8 bits per radix)." +type: archive +source: twitter +source_url: "https://x.com/kechogarcia/status/1737288025107816560" +author: "Kecho" +handle: kechogarcia +post_id: "1737288025107816560" +date: 2023-12-20 +archived: 2026-08-26 +draft: false +tags: + - archive + - twitter + - kechogarcia +description: "Stable radix sorting in the GPU (8 bits per radix)." +in_reply_to: "" +--- + +## Source + +- URL: https://x.com/kechogarcia/status/1737288025107816560 +- Author: Kecho (@kechogarcia) +- Posted: 2023-12-20 01:45:17 + +## Thread + +**1/** **@kechogarcia** ^1737288025107816560 + +Stable radix sorting in the GPU (8 bits per radix). Not many accessible resources to do it stable. Here it goes: + + 1. Count & Scatter + 2. Prefix batch table + 3. Global Prefix + 4. Scatter Output + 5. Repeat 1-4 for each radix. + +(Details & code in thread) + +**2/** **@kechogarcia** ^1737288026416398688 + +Step1 (Count & Scatter). +Split input in batches. Have a thread group in your compute shader tackle a set of batches: + +for (uint i = groupThreadID ; i < totalInBatch; i+= GROUPSIZE) +{ + //do some work +} + +![](https://pbs.twimg.com/media/GBwMbVqWAAAL2-l?format=png&name=orig) + +**3/** **@kechogarcia** ^1737288028081598820 + +The work in this kernel involves filling a "local offset buffer" (LOB) (source of stability) and a "radix count batch table" ) RBT + +RBT is a 256 table of dwords (histogram for each batch) and the LOB has 1 entry for each input entry. + +**4/** **@kechogarcia** ^1737288029423747502 + +The LOB's goal is to have the local offset for each element with respect to its radix. +i.e. a list that has [0, 1, 3, 1, 3, 4] will have a lob of [0,0,0,1,1,0]. The 1's are there because these are the stable offsets for repeated radixes. + +**5/** **@kechogarcia** ^1737288030744920498 + +To compute the LOB, we first have a Local Thread Bit Cache (LTBC) in LDS. This LTBC is 256 elements (1 for each radix) and gets initialized to 0 on each iteration of the loop. +Each LTBC will have 1 bit for each thread in the group. So if a group has 64 threads, its 2 dwords. + +**6/** **@kechogarcia** ^1737288032149967144 + +On each iteration of the loop, we will InterlockOr the bit of the corresponding thread for the corresponding entry. We sync groups, then do a prefix popcnt (countbits in hlsl) to count the number of previous elements seen for that particular radix, at that iteration. + +**7/** **@kechogarcia** ^1737288033651642415 + +g_threadBit to 0 +InterlockOr(g_threadBit[radix], 1 << groupThreadID); +Sync(); +g_lobOutput[i] = g_lobOffset[radix] + countbits(g_threadBit[radix] - 1); +g_lobOffset[radix] += countbits(g_localRadix[radix]); + +**8/** **@kechogarcia** ^1737288035111260177 + +Step 2 (Prefix batch table) +We do a Hillis Steele Scan on the RBT of step 1, and spit out a prefix batch table (PBT) and a Global Radix count (GRC) + +![](https://pbs.twimg.com/media/GBwUua9WcAA7pyp?format=jpg&name=orig) + +**9/** **@kechogarcia** ^1737288036700787084 + +Step 3 global prefix table (GPT), this holds the global offset of each radix. Again a hillis steele scan. + +![](https://pbs.twimg.com/media/GBwVSwTXUAAqqEl?format=png&name=orig) + +**10/** **@kechogarcia** ^1737288038307213755 + +Step 4: Scatter ouput. + +outputIndex = + GPT[radix] + + PBT[radix*batchCount + batchIndex] + + LOB[inputIndex]; + +g_output[outputIndex] = g_input[iputIndex] + +**11/** **@kechogarcia** ^1737288039603269648 + +Go to the next radix, use the output as the new input and repeat steps 1 through 4. + +Code can be found here: +https://github.com/kecho/gpu_algorithms/blob/main/gpu_algorithms/gpu/radix_sort.hlsl + +Kernel scheduling in coalpy: +https://github.com/kecho/gpu_algorithms/blob/main/gpu_algorithms/gpu/radix_sort.py + +Perf numbers for 20 million items on a RTX 2070 Super + +![](https://pbs.twimg.com/media/GBwWOXJXAAESwoQ?format=png&name=orig) + +Branches: [[archive/threads/kechogarcia/2023-12-20-stable-radix-sorting-in-the-gpu-8-bits-per-radix/2023-12-20-SebAaltonen-i-like-the-single-pass-per-pass-approach-but]], [[archive/threads/kechogarcia/2023-12-20-stable-radix-sorting-in-the-gpu-8-bits-per-radix/2023-12-20-aras_p-noice-any-ideas-how-it-compares-to-say-amd]] diff --git a/archive/threads/kechogarcia/index.md b/archive/threads/kechogarcia/index.md new file mode 100644 index 0000000..29e8f29 --- /dev/null +++ b/archive/threads/kechogarcia/index.md @@ -0,0 +1,12 @@ +--- +title: kechogarcia +type: note +draft: false +description: Archived threads by kechogarcia. +tags: + - archive + - twitter + - kechogarcia +--- + +- [[archive/threads/kechogarcia/2023-12-20-stable-radix-sorting-in-the-gpu-8-bits-per-radix]]