1898272635277967509, 1884730404587909544, 1884733082474775028
This commit is contained in:
ed committed
1 parent
c1aed52bd3
commit
f44a6f55d1
7 files changed
+344
No files matched your search
+69
@@ -0,0 +1,69 @@
|
||||
---
|
||||
title: "@FFmpeg I know a person with a PhD that might disagree with you"
|
||||
type: archive
|
||||
source: twitter
|
||||
source_url: "https://x.com/mu_chrinovic/status/1884733082474775028"
|
||||
author: "Chrinovic Mukanya"
|
||||
handle: mu_chrinovic
|
||||
post_id: "1884733082474775028"
|
||||
date: 2025-01-29
|
||||
archived: 2026-08-25
|
||||
draft: false
|
||||
tags:
|
||||
- archive
|
||||
- twitter
|
||||
- FFmpeg
|
||||
description: "@FFmpeg I know a person with a PhD that might disagree with you"
|
||||
in_reply_to: ""
|
||||
parent_post_id: "1884730404587909544"
|
||||
---
|
||||
|
||||
## Source
|
||||
|
||||
- URL: https://x.com/mu_chrinovic/status/1884733082474775028
|
||||
- Author: Chrinovic Mukanya (@mu_chrinovic)
|
||||
- Posted: 2025-01-29 22:39:18
|
||||
|
||||
## Branch
|
||||
|
||||
**1/**
|
||||
|
||||
@FFmpeg I know a person with a PhD that might disagree with you
|
||||
|
||||
**2/**
|
||||
|
||||
@mu_chrinovic @FFmpeg Yeah PTX is not assembly, it's a mid level language that is closer to a 1:1 match for the HW ISA --- BUT --- the actual register allocation and management is still done by a high-level compiler ...
|
||||
|
||||
**3/**
|
||||
|
||||
@NOTimothyLottes @mu_chrinovic @FFmpeg That's almost semantics at this point though? If you're writing x86 assembly it gets translated to microcode in the CPU as well, and will do register remapping etc. Only difference in Nvidia case that's happening in the driver, right?
|
||||
|
||||
**4/**
|
||||
|
||||
@janekm @mu_chrinovic @FFmpeg Sounds like you are trying to claim software compiler register allocation is the same as HW runtime register renaming? No. CPU register renaming maps a small set of registers to a larger set to get IPC. Compiler GPU reg allocation maps a massive SSA mess to a small set.
|
||||
|
||||
**5/**
|
||||
|
||||
@NOTimothyLottes @mu_chrinovic @FFmpeg Fair point, my argument was merely that both cases of assembly language are really an intermediate language that gets translated to the final instructions executed by the CPU/GPU. Feels quite pedantic to claim that PTX assembly is not assembly, merely because of reg allocation 🤷♂️
|
||||
|
||||
**6/**
|
||||
|
||||
@janekm @mu_chrinovic @FFmpeg SOFTWARE
|
||||
Assembler = human writes the physical HW instructions
|
||||
Compiler = program figures out the instructions from some other form
|
||||
----DIFFERENT-THINGS----
|
||||
HW
|
||||
Instruction-reordering / wave-scheduling
|
||||
Register renaming
|
||||
|
||||
**7/**
|
||||
|
||||
@NOTimothyLottes @mu_chrinovic @FFmpeg We've been calling it Assembly since at least when JVM byte code was first a thing. Doesn't matter that a JIT compiler will still do register allocation.
|
||||
|
||||
**8/**
|
||||
|
||||
@janekm @mu_chrinovic @FFmpeg Sure people have been incorrectly labeling stuff for a long time
|
||||
|
||||
## Related
|
||||
|
||||
- Spine: [[archive/threads/FFmpeg/2025-01-29-ptx-is-a-form-of-assembly-language]]
|
||||
@@ -0,0 +1,36 @@
|
||||
---
|
||||
title: "PTX is a form of assembly language"
|
||||
type: archive
|
||||
source: twitter
|
||||
source_url: "https://x.com/FFmpeg/status/1884730404587909544"
|
||||
author: "FFmpeg"
|
||||
handle: FFmpeg
|
||||
post_id: "1884730404587909544"
|
||||
date: 2025-01-29
|
||||
archived: 2026-08-25
|
||||
draft: false
|
||||
tags:
|
||||
- archive
|
||||
- twitter
|
||||
- FFmpeg
|
||||
description: "PTX is a form of assembly language"
|
||||
in_reply_to: ""
|
||||
---
|
||||
|
||||
## Source
|
||||
|
||||
- URL: https://x.com/FFmpeg/status/1884730404587909544
|
||||
- Author: FFmpeg (@FFmpeg)
|
||||
- Posted: 2025-01-29 22:28:39
|
||||
|
||||
## Thread
|
||||
|
||||
**1/**
|
||||
|
||||
PTX is a form of assembly language
|
||||
|
||||
Branches: [[archive/threads/FFmpeg/2025-01-29-ptx-is-a-form-of-assembly-language/2025-01-29-mu_chrinovic-i-know-a-person-with-a-phd-that-might-disagree]]
|
||||
|
||||
**2/**
|
||||
|
||||
https://docs.nvidia.com/cuda/inline-ptx-assembly/index.html
|
||||
@@ -0,0 +1,12 @@
|
||||
---
|
||||
title: FFmpeg
|
||||
type: note
|
||||
draft: false
|
||||
description: Archived threads by FFmpeg.
|
||||
tags:
|
||||
- archive
|
||||
- twitter
|
||||
- FFmpeg
|
||||
---
|
||||
|
||||
- [[archive/threads/FFmpeg/2025-01-29-ptx-is-a-form-of-assembly-language]]
|
||||
+182
@@ -0,0 +1,182 @@
|
||||
---
|
||||
title: "@SebAaltonen Actually this is a super important question IMO."
|
||||
type: archive
|
||||
source: twitter
|
||||
source_url: "https://x.com/NOTimothyLottes/status/1898352640825651644"
|
||||
author: "NOTimothyLottes"
|
||||
handle: NOTimothyLottes
|
||||
post_id: "1898352640825651644"
|
||||
date: 2025-03-08
|
||||
archived: 2026-08-25
|
||||
draft: false
|
||||
tags:
|
||||
- archive
|
||||
- twitter
|
||||
- SebAaltonen
|
||||
description: "@SebAaltonen Actually this is a super important question IMO."
|
||||
in_reply_to: ""
|
||||
parent_post_id: "1898272635277967509"
|
||||
---
|
||||
|
||||
## Source
|
||||
|
||||
- URL: https://x.com/NOTimothyLottes/status/1898352640825651644
|
||||
- Author: NOTimothyLottes (@NOTimothyLottes)
|
||||
- Posted: 2025-03-08 12:38:33
|
||||
|
||||
## Branch
|
||||
|
||||
**1/**
|
||||
|
||||
@SebAaltonen Actually this is a super important question IMO.
|
||||
|
||||
For a review of how this works on NVIDIA: https://arxiv.org/pdf/1903.07486
|
||||
6 physical dependency barriers
|
||||
opcodes have {6-bit wait barrier mask, 3-bit read barrier index, 3-bit write barrier index}
|
||||
|
||||
So don't need a u64 bitmask
|
||||
|
||||
**2/**
|
||||
|
||||
@SebAaltonen AMD pre-GCN was a fixed TEX/ALU clause based arch. GCN went to their counter system (decade+ back). So there is a very long compiler & HW design technical debt inertia probably keeping the HW design on this counter based evolutionary tract
|
||||
|
||||
**3/**
|
||||
|
||||
@SebAaltonen K$ loads are out of order, so the only counter that makes any sense is 0, ie waiting until all prior loads have finished.
|
||||
|
||||
VMEM loads had traditionally been in-order only so counter made more sense. AMD's compiler would trickle-feed waits so tiny bits of VALU could move forward..
|
||||
|
||||
**4/**
|
||||
|
||||
@SebAaltonen Vs say pre-GCN which was all-or-nothing. And vs NV which is all (for the mask of 6 barriers or nothing). So I think the counter design mixed with in-order VMEM was an attempt to reduce latency at the time
|
||||
|
||||
**5/**
|
||||
|
||||
@SebAaltonen And also keep in mind TEX$ had traditionally been very small with respect to the number of clients (waves). So cacheline life was quite short, so in-order with counter like this might have had line-life benefits
|
||||
|
||||
**6/**
|
||||
|
||||
@SebAaltonen However in modern times TEX$ is bigger, and ALU issue is faster (compared to GCN's round robin wave64 on simd16). Also modern GPUs have operand caching. And trickle feeding waitcnts would break operand caching (more irregular switching to other waves) ...
|
||||
|
||||
**7/**
|
||||
|
||||
@SebAaltonen And at the point where vector memory requests can go out-of-order (hit-under-miss, etc), counters are just legacy baggage that might not really make sense any more ...
|
||||
|
||||
**8/**
|
||||
|
||||
@SebAaltonen Also the elephant in the room, dynamic wave scheduling. In theory with a chunky granularity design like NVIDIA, NVIDIA could build in hardware wave scheduling something that tries to keep waves at similar barriers! ->
|
||||
|
||||
**9/**
|
||||
|
||||
@SebAaltonen -> Meaning better uniform forward progress of groups of waves. Trying to keep higher cache reuse. Where as AMD oldest first does allow the older wave(s) to get extremely out-of-sync with possible cache reuse across waves
|
||||
|
||||
**10/**
|
||||
|
||||
@SebAaltonen On AMD I've always wanted to try to play with waitcnt placement for better dynamic scheduling (chunky vs fine-granularity, etc) BUT AMD refuses to ever allow ASM shaders in CS VK (even though ROCm supports ASM shaders without VK) ->
|
||||
|
||||
**11/**
|
||||
|
||||
@SebAaltonen I do know traditionally that AMD often has a lot of trouble getting to 100% utilization of ALU-bound stuff. Typically sitting in the low 90%s instead, all due to wave scheduling (oldest first VALU base camping, preventing other waves to get VMEM issue out = bubbles) ->
|
||||
|
||||
**12/**
|
||||
|
||||
@SebAaltonen AMD also wouldn't provide any access to s_setprio via intrinsics. So the two most important scheduling tools {waitcnt placement/grouping,setprio} are simply non-accessible to us "idiots" (who could never actually responsibly use intrinsics)
|
||||
|
||||
**13/**
|
||||
|
||||
@SebAaltonen So what we actually do on AMD, is to do semi-persistent (or fully persistent) waves that instead of exiting, loop through groups of work instead. And this can sometimes actually fix HW scheduling issues by depending on oldest-first schedule behavior ->
|
||||
|
||||
**14/**
|
||||
|
||||
@SebAaltonen Effectively the more scheduling one takes away from the dynamic HW scheduler and places instead into the static scheduling of a compiler, often seriously improves behavior by removing the worst case dynamic irregular behavior at runtime ->
|
||||
|
||||
**15/**
|
||||
|
||||
@SebAaltonen I say "sometimes" because if you get out of I$ you hit perf death. And if the compiler explodes VGPR count then typically it can also be perf-death too
|
||||
|
||||
**16/**
|
||||
|
||||
@SebAaltonen Historically the highest utilization stuff on AMD hardware would be 4-2 waves/SIMD actually [multiple work/wave] ... which will come to a surprise to UE material shader people who are dead in the water on performance at that wave count [single work/wave] ...
|
||||
|
||||
**17/**
|
||||
|
||||
@SebAaltonen [IRONY] those who actually know what they are doing on AMD are fixed high-register-count low-wave-count persistent-wave folks with unrolling and static scheduling-focused (VS) the masses are lower-register-count high-wave-count non-persistent-wave dynamic-sched focused
|
||||
|
||||
**18/**
|
||||
|
||||
@SebAaltonen One other point: in HW, how many waves should one actually keep live? The answer is only enough to reach high highest perf. But how does one build hardware to do that? With NVIDIA's design I think it is a lot easier, with the chunky barriers
|
||||
|
||||
**19/**
|
||||
|
||||
@NOTimothyLottes I said WG_RR_EN (round-robin arbitration) in the RDNA4 ISA-docs, but no explanation. Wondering what this is.
|
||||
|
||||
**20/**
|
||||
|
||||
@SebAaltonen I think it means oldest-workgroup first scheduling with round-robin scheduling across waves in the workgroups at the given workgroup priorities.
|
||||
|
||||
**21/**
|
||||
|
||||
@NOTimothyLottes @SebAaltonen What about losing shader occupancy and the latency-hiding benefits it provides for memory ops?
|
||||
|
||||
**22/**
|
||||
|
||||
@AgileJebrim @SebAaltonen The higher the occupancy the higher the clients on the caches = the lower the cache/client. So you never actually want highest occupancy, you want lowest occupancy that enables just enough latency hiding to keep highest utilization ...
|
||||
|
||||
**23/**
|
||||
|
||||
@AgileJebrim @SebAaltonen ... To be more detailed, what you really really want, is to keep the memory requests grouped by locality (for highest cache hit rate) and NO unnecessary bubbles in the memory request queues that would result in ALU stalling !!! - this is basic real-world supply chain management
|
||||
|
||||
**24/**
|
||||
|
||||
@NOTimothyLottes @SebAaltonen Now what if you want to reuse the same shared memory buffer for a larger workgroup rather than needlessly duplicating it with many smaller workgroups? Wouldn’t that be an argument for a larger workgroup size than 64,1,1?
|
||||
|
||||
**25/**
|
||||
|
||||
@AgileJebrim @SebAaltonen Right, that is the CU-sized workgroup. Something that takes over the full CU. That is one of the persistent-wave usage models. But a CU-sized workgroup for non-persistent work is perf-death, because of the idle time during workgroup barriers (horrible with oldest-wave sched)
|
||||
|
||||
**26/**
|
||||
|
||||
@NOTimothyLottes @SebAaltonen I’m not as well versed in AMD. 4 SIMDs per CU, so do 64x4 = 256 invocations per workgroup?
|
||||
|
||||
**27/**
|
||||
|
||||
@AgileJebrim @SebAaltonen That topic gets complex with RDNA: CU vs WGP mode, wave32 vs wave64, 8 vs 12 VGPRs per allocation granularity, etc
|
||||
|
||||
**28/**
|
||||
|
||||
@NOTimothyLottes @SebAaltonen Managed to get this info out of Grok.
|
||||
|
||||
|
||||
|
||||
**29/**
|
||||
|
||||
@NOTimothyLottes @SebAaltonen Oh.
|
||||
|
||||
|
||||
|
||||
**30/**
|
||||
|
||||
@NOTimothyLottes @SebAaltonen Seems like just using 128 is a safe bet for most hardware?
|
||||
|
||||
**31/**
|
||||
|
||||
@AgileJebrim @SebAaltonen Yeah safe bet like one-size-fits-all unisex underwear
|
||||
|
||||
**32/**
|
||||
|
||||
@NOTimothyLottes @SebAaltonen 😆
|
||||
|
||||
**33/**
|
||||
|
||||
On the flip side, it appears that using 128 would only use half the register file capacity on each. If we did 256 instead, then when the hardware scheduler became active, I imagine it’d only have a single choice to choose from for another thread to schedule and can use 255-256 registers each?
|
||||
|
||||
|
||||
|
||||
|
||||
**34/**
|
||||
|
||||
@NOTimothyLottes @SebAaltonen I’m imagining scheduler going back and forth between the same two SIMDs but dunno if such even distribution is what would actually happen in practice?
|
||||
|
||||
## Related
|
||||
|
||||
- Spine: [[archive/threads/SebAaltonen/2025-03-08-stupid-hardware-question-why-does-amd-use-several]]
|
||||
+43
@@ -0,0 +1,43 @@
|
||||
---
|
||||
title: "Stupid hardware question: Why does AMD use several counters instead of one u64 bitmask for memory ops? "
|
||||
type: archive
|
||||
source: twitter
|
||||
source_url: "https://x.com/SebAaltonen/status/1898272635277967509"
|
||||
author: "Sebastian Aaltonen"
|
||||
handle: SebAaltonen
|
||||
post_id: "1898272635277967509"
|
||||
date: 2025-03-08
|
||||
archived: 2026-08-25
|
||||
draft: false
|
||||
tags:
|
||||
- archive
|
||||
- twitter
|
||||
- SebAaltonen
|
||||
description: "Stupid hardware question: Why does AMD use several counters instead of one u64 bitmask for memory ops? "
|
||||
in_reply_to: ""
|
||||
---
|
||||
|
||||
## Source
|
||||
|
||||
- URL: https://x.com/SebAaltonen/status/1898272635277967509
|
||||
- Author: Sebastian Aaltonen (@SebAaltonen)
|
||||
- Posted: 2025-03-08 07:20:39
|
||||
|
||||
## Thread
|
||||
|
||||
**1/**
|
||||
|
||||
Stupid hardware question: Why does AMD use several counters instead of one u64 bitmask for memory ops?
|
||||
Each load/store would have a bit index and wait would have a bitmask (marked bits need to be set = finished).
|
||||
|
||||
|
||||
|
||||
Branches: [[archive/threads/SebAaltonen/2025-03-08-stupid-hardware-question-why-does-amd-use-several/2025-03-08-NOTimothyLottes-actually-this-is-a-super-important-question-imo]]
|
||||
|
||||
**2/**
|
||||
|
||||
I am glad that RDNA4 has separate counters for sampling (long latency) and lower latency raw loads now. That's a good improvement.
|
||||
|
||||
**3/**
|
||||
|
||||
Some thoughts: Compiler managed bitfield slot indices would be iffy with loops and branches. One bit per register (big bitfield) would always work, but that would be much more than 64 bits. And wait instructions would be iffy (how to describe a massive bitfield).
|
||||
@@ -11,4 +11,5 @@ tags:
|
||||
|
||||
- [[archive/threads/SebAaltonen/2024-04-04-this-is-how-slow-code-feels-like]]
|
||||
- [[archive/threads/SebAaltonen/2024-05-03-why-should-one-be-able-to-code-without]]
|
||||
- [[archive/threads/SebAaltonen/2025-03-08-stupid-hardware-question-why-does-amd-use-several]]
|
||||
- [[archive/threads/SebAaltonen/2025-03-28-indoor-scenes-in-hypehype-are-starting-to-look]]
|
||||
@@ -13,6 +13,7 @@ How to read a folder: [[notes/How threads are organized]].
|
||||
|
||||
Example of the note shape (single spine, no branches): [[archive/threads/How to archive a thread]]
|
||||
|
||||
- [[archive/threads/FFmpeg]]
|
||||
- [[archive/threads/NOTimothyLottes]]
|
||||
- [[archive/threads/SebAaltonen]]
|
||||
- [[archive/threads/VPCOMPRESSB]]
|
||||
|
||||
Reference in new issue
Block a user